Merge branch 'main' into worktree-messaging-platform

# Conflicts:
#	src/config/spotbugs/exclude.xml
This commit is contained in:
DongHyeonka
2026-08-14 15:15:37 +09:00
1342 changed files with 94911 additions and 374 deletions
@@ -0,0 +1,35 @@
# ADR-JPA-001 — The domain owns the persistence model
- Status: Accepted
- Date: 2026-08-11
- Design: §10.1, §23.3
## Context
A persistence platform can either own the repository abstraction — a `GenericRepository<T, ID>`
every aggregate inherits — or provide only the pieces domains assemble themselves.
## Decision
The domain owns entities, embeddables, repositories, queries, index requirements, and lock,
soft-delete, and audit policy. The platform provides no generic CRUD repository and no base
repository. `JpaRepositoryFragmentSupport` exists, has no `save`, `findById`, `findAll`, or
`delete`, and is enforced not to acquire them.
## Consequences
A generic base repository has one property that looks like a benefit and is not: every aggregate
gets the same operations. That means each aggregate is offered operations that may be wrong for it —
a `delete` on an append-only ledger, a `findAll` on a table that will never be small — and, worse,
one aggregate's later requirement changes the shared base and therefore changes behaviour for
aggregates nobody reviewed.
Spring Data already implements CRUD. Re-implementing it adds a layer whose only function is to be
harder to opt out of.
The cost is a small amount of repetition: each domain declares the repository interface it needs.
That repetition is the thing that makes each aggregate's persistence surface reviewable.
## Enforcement
`JpaArchitectureRules.noGenericRepository()`; `JpaRepositoryFragmentSupportTest`.
@@ -0,0 +1,37 @@
# ADR-JPA-002 — Retry re-runs the whole use case
- Status: Accepted
- Date: 2026-08-11
- Design: §19.2
## Context
Optimistic conflicts, deadlocks, and serialization failures are recoverable. The question is what
unit gets retried: the failed statement, the transaction, or the use case.
## Decision
The whole use case, in a new transaction with a new Persistence Context.
`FullTransactionRetryCoordinator` re-enters `JpaTransactionExecutor` for every attempt, and the
retry advice is ordered outside Spring's transaction advice so each attempt begins a new
transaction.
## Consequences
Statement-level retry is wrong for exactly the failures being retried. An optimistic conflict means
the state the attempt computed against is no longer the committed state; re-issuing the same
statement computes the same wrong answer against a version that has moved on. The domain rules have
to run again over reloaded data, which means the whole use case.
Reusing the Persistence Context would be equally wrong: the second attempt would read the first
attempt's stale entities out of the first-level cache. And with the advice ordering inverted, the
retry loop would run inside one transaction that has already been marked rollback-only, so the
second attempt fails immediately without executing anything.
The cost is that a retryable use case must be safe to run from scratch — no irreversible external
effect before the commit. `IrreversibleSideEffectContext` lets a use case declare when that does not
hold, and the policy then refuses to retry it whatever budget remains.
## Enforcement
`FullTransactionRetryCoordinatorTest`; `RetryableJpaTransactionInterceptor.DEFAULT_ORDER`.
@@ -0,0 +1,41 @@
# ADR-JPA-003 — Completion unknown is never retried
- Status: Accepted
- Date: 2026-08-11
- Design: §17
## Context
A connection can break while a commit is in flight. The server may have committed; the
acknowledgement may simply have been lost. The driver cannot tell the two apart.
## Decision
`TransactionCompletionUnknownException` is never retried, automatically or otherwise. It is
produced only by a failure observed while the transaction phase is `COMMITTING`, and only for
SQLSTATE `40003`, a connection-class (`08*`) state, or a transport break. Recovery is
domain-specific reconciliation through `TransactionCompletionResolver`.
## Consequences
Retrying a possibly-committed write is the most damaging thing this platform could do: a duplicate
payment, a duplicate order, a double decrement. There is no budget or backoff that makes it safe,
because the failure is epistemic rather than transient.
The invariant is enforced at the type level rather than by policy alone. `JpaFailureContext` refuses
to construct a retryable completion-unknown context, and the exception rebuilds its context through
the safe factory whatever it is handed. A future policy bug therefore cannot produce an unsafe
retry — the value it would need does not exist.
The rule is deliberately narrow in the other direction too. Classifying every connection failure as
completion-unknown would push ordinary pool exhaustion and server restarts into the reconciliation
queue, which trains operators to clear that queue without reading it — and then the one entry that
mattered gets cleared with the rest.
The cost is that the domain must supply the resolver. The platform cannot: only the domain knows
which idempotency record, business row, or outbox entry proves the write happened.
## Enforcement
`JpaFailureContextTest`; `DefaultJpaRetryPolicyTest`; `CommitFailureClassifierTest`; release gate
`completion-unknown-no-retry`.
@@ -0,0 +1,37 @@
# ADR-JPA-004 — Flyway is the schema source of truth
- Status: Accepted
- Date: 2026-08-11
- Design: §31
## Context
Hibernate can create and alter schema from the entity mapping. Flyway can apply versioned scripts.
Both cannot own the schema.
## Decision
Flyway owns every schema change. Hibernate validates and never mutates: `ddl-auto` is `validate` or
`none`, enforced at startup. The runtime database credential holds no DDL privilege, so the rule is
enforced by the server as well as by configuration.
## Consequences
`ddl-auto=update` fails in a specific and expensive way: it adds but never drops or narrows, so the
result is a schema that is neither the previous one nor the one the mappings describe — produced
silently, by whichever instance started first, with no record of what it did.
Two credentials rather than one is what makes this more than a convention. A configuration rule can
be overridden by a property; a role without `CREATE` cannot be overridden by anything the
application does.
Validation fails closed and never repairs. `repair` rewrites the schema history to match the scripts
on disk, which resolves a checksum mismatch by deleting the evidence of which change is missing.
The cost is that a schema change requires a migration script and a deployment step. That is the
intended cost: it makes schema change reviewable and reversible.
## Enforcement
`JpaDangerousConfigurationGuard`; `FlywaySchemaPolicy`; `FlywayValidationGate`;
`PostgreSqlRuntimeRoleVerifier`; release gates `flyway-validate` and `runtime-role-no-ddl`.
@@ -0,0 +1,38 @@
# ADR-JPA-005 — Contracts run against real PostgreSQL
- Status: Accepted
- Date: 2026-08-11
- Design: §40
## Context
An in-memory database makes tests fast and hermetic. A container makes them slow and requires
Docker.
## Decision
Every persistence contract runs against real PostgreSQL 16, 17, and 18 in containers. H2 remains a
local-development convenience and never satisfies a contract. The lanes fail closed when Docker is
absent rather than skipping.
## Consequences
The behaviours these contracts verify either do not exist in H2 or differ there: SQLSTATE values for
the same violation, `FOR UPDATE SKIP LOCKED` semantics, JSONB operators, range types, concurrent
index builds, `search_path` privileges, and the generated SQL for a paged collection fetch. A green
H2 run is evidence that the code compiles and runs — not that any of the above holds.
Three versions rather than one because the platform claims three. A contract suite that ran only on
16 would make "Stable on 17 and 18" an assumption.
Skipping on missing Docker is the failure mode this decision most wants to avoid: a skipped contract
reports success, and CI eventually inherits that silence. `PostgreSqlContainerFactory.assertDockerAvailable()`
throws instead.
The cost is that the contract lanes need Docker and take minutes. The unit lane stays hermetic and
fast, and is where most tests live; the container lanes verify the things only a real server can
answer.
## Enforcement
`PostgreSqlVersion.stable()`; `PostgreSqlContainerFactory`; release gate `postgresql-contract`.
@@ -0,0 +1,63 @@
# ADR-MONGO-001 — MongoDB platform boundary
- **Status:** Accepted
- **Date:** 2026-08-13
- **Design source:** `mongodb-superpowers-package/.../2026-08-11-mongodb-document-persistence-platform-design.md` §1, §2 (D-01, D-04, D-05), §5, §6
## Context
Two failure modes are common when a team wraps MongoDB.
The first is flattening: a shared `CommonMongoRepository<T, ID>` and a generic CRUD facade, which
forces every collection to share an id strategy, a consistency profile and a query surface. MongoDB's
single-document atomicity, aggregation model and change streams stop being reachable, and the first
collection that needs something different gets a cast or a leaky generic.
The second is unrestricted exposure: the driver and `runCommand` available everywhere. Then any
service can drop a collection, run an unbounded pipeline, or issue an admin command from a request
thread, and no review catches it because there is nothing structural to catch.
## Decision
The domain owns its documents; the platform owns the cross-cutting decisions. Four exposure planes:
| Plane | Contents | Client |
|---|---|---|
| D1 Standard document persistence | Spring Data repositories, typed queries, mapping manifest, atomic update primitives, optimistic revision | Stable API V1, `apiStrict=true` |
| D2 Advanced document operations | `MongoTemplate`, transactions/sessions, bulk, aggregation, keyset cursors, change streams | Stable API V1, `apiStrict=true` |
| D3 Explicit Mongo capability | Native BSON, time series, search/vector, CSFLE/QE, shard-aware operations | Separate capability client |
| D4 Admin plane | Collection, validator, index, migration, shard, repair | Separate admin client and credential |
Specifically:
1. **No `CommonMongoRepository<T, ID>`.** Each aggregate declares its own repository.
2. **D1/D2 run on Stable API V1 with `apiStrict=true`,** so a command outside the versioned API fails
at development time instead of on the next server upgrade.
3. **D3 is not a raw-client escape.** Every call passes a fixed admission order: capability registered
→ database profile → collection allowlist → operation name → timeout → consistency profile →
result limit → trace → redaction → command category → admin-command refusal → execute.
4. **D4 is a separate client with a separate credential.** No application-plane path reaches it;
`PolicyAwareMongoNativeGateway` refuses admin-category commands regardless of capability.
5. **Advanced and Experimental capabilities are opt-in modules**, never transitive dependencies of the
Stable surface.
## Consequences
**Positive.** MongoDB's semantics stay reachable. Misuse is refused structurally rather than reviewed
for. A server upgrade cannot silently change D1/D2 behaviour. Admin operations have their own audit
trail and credential.
**Negative.** Every operation needs a registered name and profile, so a new query is a small amount of
configuration rather than zero. A genuinely new capability requires a registration before it can be
used. Both are deliberate: the cost is paid once per operation, at review time.
**Rejected alternative — "expose the driver, rely on code review."** Review does not scale to every
query in every service, and the operations that matter (unbounded pipeline, `dropCollection`,
unanchored regex on user input) look unremarkable in a diff.
## Repository adaptation
The design assumes 19 Gradle modules under `modules/mongodb/`. This repository's fail-closed registry
declares exactly 19 leaf identities, so the modules became package boundaries inside
`:adapter:outbound:persistence-mongo`, enforced by ArchUnit. See
[docs/mongodb/repository-adaptation.md](../mongodb/repository-adaptation.md).
@@ -0,0 +1,58 @@
# ADR-MONGO-002 — BSON representation is a pinned manifest
- **Status:** Accepted
- **Date:** 2026-08-13
- **Design source:** design §10, decision D-06
## Context
How a Java value is represented in BSON is a data contract, but nothing in the default toolchain
treats it as one. Spring Data and the MongoDB driver both have defaults, and those defaults have
changed across versions. A `BigDecimal` can land as a `Double`, a `String` or a `Decimal128`; a `UUID`
can land as `Binary` subtype 3 or subtype 4; an `Instant` can land as a `Date` or a `String`.
The consequences are asymmetric. A representation change is invisible in a value-equality test —
`12.30` looks like `12.30` whether it is a double or a `Decimal128` — but once a collection holds
production data, changing it is a full migration. And the UUID case is worse than a migration: legacy
Java representation byte-swaps two halves of the UUID, so a document written under one representation
and read under the other yields a *different, valid-looking* UUID. Nothing errors. You get the wrong
record.
## Decision
`MongoTypeRepresentationManifest` pins the representation for every type the platform maps, and
`MongoMappingConfiguration` builds the Spring Data converters from it. Nothing relies on a library
default.
| Java | BSON | Rationale |
|---|---|---|
| `UUID` | `Binary` subtype 4 (`STANDARD`) | Subtype 3 byte-swaps; cross-representation reads are silently wrong. |
| `BigDecimal` | `Decimal128` | A double cannot represent `12.30`; money compared as a double is eventually wrong by a cent. |
| `BigInteger` | `Decimal128`, or declared `String` when out of range | 34 significant digits; out of range fails on write instead of rounding. |
| `Instant` / `OffsetDateTime` / `ZonedDateTime` | UTC `Date` | One instant, one representation. |
| `LocalDate` | declared per field | A calendar day is not an instant. |
| `LocalDateTime` | **refused** | No offset: the stored value depends on the writing JVM's default zone. |
| `enum` | `String` name | Ordinals renumber when someone inserts a constant. |
Type metadata follows `MongoTypeMetadataPolicy``NONE`, `ALIAS` or `CLASS_NAME`. A
`@LongLivedMongoDocument` type may not use `CLASS_NAME`: writing a FQCN into a million documents makes
a package rename a data migration.
The manifest is enforced by a golden gate. `MongoBsonSnapshot` canonicalises a stored document,
preserving BSON types and keeping missing distinct from null, and
`MongoBsonSnapshotAssert.hasTypeSignature(...)` fails on any representation change. The registry
pins `UuidCodec(STANDARD)` explicitly rather than inheriting a default, since inheriting the default
is the exact drift the gate exists to catch.
## Consequences
**Positive.** A library upgrade cannot move a representation without failing a test. Money is exact.
UUIDs read back as themselves. Class moves stay refactors.
**Negative.** Every representation-affecting change requires updating a snapshot *and* writing a
migration. A new mapped type needs a manifest entry before it can be used. This is the intended
friction: the alternative is discovering the change in production.
**Rejected alternative — "snapshot the JSON."** JSON destroys exactly the distinctions the gate
protects: `Decimal128` and `String` both render as text, `Binary` UUID and `ObjectId` both render as
hex, and missing and null both disappear.
@@ -0,0 +1,68 @@
# ADR-MONGO-003 — Transaction body retry and commit retry are separate loops
- **Status:** Accepted
- **Date:** 2026-08-13
- **Design source:** design §14–§16, decisions D-07 through D-10
## Context
MongoDB reports two transaction failures that look similar and must be handled in opposite ways.
`TransientTransactionError` means the transaction definitively did not commit. The correct response is
to run the whole thing again.
`UnknownTransactionCommitResult` means the commit **may already have applied** — typically because the
primary changed while the commit was in flight. The correct response is to retry *the commit*, which
is a no-op if it already succeeded.
The common implementation wraps everything in one retry loop. That loop replays the body after an
unknown commit, and if the commit did apply, the body applies twice. In a payment or notification path
that is a duplicate charge or a duplicate message, produced by the error handler.
The related trap is session reuse: retrying on the same session after an abort carries the aborted
transaction's state into the retry.
## Decision
`MongoTransactionRetryCoordinator` implements two loops with different scopes.
```
for each body attempt within the budget:
open a NEW session
run the body
TransientTransactionError -> abort, continue to next body attempt
commitWithRetry(session):
UnknownTransactionCommitResult -> retry the COMMIT ONLY, same session
```
Rules that follow, all of them load-bearing:
1. **A new session per body attempt.** No aborted state leaks into a retry.
2. **The body is never replayed after a commit ambiguity.** `MongoRetryScope.COMMIT_ONLY` is a
distinct value from `BODY` precisely so this cannot be collapsed by accident.
3. **One budget bounds both loops.** `MongoRetryBudget` limits attempts *and* elapsed time, with
jittered backoff, so a struggling primary is not retried into the ground by every instance at once.
4. **An exhausted commit retry surfaces `TRANSACTION_COMMIT_UNKNOWN`,** never a generic failure. An
ambiguous outcome reported as a failure invites the caller to retry — the one thing that must not
happen. See [docs/mongodb/runbooks/unknown-commit.md](../mongodb/runbooks/unknown-commit.md).
5. **Transaction bodies write a deterministic marker** so `MongoCommitReconciler` can establish what
actually happened. A transaction that cannot be reconciled has no recovery path.
6. **Classification reads labels before codes.** Server error labels are the authoritative statement
about retryability; error codes vary by version.
Surrounding decisions that reduce how often this path is reached at all: single-document atomic
operations are preferred over transactions (D-09), partial changes use update operators rather than
`save()` (D-07), and whole-document replacement requires an optimistic revision (D-08).
## Consequences
**Positive.** A commit ambiguity cannot become a duplicate effect. The ambiguity reaches the caller as
an ambiguity. The retry budget is bounded in both attempts and time.
**Negative.** Callers must handle a third outcome beyond success and failure. Transaction bodies must
write a marker they would not otherwise need. Both costs are small compared with reconciling
duplicated financial effects after the fact.
**Rejected alternative — "one retry loop, at-least-once everywhere."** It requires every transaction
body to be fully idempotent, which is a much stronger and much less checkable property than writing
one marker, and it is silently violated the first time someone adds a non-idempotent step.
@@ -0,0 +1,62 @@
# ADR-MONGO-004 — Index and schema changes belong to the admin plane
- **Status:** Accepted
- **Date:** 2026-08-13
- **Design source:** design §21–§25, decisions D-11, D-13
## Context
Spring Data can create indexes automatically from annotations. On a laptop this is convenient. On a
collection with a hundred million documents, an index build is a capacity event: it consumes CPU, IO
and memory on the primary for minutes to hours, and it starts because a pod restarted.
Worse, it starts N times when N pods restart, and there is no approval step, no ordering relative to
the code that needs the index, and no record afterwards of what was created.
Schema validators have the same shape with a sharper edge: tightening a validator on a collection with
existing data rejects writes to documents that were legal when they were written.
TTL has a third shape. It looks like a scheduler and is not one: the TTL monitor runs about once a
minute and deletes in batches, so an expired document routinely remains readable for minutes or hours.
## Decision
**Indexes and validators are declared in a manifest and applied by the admin plane (D4).** Automatic
index creation in production is disabled.
1. `MongoManifestRegistry` holds the declared indexes (`MongoIndexManifest`) and validator
(`MongoSchemaManifest`) per collection. The manifest is the source of truth, reviewed in a pull
request.
2. `MongoIndexDiffEngine` compares manifest against observed state and reports missing, extra and
*changed* indexes. Changed ones are reported rather than re-issued: MongoDB will not silently
rebuild an index whose definition moved.
3. `MongoIndexApplyPolicy` sets what an environment may do — `APPLY` (local), `APPLY_WITH_DIFF`
(staging), `DIFF_WITH_APPROVED_APPLY` (production), `REPORT_ONLY` (audit).
4. **Ownership gates every drop.** `MongoMetadataOwnership` distinguishes `APPLICATION_MANAGED` from
`SEARCH_MANAGED`, `ENCRYPTION_MANAGED` and `EXTERNAL`. Only application-managed objects are
droppable on drift. A diff engine without ownership eventually proposes dropping
`enxcol_.customers.esc`, and "the drift tool cleaned it up" is a very bad incident summary.
5. **Retirement is staged.** `MongoIndexRetirementState` moves an index declared → hidden →
observed-unused → droppable, one deployment per transition. Hiding is instantly reversible;
dropping is a rebuild.
6. **Stable validation actions are `error` and `warn` only.** `errorAndLog` is not part of the Stable
contract on 7.0 or 8.0 and is refused. Tightening goes `warn`+`MODERATE` → confirm zero warnings →
`error`+`STRICT`, in two deployments.
7. **TTL is physical cleanup only** (D-13). `MongoExpirationAccessPolicy` states the rule: a
document's presence is not authorization and its absence is not a deadline. Access control checks
the expiry field; scheduling uses a scheduler.
8. **Migrations are checksummed, locked, precondition-checked and resumable.**
`MongoMigrationRunner` fails hard when an applied id's checksum changed — two environments running
different code under one id is worse than a failed deploy.
## Consequences
**Positive.** Index builds are scheduled by people who know the capacity. Rollback is possible at
every step. Drift is visible without being dangerous. Nothing drops what it does not own.
**Negative.** Adding an index is a manifest change plus an apply, not an annotation. Local development
uses `APPLY` so the friction is confined to environments where it is warranted.
**Rejected alternative — "auto-create with a feature flag."** The flag is either on in production,
which is the problem, or off, in which case the manifest is the real mechanism and the annotation is a
second, divergent source of truth.
@@ -0,0 +1,79 @@
# ADR-MONGO-ADV-001 — Advanced capability promotion
- **Status:** Accepted
- **Date:** 2026-08-13
- **Design source:** design §2 (D-15), §3.2–§3.3; Advanced expansion plan Task 15
## Context
Sharding, time series, CSFLE, Queryable Encryption, search, vector search and multi-tenancy each work
in a demo within an afternoon. What they do not do is behave the same way in production, and the
differences are not discovered by functional tests:
- Sharding changes which queries are efficient. A query that misses the shard key becomes
scatter-gather, which passes every test on a one-shard cluster.
- Encryption's failure modes are KMS failure modes — wrong key, revoked permission, mid-rotation —
none of which occur against a local key provider.
- Search and vector search can be functionally correct and useless: the index returns results, and
the results are not relevant. Recall is not visible in a pass/fail assertion.
- Database-per-tenant works until the tenant count crosses what the connection and file-handle
budget supports, which is an operational property, not a code property.
The failure mode this ADR prevents is a capability marked "done" on the strength of a green test that
never touched the environment where it will run.
## Decision
Every Advanced and Experimental capability is an **opt-in module behind its own flag**, and promotion
requires evidence, not confidence.
### Enablement
`MongoAdvancedCapabilityFlags` gates construction of every Advanced entry point. A disabled capability
does not produce a runtime warning — the type refuses to be constructed, naming the property that
enables it (`MongoAdvancedCapabilityFlags.propertyFor(capability)`). Being on the classpath is not
being enabled, and `stableNeverDependsOnAdvanced` (ArchUnit) keeps the Stable surface free of them.
### Promotion evidence
`MongoAdvancedPromotionGate.verify(evidence)` requires every category:
| Category | Means |
|---|---|
| `stable-platform` | The Stable release gate passed on the same revision. |
| `actual-topology` | The capability ran on the real topology — a real sharded cluster, the real KMS, the actual target deployment. Atlas Local is a pull-request convenience and explicitly not release evidence (`MongoAtlasCapabilityContractSuite.Environment.ATLAS_LOCAL`). |
| `security` | Privileges reviewed; the capability's admin role is separate from the application role. |
| `migration` | A documented path in and, where the capability is irreversible, an explicit statement that there is no path back. |
| `failure` | Negative cases fail closed: wrong key, missing permission, rotation, non-ready index, unrouted query. |
| `runbook` | A runbook exists for the capability's characteristic incident. |
### Additional per-capability requirements
- **Search / vector search:** relevance and performance evidence, not functional success alone.
`MongoVectorSearchBenchmarkGate` requires recall alongside latency and index size; a gate that
measures only latency certifies a fast wrong answer.
- **Database-per-tenant and reshard orchestration remain Experimental** until operational scale
evidence exists. Both are correct in the small and unbounded in the large.
- **Reshard requires an explicit `ReshardApproval`** — a named approver and a stated window. It
rewrites the collection.
### Promotion does not change the dependency boundary
A capability promoted to Stable **remains an opt-in module** unless a later starter ADR changes the
dependency boundary. Promotion is a statement about evidence, not an invitation to add a transitive
dependency to every service.
## Consequences
**Positive.** No capability reaches production on the strength of a container-only test. The evidence
list is the same for every capability, so promotion is reviewable rather than negotiated.
**Negative.** Promotion requires access to real infrastructure — a sharded cluster, a real KMS, the
target deployment. That is the cost of the guarantee: the alternative is finding out in production,
where encryption and sharding are both expensive to reverse.
## Verification
```bash
bash scripts/verify-mongodb-advanced.sh
```
+64
View File
@@ -0,0 +1,64 @@
# Entity Mapping Guide
Design §10-§13. The rules here exist because each one has a failure mode that is invisible in review
and expensive in production.
## The domain owns the model
The platform defines no business entity. Table names, column semantics, keys, unique and check
requirements, associations, cascade rules, lock policy, and soft-delete policy all belong to the
domain module. There is no `GenericRepository<T, ID>` and no platform base repository, because a
single generic API forces every aggregate through the same operations — and one aggregate's later
requirement then changes behaviour for all of them.
## Entities must be proxyable
- Not `final`. Hibernate creates a lazy proxy by generating a subclass; a final entity cannot be
subclassed, so *every* association to it loads eagerly whatever the mapping says. Nothing errors.
- A non-private no-arg constructor. The provider instantiates entities reflectively before
populating fields.
`EntityMappingCondition` in the testkit enforces both.
## Identifiers
Default to a sequence with an `allocationSize` that matches the migration's `INCREMENT BY`. When
they disagree, the provider hands out identifiers the sequence has not reserved and the collision
surfaces later as a primary-key violation under load.
`GenerationType.IDENTITY` is supported and limited: the key is assigned on insert, so the provider
must execute each insert immediately to learn it, which disables JDBC insert batching entirely.
`HibernateBatchConfigurationGuard` fails a batch profile that targets an IDENTITY entity rather than
letting the import silently run an order of magnitude slower.
UUIDv7 (`UuidV7Generator`) is the application-side option. It is preferred over UUIDv4 for a primary
key because v4 is uniformly random: every insert lands on a random leaf of the B-tree, so the index
never stays in cache and write amplification grows with the table.
## Values
- Enums are `EnumType.STRING` or an explicit converter. **Never** `ORDINAL` — it stores the
constant's position, so inserting a new constant anywhere but the end silently reinterprets every
existing row.
- Money is `BigDecimal` with explicit precision and scale. `double` cannot represent `0.1`, so sums
drift and reconciliation disagrees with the ledger.
- `Duration` goes through a converter that stores milliseconds. The ISO-8601 text form sorts and
compares wrongly in SQL.
- `Instant` and `OffsetDateTime` map differently; a column typed for one cannot faithfully store the
other.
## Associations
- To-one associations are `LAZY`. JPA's default is `EAGER`, which means every query that loads a
child also queries for its parent — the most common accidental N+1 in a JPA application.
- The owning side holds the foreign key. Adding to the inverse collection alone leaves the row
unlinked, so aggregates expose an association helper that sets both sides.
- `CascadeType.ALL` with `orphanRemoval` is correct only for a child the aggregate genuinely owns.
Between independent aggregates it deletes rows another part of the system still owns.
## Entities never leave the transaction
A controller must not return an entity, or a collection or `Optional` of one. Response serialisation
happens after the transaction closes, so a lazy association touched by the serialiser either throws
or — with OSIV on, which this platform forbids — issues a query from the view layer, one per element.
`EntityExposureCondition` checks generic type arguments, not just the erased return type.
@@ -0,0 +1,43 @@
# Experimental Promotion Checklist
`ExperimentalPromotionGate` evaluates this checklist. Every technical item, then the ADR.
## Technical evidence
- [ ] **Compatibility** — the Stable contract suite passes on the experimental target, twice, on two
supported patch releases. One passing run is a coincidence.
- [ ] **Security** — for tenancy features, cross-tenant read *and* write are both proven impossible,
including through native SQL, bulk DML, `getReference`, and the second-level cache. A filter
that covers only entity queries covers none of those.
- [ ] **Failure** — connection reuse does not leak tenant context; a failover does not silently route
a read-after-write to a stale replica; the commit-ambiguity scenarios still behave.
- [ ] **Migration** — per-tenant migration is resumable after a partial failure, and rate-limited.
With one schema per tenant, a run is N independent migrations and "it failed" is not an answer.
- [ ] **Performance** — pool capacity, replica lag under load, and per-tenant memory are measured,
not estimated. Database-per-tenant fails as a sum, not as an individual pool.
## Decision
- [ ] **Reviewed ADR** — recording what is being promised, the operational burden it carries, and
what would cause it to be withdrawn.
The ADR is not a formality. The technical suites establish that something works; the ADR records
that the platform should promise it, which is a different question with a different cost.
## What does not count as evidence
- The version being generally available.
- The feature working in one environment.
- A passing suite that skipped because Docker was unavailable.
- A green lane whose assertions were relaxed to make it pass.
## Outcomes
| Decision | Meaning |
|---|---|
| `BLOCKED_TECHNICAL` | at least one suite has not passed |
| `BLOCKED_MISSING_ADR` | evidence is complete; no reviewed decision exists |
| `ELIGIBLE_FOR_STABLE_REVIEW` | both; Stable review may begin |
The two blocked states are distinct because they need different work: one needs evidence, the other
needs a decision.
+43
View File
@@ -0,0 +1,43 @@
# Experimental Support Matrix
Everything here is off unless its `backend.jpa.experimental.*` flag is explicitly true, and none of
it is part of the Stable composition.
| Feature | Flag | State |
|---|---|---|
| Shared-schema multi-tenancy (column) | `backend.jpa.experimental.multitenancy-column` | Experimental |
| PostgreSQL RLS multi-tenancy | `backend.jpa.experimental.multitenancy-rls` | Experimental |
| Schema-per-tenant | `backend.jpa.experimental.multitenancy-schema` | Experimental |
| Database-per-tenant | `backend.jpa.experimental.multitenancy-database` | Experimental |
| Consistency-aware read replica | `backend.jpa.experimental.read-replica` | Experimental |
| Jakarta Persistence 4.0 lane | `backend.jpa.experimental.jakarta-persistence-4` | Experimental |
| Hibernate ORM 8 lane | `backend.jpa.experimental.hibernate-8` | Experimental |
| PostgreSQL 19 lane | `backend.jpa.experimental.postgresql-19` | Experimental |
Presence on the classpath is not consent. `ExperimentalFeatureGate` fails startup when a module is
present and its flag is not set, because an experimental module can arrive transitively and a
tenant-isolation feature that switched itself on would be the worst possible default.
## Known constraints
- Tenant context is fail-closed. An unbound tenant in a shared-schema deployment means a query with
no tenant predicate, which returns every tenant's rows.
- A Hibernate filter is not the security boundary. It does not apply to native SQL, bulk DML,
`getReference`, or the second-level cache.
- RLS requires all three of: `ENABLE ROW LEVEL SECURITY`, `FORCE ROW LEVEL SECURITY` (the owner is
otherwise exempt from its own policies), and a runtime role without `BYPASSRLS`.
- Tenant bindings are transaction-local. A session-local setting survives the connection's return to
the pool.
- `readOnly=true` never routes to a replica on its own. Read-after-write uses a consistency token or
the primary.
- Unavailable replica lag evidence means the primary. Absence of evidence is not evidence of
freshness.
- Per-tenant pools are bounded globally. Fifty tenants with a modest pool each is five hundred
connections against a server that permits a hundred.
- Tenant ids never become metric tags. Tenant cardinality is unbounded by definition.
## Lanes never change Stable
A compatibility lane publishes nothing and changes no Stable contract. If Hibernate 8 generates
different SQL for the fetch-pagination gate, that is a finding about Hibernate 8 — the 7.x gate keeps
asserting what 7.x must do, because that is what deployments run.
+64
View File
@@ -0,0 +1,64 @@
# Migration Guide
Design §31-§32. Flyway owns the schema; Hibernate only validates.
## Who may change the schema
| Environment | Mode |
|---|---|
| local, test, dev | migrate at startup with the migration credential |
| staging, prod | deployment-owned migration; the application validates only |
Migrating from inside the application in production means every instance of a rolling deploy races
to apply the same script, and the loser's failure is indistinguishable from a real one.
`ddl-auto` is `validate` or `none`. Never `update`: it never drops or narrows anything, so it
produces a schema that is neither the old one nor the one the migrations describe — silently, on
whichever instance started first.
## Validation fails closed and never repairs
`FlywayValidationGate` throws `SchemaMismatchException` on a checksum mismatch, a missing migration,
or a schema Hibernate disagrees with. It never calls `repair`.
Repair rewrites the schema history table to match whatever scripts are on disk. That resolves the
symptom by deleting the evidence: a checksum mismatch means the deployed script differs from the
applied one, and the interesting question is which change is missing from this database. Repair
makes that question unaskable. It exists only as an explicit admin operation with an operator, a
reason, and an approval (design §8.4).
Only Flyway's structured error codes reach the exception. Its messages embed the script path and
part of the failing statement.
## Concurrent index builds
`CREATE INDEX CONCURRENTLY` cannot run inside a transaction block, and Flyway wraps migrations in
one by default. The migration therefore needs a companion configuration:
```conf
# V42__order_index.sql.conf
executeInTransaction=false
```
`ConcurrentIndexMigrationInspector` fails validation without it, and additionally requires the
migration to contain nothing else. A failed concurrent build leaves an invalid index behind;
recovering is a single `DROP INDEX` when the migration did nothing else, and a manual reconstruction
of partial state when it did.
An invalid index is not merely useless — the planner ignores it while every write still maintains
it. `FailedConcurrentIndexRecovery` reports them with the statement to run, and deliberately does
not drop them: an invalid index can also mean a build is still running, and the two are
indistinguishable from the catalog alone.
## Upgrade scenarios
Three, each catching something the others do not:
| Scenario | Catches |
|---|---|
| `empty` | an early migration edited to match a later one, no longer applying to a fresh database |
| `previous-release` | the actual deployment path; the only one exercising this release's migrations |
| `oldest-supported` | a migration that silently assumes state only recent databases have |
Each asserts a data invariant, not just the schema version. A migration that renames a column and
loses its contents leaves the version correct and the data gone.
+61
View File
@@ -0,0 +1,61 @@
# Observability
Design §37. What is measured, and what must never appear in a measurement.
## Bounded tags, always
Every JPA metric carries exactly five tags: persistence unit, operation, query, outcome, failure
category. All five are registered identifiers, validated by `LowCardinality` at construction rather
than at the registry — so an unbounded value fails where it was introduced instead of surviving
until a dashboard stops loading.
Never a tag: entity id, tenant id, SQL parameter, exception message, JDBC URL. Each is unbounded, so
each creates a time series per row or per failure; several are also the data the platform keeps out
of logs, which a metrics backend would store just as durably and export just as widely.
## Transaction metrics
| Meter | Why it exists |
|---|---|
| `jpa.transaction.duration` | the baseline |
| `jpa.transaction.rollback` | rollback rate by failure category |
| `jpa.transaction.timeout` | timeouts, distinct from other rollbacks |
| `jpa.transaction.completion.unknown` | its own counter, deliberately |
Completion-unknown gets a separate counter rather than being folded into failures. It is the one
outcome that means a human has to look: every other failure is a transaction that definitely did not
happen, while this one is a transaction that may have.
## Query metrics
`jpa.query.duration` and `jpa.query.rows`. Rows are measured as well as duration because a query
that issues one statement and hydrates twenty thousand rows is fast per statement and catastrophic
per request — a duration metric alone reports it as merely slow.
## Retry metrics
Attempts are metrics, not warnings. Optimistic conflicts and serialization failures are the expected
cost of concurrency; logging each at WARN pages someone for a system working as designed, after
which the retry log gets filtered out and takes the genuinely interesting entries with it.
`jpa.retry.attempt`, `jpa.retry.attempts` (distribution per operation), `jpa.retry.exhausted`.
## Query names in SQL
`NamedStatementInspector` prefixes each statement with its registered query name as a SQL comment,
which travels into `pg_stat_activity`, `auto_explain`, and the slow-query log. Without it, "which
endpoint issues this query" is answered by grepping the codebase for fragments of SQL.
## Diagnostics
`SqlDiagnosticRedactor` removes string literals, numbers, and anything email-shaped before SQL
reaches a log. Redaction is blunt on purpose: preserving "harmless" values would require knowing
which columns hold personal data.
## The actuator endpoint
`jpaplatform` reports database major version, provider version, schema version, OSIV state, runtime
role verification, and capability levels. It reports no JDBC URL, no username, no SQL, and no entity
catalog — an actuator endpoint is reachable by anyone who reaches the management port, and each of
those would be a free reconnaissance answer. It is read-only: an endpoint that could trigger a
migration or a repair would be an admin capability exposed over HTTP.
+72
View File
@@ -0,0 +1,72 @@
# PostgreSQL Extensions
Design §8.3, §21, §30. What the platform uses beyond portable JPA, and what each is guarded by.
Everything here is core PostgreSQL. No server extension is required.
## Locking
`SELECT ... FOR UPDATE` with a finite bound, always. `PostgreSqlLockOptions` refuses an unbounded
lock request because it waits as long as the holder holds it, turning one slow transaction into a
pile-up of blocked connections.
`NOWAIT` and a wait timeout are separate requests, not two spellings of one — modelling them as a
single field with a magic zero is how "no wait" becomes "wait forever".
`55P03` (lock not available) and `40P01` (deadlock) drive opposite recovery and are never collapsed:
the first leaves the transaction alive and the caller in control; the second has already been rolled
back by the server.
## Work claims
`FOR UPDATE SKIP LOCKED` is reachable only through a registered `WorkQueueName`, never as a
repository flag. It deliberately returns an incomplete view of the table: correct for handing
disjoint work to competing workers, silently wrong for anything that needs to see every matching
row. A registered claim statement must skip locked rows and impose a deterministic `ORDER BY`.
## Upserts
`INSERT ... ON CONFLICT ... RETURNING` under a registered `NativeWriteName` with a fixed conflict
target and update column set. The conflict target cannot be a bound parameter, so accepting one from
a caller would mean building SQL from input.
An upsert is the correct answer to a create race precisely because the database decides.
Read-then-write cannot be made correct: another transaction can commit between the read and the
write. `(xmax = 0) AS inserted` in the `RETURNING` list is what lets the platform report
insert-versus-update without a second query.
The executor flushes before and clears after: a native write is invisible to the Persistence
Context, so a pending managed change would otherwise overwrite it, and a managed entity loaded
beforehand would keep serving pre-upsert values.
## JSONB
`JsonDocument` carries a schema name and version alongside the payload. A JSONB column is schemaless
at the database level, so without an envelope the only record of what a stored document means is the
code that wrote it — and a document written two releases ago is indistinguishable from a current one.
The payload never carries a Java class name. Type metadata in a JSONB column is a deserialization
gadget: whoever can write a row chooses the class the reader instantiates.
Query paths are registered. A JSON path is part of the SQL text and cannot be bound, so forwarding a
request field into one is concatenating untrusted input into a statement. Values are always bound.
## Arrays and ranges
Arrays are built with `Connection.createArrayOf`, never by formatting a literal — hand-formatting is
where quoting bugs live, and a tag containing a comma changes the array's shape rather than its
content.
`PgRange` models both endpoints as independently optional and independently inclusive, because that
is what a PostgreSQL range is. Whether `[09:00, 10:00)` and `[10:00, 11:00)` overlap depends on the
bracket, not the values, and a pair of `timestamptz` columns cannot express it.
## COPY (J4 admin)
`COPY` bypasses the Persistence Context, entity callbacks, version checks, and Envers entirely. That
is why it is fast and why it is an admin capability with a registered statement, a bounded stream, a
row and byte cap, a finite server-side `statement_timeout`, and a named operator.
The registry accepts only `COPY ... FROM STDIN`. `COPY ... FROM '/path'` reads a file on the
*database server* as the server's OS user; it is superuser-only for exactly that reason and does not
belong behind an application API.
+74
View File
@@ -0,0 +1,74 @@
# Query and Fetch Guide
Design §23-§28. How queries are chosen, bounded, and proven.
## Named queries
Every registered query carries a `QueryName`. It becomes the metric tag, the trace attribute, and
the SQL comment that appears in `pg_stat_activity` and the slow-query log — which is the only thing
that connects a statement on the server back to the use case that issued it. The format rejects raw
SQL for a reason: a metric tag built from a query string is unbounded by construction, and one built
from a parameterised value leaks row data into telemetry.
## Fetch plans, not eager mappings
N+1 is solved per use case with a registered entity graph, not by making an association `EAGER` in
the mapping. The eager fix repairs the one query that needed it and imposes the extra join on every
other query against that entity, including the ones that only wanted the id.
`fetchgraph` and `loadgraph` are different: a fetch graph is exhaustive (attributes outside it are
lazy whatever the mapping says), a load graph is additive. Choosing the wrong one produces either
missing data or the amplification the graph was meant to avoid.
## Measuring, not guessing
`QueryMeasurement` records statements, hydrated entities, rows, fetches, and elapsed time. Statement
count alone cannot distinguish the two failures that matter:
- **N+1** — many statements, few rows.
- **Cartesian fetch** — one statement, an enormous number of rows.
A suite asserting only on statement count passes the second one every time.
## Pagination
Offset pagination makes the database walk and discard `n` rows before returning any. Keyset
pagination replaces it:
- The predicate is lexicographic. For an ordering of `(createdAt, id)`, "after `(t, x)`" is
`createdAt < t OR (createdAt = t AND id < x)`**not** `createdAt <= t AND id < x`, which reads
plausibly and silently drops rows from the middle of the result set.
- The ordering must end in a unique column. Without one, a page boundary inside a run of equal
values duplicates and skips rows.
- `size + 1` rows are fetched and `size` returned. That extra row answers `hasNext` without a count
query, which would be a second full scan whose answer is stale on arrival.
Cursors are signed. An unsigned cursor is client-controlled ordering state: rewriting it lets a
caller seek to arbitrary keys.
## Sorting
Client sort parameters are mapped through `SafeSortRegistry`, never passed through. A sort field
reaches the query as part of the ORDER BY clause rather than as a bound value, so forwarding the
client's string means the client writes part of the statement. `JpaSort.unsafe` has no call site in
this platform.
The registry's tie-breaker is always appended, because a sort that does not end in a unique column
has no total order and paging over a non-total order duplicates and skips rows.
## Streaming
A JPA `Stream` is a live cursor holding a `ResultSet`, a statement, and a connection. `JpaStreamExecutor`
consumes it inside a try-with-resources and never returns it, because a stream returned past the
transaction boundary is a connection leak that presents as unrelated timeouts elsewhere. A read-only
transaction is required: streaming inside a write transaction pins a write connection for the whole
traversal.
## Batching
Configuring `hibernate.jdbc.batch_size` proves nothing. `BatchExecutionResult.jdbcBatches` comes from
counting real `executeBatch()` calls at the JDBC layer, because an IDENTITY generator, an interleaved
select, or a mid-loop flush disables batching while the configuration still says it is on.
Flush and clear are separate boundaries. Flushing alone sends the statements and keeps every entity
in the Persistence Context — the classic bulk-import out-of-memory.
+156
View File
@@ -0,0 +1,156 @@
# JPA Relational Persistence Platform — Repository Adaptation Contract
**Design source:** `jpa-superpowers-package/docs/superpowers/specs/2026-08-11-jpa-persistence-platform-design.md`
(copied to `docs/superpowers/specs/`)
**Stable plan source:** `jpa-superpowers-package/docs/superpowers/plans/2026-08-11-jpa-persistence-platform-implementation-plan.md`
(copied to `docs/superpowers/plans/`)
**Experimental plan source:** `jpa-superpowers-package/docs/superpowers/plans/2026-08-11-jpa-persistence-experimental-expansion-plan.md`
(copied to `docs/superpowers/plans/`)
The design package states its own adaptation rule (§3.2): the assumed package paths and Gradle
structure are explicit implementation *assumptions* made because the real Backend Skeleton
repository was not supplied. Before implementing, paths are adjusted to the repository's existing
conventions and root package while the public contracts and policy semantics are preserved.
This file is the single record of *how* that mapping was performed. Only paths, build DSL, and
composition-root ownership changed. Public contracts, policy order, retry semantics, and error
semantics are implemented as specified.
## 1. Why the module layout differs
The plan assumes a greenfield library with 18 Stable Gradle projects under `modules/jpa/` plus 7
Experimental projects under `modules/jpa-experimental/`. This repository is a Clean Architecture
template whose **fail-closed registry** (`src/config/architecture/modules.json`, enforced by
`src/settings.gradle` and `verifyCleanArchitectureDependencies`) declares **exactly 19 leaf
identities**, and `src/settings.gradle` throws when the registry does not contain exactly 19
modules. Creating 25 more Gradle projects would violate HARD-STOP #5 in `AGENTS.md`.
Therefore the plan's library modules become **package boundaries inside the registered leaf**
`:adapter:outbound:persistence-jpa`, with two exceptions driven by this repository's own rules.
This is the same adaptation already applied to the HTTP client platform
(`docs/httpclient/repository-adaptation.md`).
| Plan module | Repository home | Reason |
|---|---|---|
| `jpa-spring-boot-starter` | `:app-bootstrap` (`dev.caskeleton.bootstrap.autoconfigure.jpa`) | This repository's composition root owns wiring, startup validation, and actuator surface; an adapter leaf must not auto-configure itself. `AGENTS.md` assigns composition to `app-bootstrap`. |
| `jpa-testkit`, `jpa-testkit-postgresql`, `jpa-testkit-migration`, `jpa-testkit-queryplan` | `:adapter:outbound:persistence-jpa` `src/testkit/java/**/testkit` | The plan forbids production modules depending on the testkit. A source set whose dependencies are declared only on test configurations gives the same guarantee without a new Gradle project, and more than one lane consumes it. |
The package boundary is enforced by `JpaModuleBoundaryTest`, which reproduces the plan's
§3 module dependency map as package rules.
## 2. Package mapping
Root package: `io.backend.skeleton.jpa``dev.caskeleton.adapter.outbound.persistence`.
| Plan module | Plan package | Repository package |
|---|---|---|
| `jpa-core-api` | `…jpa.api` (+ `.capability`, `.error`, `.query`, `.transaction`) | `dev.caskeleton.adapter.outbound.persistence.api` (+ same subpackages) |
| `jpa-transaction` | `…jpa.transaction` | `…persistence.transaction` |
| `jpa-spring-data` | `…jpa.springdata` | `…persistence.springdata` |
| `jpa-querydsl` | `…jpa.querydsl` | `…persistence.querydsl` |
| `jpa-hibernate` | `…jpa.hibernate` (+ `.batch`, `.bulk`, `.stateless`) | `…persistence.hibernate` (+ same subpackages) |
| `jpa-postgresql` | `…jpa.postgresql` (+ `.error`, `.lock`, `.constraint`, `.json`, `.array`, `.range`, `.write`) | `…persistence.postgresql` (+ same subpackages) |
| `jpa-postgresql-copy` | `…jpa.postgresql.copy` | `…persistence.postgresql.copy` |
| `jpa-migration-flyway` | `…jpa.migration` | `…persistence.migration` |
| `jpa-auditing` | `…jpa.auditing` | `…persistence.auditing` |
| `jpa-envers` | `…jpa.envers` | `…persistence.envers` |
| `jpa-cache-hibernate` | `…jpa.cache` | `…persistence.cache` |
| `jpa-observability` | `…jpa.observation` | `…persistence.observation` |
| `jpa-security` | `…jpa.security` | `…persistence.security` |
| `jpa-spring-boot-starter` | `…jpa.autoconfigure` | `dev.caskeleton.bootstrap.autoconfigure.jpa` |
| `jpa-testkit*` | `…jpa.testkit` (+ `.id`, `.mapping`, `.lifecycle`, `.query`, `.fetch`, `.postgresql`, `.migration`, `.queryplan`, `.failure`, `.pool`, `.release`) | `…persistence.testkit` (+ same subpackages), `testkit` source set |
| `jpa-experimental/*` | `…jpa.experimental` (+ `.tenant`, `.rls`, `.schema`, `.database`, `.replica`, `.next`) | `…persistence.experimental` (+ same subpackages) |
The existing `…persistence.transaction` and `…persistence.postgresql` packages already hold this
leaf's `TransactionPort` implementation and PostgreSQL vendor composition. The platform types are
**additive**: no existing type was renamed, moved, or replaced, and no plan type collides with an
existing name.
## 3. Test-suite mapping
The plan declares seven JVM test suites (`test`, `integrationTest`, `contractTest`,
`migrationTest`, `failureTest`, `performanceTest`, `compatibilityTest`). This leaf already owns a
Docker-backed `postgresqlIntegrationTest` source set and its readiness Gradle tasks are registered
in a fail-closed contract (`verifyJpaReadinessRegistry` in `src/build.gradle`).
| Plan suite | Repository lane |
|---|---|
| `test` | `src/test` — hermetic unit lane, `./gradlew :adapter:outbound:persistence-jpa:test` |
| `contractTest`, `integrationTest`, `migrationTest`, `failureTest`, `compatibilityTest` | `src/postgresqlIntegrationTest` — real PostgreSQL containers; selected by the `jpaPlatform*` Gradle tasks |
| `performanceTest` | `src/jpaPlatformPerformanceTest` — machine-dependent bounds, never part of `check` |
Docker-dependent lanes fail closed rather than skipping, matching the existing
`PostgreSqlReadinessSupport.assertDockerAvailable()` convention in this leaf.
## 4. Other deliberate substitutions
| Plan assumption | Repository reality | Adaptation |
|---|---|---|
| Gradle Kotlin DSL, `build-logic` convention plugin, `jpa-library-conventions.gradle.kts` | Groovy DSL, root `src/build.gradle` conventions (spotless google-java-format, checkstyle, SpotBugs + FindSecBugs, ErrorProne, `-Werror`, one-type-per-file), `LockMode.STRICT` dependency locking | Source sets and dependencies declared in `src/adapter/outbound/persistence-jpa/build.gradle`; `gradle.lockfile` regenerated with `resolveAndLockAll --write-locks`. |
| Spring Boot 4.1 dependency management, Spring Data JPA 4.1 | Repository baseline is Spring Boot 4.0.0 | Versions are inherited from the repository BOM and never pinned per module, exactly as the plan requires ("do not override Hibernate/Flyway/Hikari versions outside the Boot BOM"). |
| Hibernate ORM 7.4 is the Stable provider | Boot 4.0.0 resolves `org.hibernate.orm:hibernate-core:7.1.8.Final` | The *declared* Stable provider baseline of the design stays 7.4 in `HibernateProviderPolicy`; the runtime provider version is read from Hibernate itself and reported. The collection-fetch-pagination gate runs against whatever provider the BOM resolves, and `HibernateProviderPolicy.driftsFromDeclaredBaseline()` makes the difference visible instead of hiding it behind a green check. |
| PostgreSQL 16·17·18 Stable matrix | This leaf's existing evidence image is `postgres:16-alpine` | `PostgreSqlVersion` declares exactly PG 16, 17, 18. The default lane runs the repository's existing 16 image; 17 and 18 are selected by `-Pjpa.matrix.versions=16,17,18`, and an unknown or empty selection is an error rather than a skip. |
| `settings.gradle.kts` module registration | Fail-closed 19-leaf registry | No registry change: leaf identity, Gradle path, allowed dependencies, and runtime memberships are unchanged. |
| `infra/jpa/{postgres,roles,toxiproxy}` | Repository already owns `infra/` | Created at the same repository-relative paths. |
| `docs/jpa/**`, `docs/adr/ADR-JPA-*`, `.github/workflows/jpa-*.yml` | Repository already owns `docs/` and `.github/workflows/` | Created at the same repository-relative paths. |
| `build.gradle.kts` release aggregate `jpaReleaseGate` | Root is `src/build.gradle` | Registered there against the repository lane names in §3. |
| Per-task `git add` + `git commit` | `AGENTS.md`: commit policy is `human-only`; agents do not stage, commit, amend, or push | Implementation is delivered unstaged. This is the only plan step intentionally not executed, and it is recorded here. |
| Querydsl as an optional module dependency | Querydsl is not part of this repository's dependency set | `querydsl` is implemented against the plan's contracts with the Querydsl types kept behind `compileOnly`, so the Stable runtime classpath never carries Querydsl and a deployment opting in adds the artifact itself. |
| Hibernate Envers as a module dependency | Envers is not part of this repository's dependency set | Same treatment as Querydsl: `compileOnly` + explicit opt-in, matching the plan's "Envers is opt-in and never enabled by a global base class". |
| `build-logic/src/test/kotlin/JpaModuleBoundaryTest.kt` | There is no `build-logic` project and no Kotlin source set; module boundaries are enforced by the registry itself | `verifyCleanArchitectureDependencies` plus `:app-bootstrap:test --tests '*CleanArchitectureTest'` assert the same property against `src/config/architecture/modules.json`, which is the authority the plan's test would have had to duplicate. |
| `PostgreSqlRuntimeRoleVerifierIntegrationTest` (Task 45) | The security lane is one suite in this leaf rather than a per-module `integrationTest` | `PostgreSqlSecurityContractTest` (tag `jpa-security`) exercises `PostgreSqlRuntimeRoleVerifier.verify` and `.requireSafe` against a real restricted role on a real server. |
| `JpaSafetyProperties`, `JpaDataSourceProperties` | `NamingConventionTest` requires every `@ConfigurationProperties` type to end in `Settings` or `Policy` | Renamed to `JpaSafetySettings` and `JpaDataSourceSettings`. The bound property prefixes and every field are unchanged; only the class names move to this repository's convention. |
### Types relocated to keep the dependency direction legal
The plan's module map forbids `jpa-core-api` from depending on any other platform module. Three
value-only types the design places in a downstream module are consumed by a core contract, so they
live in the core here instead. Each is a pure value with no framework dependency, so the relocation
costs nothing and the alternative — a core contract importing an adapter package — would break the
boundary the module map exists to hold.
| Type | Plan module | Repository package | Consumed by |
|---|---|---|---|
| `TransactionCompletionEvidence` | `jpa-transaction` | `…persistence.api.transaction` | `TransactionCompletionUnknownException` (design §17.3 types the field) |
| `ConstraintCode` | `jpa-postgresql` | `…persistence.api.error` | `ConstraintViolationDetails` (design §22.4) |
| `SqlStateResolver`, `SqlExceptionSqlStateResolver` | `jpa-transaction` | `…persistence.api.error` | both the transaction module's commit classifier and the PostgreSQL translator |
The ArchUnit rule pack (`JpaArchitectureRules`, `EntityMappingCondition`, `EntityExposureCondition`)
is placed in the `testkit` source set rather than in `…persistence.security` production code. ArchUnit
is a test library; putting the rule pack in `main` would drag it onto every deployment's runtime
classpath to serve code that only ever runs in a test.
### Findings the contracts produced against a real server
Two of the design's rules turned out to be stated slightly wrong, and the container lanes are what
showed it. Both are recorded here because the design text still reads the old way.
- **§17.2 commit ambiguity is not only SQLSTATE class `08`.** `pg_terminate_backend` on a backend
with a commit in flight reports `57P01` (`admin_shutdown`), not a connection-class state — and the
commit record may already be in the WAL when it arrives. `CommitFailureClassifier` now treats
`57P01`/`57P02`/`57P03` as completion-unknown alongside `40003`, class `08`, and transport breaks.
`CommitAmbiguityContractTest` asserts the SQLSTATE directly so the rule cannot silently narrow
again.
- **Schema-per-tenant status must be read back, not inferred from the run.** `MigrateResult`'s
target version is empty for a tenant that was already current, so recording it reported migrated
tenants as unmigrated during a partial rollout. `SchemaTenantMigrationOrchestrator` now reads the
applied version from the tenant's schema history.
## 5. What is unchanged from the design
- Domain owns Entity, Embeddable, Repository, Query, index requirements, lock/soft-delete/audit
policy. No `GenericRepository<T, ID>` and no Spring Data CRUD re-implementation exists.
- Application Service owns the transaction boundary; OSIV is false in every runtime profile.
- `TransactionCompletionUnknownException` always reports `completionUnknown=true`,
`retryable=false`, and is never automatically retried — reconciliation handles it.
- Retry re-executes the whole use case in a new transaction and a new Persistence Context.
- SQLSTATE classification is structural (`40001`, `40003`, `40P01`, `23505`, `23503`, `23514`,
`55P03`) and never parses localized message text.
- Flyway is the source of truth for production schema change; Hibernate only validates;
`ddl-auto` never mutates a deployed schema.
- `CREATE INDEX CONCURRENTLY` requires an explicit non-transactional migration marker.
- Metric labels and ordinary logs never carry SQL parameters, entity IDs, tenant IDs, or PII.
- Experimental features (multi-tenancy, RLS, schema/database tenancy, read replica, JPA 4,
Hibernate 8, PostgreSQL 19) stay behind `backend.jpa.experimental.*` flags and never enter the
Stable composition.
+85
View File
@@ -0,0 +1,85 @@
# JPA Platform Runbooks
Operator procedures for the failures this platform is designed to surface rather than hide.
## A transaction reported completion unknown
**Signal:** `jpa.transaction.completion.unknown` incremented; a `CompletionUnknownRecord` in the
reconciliation channel.
**What it means:** the commit may or may not have happened. It is not a rollback.
**Do not** re-run the use case. That is what the platform refused to do automatically, for the same
reason.
**Procedure:**
1. Take the `transactionKey` from the record.
2. Check the idempotency record for that key.
3. Check the business row the use case would have written.
4. Check the outbox for a corresponding event.
5. If all three agree the write happened, mark the record `COMMITTED` and stop.
6. If all three agree it did not, the use case may be re-run.
7. If they disagree or are inconclusive, leave it `STILL_UNKNOWN` and escalate. An inconclusive
answer is a legitimate outcome; guessing is not.
A record with no `transactionKey` cannot be resolved automatically — use the operation name and
timestamp.
## Deadlock or serialization rate rising
**Signal:** `jpa.retry.attempt` rising; `jpa.retry.exhausted` non-zero.
Retries are expected. Exhaustion is not.
1. Group `jpa.retry.attempt` by operation. A single operation dominating means a hot row or an
inconsistent lock order.
2. For deadlocks, check whether two operations take the same rows in opposite orders — that is a
code fix, not a tuning one.
3. For serialization failures under `SERIALIZABLE`, confirm the isolation is actually required.
4. Only then consider raising `maxAttempts`. A larger budget on a hot row converts a fast failure
into a slow one.
## Pool exhaustion
**Signal:** connection acquisition timeouts; `PoolMeasurement.pending` non-zero.
1. Check `REQUIRES_NEW` usage. It takes a second connection while pinning the first, so the pool
must satisfy `(threads x (1 + depth)) + 1`.
2. Check for streaming outside a bounded scope — a `Stream` returned past the transaction holds its
connection until the pool notices.
3. Check for external calls inside a DB transaction. The design forbids them precisely because an
HTTP timeout then holds a connection for its whole duration.
## Flyway validation failed at startup
The deployment is running against a schema it was not built for. It failed closed, which is correct.
1. Read the reported error codes (the messages are deliberately not propagated).
2. `CHECKSUM_MISMATCH` — an applied migration was edited afterwards. Find which change is missing
from this database. **Do not run `repair`**: it rewrites history to match the scripts, which
resolves the symptom by deleting the evidence.
3. `MISSING_SCRIPT` — a migration applied here is not in this build. Usually a rollback to an older
artifact.
## An invalid index exists
**Signal:** `FailedConcurrentIndexRecovery.invalidIndexes()` is non-empty.
A concurrent build failed. The index is ignored by the planner and maintained by every write.
1. Confirm no build is currently running. An in-progress build looks identical in the catalog.
2. Run the reported `DROP INDEX CONCURRENTLY` outside a migration.
3. Re-apply the index migration.
The platform does not drop these automatically: on a rolling deploy every instance would race to
drop an index another instance was about to finish building.
## The runtime role failed verification
Startup refused because the runtime credential holds `CREATE`, or `search_path` contains an
unapproved schema.
This is not a false positive to be worked around. Re-provision from
`infra/jpa/roles/runtime-roles.sql`; the application's credential having DDL is the condition that
makes every other schema guarantee unenforceable.
+66
View File
@@ -0,0 +1,66 @@
# Security
Design §36. Credential separation, privilege verification, and what never leaves the process.
## Three credentials
| Role | May |
|---|---|
| `app_migration` | own the schema, apply migrations (DDL) |
| `app_runtime` | select, insert, update, delete (DML only) |
| `app_admin` | J4 operations — COPY, backfill, maintenance |
The separation is what makes "Flyway owns schema change" enforceable rather than aspirational. If
the application's own credential cannot execute DDL, then no code path, no library, and no injected
statement can alter the schema at runtime, regardless of what the application intended.
`infra/jpa/roles/runtime-roles.sql` provisions them.
## Startup verification
`PostgreSqlRuntimeRoleVerifier` asks the *server* what the connection can do:
```sql
select current_user,
current_setting('search_path'),
has_schema_privilege(current_user, current_schema(), 'CREATE'),
has_database_privilege(current_user, current_database(), 'CREATE')
```
Configuration cannot answer this. Effective privileges come from direct grants, inherited role
memberships, `PUBLIC` grants, and schema ownership, and no reading of a deployment manifest
reconstructs that combination reliably.
Startup fails when the runtime role is not on the allowlist, or holds `CREATE` on the schema or the
database.
## search_path
`SearchPathPolicy` is an allowlist. `search_path` decides which schema an unqualified name resolves
to, so a writable untrusted schema on it — classically `public`, where `CREATE` was granted broadly
before PostgreSQL 15 — lets a planted table, function, or operator shadow the real one, and the
application executes it without noticing. `$user` is exempt: only the connected role owns it.
Refusing the runtime role `CREATE` closes the same route from the other side.
## What never leaves the process
- SQL parameter values, entity ids, tenant ids, and PII: not in exception messages, not in metric
tags, not in logs. `JpaFailureContext` composes messages from bounded values only.
- Constraint names reach the application as registered `ConstraintCode`s; an unregistered physical
name maps to a bounded unknown code rather than being passed through.
- Cursors are HMAC-signed. An unsigned cursor is client-controlled ordering state.
- The actuator report carries no JDBC URL, username, or SQL.
## Injection surfaces, and how each is closed
| Surface | Why it cannot be a parameter | Closed by |
|---|---|---|
| sort field | part of ORDER BY | `SafeSortRegistry` allowlist |
| JSON path | part of the statement | registered `JsonPathName` |
| schema name | an identifier | registered `SchemaTenantRegistry` |
| upsert conflict target | an identifier list | registered `UpsertConflictTarget` |
| COPY table | an identifier | registered `RegisteredCopyStatement` |
| queue claim SQL | a whole statement | registered `WorkQueueDefinition` |
Values are always bound. Identifiers are always registered.
+79
View File
@@ -0,0 +1,79 @@
# JPA Persistence Platform — Support Matrix
The machine-readable source for `JpaReleaseManifest`. A release gate parses this file, so a version
or gate that stops being named here stops being claimed — and the build fails rather than the
document quietly drifting from the code.
## Database
| Database | Support | Evidence |
|---|---|---|
| PostgreSQL 16 | Stable | full contract suite, release lane |
| PostgreSQL 17 | Stable | full contract suite, release lane |
| PostgreSQL 18 | Stable | full contract suite, release lane |
| PostgreSQL 19 | Experimental | compatibility lane only; promotion requires an ADR |
| H2 | Local convenience | **never** evidence of PostgreSQL behaviour |
H2 is not a second production target. It reports different SQLSTATEs for the same violation, has no
`SKIP LOCKED` guarantee the platform relies on, no JSONB operators, no range types, and no
concurrent index builds. A green H2 run is evidence that the code compiles and runs, and nothing
more.
## Specification and provider
| Component | Stable | Experimental |
|---|---|---|
| Jakarta Persistence | 3.2 | 4.0 (lane) |
| Hibernate ORM | 7.4 declared baseline | 8 (lane) |
| Spring Boot | repository BOM | — |
The Hibernate row needs a note. The design declares 7.4 as the Stable provider; this repository's
Spring Boot BOM resolves 7.1.x. `HibernateProviderPolicy` holds both — the declared baseline as a
constant, the resolved version read from Hibernate itself — and `driftsFromDeclaredBaseline()` makes
the difference visible instead of asserting a constant against itself. See
[repository-adaptation.md](repository-adaptation.md) §4.
## Capability support levels
| Capability | Level |
|---|---|
| Full-transaction retry | Stable |
| Commit completion evidence | Stable |
| Keyset pagination | Stable |
| JDBC batch | Stable |
| Flyway schema gate | Stable |
| Runtime role verification | Stable |
| Observability | Stable |
| PostgreSQL native write (`ON CONFLICT`/`RETURNING`) | Advanced |
| PostgreSQL work claim (`SKIP LOCKED`) | Advanced |
| PostgreSQL JSONB | Advanced |
| PostgreSQL array and range | Advanced |
| Bulk DML | Advanced |
| Hibernate `StatelessSession` | Advanced |
| PostgreSQL `COPY` | Admin (J4) |
| Hibernate second-level cache | Advanced |
| Hibernate Envers | Advanced |
| Multi-tenancy (column, RLS, schema, database) | Experimental |
| Consistency-aware read replica | Experimental |
## Release gates
Each row is a way the platform could pass its tests and still be wrong in production.
| Gate | Kind | What it prevents |
|---|---|---|
| `postgresql-contract` | gate | a release whose only database evidence came from H2 |
| `completion-unknown-no-retry` | gate | automatically re-running a write that may already have committed |
| `osiv-disabled` | gate | lazy loading from the view layer, one query per rendered row |
| `flyway-validate` | gate | Hibernate mutating a deployed schema, or running against one it was not built for |
| `runtime-role-no-ddl` | gate | the application's own credential being able to alter or drop schema objects |
| `hibernate-7.4-fetch-pagination` | gate | a paged collection fetch silently reading the whole table and paginating in memory |
## Explicitly unsupported
- Reactive JPA. JPA is a blocking specification; a reactive facade over it moves the blocking call
onto an event loop rather than removing it.
- Hibernate as the production schema writer. `ddl-auto` never mutates a deployed schema.
- A platform-owned generic CRUD repository. Domains own their repositories (design §10.1).
- Automatic reconciliation of a completion-unknown transaction. The platform records; the domain
resolves.
+66
View File
@@ -0,0 +1,66 @@
# Transaction Guide
Design §15-§20. What owns a transaction, what may be retried, and what must never be.
## The application service owns the boundary
Repository adapters do not open transactions. The use case does, through `TransactionPort` or
`JpaTransactionExecutor`, because the unit of work is a business decision and only the use case
knows where it starts and ends.
Open Session In View is off in every runtime profile. It is on by default in Spring Boot, which is
why `JpaDangerousConfigurationGuard` fails startup rather than trusting configuration review.
## Profiles
A `TransactionProfile` fixes propagation, isolation, timeout, read-only, and the retry budget. A
write profile must carry a positive finite timeout — the type refuses to represent one without —
because an unbounded write transaction holds a connection, its locks, and its row versions for as
long as one stuck statement takes.
`REQUIRES_NEW` is opt-in. It acquires a second physical connection while pinning the first, so a
profile using it must be paired with the pool-pressure evidence in design §38:
```text
maximumPoolSize >= (concurrent_threads x (1 + max_requires_new_depth)) + 1
```
## Retry is per use case, never per statement
`FullTransactionRetryCoordinator` re-enters the executor, which produces a new transaction and a new
Persistence Context for every attempt. That granularity is the whole point: an optimistic conflict
means the state the attempt computed against is no longer the committed state, so re-issuing the
same statement would compute the same wrong answer. The domain rules have to run again against
reloaded data.
Retryable: serialization failure (`40001`), deadlock (`40P01`), optimistic conflict.
Not retryable: constraint violations, schema mismatch, query timeout, and anything unclassified.
Two additional refusals, independent of budget:
- An attempt that declared an irreversible external effect through `IrreversibleSideEffectContext`.
Rollback reverses database work only; an email or a card charge has already changed the world.
- Anything completion-unknown.
## Completion unknown
`TransactionCompletionUnknownException` is never retried, and the type system enforces it twice:
`JpaFailureContext` refuses to represent a retryable completion-unknown failure, and the exception
rebuilds its context through the safe factory whatever it is handed.
`EvidenceAwareJpaTransactionManager` marks the phase `COMMITTING` immediately before delegating to
the provider commit and never after. If the network, the JVM, or the server dies inside that call,
the last thing written is "we asked, we do not know" — which is exactly the state that must not be
mistaken for a rollback.
Recovery is reconciliation, not retry:
```text
record the transaction key -> check the idempotency record
-> check the business row
-> check the outbox
-> still undetermined? reconciliation queue
```
`CompletionUnknownRecorder` writes that record through a channel outside the unknown transaction.
Writing it through the same connection would make the audit trail share the failure it documents.
+91
View File
@@ -0,0 +1,91 @@
# Advanced — CSFLE and Queryable Encryption
**Capabilities:** `MongoCapability.CSFLE`, `MongoCapability.QUERYABLE_ENCRYPTION`
**Properties:** `ca-skeleton.persistence-mongo.advanced.csfle.enabled`,
`ca-skeleton.persistence-mongo.advanced.queryable-encryption.enabled`
**Status:** Advanced.
## Requirements
| | |
|---|---|
| Topology | Replica set or sharded cluster. |
| Server | MongoDB 7.0 or 8.0 (see §4 for the 8.0 query-type limits). |
| Privilege | `MongoPrincipalRole.ENCRYPTION_ADMIN` for the key vault; the application role never holds it. |
| Environment | A real KMS and key vault. A local key provider does not exercise any of the failure modes that matter. |
## 1. CSFLE
`MongoCsfleProfile` binds a collection to its `MongoCsfleFieldPolicy` list, a key vault
`MongoCredentialReference` and the key vault namespace. `MongoCsfleClientFactory` builds the encrypted
client; `MongoDataKeyResolver` resolves data keys.
`MongoCsfleMode`:
| Mode | Queryable | Trade-off |
|---|---|---|
| `RANDOMIZED` | no | Same plaintext encrypts differently each time. The safe default. |
| `DETERMINISTIC` | equality only | Same plaintext always yields the same ciphertext, so equality works — and so does frequency analysis. |
| `UNINDEXED` | no | Stored encrypted, excluded from any index. |
`MongoCsfleFieldPolicy.forPii(field, queryable)` defaults to `RANDOMIZED` when the field is not
queried. Deterministic encryption requires a written `equalityQueryJustification`; the constructor
refuses a blank one, naming frequency analysis. A low-cardinality deterministic field (a status, a
country, a boolean) leaks its distribution to anyone who can read the collection, which is the party
encryption was protecting against.
## 2. Queryable Encryption
`MongoQueryableEncryptionProfile` binds a collection to `MongoEncryptedFieldDescriptor` entries.
`MongoQueryableEncryptionQueryType` has exactly two values:
- `EQUALITY`
- `RANGE` — must declare its domain (`min`, `max`). The constructor refuses a range field without one,
because changing the domain later means re-encrypting the field.
`MongoQueryableEncryptionCollectionManager` owns the collection's lifecycle, because a QE collection
is not just a collection: it carries metadata collections.
## 3. Metadata ownership
`MongoEncryptionMetadataOwnership` maps `customers` to `enxcol_.customers.esc` and
`enxcol_.customers.ecoc`, and reports `__safeContent__`-prefixed indexes as
`MongoMetadataOwnership.ENCRYPTION_MANAGED`.
These are never application-owned and never droppable by drift reconciliation. A drift tool that
drops `enxcol_.customers.ecoc` corrupts the collection's queryability. This is the single most
important integration point between encryption and
[ADR-MONGO-004](../../adr/ADR-MONGO-004-index-schema-admin-plane.md).
## 4. Unsupported combinations
Refused at declaration, not discovered at runtime:
| Combination | Why |
|---|---|
| CSFLE **and** QE on the same collection | Two incompatible encryption schemes over one namespace. Both profile constructors refuse it. |
| CSFLE on a time series collection | `requireNotTimeSeries(true)` raises `MongoOperationRejectedException`. |
| QE `prefix` / `suffix` / `substring` | Not available on the platform's 8.0 baseline. The factory methods throw `UnsupportedOperationException` rather than returning a profile that fails later. |
| Deterministic CSFLE without a justification | `IllegalArgumentException` naming frequency analysis. |
| Range QE without a declared domain | `IllegalArgumentException` naming re-encryption. |
## 5. Failure recovery
| Symptom | Cause | Action |
|---|---|---|
| `MongoEncryptionException` on read | Wrong data key, or the key vault is unreachable | Check KMS reachability and the key vault credential. Data is intact; the client cannot decrypt it. |
| `MongoEncryptionException` on write | KMS permission revoked mid-operation | Restore the grant. Writes fail closed — nothing was written in plaintext. |
| Queries return nothing on a deterministic field | The field was re-keyed | Equality matching is over ciphertext; a new key produces different ciphertext. Re-encrypt the field. |
| QE queries fail after a drift reconciliation | A metadata collection was dropped | Restore from backup. This is why ownership gates drops. |
**Key rotation.** Rotating the customer master key re-wraps the data keys and does not require
re-encrypting documents. Rotating a *data* key does require re-encrypting every document that used
it. These are different operations with different costs, and confusing them is how a rotation becomes
an outage.
## 6. Promotion evidence
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md), promotion requires the
real KMS and key vault, plus negative cases that fail closed: wrong key, missing permission, rotation
mid-operation (`MongoAtlasCapabilityContractSuite.kmsFailureModes`). A local key provider certifies
none of these — it never rejects anything.
+63
View File
@@ -0,0 +1,63 @@
# Advanced — GridFS compatibility and migration
**Capability:** `MongoCapability.GRIDFS_COMPATIBILITY`
**Property:** `ca-skeleton.persistence-mongo.advanced.gridfs-compatibility.enabled`
**Status:** Advanced, compatibility only. Decision D-14.
## Position
GridFS is a **compatibility adapter for files that already exist there**. New files use the existing
Fileserver / Object Storage adapter, which is the source of truth for binary content.
The reason is not preference. GridFS stores file chunks in the same collections, on the same replica
set, competing for the same working set as your documents. A large file read evicts document pages
from cache, and file storage growth becomes replica-set growth — which means it becomes oplog
pressure, backup duration and failover time. Object storage was built for this and MongoDB was not.
## Reading legacy files
`MongoGridFsCompatibilityReader` reads existing GridFS content as
`GridFsLegacyContent(legacyId, filename, sizeBytes, checksum, stream)`. It reads; it does not write.
## Migration
`MongoGridFsMigrationJob` moves a file to object storage in a fixed order:
```
read legacy content
→ write to object storage
→ verify the target checksum matches the source
→ switch the reference
→ (later, separately) delete the source
```
Three properties, each of which exists because of a specific way this goes wrong:
1. **Verify before switching.** `MongoGridFsObjectReference` requires a non-blank checksum, and the
job returns empty and writes no reference when the target checksum does not match the source. A
migration that switches the reference on a successful *write* rather than a verified *copy*
silently points at a truncated object.
2. **The source is never deleted here.** Deletion is a separate, later decision after the new
location has been serving reads long enough to be trusted. A migration that deletes as it goes has
no rollback.
3. **The checkpoint separates migrated from failed.** `MongoGridFsMigrationCheckpoint` tracks
`migratedCount()`, `failedCount()`, `clean()` and `lastMigratedLegacyId()`, so a restart continues
from the last completed file rather than starting over, and a partially failed run is visible as
partial rather than as "done".
## Failure recovery
| Symptom | Cause | Action |
|---|---|---|
| `migrate` returns empty | Checksum mismatch | The copy is bad. Investigate before retrying; do not force the reference. |
| `IllegalArgumentException` on the reference | Missing checksum | A reference without a checksum cannot be verified and is refused. |
| Checkpoint not `clean()` | Some files failed | Re-run for the failed ids only; the checkpoint names the last successful one. |
| Reference switched but content missing | Source deleted too early | Restore from backup. This is what rule 2 prevents. |
## Promotion evidence
Actual-topology evidence against the real object storage backend, a security review of the storage
credential, the migration path above, failure cases (checksum mismatch refused, missing checksum
refused, restart resumes), and this document as the runbook.
New file storage does not go through here at all — see the fileserver adapter.
+90
View File
@@ -0,0 +1,90 @@
# Advanced — Multi-tenancy
**Capabilities:** `MongoCapability.SHARED_COLLECTION_TENANCY` (Advanced),
`MongoCapability.DATABASE_PER_TENANT` (Experimental)
**Properties:** `ca-skeleton.persistence-mongo.advanced.shared-collection-tenancy.enabled`,
`ca-skeleton.persistence-mongo.advanced.database-per-tenant.enabled`
## 1. Shared collection
Every tenant's documents live in one collection, discriminated by a tenant field.
`MongoTenantContext` carries the tenant. `MongoTenantPredicateInjector` adds the tenant predicate to
every query, every atomic filter and the **first** aggregation stage. `TenantScopedMongoOperations`
is the entry point, so a caller cannot construct an unscoped operation by forgetting.
Three details are load-bearing:
- **Injection, not convention.** A tenant predicate that each query is expected to add itself is a
cross-tenant leak waiting for one missed `where(...)`. The injector adds it structurally.
- **First aggregation stage.** `firstStageMatch(...)` places the tenant `$match` before anything else.
A `$lookup` or `$group` that runs before the tenant filter has already crossed the boundary, even if
a later stage filters the output.
- **An absent tenant is not "all tenants".** The injector takes an `Optional<MongoTenantContext>` so
the missing case is a decision the policy makes explicitly, not a predicate that quietly disappears.
`MongoTenantManifestValidator.validate(manifest, tenantScopedUniqueIndexes)` checks that every unique
index that should be per-tenant actually includes the tenant field. A unique index on `email` alone in
a shared collection makes an email globally unique across tenants — tenant B cannot register an
address tenant A already used, which is both a bug and an information leak.
`requireShardKeyAnalysed(...)` requires a shard-key readiness report before a shared-collection tenant
model is sharded: tenant id as a shard key prefix concentrates the largest tenant on one shard.
### Observability
`tenantId` and `rawTenantId` are on `MongoObservationConvention`'s forbidden tag list. Cardinality
grows with the customer list, and the tag ships tenant identity into the metrics backend.
## 2. Database per tenant (Experimental)
`MongoTenantDatabaseResolver` maps a tenant to its database; `MongoTenantClientRegistry` holds the
clients.
Experimental for a specific reason: it is correct in the small and unbounded in the large. Each tenant
database costs connections, file handles and monitoring cardinality. It works beautifully at 20
tenants and falls over at 2,000, and nothing in a functional test distinguishes the two. Promotion
requires operational scale evidence.
`MongoTenantLifecyclePolicy`:
- `requireActivationReady(tenantKey, schemaAndIndexesValidated)` — a tenant is not activated until its
schema and indexes are validated. Activating first means the first customer request is the migration
test.
- `requireDeleteAllowed(...)` — deletion requires an explicit retention decision. Dropping a tenant
database is irreversible and takes the backup surface with it.
`MongoTenantMigrationCoordinator` runs a migration across tenant databases with per-tenant results.
Partial failure is normal and must be reported per tenant: "migration failed" across 500 databases is
not a report anyone can act on.
## 3. Choosing
| | Shared collection | Database per tenant |
|---|---|---|
| Isolation | Logical, enforced by injection | Physical |
| Tenant count | Unbounded | Bounded by connections and file handles |
| Per-tenant restore | Hard | Natural |
| Noisy neighbour | Shared resources | Isolated |
| Migration | One collection | N databases, partial failures |
| Cross-tenant query | Possible (and must be forbidden) | Structurally impossible |
Shared collection is the default. Database-per-tenant is for a small number of tenants with a
contractual isolation or per-tenant-restore requirement.
## 4. Failure recovery
| Symptom | Cause | Action |
|---|---|---|
| Cross-tenant data visible | An operation bypassed `TenantScopedMongoOperations` | Treat as a security incident. Find the path, close it, audit access. |
| Unique constraint fires across tenants | Unique index missing the tenant field | Rebuild the index with the tenant field as prefix; the validator catches this before it ships. |
| One shard holds most data | Tenant id as shard-key prefix with a dominant tenant | Refine the shard key with a high-cardinality suffix. |
| Connection exhaustion | Database-per-tenant beyond the connection budget | The scale limit. Consolidate or move to shared collections. |
| Migration partially applied across tenants | Normal | `MongoTenantMigrationCoordinator` reports per tenant; re-run for the failures only. |
## 5. Promotion evidence
Shared-collection tenancy: actual-topology evidence, a security review covering cross-tenant access,
a migration path, failure cases (injection proven on query, atomic filter and first aggregation
stage), this runbook. Database-per-tenant additionally requires **operational scale evidence** and
stays Experimental until it exists.
+81
View File
@@ -0,0 +1,81 @@
# Advanced — Search and Vector Search
**Capabilities:** `MongoCapability.SEARCH`, `MongoCapability.VECTOR_SEARCH`
**Properties:** `ca-skeleton.persistence-mongo.advanced.search.enabled`,
`ca-skeleton.persistence-mongo.advanced.vector-search.enabled`
**Status:** Experimental (design §3.3). Hybrid search likewise.
## Requirements
| | |
|---|---|
| Topology | A deployment with the search service. Atlas Local in a container is a pull-request convenience and is **not** release evidence. |
| Privilege | `MongoPrincipalRole.SEARCH_ADMIN` for index management; the application role queries only. |
| Gate | `MongoAtlasCapabilityContractSuite` on the actual target deployment. |
## 1. Created is not ready
`MongoSearchIndexState`: `CREATED``BUILDING``READY`, plus `FAILED` and `DELETING`.
`MongoSearchReadinessGate.requireReady(state)` refuses anything but `READY`. A search index is built
asynchronously: the create call returns immediately and the index answers queries with *partial*
results while building. Not an error, not empty — partial. A deployment that creates an index and
starts querying serves incomplete results for as long as the build takes, and nothing reports it.
## 2. Search indexes are not application-owned
`MongoSearchIndexDescriptor.metadataOwnership()` is `MongoMetadataOwnership.SEARCH_MANAGED`, and
`droppableByApplicationDrift()` is false. The index reconciliation described in
[ADR-MONGO-004](../../adr/ADR-MONGO-004-index-schema-admin-plane.md) must not drop it.
## 3. Query guardrails
`MongoSearchQuery` binds an index, an allowlist of paths, the search text and a result limit.
- `requireAllowedPaths(allowed)` raises `MongoOperationRejectedException` on a path outside the
allowlist. Without it, a caller can search any indexed field, including ones indexed for a
different purpose.
- Search text length and result count are bounded at construction. An unbounded search text is a
cost multiplier on someone else's service.
## 4. Vector search
`MongoVectorIndexDescriptor.cosine(path, dimensions)` declares the index.
`MongoEmbedding.forIndex(index, values)` binds an embedding to it and **rejects a dimension
mismatch** — a 1536-dimension embedding against a 768-dimension index is not a runtime degradation,
it is a category error, and catching it at construction beats catching it as a confusing server
message.
`MongoEmbedding` copies its backing array in and out. A vector that shares an array with its caller
can be mutated after the query is built, which produces a query nobody wrote.
`MongoVectorQuery` requires `numCandidates > limit` — searching 10 candidates to return 10 results is
an exhaustive scan wearing an ANN index's name. `MongoVectorQuery.nearest(embedding, 10)` uses the
standard 20× ratio (200 candidates for 10 results).
## 5. Relevance is the gate, not functionality
`MongoVectorSearchBenchmarkGate.standard()` requires **recall** alongside latency and index size.
`requiredEvidence()` names recall explicitly.
This is the difference between search and everything else in the platform. A vector index can be
functionally perfect — it accepts the index, accepts the query, returns k results, within the latency
budget — and return the wrong k. A gate that measures only latency certifies a fast wrong answer.
`failures(recall, latencyMs, indexMb)` reports which dimension failed so the finding is actionable.
## 6. Failure recovery
| Symptom | Cause | Action |
|---|---|---|
| Incomplete results after a deploy | Queried a `BUILDING` index | Wait for `READY`. The gate prevents this; if it fired, something bypassed it. |
| `MongoOperationRejectedException` on a path | Path not in the allowlist | Add it deliberately, or fix the caller. |
| Dimension mismatch | Model changed | A new model means a new index. Build alongside, cut over, then retire. |
| Recall dropped without a code change | The index was rebuilt with different parameters, or the data distribution shifted | Re-run the benchmark gate; treat a recall regression like a failing test. |
| Index `FAILED` | Build error on the search service | Search-side diagnosis; the application must not fall back to a scan silently. |
## 7. Promotion evidence
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md): the actual target
deployment (not Atlas Local), security review of `SEARCH_ADMIN`, a rebuild path, failure cases
(non-ready index refused, disallowed path refused, dimension mismatch refused), this document as the
runbook, **and** relevance evidence. Search and vector search do not promote on functional success.
+82
View File
@@ -0,0 +1,82 @@
# Advanced — Sharding
**Capability:** `MongoCapability.SHARDING`
**Property:** `ca-skeleton.persistence-mongo.advanced.sharding.enabled`
**Status:** Advanced. Reshard orchestration remains Experimental.
## Requirements
| | |
|---|---|
| Topology | A real sharded cluster. A replica set cannot exercise routing. |
| Server | MongoDB 7.0 or 8.0. |
| Privilege | `MongoPrincipalRole.SHARD_ADMIN` for the admin plane; the application role is unchanged. |
| Gate | `mongoShardedTest` lane with `MongoShardingContractSuite`. |
## Shard key
`ShardKeyDescriptor` declares the key as an ordered list of `ShardKeyPart` plus a `ShardStrategy`:
| Strategy | Distributes | Cost |
|---|---|---|
| `RANGE` | by value ranges | Range queries stay targeted; a monotonic key (a timestamp, an `ObjectId`) sends every insert to one shard. |
| `HASHED` | by hash of the key | Inserts spread evenly; every range query becomes scatter-gather. |
There is no strategy that is good at both, which is why the choice is a declaration rather than a
default.
## Routing classification
`ShardAwareQueryValidator` classifies each query before execution:
| `MongoRoutingClassification` | Meaning |
|---|---|
| `TARGETED` | The full shard key is present. One shard answers. |
| `PREFIX_TARGETED` | A prefix of a compound key is present. A subset of shards answers. |
| `SCATTER_GATHER` | No shard-key predicate. Every shard answers. |
| `REJECTED` | Scatter-gather where the profile forbids it. |
A scatter-gather query is not an error — some queries legitimately need every shard — but it must be
declared. Undeclared scatter-gather raises `MongoShardRoutingException`. The reason is that
scatter-gather passes every test on a single-shard development cluster and only degrades once the
cluster grows, at which point the query is already in production and the fix is a schema change.
## Unsupported combinations
- Unique index on a field that is not a prefix of the shard key. MongoDB cannot enforce it across
shards, and it fails at index creation, not at query time.
- Transactions that touch documents on multiple shards remain supported but cost a cross-shard
two-phase commit. Prefer a shard key that keeps a transaction's documents co-located.
- CSFLE on a sharded collection: see [encryption.md](encryption.md) for the combinations that are
refused.
## Admin plane
`MongoShardingAdminGateway` (D4, `SHARD_ADMIN` credential) covers shard-collection, refine-shard-key
and reshard.
`ShardKeyAnalyzer` produces a `ShardKeyReadinessReport` before sharding a collection: cardinality,
frequency skew and monotonicity. A key with low cardinality creates jumbo chunks that cannot be split;
a monotonic key creates a hot shard. Both are visible in the report and invisible in a functional
test.
`ReshardApproval` is required for a reshard — a named approver and a stated window. Resharding
rewrites the collection: it duplicates the data during the operation and saturates IO. It is not a
runtime operation and the type refuses to pretend otherwise.
## Failure recovery
| Symptom | Cause | Action |
|---|---|---|
| `MongoShardRoutingException` | Undeclared scatter-gather | Add the shard key to the predicate, or declare the query as scatter-gather in its profile after review. |
| Jumbo chunks | Low-cardinality shard key | Refine the shard key (adds a suffix, non-destructive) before considering a reshard. |
| One hot shard | Monotonic range key | Refine with a high-cardinality prefix, or reshard to hashed if range queries are not needed. |
| Balancer never converges | Chunk migration blocked by long-running operations | Check for long transactions and cursors; the balancer waits on them. |
## Promotion evidence
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md): actual sharded-cluster
evidence, security review of the `SHARD_ADMIN` role, a migration path for an existing unsharded
collection, failure cases (undeclared scatter-gather refused, jumbo chunk detected), and this
document as the runbook. Reshard orchestration stays Experimental until operational scale evidence
exists.
+73
View File
@@ -0,0 +1,73 @@
# Advanced — Time Series
**Capability:** `MongoCapability.TIME_SERIES`
**Property:** `ca-skeleton.persistence-mongo.advanced.time-series.enabled`
**Status:** Advanced.
## Requirements
| | |
|---|---|
| Topology | Replica set or sharded cluster. |
| Server | MongoDB 7.0 or 8.0. |
| Privilege | Standard application role; collection creation goes through the admin plane. |
## Descriptor
`MongoTimeSeriesDescriptor` declares:
- **timeField** — required, a BSON date. This is the bucketing axis.
- **metaField** — optional but nearly always wanted: the series identity (device id, tenant, sensor).
Documents sharing a `metaField` value bucket together, which is where the compression comes from.
- **granularity** — `MongoTimeSeriesGranularity`:
| Granularity | Bucket span | Use for |
|---|---|---|
| `SECONDS` | 1 hour | Sub-second to per-second ingest. |
| `MINUTES` | 24 hours | Per-minute metrics. |
| `HOURS` | 30 days | Hourly rollups. |
Granularity that is too fine produces many small buckets and loses the compression; too coarse
produces oversized buckets that must be read whole to answer a narrow query.
## What a time series collection is not
`MongoTimeSeriesCapabilityValidator` refuses the operations the collection type does not support, at
declaration time rather than at first use:
- **No arbitrary updates.** Time series data is append-mostly. Delete and limited update support
exists on recent servers but is not part of this platform's contract.
- **No unique index on the measurement.** There is no `_id` to be unique on in the usual sense.
- **No CSFLE.** Refused — see [encryption.md](encryption.md).
- **No change stream on the raw buckets** as a business event source. The bucket documents are a
storage representation, not your measurements.
Converting an existing regular collection to a time series collection is a copy, not an alter. Plan
it as a migration with a dual-write window.
## TTL
Time series collections use `expireAfterSeconds` on the collection rather than a TTL index on a
field. The [TTL rules](../schema-index-migration-guide.md#4-ttl) still apply: expiry is physical
cleanup on a bucket boundary, so a measurement can outlive its expiry by up to a bucket span plus the
monitor interval. Do not treat absence as a deadline.
## Operations
`MongoTimeSeriesOperations` is the port for insert and windowed read. Reads are bounded by the same
`MongoOperationBudget` as everything else: an unbounded time-range query on a time series collection
is the fastest way to read a year of data into heap.
## Failure recovery
| Symptom | Cause | Action |
|---|---|---|
| Writes rejected with an unsupported-operation error | An update or unique-index expectation | The collection type does not support it; change the access pattern. |
| Poor compression / large storage | Missing `metaField`, or granularity too fine | Both require a rebuild; measure on a copy before committing. |
| Slow range queries | Granularity too coarse for the query window | Same: rebuild with the granularity matched to the dominant query. |
## Promotion evidence
Actual-topology evidence on the target deployment, a migration path from the existing collection,
failure cases (unsupported update refused, CSFLE combination refused), and this document as the
runbook.
+92
View File
@@ -0,0 +1,92 @@
# BSON Mapping Guide
Design §10 and decision D-06. The representation of a value in BSON is a data contract, not an
implementation detail: once a collection holds a million documents, changing how a `BigDecimal` is
stored is a migration with downtime, not a code change. `MongoTypeRepresentationManifest` pins the
representation so a library upgrade or a different default cannot move it.
## 1. The manifest
`MongoTypeRepresentationManifest.standard()` fixes:
| Java type | BSON | Representation type |
|---|---|---|
| `UUID` | `Binary` subtype 4 | `MongoUuidRepresentation.STANDARD` |
| `BigDecimal` | `Decimal128` | `MongoDecimalRepresentation.DECIMAL_128` |
| `BigInteger` | `Decimal128` (or `String` when out of range, declared) | `MongoBigIntegerRepresentation` |
| `Instant` / `OffsetDateTime` / `ZonedDateTime` | UTC `Date` | `MongoTemporalRepresentation.UTC_DATE` |
| `LocalDate` | `String` (ISO-8601) or UTC `Date`, declared per field | `MongoTemporalRepresentation` |
| `enum` | `String` name | `MongoEnumRepresentation.NAME` |
`MongoMappingConfiguration` and `MongoCustomConversionsFactory` build the Spring Data converters from
the manifest, so there is one place to read and one place to change.
## 2. UUID
`UuidRepresentation.STANDARD` (subtype 4), always. The driver's legacy Java representation
(subtype 3) byte-swaps two halves of the UUID, so a document written by one representation and read
by the other yields a different — and valid-looking — UUID. Nothing errors; you just get the wrong
row. The golden snapshot kit pins the codec explicitly for this reason
(`MongoBsonSnapshot.defaultRegistry()`).
## 3. Decimal
`BigDecimal``Decimal128`, never `Double`. `12.30` stored as a double is `12.299999999999999`, and
a monetary comparison written against it will one day be wrong by a cent for a customer who notices.
`BigDecimalToDecimal128Converter` / `Decimal128ToBigDecimalConverter` are registered from the
manifest.
`Decimal128` has 34 significant digits; a `BigDecimal` beyond that range fails on write rather than
rounding silently.
## 4. Time
Store instants, not local times. `LocalDateTimeMappingGuard` refuses `LocalDateTime` fields on a
mapped document: a `LocalDateTime` has no offset, so the value that goes in depends on the JVM
default zone of whichever instance wrote it, and the two instances in a rolling deploy can disagree.
Use `Instant` when the moment matters and `LocalDate` when the calendar day matters.
## 5. Type metadata
`MongoTypeMetadataPolicy` decides what goes in `_class`:
| Policy | Stored | Use when |
|---|---|---|
| `NONE` | nothing | The collection holds exactly one type and never will hold a subtype. |
| `ALIAS` | a registered short alias | A polymorphic hierarchy in a long-lived collection. |
| `CLASS_NAME` | the FQCN | Short-lived or internal collections only. |
`PolicyAwareMongoTypeMapper` enforces it, and `MongoTypeMetadataRegistry` holds alias → class.
A `@LongLivedMongoDocument` type with `CLASS_NAME` is refused: writing `com.example.OrderV2` into a
million documents means that renaming the package is a data migration.
## 6. Missing versus null
The golden kit keeps these apart deliberately. `MongoBsonSnapshotAssert.hasNoField(...)` and
`hasExplicitNull(...)` are different assertions, because in MongoDB they are different documents:
`{"a": null}` matches `{a: null}` and `{a: {$exists: true}}`, while `{}` matches only the first.
A mapper change that starts writing explicit nulls silently changes what your queries return.
## 7. Golden representation tests
Every collection with a fixed representation should have a snapshot test:
```java
MongoBsonSnapshot snapshot = MongoBsonSnapshot.of(storedDocument);
MongoBsonSnapshotAssert.assertThat(snapshot)
.hasBsonType("amount", "DECIMAL128")
.hasBsonType("externalId", "BINARY")
.hasNoJavaClassName("dev.caskeleton")
.hasTypeSignature("_id:OBJECT_ID,amount:DECIMAL128,createdAt:DATE_TIME,externalId:BINARY");
```
`hasTypeSignature` is the regression gate: it fails on *any* representation change, including ones a
value-equality assertion would pass. When it fails, the question is whether the change was intended
and has a migration — not whether to update the string.
## 8. Round trips
`MongoRoundTripContract` asserts that `write → read` returns an equal domain object *and* that
`write → read → write` produces an identical BSON document. The second half is what catches an
asymmetric converter: a value that reads back equal but re-serialises differently makes every
subsequent `save()` a spurious update, and turns change streams into a noise generator.
+105
View File
@@ -0,0 +1,105 @@
# Change Stream Guide
Design §20, decision D-12. A change stream is an **at-least-once projector**, not an event bus.
## 1. What a change stream is not
D-12 is explicit: a physical change event is not a business integration event. The two differ in
ways that matter to every consumer:
| Change event | Integration event |
|---|---|
| Emitted per document write | Emitted per business fact |
| Shape follows the storage schema | Shape is a published contract |
| A refactor of the document changes it | A refactor of the document does not change it |
| Replayed on resume, duplicated on retry | Versioned and deliberately evolved |
Publishing raw change events externally makes your storage schema a public API, and the first time
someone renames a field the downstream consumers break. If you need to bridge to messaging, use the
Advanced bridge, which maps to an owned envelope
([advanced/multi-tenancy.md](advanced/multi-tenancy.md) is separate;
the bridge is described in §7 below).
## 2. Subscription and resume
`MongoChangeStreamSubscription` declares the collection, pipeline and consistency. `MongoResumePosition`
is either a resume token or a cluster time; `MongoResumeCheckpoint` is what gets persisted and
`MongoResumeCheckpointStore` persists it.
The checkpoint stores the token as a Base64 `encodedToken` string rather than a byte array — a record
with an array component has broken equality, and a checkpoint that does not compare correctly is a
checkpoint that silently fails its own dedup test.
## 3. Checkpoint after processing, not after receiving
The ordering rule that makes at-least-once actually hold:
```
receive event
→ process it (idempotently)
→ persist the checkpoint
```
Checkpointing on receipt turns the delivery guarantee into at-most-once, and the events lost are
exactly the ones the process died while handling.
## 4. Idempotency
`MongoChangeEventIdentity` is the dedup key: `(resumeToken, documentKey, clusterTime, operationType)`.
`MongoChangeDeduplicationStore` records what has been applied. Duplicates are not an edge case — every
resume after any interruption replays at least one event, so a projector that is not idempotent is
wrong on its first restart, not on some rare day.
`MongoChangeProjector` returns a `MongoChangeProjectionResult` so the runner can distinguish applied
from skipped-as-duplicate, and the skip count is worth a metric: a sudden rise means something is
looping.
## 5. States and recovery
`MongoChangeStreamState`: `STARTING`, `RUNNING`, `RESUMING`, `STOPPED`, `HISTORY_LOST`.
`MongoChangeStreamRecoveryPolicy` returns a `MongoChangeStreamRecoveryDecision`, which is either
`resume()` (auto-resume from the checkpoint) or `halt(state, runbook)`. A halting decision **must**
name a runbook — a decision that only says "stopped" leaves the on-call engineer to work out from
scratch whether the projection can be rebuilt and from what.
| Situation | Decision |
|---|---|
| Transient network error, token still valid | `resume()` |
| Primary failover | `resume()` — the token survives an election |
| `invalidate` (collection dropped/renamed) | `halt(STOPPED, …)``MongoInvalidateRecovery` |
| Token no longer in the oplog | `halt(HISTORY_LOST, "history-lost")``MongoChangeHistoryLostException` |
## 6. History lost
`MongoChangeHistoryLostException` is raised when the resume token predates the oldest oplog entry.
The stream **cannot** be resumed: the events between the checkpoint and now are gone from the server,
and no amount of retrying brings them back.
What the platform will not do is silently restart from "now". That looks like a recovery and is
actually a silent gap in the projection — the worst possible outcome, because nothing reports it. The
runner halts and requires an operator decision. See
[runbooks/history-lost.md](runbooks/history-lost.md).
## 7. Bridging to messaging (Advanced)
`MongoChangeMessagingBridge` is opt-in behind `MongoCapability.CHANGE_STREAM` plus the bridge's own
flag. It maps a change event to a platform-owned `MongoIntegrationEventEnvelope` through
`MongoChangeToIntegrationEventMapper` and hands it to a `MongoIntegrationEventPublisher` port.
The port is defined in the bridge package rather than imported from the messaging adapter because the
architecture registry forbids adapter-to-adapter dependencies; the composition root supplies the
implementation.
`MongoBridgeOutboxPolicy` and `MongoBridgeCheckpointPolicy` state the delivery contract: publish then
checkpoint, at-least-once, consumers must dedup on the envelope's event id.
## 8. Operating notes
- Change streams require a replica set. `MongoStartupValidator` refuses a change-stream profile on
`STANDALONE`.
- The change-stream principal is its own role (`MongoPrincipalRole.CHANGE_STREAM`) with
`changeStream` and `find` — not the application write credential.
- Oplog window is the recovery budget. If the oplog holds four hours, a consumer that is down for five
hours needs a rebuild, not a resume. Alert on consumer lag against the oplog window, not against
wall-clock.
@@ -0,0 +1,126 @@
# Consistency and Transaction Guide
Design §12–§16, decisions D-07 through D-10. This is the part of the platform where the wrong
default is most expensive and the least visible in testing, because every failure mode here needs a
primary change to reproduce.
## 1. Prefer a single-document atomic operation
D-09: a transaction is for a **multi-document invariant**, nothing else. A single document is already
atomic in MongoDB, so wrapping a one-document update in a transaction buys nothing and costs a
session, a two-phase commit and a new ambiguous outcome.
D-07: partial change uses update operators, not `save()`. `MongoAtomicOperations` /
`MongoAtomicOperationsTemplate` expose the operator set through `MongoUpdateOperator`
(`$set`, `$inc`, `$push`, `$pull`, `$addToSet`, `$min`, `$max`, `$currentDate`, …) with an
`AtomicFilter` precondition and a `ReturnDocumentMode`. Read-modify-write through `save()` replaces
the whole document and silently discards any field another writer changed in between — a lost update
with no error.
## 2. Whole-document replacement needs a revision
D-08. `VersionedMongoUpdater` requires a `MongoRevision`: either a Spring Data `@Version` field or an
explicit expected-revision predicate in `VersionedUpdateCommand`. A replacement whose filter matched
zero documents is not "nothing to do" — `MongoOptimisticConflictTranslator` distinguishes:
- filter matched nothing and the id does not exist → `MongoDocumentNotFoundException`
- filter matched nothing and the id exists → `MongoOptimisticConflictException`
Collapsing these two into one is how a concurrent overwrite becomes a 404.
## 3. Consistency profiles
`MongoConsistencyProfile` names the read/write concern pair; `MongoConsistencyRegistry` binds a
profile to an operation or collection, and `MongoConsistencyBinder` /
`ReactiveMongoConsistencyBinder` apply it at execution.
| Profile | Meaning | Use for |
|---|---|---|
| `PRIMARY_LOCAL` | primary read, local concern | Throughput-sensitive reads that tolerate a rollback window. |
| `PRIMARY_MAJORITY` | primary read, majority write | The default for anything a user will see again immediately. |
| `CAUSAL_MAJORITY` | majority inside a causal session | Read-your-writes across separate operations. |
| `STALE_READ_ALLOWED` | secondary reads permitted | Reporting and analytics that state their staleness. |
| `SNAPSHOT_TRANSACTION` | snapshot isolation | Multi-document reads inside a transaction. |
A profile is a declaration, not a hint: the registry is consulted per operation and an operation
without a registered profile is rejected rather than defaulting.
## 4. Causal sessions
`MongoCausalSessionContext` plus `SpringMongoCausalSessionExecutor` /
`ReactiveMongoCausalSessionExecutor` carry the cluster time and operation time between operations, so
"write then read" returns the write even when the read lands on a different node. Without a causal
session, `PRIMARY_MAJORITY` gives you durability but not read-your-writes across two calls.
In the reactive path the session travels in the Reactor context (`ReactiveMongoContextKeys`), not in
a thread local — a thread local is empty on the next operator in the chain.
## 5. Transactions
`MongoTransactionExecutor` / `ReactiveMongoTransactionExecutor` open a session through the session
factory, run the body, and commit. `MongoTransactionProfile` carries the consistency profile, the
`maxCommitTime` and the retry budget. Topology matters: a transaction requires a replica set, and
`MongoStartupValidator` refuses a transaction-declaring profile on `STANDALONE` at startup rather
than at the first call.
## 6. Retry: body and commit are different loops
D-10, and the single most consequential rule in the design.
```
for each body attempt:
open a NEW session
run the body
TransientTransactionError -> abort, next body attempt
commit
UnknownTransactionCommitResult -> retry COMMIT ONLY, same session
```
`MongoTransactionRetryCoordinator` implements exactly this:
- **A new session per body attempt.** Reusing the session after an abort carries the aborted
transaction's state into the retry.
- **The body is never replayed after a commit ambiguity.** An unknown commit means the commit may
already have applied. Re-running the body would apply it a second time. Only the commit is retried,
and a commit retry on an already-committed transaction is a no-op by design.
- **A budget bounds both loops.** `MongoRetryBudget` limits attempts *and* elapsed time, with jittered
backoff (`delayBefore(attempt, random)`), so a struggling primary is not retried into the ground.
`MongoRetryDecision` and `MongoRetryScope` (in `…api.error`) say what may be retried:
`MongoRetryScope.BODY`, `COMMIT_ONLY`, or `NONE`.
## 7. Ambiguous outcomes
`MongoExecutionOutcome` has six values, two of which are ambiguous and must not be collapsed:
| Outcome | Did the write happen? |
|---|---|
| `NOT_SENT` | No. Safe to retry. |
| `NO_WRITE_PERFORMED` | No — the server answered and did nothing. |
| `WRITE_CONFIRMED` | Yes. |
| `PARTIAL_BULK_WRITE` | Some of it. See `MongoBulkResult`. |
| `WRITE_RESULT_UNKNOWN` | **Unknown.** |
| `TRANSACTION_COMMIT_UNKNOWN` | **Unknown.** |
An unknown outcome is not a failure and must not be reported to a caller as one. The caller either
reconciles (`MongoCommitReconciler` re-reads a deterministic marker the body wrote) or surfaces the
ambiguity. See [runbooks/unknown-commit.md](runbooks/unknown-commit.md).
`MongoFailureContext` records only the design-permitted fields — outcome, category, operation name,
collection profile, retry scope, attempt — never the query, the document, or the values.
## 8. Failure translation
`DefaultMongoFailureClassifier` classifies **labels before codes**. The server's error labels
(`TransientTransactionError`, `UnknownTransactionCommitResult`, `RetryableWriteError`) are the
authoritative statement about retryability; an error code is a secondary signal whose meaning varies
by server version. `DefaultMongoFailureTranslator` maps a classification onto the stable exception
hierarchy, and anything unmatched becomes `MongoUnclassifiedFailureException` rather than leaking a
driver type.
## 9. Bulk writes
`MongoBulkExecutor` returns a `MongoBulkResult` with per-item `MongoBulkItemFailure` entries. An
unordered bulk write that partially fails is `PARTIAL_BULK_WRITE`, not a failure: some documents were
written. `MongoBulkPartialFailureException` carries the succeeded and failed indexes so a caller can
resume rather than replay.
+85
View File
@@ -0,0 +1,85 @@
# Document Modeling Guide
Design §7–§9. The platform does not own your documents — D-01 is explicit that the domain owns
`@Document`, repositories, queries, index requirements and schema version. What the platform owns is
the set of modeling decisions that are expensive to reverse once a collection holds production data.
## 1. There is no `CommonMongoRepository`
A generic `CommonMongoRepository<T, ID>` is listed under explicitly unsupported (§3.4), and the
reason is not purity. A shared supertype forces every collection to share an id strategy, a
consistency profile and a query surface, and the first collection that needs a different one either
gets a cast or a leaky generic parameter. Declare a Spring Data repository per aggregate.
## 2. Embed or reference
`MongoDocumentModelManifest` records the decision per collection so it is reviewable, and
`MongoDocumentModelValidator` refuses the combinations that do not survive growth.
| Descriptor | Use when |
|---|---|
| `EmbeddedCollectionDescriptor` | The child is read with the parent, is bounded, and has no independent lifecycle. Declare `maxElements`; an unbounded array is the single most common way a document reaches the size limit. |
| `MongoReferenceDescriptor` | The child is queried independently, is unbounded, or outlives the parent. Declare `MongoReferenceLifecycle` so the deletion story is written down rather than discovered. |
The validator rejects an embedded collection without a bound, and a reference whose lifecycle says
the child is owned by the parent but which is also referenced from elsewhere.
## 3. Size budget
`MongoDocumentSizeBudget`:
| Constant | Bytes | Meaning |
|---|---|---|
| `MONGODB_HARD_LIMIT_BYTES` | 16 MiB | MongoDB's own limit. |
| `PLATFORM_CEILING_BYTES` | 4 MiB | The largest budget the platform will accept. |
| `DEFAULT_BYTES` | 2 MiB | `MongoDocumentSizeBudget.standard()`. |
A budget above the ceiling is refused at construction. Budgeting to 16 MiB means the failing write
is the first symptom, and by then the collection is already full of near-limit documents.
## 4. Identity
`DomainDocumentId` and `MongoIdRepresentation` fix how a domain identifier becomes `_id`. Pick the
representation once per collection and record it in the manifest:
- `OBJECT_ID` — server-generated, monotonic, 12 bytes. Good default when the domain has no natural id.
- `UUID_BINARY` — a domain UUID stored as `Binary` subtype 4 (`STANDARD`). Never store a UUID as a
string "because it is easier to read"; it doubles the index size and loses the type.
- `STRING` — a natural key that is genuinely a string (a slug, an external system's id).
An `_id` choice is effectively permanent: it is the shard key candidate, the resume-token join key
and the pagination tie-breaker.
## 5. Schema version
Every long-lived collection carries `DocumentSchemaVersion`. `MongoSchemaVersionPolicy` and
`MongoSchemaVersionRange` say which versions the running code can read; a document outside the range
raises `MongoDataSchemaUnsupportedException` rather than being silently mapped with missing fields.
Write the range down before the migration, not after: the range is what lets old and new instances
run at once during a rolling deploy.
## 6. Type metadata
`@LongLivedMongoDocument` marks a document whose stored type alias must not be a Java class name.
`MongoTypeMetadataRegistry` maps alias → class. Storing the FQCN means moving or renaming the class
becomes a data migration; storing an alias keeps it a refactor. See
[bson-mapping-guide.md](bson-mapping-guide.md) §4.
## 7. Collection profiles
`MongoCollectionProfileRegistry` binds a `CollectionProfileName` to its consistency profile, budget
and allowlist. A collection that is not registered cannot be reached through
`MongoImperativeExecutor` or `ReactiveMongoExecutor` — the allowlist is the mechanism that keeps an
unreviewed collection from appearing in production by accident.
## 8. What to write down before the first insert
1. Embed/reference decision per child collection, with bounds.
2. Size budget.
3. `_id` representation.
4. Schema version range.
5. Index manifest (see [schema-index-migration-guide.md](schema-index-migration-guide.md)).
6. Consistency profile (see [consistency-transaction-guide.md](consistency-transaction-guide.md)).
Each of these is cheap now and a migration later.
+140
View File
@@ -0,0 +1,140 @@
# Query and Aggregation Guide
Design §17–§19, decision D-11. Every query and every pipeline is a registered, bounded thing. Free-form
JSON queries and unbounded pipelines are explicitly unsupported (§3.4).
## 1. Registered operations
Every execution carries a `MongoOperationContext`: a `MongoOperationName`, a `DatabaseProfileName`, a
`CollectionProfileName`, a `MongoOperationType` and a `MongoOperationScope`.
`MongoOperationName` matches `[a-z][a-z0-9.-]{2,95}`. It is the join key for the budget registry, the
consistency registry, the metric tag and the log line — a free-form or interpolated name breaks all
four at once, which is why the pattern is enforced at construction.
`MongoOperationScope` uses an `UNSPECIFIED` sentinel rather than `null`, so "the caller did not say"
is a value the policy layer can reject rather than an NPE further down.
## 2. Query guardrails
`PolicyAwareMongoQueryBuilder` builds a query from `MongoFieldDescriptor` + `MongoOperator` pairs
against a `MongoQueryPolicy`. The policy refuses:
- a field not in the collection's allowlist
- an operator not allowed for that field
- a sort on an unindexed field
- `$where`, `$expr` with arbitrary JavaScript, and server-side evaluation generally
- an unbounded `$regex`
`MongoRegexPolicy` requires an anchored prefix pattern and bounds the pattern length. An unanchored
regex is a collection scan wearing an index's clothes, and a user-supplied one is a denial-of-service
primitive.
`MongoSortDescriptor` pairs a field with a direction and is validated against the index manifest, so
a sort that would spill to disk fails review rather than production.
## 3. Operation budgets
`MongoOperationBudget` bounds four things at once:
| Bound | Why |
|---|---|
| `maxTimeMS` | The server stops working on a query nobody is waiting for. |
| result limit | An unbounded result set is an OOM with extra steps. |
| batch size | Bounds the per-round-trip memory. |
| examined-document ceiling | Catches an index regression that a time limit alone would hide on a fast day. |
`MongoBudgetPolicyRegistry` binds a budget to an operation name; `MongoBudgetEnforcer` applies it and
raises `MongoOperationRejectedException` before execution when a request exceeds it, and
`MongoTimeoutException` when the server enforces it.
## 4. Keyset pagination
Unbounded `skip` is unsupported: `skip(1_000_000)` makes the server walk a million documents to throw
them away, so page 1000 costs a thousand times page 1.
`MongoKeysetQueryBuilder` builds the resume predicate lexicographically. For a sort on `(a DESC, _id
DESC)` resuming after `(A, I)`:
```
(a < A) OR (a = A AND _id < I)
```
`validate()` rejects a `MongoKeysetSort` without a unique tie-breaker. Without one, two documents with
the same sort value straddle the page boundary and one of them is skipped or repeated — invisibly,
and only under concurrency.
`MongoNullSortOrdering` makes null placement explicit, because MongoDB's own ordering of missing
versus null versus present is not what most people assume.
### Cursors are authenticated
`MongoKeysetCursorCodec` signs the cursor with HMAC-SHA256 and compares with
`MessageDigest.isEqual` (constant time). An unsigned cursor is a client-controlled query predicate: a
caller can edit it to read a range they were never offered. A tampered or truncated cursor yields
`MongoCursorException`, never a partially-decoded resume position.
## 5. Aggregation guardrails
`MongoAggregationPlan` is a registered pipeline: an ordered list of `MongoAggregationStageDescriptor`
validated against a `MongoAggregationProfile`. `PolicyAwareMongoAggregationExecutor` runs only a
registered plan.
`MongoAggregationRisk` grades each stage, and the profile sets the ceiling:
| Risk | Stages | Policy |
|---|---|---|
| low | `$match` on an indexed prefix, `$limit`, `$project` | Always allowed. |
| moderate | `$group`, `$sort` with an index, `$unwind` with a bound | Allowed within budget. |
| high | `$lookup`, `$graphLookup`, `$facet`, unindexed `$sort` | Requires explicit approval in the profile. |
| forbidden | `$out`, `$merge` outside the admin plane, `$function`, `$accumulator` | Refused. |
`allowDiskUse` is a declared property of the plan, not a runtime flag. A pipeline that needs disk is a
pipeline whose shape should be reviewed.
## 6. Reactive execution and cursors
`ReactiveMongoExecutor` / `DefaultReactiveMongoExecutor` carry the operation context in the Reactor
context. `MongoCursorGuard` and `MongoCursorLease` bound cursor lifetime:
- a cursor has a lease with a deadline
- cancellation closes the server-side cursor (`MongoCursorTermination`)
- an abandoned cursor is a server-side resource, so the lease is released on cancel, error *and*
completion — `MongoReactiveCursorPublisher` uses `Flux.using` so all three paths run the same
release
A leaked cursor does not fail anything locally; it consumes a connection and a snapshot on the server
until the server's own timeout, which is why the guard is not optional.
## 7. Geospatial
`MongoGeoQuery` + `MongoGeoPoint` + `MongoGeoDistance` over a `2dsphere` index. Distances are metres
on a sphere (`nearSphere` with `maxDistance`), never degrees — a degree of longitude is a different
distance in Oslo than in Nairobi, and a radius expressed in degrees is a bug that only shows up away
from the equator. `SpringMongoGeospatialOperations` is the Spring Data binding;
`MongoGeospatialOperations` is the port.
## 8. Native capability gateway
When a registered operation genuinely needs something outside the Stable API, it goes through
`MongoNativeCapabilityGateway` (`PolicyAwareMongoNativeGateway`), never through the driver directly.
The admission order is fixed:
```
capability registered
→ database profile
→ collection allowlist
→ operation name present
→ timeout / maxTimeMS
→ consistency profile
→ result / batch limit
→ trace
→ log redaction
→ command category (MongoNativeCommandCategory)
→ D4 admin command refused
→ execute
```
`ApprovedMongoNativeOperation` is the registration record; `MongoNativeOperationPolicy` is the policy.
An admin-plane command reaching this gateway is refused regardless of capability — the admin plane has
its own credential and its own client (see [security-observability.md](security-observability.md)).
+124
View File
@@ -0,0 +1,124 @@
# MongoDB Document Persistence Platform — Repository Adaptation Contract
**Design source:** `mongodb-superpowers-package/docs/superpowers/specs/2026-08-11-mongodb-document-persistence-platform-design.md`
**Stable plan:** `mongodb-superpowers-package/docs/superpowers/plans/2026-08-11-mongodb-document-persistence-platform-implementation-plan.md`
**Advanced plan:** `mongodb-superpowers-package/docs/superpowers/plans/2026-08-11-mongodb-advanced-capabilities-expansion-plan.md`
The design package declares its own module root (`modules/mongodb`) and root package
(`io.backend.skeleton.mongodb`) as *implementation assumptions*, not as contract. This file is the
single record of how that assumed layout was mapped onto this repository. Only paths, build DSL,
and composition-root ownership changed. Public contracts, policy order, and error semantics are
implemented exactly as specified.
## 1. Why the module layout differs
The design assumes 19 Stable Gradle projects under `modules/mongodb/` and 12 Advanced projects
under `modules/mongodb-advanced/`. This repository is a Clean Architecture template whose
**fail-closed registry** (`src/config/architecture/modules.json`, enforced by `src/settings.gradle`
and `verifyCleanArchitectureDependencies`) declares **exactly 19 leaf identities**. Creating 31 more
Gradle projects would violate HARD-STOP #5 in `AGENTS.md`.
Therefore the design's 31 modules become **package boundaries inside the registered leaf**
`:adapter:outbound:persistence-mongo`, following the precedent already set by
[docs/httpclient/repository-adaptation.md](../httpclient/repository-adaptation.md). The design's
module dependency table (§6.3) is reproduced as ten ArchUnit rules in `MongoModuleBoundaryTest`, so
a forbidden edge fails the build the same way a missing Gradle dependency would.
## 2. Package mapping
Root package: `io.backend.skeleton.mongodb``dev.caskeleton.adapter.outbound.mongo`.
### 2.1 Stable modules
| Design module | Repository package |
|---|---|
| `mongodb-core-api` | `…outbound.mongo.api` (+ `.capability`, `.consistency`, `.error`, `.mapping`, `.observation`, `.profile`, `.schema`) |
| `mongodb-spring-data` | `…outbound.mongo.mapping` (+ `.type`), `…outbound.mongo.failure` |
| `mongodb-imperative` | `…outbound.mongo.imperative` (+ `.atomic`, `.bulk`, `.revision`) |
| `mongodb-reactive` | `…outbound.mongo.reactive` (+ `.cursor`) |
| `mongodb-query` | `…outbound.mongo.query` (+ `.budget`, `.pagination`) |
| `mongodb-aggregation` | `…outbound.mongo.aggregation` |
| `mongodb-transaction` | `…outbound.mongo.transaction` (+ `.retry`, `.session`) |
| `mongodb-index-schema` | `…outbound.mongo.schema` (+ `.index`, `.manifest`, `.model`, `.ttl`, `.validation`) |
| `mongodb-change-stream` | `…outbound.mongo.changestream` (+ `.projector`, `.recovery`) |
| `mongodb-geospatial` | `…outbound.mongo.geo` |
| `mongodb-migration-core` | `…outbound.mongo.migration` |
| `mongodb-migration-flamingock` | `…outbound.mongo.migration.flamingock` |
| `mongodb-observability` | `…outbound.mongo.observation` |
| `mongodb-security` | `…outbound.mongo.security` (+ `.admin`), `…outbound.mongo.nativecap` |
| `mongodb-spring-boot-starter` | `…outbound.mongo.autoconfigure` |
| `mongodb-testkit-core` | `…outbound.mongo.testkit.mapping`, `.compat`, `.performance` (`testkit` source set) |
| `mongodb-testkit-replicaset` | `…outbound.mongo.testkit.rs` (`testkit` source set) |
| `mongodb-testkit-failover` | `…outbound.mongo.testkit.failover` (`testkit` source set) |
| `mongodb-testkit-migration` | `…outbound.mongo.testkit.migration` (`testkit` source set) |
`…outbound.mongo.architecture` has no design counterpart: it holds the `@MongoOperation` marker and
the reusable ArchUnit rule set a fork applies to its own document/repository code.
### 2.2 Advanced modules
| Design module | Repository package |
|---|---|
| `mongodb-sharding` | `…outbound.mongo.advanced.sharding` (+ `.admin` for the D4 shard plane) |
| `mongodb-timeseries` | `…outbound.mongo.advanced.timeseries` |
| `mongodb-csfle` | `…outbound.mongo.advanced.encryption.csfle` |
| `mongodb-queryable-encryption` | `…outbound.mongo.advanced.encryption.qe` |
| `mongodb-search` | `…outbound.mongo.advanced.search` |
| `mongodb-vector-search` | `…outbound.mongo.advanced.vector` |
| `mongodb-tenancy-shared` | `…outbound.mongo.advanced.tenancy.shared` |
| `mongodb-tenancy-database` | `…outbound.mongo.advanced.tenancy.database` |
| `mongodb-change-stream-messaging-bridge` | `…outbound.mongo.advanced.bridge` |
| `mongodb-gridfs-compat` | `…outbound.mongo.advanced.gridfs` |
| `mongodb-testkit-sharded` | `…outbound.mongo.testkit.sharded` (`testkit` source set) |
| `mongodb-testkit-atlas` | `…outbound.mongo.testkit.atlas` (`testkit` source set) |
The design's rule that a Stable module never depends on an Advanced one survives as an ArchUnit rule
(`stableNeverDependsOnAdvanced`) plus the opt-in flag: every Advanced entry point requires
`MongoAdvancedCapabilityFlags` to have the matching capability enabled and refuses construction
otherwise. Being on the classpath is not being enabled.
## 3. Other deliberate substitutions
| Design assumption | Repository reality | Adaptation |
|---|---|---|
| Gradle Kotlin DSL under `modules/mongodb*` | Groovy DSL, root `build.gradle` conventions, `LockMode.STRICT` locking | Dependencies declared in `src/adapter/outbound/persistence-mongo/build.gradle`; `gradle.lockfile` regenerated. |
| `mongodb-spring-boot-starter` is a separate module the app depends on | `modules.json` gives `adapter-outbound-persistence-mongo` `runtime_memberships: []` and does **not** list it among `app-bootstrap`'s allowed dependencies | The `autoconfigure` package stays inside the leaf and registers through the leaf's own `META-INF/spring/…AutoConfiguration.imports`. This differs from the httpclient precedent, where the starter moved to `:app-bootstrap`; here the registry forbids that edge. |
| Spring Boot 4.1 / Spring Data MongoDB 5.1 baseline | Repository baseline is Spring Boot 4.0.0 / Spring Data MongoDB 5.0.0 | The platform targets the Spring Data MongoDB **API surface** common to both; no 5.1-only type is referenced. The support matrix records the actual pinned versions. |
| `MongoRetryScope` lives in `mongodb-transaction` | The `mongodb-spring-data` failure translator must classify retry scope, and it cannot depend on `mongodb-transaction` | `MongoRetryScope` lives in `…api.error` (core-api), which both packages already depend on. Same values, same meaning, one legal position in the DAG. |
| `mongodb-migration-flamingock` depends on Flamingock | Adding an unvetted external dependency is out of scope for this task, and the design itself requires the public contract not to depend on Flamingock types | The adapter is provider-neutral: it consumes a platform-owned `FlamingockChangeUnitView`. Wiring an actual Flamingock distribution is a one-file change behind that view. |
| Testkit as its own Gradle module | The design forbids production modules depending on the testkit | A dedicated `testkit` source set whose output is on the test compile/runtime classpaths only. ArchUnit rule `productionNeverDependsOnTestkit` enforces the direction. |
| Per-task `git commit` | `AGENTS.md`: commit policy is `human-only` | Implementation is delivered unstaged; commits are the human's action. This is the only plan step intentionally not executed, and it is recorded here. |
| `docs/mongodb/**`, `scripts/verify-mongodb-*.sh` | Repository already owns `docs/` and `scripts/` | Created at the same repository-relative paths. |
## 4. What is unchanged from the design
- D1 / D2 / D3 / D4 exposure planes and the ordered D3 admission sequence (§5).
- Stable API V1 with `apiStrict=true` on the D1/D2 client generation; D3/D4 on separate generations.
- `MongoExecutionOutcome`, including both ambiguous outcomes (`WRITE_RESULT_UNKNOWN`,
`TRANSACTION_COMMIT_UNKNOWN`), and `MongoFailureContext`'s permitted-field list.
- The complete stable exception hierarchy and the label-before-code classification order.
- The BSON representation manifest (UUID `STANDARD`, `Decimal128`, UTC instants, alias type metadata)
and the document-size budget.
- Update-operator-first writes, and optimistic revision as the precondition for whole-document
replacement.
- Transaction body retry and commit retry as separate loops: a new session per body attempt, and
commit-only retry on unknown commit. The body is never replayed after a commit ambiguity.
- Registered operation names and manifests for query, aggregation and index; no free-form JSON query
and no unbounded pipeline.
- Keyset pagination with an authenticated cursor and a unique tie-breaker requirement.
- Change stream as an at-least-once projector with resume-token checkpointing and explicit
history-lost handling.
- TTL as physical cleanup only, never the sole basis for access denial or business scheduling.
- Manifest-owned index/validator state with an apply policy that never drops what it does not own.
- Low-cardinality observation tags, command redaction, and the credential reference indirection.
- The Stable release gate's evidence categories, and the Advanced promotion gate's requirement for
actual-topology evidence.
## 5. Verification
```bash
bash scripts/verify-mongodb-platform.sh # Stable gate
bash scripts/verify-mongodb-advanced.sh # Advanced gate (opt-in lanes)
```
Both scripts run from the repository root and delegate to `src/gradlew`.
+98
View File
@@ -0,0 +1,98 @@
---
title: Runbook — MongoDB primary failover
category: mongodb
severity: P2
owner: oncall
last_updated: 2026-08-13
status: active
---
# Runbook: MongoDB primary failover
Design §29, scenarios `PRIMARY_KILL`, `NETWORK_PARTITION`, `SERVER_SELECTION_TIMEOUT`,
`WRITE_RESPONSE_LOSS`.
## Symptoms
- `MongoServerSelectionException` / `MongoConnectionException` spike, then recovery within seconds.
- `MongoSdamObservationListener` reports a topology change (primary removed, new primary elected).
- `MongoPoolObservationListener` shows checkout wait times rising while server-side command duration
stays flat — the wait is topology, not query cost.
- Latency spike on writes with no corresponding rise in read latency.
A failover that resolves in under ~15 s and produces no `WRITE_RESULT_UNKNOWN` is normal replica-set
behaviour and needs no action beyond confirming it self-healed.
## Diagnosis
1. Confirm an election actually happened. SDAM events distinguish an election from "the database got
slow"; without them the two are indistinguishable in application metrics.
2. Split the failure categories. Metric tag `failureCategory`:
- `SERVER_SELECTION` / `CONNECTION` → the driver could not reach a primary. `NOT_SENT`; safe.
- `TIMEOUT` with outcome `WRITE_RESULT_UNKNOWN` → a write may have applied. Not safe; see below.
- `TRANSACTION_COMMIT_UNKNOWN` → go to [unknown-commit.md](unknown-commit.md) instead.
3. Check the election duration against `MongoRetryBudget`. If the election outlasted the budget, the
retries were exhausted before a primary existed and callers saw errors that a longer budget would
have absorbed.
4. Check whether the new primary is in the expected region/AZ. A failover to a distant node changes
write latency permanently, not transiently.
## Action
**Self-healed (the common case).**
Confirm outcome distribution contains no `WRITE_RESULT_UNKNOWN`, record the election in the incident
log, and close. Nothing to replay.
**Writes with `WRITE_RESULT_UNKNOWN`.**
These writes may or may not have applied. Do not blind-retry.
- Idempotent operation (registered `MongoUpdateOperator` with an `AtomicFilter` precondition): retry.
The precondition makes the second application a no-op.
- Non-idempotent operation: reconcile by reading the target document and comparing against the
intended post-state. Retry only if it does not reflect the write.
**Server selection never recovers.**
The set has lost quorum — two of three nodes are down or partitioned. No client-side action fixes
this; escalate to the database owner to restore a majority. The application should be failing closed,
not queueing.
**Elections are frequent (more than one a day, unprompted).**
This is an infrastructure symptom, not an application one: check node resource saturation, disk
latency on the primary, and network stability between members. Repeated elections cause repeated
unknown-outcome windows.
## Escalation
- P2 → P1 if server selection has failed for more than 2 minutes, or if any non-idempotent write
returned `WRITE_RESULT_UNKNOWN` and cannot be reconciled.
- Page the database owner for quorum loss, and the service owner for reconciliation of ambiguous
writes.
## Verification
The failover lane reproduces this deliberately:
```bash
cd src
./gradlew :adapter:outbound:persistence-mongo:mongoFailoverTest --console=plain
```
It starts a real three-node set (`MongoThreeNodeReplicaSet`), stops the primary
(`MongoPrimaryController`), and injects network faults through Toxiproxy
(`ToxiproxyMongoNetworkFaultController`). A single-node set is not sufficient for the election: it
never holds one, so every guarantee that depends on a primary change goes untested.
The network faults need their own fixture (`MongoProxiedReplicaSetNode`) because a stopped container
cannot produce them. Stopping a node tells the client the write did not happen; cutting the *path*
while the server keeps running produces a client that cannot tell. `MongoNetworkFaultLaneTest`
asserts the difference by reaching the same server twice — once through the proxy, once directly:
- **Partition**: the proxied client fails, the direct client finds the server healthy and the earlier
write intact. The path was cut, not the server.
- **Response loss**: the proxied client fails, and the direct client then finds the document
*present*. The write applied and only the acknowledgement was lost —
`DefaultMongoFailureClassifier` returns `WRITE_RESULT_UNKNOWN`, and a retry would have inserted a
second document.
One detail the lane depends on: the connection is warmed before the toxic is applied. On a cold
connection it is the driver's handshake whose response is dropped, so the write is never transmitted
`NOT_SENT`, the opposite of the ambiguity being tested.
+90
View File
@@ -0,0 +1,90 @@
---
title: Runbook — MongoDB change stream history lost
category: mongodb
severity: P1
owner: oncall
last_updated: 2026-08-13
status: active
---
# Runbook: change stream history lost
Design §20.3, scenarios `OPLOG_HISTORY_LOSS`, `RESUME_TOKEN_LOSS`.
The stored resume token predates the oldest entry in the oplog. The events between the checkpoint and
now are gone from the server; no retry recovers them. `MongoChangeStreamRecoveryPolicy` returns
`halt(HISTORY_LOST, "history-lost")` and the runner stops.
**The platform will not silently restart from "now".** That looks like a recovery and is actually a
permanent, unreported gap in the projection.
## Symptoms
- `MongoChangeHistoryLostException`.
- `MongoChangeStreamState.HISTORY_LOST`; the consumer is stopped, not looping.
- Precedes it: consumer lag approaching the oplog window, or a consumer that was down for a long
period (a deploy that failed, a scaled-to-zero worker, a long outage).
## Diagnosis
1. **Determine the gap.** The checkpoint's cluster time is the start; the oldest oplog entry is the
end of what is unrecoverable. Everything in between was never processed.
2. **Determine the oplog window.** `rs.printReplicationInfo()` on the primary gives the first and last
oplog timestamps. If the window is materially smaller than it was, the write rate rose or the
oplog was resized — the consumer may be fine and the server changed.
3. **Determine what the projection is missing.** Which collections and which operations does this
projector consume? The gap is bounded by that, not by everything that happened.
4. **Check for a second consumer.** If another projector on the same collection is healthy, its
checkpoint tells you whether the problem is this consumer or the oplog.
## Action
Resuming is not an option. The choices are:
**Rebuild from source.** If the projection is derivable from the current state of the source
collections, rebuild it: stop the consumer, rebuild the projection, then start the stream from the
cluster time at which the rebuild snapshot was taken. This is the correct answer whenever the
projection is a materialised view rather than an event log, and it is the reason a projection should
be derivable.
**Backfill the gap.** If the source documents carry a timestamp covering the gap, run a bounded
backfill for that window through the migration runner (checkpointed, resumable — see
[schema-index-migration-guide.md](../schema-index-migration-guide.md) §5), then resume from the
current cluster time.
**Accept the gap explicitly.** Only when the projection is advisory and the business owner says so.
Record the window in the incident log and reset the checkpoint. This is a decision someone signs, not
a default.
Never: reset the checkpoint to "now" and restart quietly. That converts a visible P1 into an
invisible data-quality defect that surfaces months later as "the report has been wrong since March".
## Prevention
- **Alert on lag against the oplog window, not wall-clock.** "Consumer is 30 minutes behind" is fine
with a 24-hour oplog and an emergency with a 45-minute one. The threshold that matters is
`lag / oplogWindow`.
- **Size the oplog for the longest tolerable consumer outage**, including a failed deploy discovered
the next morning.
- **Checkpoint after processing, never on receipt** — see
[change-stream-guide.md](../change-stream-guide.md) §3.
- **Back up the checkpoint store.** `RESUME_TOKEN_LOSS` is the same incident reached from the other
direction: the oplog is fine, the checkpoint is gone.
- **Make the projection rebuildable.** A projection that can only be built by replaying every event
has no recovery path once the oplog rolls.
## Escalation
- P1 on detection. The consumer is stopped, so lag grows for as long as this is unresolved.
- Page the service owner for the rebuild decision, and the database owner if the oplog window shrank
unexpectedly.
## Verification
```bash
cd src
./gradlew :adapter:outbound:persistence-mongo:mongoFailoverTest --console=plain
```
`MongoFailoverScenario.OPLOG_HISTORY_LOSS` and `RESUME_TOKEN_LOSS` assert the runner halts and names
this runbook rather than restarting from the current position.
+87
View File
@@ -0,0 +1,87 @@
---
title: Runbook — MongoDB unknown transaction commit result
category: mongodb
severity: P1
owner: oncall
last_updated: 2026-08-13
status: active
---
# Runbook: unknown transaction commit result
Design §16, decision D-10, scenario `UNKNOWN_TRANSACTION_COMMIT_RESULT`.
`MongoExecutionOutcome.TRANSACTION_COMMIT_UNKNOWN` means the commit **may have applied**. It is not a
failure and must never be reported to a caller as one. The single worst response is to re-run the
transaction body: if the commit did apply, the body applies a second time.
## Symptoms
- `MongoTransactionCommitUnknownException` in logs.
- Metric `failureCategory=TRANSACTION_COMMIT_UNKNOWN`.
- Usually accompanies a primary election — see [failover.md](failover.md).
- Downstream reports of duplicated effects (double charge, double increment) are the symptom of this
being handled wrongly, not of the condition itself.
## Diagnosis
1. **Confirm the platform did the right thing automatically.**
`MongoTransactionRetryCoordinator` retries the *commit only*, on the same session, within
`MongoRetryBudget`. A commit retry against an already-committed transaction is a no-op by design.
Most occurrences resolve here and never reach a human.
2. **If the budget was exhausted, determine the actual state.** The commit either applied or it did
not; you must find out which, not guess.
- If the transaction body wrote a deterministic marker (an idempotency key, a business id, a
revision), read it back. That is exactly what `MongoCommitReconciler` does, and it is the
reason the design requires transactions to write one.
- If there is no marker: reconstruct from a downstream artefact — an outbox row, an audit record,
an external side effect. If nothing exists to compare against, the transaction was not
designed to be reconcilable and that is the finding to record.
3. **Check whether the body was replayed.** Grep for a second execution with the same operation name
and correlation id. If the body ran twice, the effects need reversing, and the code path that
replayed it is a defect: an ambiguous commit is `COMMIT_ONLY` scope
(`MongoRetryScope.COMMIT_ONLY`), never `BODY`.
## Action
**Commit applied.** Nothing to do. Record the reconciliation.
**Commit did not apply.** Re-run the whole operation from the top — a new session, a new body
attempt. This is safe precisely because you established the previous attempt left no trace.
**Cannot determine.** Do not retry. Escalate. A blind retry here is a coin flip between "no effect"
and "duplicate effect", and duplicates in a financial or notification path are worse than a delay.
Freeze the affected entity if the domain supports it, and hand off with: operation name, correlation
id, document id, the time window, and what you checked.
**Recurring.** More than one an hour means the commit path is racing something structural — a
`maxCommitTime` shorter than the observed election duration, an oversized transaction, or an
undersized retry budget. Fix the budget or the transaction shape; do not raise the retry count and
call it resolved.
## Prevention
- Every transaction body writes a deterministic marker that identifies its own commit.
- `maxCommitTime` exceeds the observed p99 election duration.
- Callers surface the ambiguity to their own callers rather than mapping it to a generic 500 — an
ambiguous outcome reported as a failure invites the caller to retry, which is the one thing that
must not happen.
- Prefer a single-document atomic operation (D-09). A transaction that exists only to wrap one
document write has invented this failure mode for nothing.
## Escalation
- Always P1 when the state cannot be determined and the operation has an external effect.
- Page the service owner immediately; the database owner only if elections are the trigger.
## Verification
```bash
cd src
./gradlew :adapter:outbound:persistence-mongo:mongoFailoverTest --console=plain
```
`MongoFailoverScenario.UNKNOWN_TRANSACTION_COMMIT_RESULT` runs this path against a real three-node
set, and the coordinator test asserts the body is never replayed after a commit ambiguity.
@@ -0,0 +1,137 @@
# Schema, Index and Migration Guide
Design §21–§25, decision D-13. Indexes and validators are **declared** in a manifest and **applied**
by an explicit plane. Automatic index creation in production is explicitly unsupported (§3.4): an
index build on a large collection is a capacity event, and discovering it because a deployment
started one is not an operating model.
## 1. The manifest is the source of truth
`MongoManifestRegistry` holds one `MongoCollectionManifest` per collection, containing:
- `MongoIndexManifest` — the declared indexes (`MongoIndexKey`, `MongoIndexDirection`, uniqueness,
partial filter, collation)
- `MongoSchemaManifest` — the declared `$jsonSchema` validator
- `MongoMetadataOwnership` — who owns each observed object
Ownership is the field that makes drift handling safe:
| Ownership | Owner | Droppable on drift |
|---|---|---|
| `APPLICATION_MANAGED` | this manifest | yes |
| `SEARCH_MANAGED` | the search service | no |
| `ENCRYPTION_MANAGED` | Queryable Encryption | no |
| `EXTERNAL` | someone else (a DBA, another service) | no |
A diff engine that does not know about ownership eventually proposes dropping
`enxcol_.customers.esc` or a search index, and "the drift tool cleaned it up" is a very bad incident
summary.
## 2. Index diff and apply
`MongoIndexDiffEngine` compares the manifest against `MongoIndexDescriptorView` observations and
produces a `MongoIndexDiff`: missing, extra, and *changed* (same name, different definition —
MongoDB will not silently rebuild these, so they must be reported rather than re-issued).
`MongoIndexApplyPolicy` decides what happens with a diff:
| Policy | Behaviour | Environment |
|---|---|---|
| `APPLY` | create what is missing | local / test |
| `APPLY_WITH_DIFF` | create what is missing and report the rest | staging |
| `DIFF_WITH_APPROVED_APPLY` | apply only what a human approved | production |
| `REPORT_ONLY` | never write | audit |
Dropping is never implicit. `MongoIndexRetirementPlan` moves an index through
`MongoIndexRetirementState` — declared → hidden → observed-unused → droppable — and each transition
is a separate deployment. Hiding an index makes the planner ignore it while keeping it maintained, so
an unexpected regression is one command to undo. Dropping it is not.
## 3. Validators
`MongoValidatorDescriptor` carries the `$jsonSchema`, a `MongoValidationLevel`
(`OFF` / `MODERATE` / `STRICT`) and a `MongoValidationAction`.
**Stable validation actions are `error` and `warn` only.** `errorAndLog` is not part of the Stable
contract on MongoDB 7.0 or 8.0 and the descriptor refuses it.
`MongoValidatorDiffEngine` produces a `MongoValidatorDiff`; `MongoValidatorApplyPolicy` gates the
apply. Tightening a validator on a collection with existing data is the dangerous direction: introduce
it as `warn` + `MODERATE`, confirm the warning count is zero, then promote to `error` + `STRICT` in a
second deployment.
## 4. TTL
D-13: TTL is **physical cleanup**, nothing else.
`MongoTtlIndexDescriptor` declares the field and `expireAfterSeconds`. `MongoTtlPolicyValidator`
enforces what `MongoTtlPolicy` allows, and `MongoExpirationAccessPolicy` states the rule that matters:
> A document's presence is not authorization, and its absence is not a deadline.
The TTL monitor runs about once a minute and deletes in batches, so a document can outlive its
expiry by minutes to hours under load. Consequences:
- Access control must check the expiry field, not the document's existence. A still-present expired
session is a valid document and an invalid session.
- Business scheduling must not be built on TTL. If something must happen at a time, schedule it.
- A TTL field must be a BSON date. A TTL index on a string silently never deletes anything.
## 5. Migrations
`MongoMigrationRunner` executes `MongoMigration` units with:
- `MongoMigrationId` — ordered, unique
- `MongoMigrationChecksum` — content hash; a changed checksum for an applied id is a hard failure, not
a re-run. Editing an applied migration means two environments ran different code under the same id.
- `MongoMigrationLedger` — what has been applied
- `MongoMigrationLock` — one runner at a time; a rolling deploy starts several instances at once
- `MongoMigrationPrecondition` / `MongoMigrationPostcondition` — checked before and after; a migration
that cannot verify its own result is a migration whose failure is discovered by a customer
- `MongoMigrationCheckpoint` — a resumable position for a backfill
`MongoMigrationResult` reports applied / incomplete / dry-run with the reason. `INCOMPLETE` is not a
failure: a rate-limited backfill that ran out of its time budget has done real work and stored a
checkpoint, and reporting it as failed would send the next run back to the beginning.
`MongoCollectionMigrationLedger` and `MongoCollectionMigrationLock` are the MongoDB-backed
implementations. Two details are load-bearing and only exist on a server:
- The ledger's **unique index** on the migration id, created by `ensureIndexes()`. Without it, two
runners that both pass the "not applied yet" read both insert, and the ledger then reports one
migration applied twice with two checksums — indistinguishable from tampering.
- The lease is taken with **one conditional update**, not read-then-write. A filter matching only a
free or expired lease lets the server pick the winner; two runners that each read "free" and then
write would both believe they hold it.
The lease expires so a runner killed mid-migration does not block every future deployment, and
`refresh` between batches is what proves the holder is still alive.
### Backfills restart, they do not restart-from-zero
A long backfill will be interrupted — a deploy, an OOM, a node replacement. The checkpoint records
the last completed key so the restart continues rather than re-processing from the beginning.
`MongoBackfillRestartFixture` in the testkit asserts exactly this: kill mid-run, restart, and the
result is identical to the uninterrupted run and does not re-apply completed work.
### Flamingock
`FlamingockMongoMigrationAdapter` bridges to Flamingock through the platform-owned
`FlamingockChangeUnitView`, with `FlamingockLedgerAdapter` and `FlamingockLockAdapter` mapping the
ledger and lock. The public contract does not reference Flamingock types, so the provider can be
replaced without touching a migration.
## 6. Ordering with deployments
```
1. Add the index (hidden if it is large) -> deployment N
2. Unhide / verify usage -> deployment N+1
3. Ship code that depends on the index -> deployment N+1
4. Backfill data -> migration, resumable
5. Tighten the validator from warn to error -> deployment N+2
6. Retire the old index through the retirement states -> deployments N+3…
```
Each step is independently reversible. A deployment that adds an index and the code that requires it
at the same time has no safe rollback: rolling back the code leaves the index build running, and
rolling back the index breaks the code that is still live on half the fleet.
+147
View File
@@ -0,0 +1,147 @@
# Security and Observability
Design §26–§28, decision D-05. The application plane and the admin plane are different credentials on
different clients, and telemetry never becomes an exfiltration path.
## 1. Roles
`MongoPrincipalRole` — one credential per role, least privilege:
| Role | Grants |
|---|---|
| `APP_READ` | `find` on allowlisted collections |
| `APP_WRITE` | `insert`, `update`, `delete` on allowlisted collections |
| `CHANGE_STREAM` | `changeStream`, `find` |
| `MIGRATION` | index and validator management on the target collections |
| `SEARCH_ADMIN` | search index management |
| `SHARD_ADMIN` | shard key operations |
| `ENCRYPTION_ADMIN` | key vault access |
| `DBA` | the human plane; never used by an application |
`MongoSecurityProfileValidator` checks the profile at startup. `forbiddenPrivilegesHeld()` names the
privileges the profile holds and must not — the validator reports *which* one, because "your
credential is over-privileged" without a name is an unactionable finding.
The privileges that must never appear on an application credential: `dropDatabase`,
`dropCollection`, `shutdown`, `killop`, `root`, `__system`, `dbOwner`, `userAdminAnyDatabase`.
## 2. Credentials are references, not values
`MongoCredentialReference` holds a `secret://…` reference plus the role. The reference is resolved at
connection time by the secret provider; the password is never a property value, a log field, or a
constructor argument that could end up in a stack trace.
`MongoCredentialRotationPolicy` states the rotation contract: overlapping validity, a drain window,
and a rotation that never requires a restart. `MongoClientGenerationRegistry` implements the swap —
a new `MongoClientGeneration` starts serving new operations while the previous generation is
`markDraining()` until its in-flight operations finish. Killing the old client immediately fails every
in-flight request, which is why rotation without generations is an outage.
Rotation is a failover scenario in the release gate (`MongoFailoverScenario.CREDENTIAL_ROTATION`),
not a runbook step people hope works.
## 3. TLS and connection policy
`MongoSecurityProfile.production(...)` requires TLS and refuses `tlsAllowInvalidCertificates` /
`tlsAllowInvalidHostnames`. `MongoSecurityProfile.local(...)` exists so a developer does not have to
weaken the production factory to get a container to connect; the startup validator refuses a local
profile on a production runtime profile.
## 4. Admin plane (D4)
`MongoAdminGateway` is the only path to `MongoAdminOperation`, and it runs on the D4 client with the
DBA-scoped credential — not the application's.
- `MongoAdminAuthorization` checks the caller's role against the operation.
- `MongoAdminRuntimeGuard` refuses high-risk operations (`highRisk()`) unless the runtime profile
explicitly permits them; a `dropCollection` reachable from a running application is a data-loss
vector regardless of how well-reviewed the calling code is.
- `MongoAdminAuditRecord` records who ran what, when and against which collection profile — before
execution, so a failed attempt is recorded too.
The native capability gateway (D3) refuses any admin-category command, so there is no path from the
application plane into the admin plane.
## 5. Observability tags
`MongoObservationConvention` allowlists exactly eight tag names:
```
mongoProfile, databaseProfile, collectionProfile, operationName,
operationType, result, failureCategory, consistencyProfile
```
and explicitly forbids:
```
documentId, rawTenantId, tenantId, dynamicCollectionName, queryParameter,
query, fullBson, resumeToken, shardKeyValue, plaintextPII, credential
```
Two reasons, and both matter. Cardinality: a tag whose values are document ids produces one time
series per document, which is how a metrics backend falls over. Confidentiality: a metric label is
stored, shipped and retained by systems with a different access model than the database.
`requireAllowed(tagName)` throws on anything outside the list, so a new tag is a deliberate change to
the convention rather than a line in a service.
`MicrometerMongoOperationObserver` implements the `MongoOperationObserver` port;
`NoOpMongoOperationObserver` is the default so observation is opt-in and never a hard dependency.
## 6. Driver-native listeners
`MongoDriverObservabilityConfiguration` registers three driver listeners, because they answer
questions the application-level timer cannot:
| Listener | Answers |
|---|---|
| `MongoCommandObservationListener` | How long did the *server* take, versus how long the caller waited? |
| `MongoPoolObservationListener` | Was the wait time connection checkout rather than query execution? |
| `MongoSdamObservationListener` | Did the topology change — an election, a node removed — during the window? |
Without pool and SDAM events, every failover looks like "the database got slow", and the difference
between "we need a bigger pool" and "we lost a primary" is invisible.
## 7. Command redaction
`MongoObservationRedactor.describe(commandName)`:
- Authentication and user-management commands (`authenticate`, `saslStart`, `saslContinue`,
`getnonce`, `createUser`, `updateUser`, `copydb*`) render as `<redacted>` — their arguments carry
credentials and key material.
- Structural commands (`ping`, `hello`, `buildInfo`, `listCollections`, `listIndexes`, `collStats`)
render by name; their arguments are not data-bearing.
- Everything else renders as `name(...)`: you get the command, never the filter or the document.
`isAlwaysRedacted(...)` is the assertion hook so a test can prove no logging path can render an auth
command's arguments.
## 8. Startup validation
`MongoStartupValidator` runs at context refresh, before the first request:
1. `MongoTopologyProbe` reports the actual `MongoTopology`.
2. Each declared `MongoTopologyRequirement` is checked against it — a transaction, causal-session or
change-stream requirement fails closed on `STANDALONE`.
3. `MongoSecurityProfileValidator` checks credentials and TLS.
4. `MongoCapabilitySupport` checks declared capabilities against the server version, with
`MongoSupportLevel` distinguishing `STABLE` / `ADVANCED` / `EXPERIMENTAL` / `UNSUPPORTED`.
5. `MongoPlatformHealthIndicator` reports the outcome for the readiness probe.
A misconfiguration found at startup costs a failed deploy. The same misconfiguration found at runtime
costs an incident, and the failing operation is rarely the one that reveals the cause.
## 9. How the security lane proves any of this
```bash
cd src
./gradlew :adapter:outbound:persistence-mongo:mongoSecurityIntegrationTest --console=plain
```
The lane runs against `MongoAuthenticatedReplicaSetContainer`, which starts mongod with `--auth` and a
generated keyfile. That detail is the whole lane: Testcontainers' `MongoDBContainer` starts mongod
*without* `--auth`, so users created on it all have every privilege and a least-privilege assertion
passes no matter how wrong the roles are. A security test that cannot fail is not a security test.
What the lane asserts is the refusal: the `read` role's insert is rejected, and the application role's
`dropDatabase` is rejected. Then it checks that `MongoSecurityProfileValidator` names the same
privilege the server just refused.
+91
View File
@@ -0,0 +1,91 @@
# MongoDB Platform — Support Matrix
Design §4. This file records what the platform is *certified* on, not what it happens to run on.
A configuration absent from this table is unsupported until someone runs the gate against it and
adds a row.
## 1. Runtime baseline
| Component | Version | Policy |
|---|---|---|
| Java | 21 | Repository runtime baseline. |
| Spring Boot | 4.0.0 | BOM-managed. Individual driver overrides are forbidden. |
| Spring Data MongoDB | 5.0.0 | Repository and `MongoTemplate` integration. Version comes from the Boot BOM. |
| MongoDB Java Driver | 5.6.1 | BOM-managed. Never pinned directly in the module. |
| Reactor | 3.8.0 | Reactive execution path. |
| Micrometer | 1.16.0 | Driver-native observability. |
| Testcontainers | 2.0.2 | Replica-set, failover, migration and compatibility lanes. |
The design's baseline is Spring Boot 4.1.x / Spring Data MongoDB 5.1.x. This repository is on
4.0.0 / 5.0.0, so the platform targets only the API surface common to both. See
[repository-adaptation.md](repository-adaptation.md) §3.
## 2. Server versions
| Lane | Version | Pinned image | Gradle task |
|---|---|---|---|
| Primary certification | MongoDB 8.0 | `mongo:8.0.16` | `mongoReplicaSetTest`, `mongoFailoverTest` |
| Compatibility | MongoDB 7.0 | `mongo:7.0.28` | `mongoCompatibilityTest` |
| Network fault injection | — | `ghcr.io/shopify/toxiproxy:2.12.0` | `mongoFailoverTest` |
Images are pinned, never `latest`: a mutable tag means the certification result describes whatever
was pulled that morning, not the version in the row. Override with
`-PmongoPrimaryImage=…` / `-PmongoCompatibilityImage=…` when testing a new patch level, and update
the row once the gate passes.
`MongoVersionMatrix.standard()` is the machine-readable form of this table; a version outside it
fails `certifies()`.
## 3. Topologies
| Topology | Status | What is certified | What is not |
|---|---|---|---|
| Standalone | **Smoke only** | Basic CRUD and mapping. | Not a production profile and never counts as Stable release evidence (D-03). Transactions, retryable writes and change streams are refused at startup by `MongoStartupValidator`. |
| Single-node replica set | **Local default** (D-02) | Transactions, retryable writes, change streams — the same semantics as production. | Elections. A single-node set never holds one, so failover behaviour is untested here. |
| 3-node replica set | **Stable production gate** | Everything above plus primary failover, unknown-commit handling and change-stream resume across an election. | Shard routing. |
| Sharded cluster | **Advanced gate** | Shard-key routing classification, scatter-gather refusal, `admin` plane operations. | Not included in the Stable gate. |
| Atlas / provider-managed | **Per-capability gate** | Search, vector search and encryption against the actual target deployment. | Atlas Local in a container is a pull-request convenience, explicitly **not** release evidence (`MongoAtlasCapabilityContractSuite.Environment`). |
## 4. Stable API and client generations
| Plane | Stable API | Purpose |
|---|---|---|
| D1 Standard document persistence | V1, `apiStrict=true` | Repositories, typed queries, atomic updates, optimistic revision. |
| D2 Advanced document operations | V1, `apiStrict=true` | `MongoTemplate`, transactions, bulk, aggregation, keyset cursors, change streams. |
| D3 Explicit Mongo capability | Not strict | Native BSON, time series, search/vector, CSFLE/QE, shard-aware operations — each behind a registered capability. |
| D4 Admin plane | Not strict | Collection, validator, index, migration, shard and repair commands. Separate credential, separate client. |
D3 is not a raw-client escape. Every call passes capability registration → database profile →
collection allowlist → operation name → timeout → consistency profile → result limit → trace →
redaction → command category → D4 refusal, in that order.
## 5. Validation actions
Stable validation actions are `error` and `warn`. `errorAndLog` is **not** part of the Stable
contract on 7.0 or 8.0 and `MongoValidatorDescriptor` refuses it.
## 6. Explicitly unsupported
Per design §3.4, none of the following is provided, and adding one is a design change rather than a
feature request:
- A generic `CommonMongoRepository<T, ID>`.
- Arbitrary runtime `runCommand`.
- Automatic index creation in production.
- A Standalone production contract.
- TTL as an exact business scheduler or as the only access control.
- Publishing raw change events as external business integration events.
- GridFS as the source of truth for new files.
- Java fully-qualified class names as a long-lived BSON schema.
- Unbounded skip pagination, unbounded aggregation pipelines, unbounded regex, unbounded results.
## 7. Capability tiers
| Tier | Capabilities | Enablement |
|---|---|---|
| Stable | Mapping, imperative/reactive execution, atomic update, optimistic lock, transactions, consistency profiles, retry/translation, query and aggregation guardrails, schema/index manifests, keyset pagination, bulk partial results, change streams, TTL contract, GeoJSON, security, observability | On when `ca-skeleton.persistence-mongo.enabled=true`. |
| Advanced | Sharding-aware query, time series, CSFLE, Queryable Encryption (equality/range), change-stream→messaging bridge, shared-collection multi-tenancy | Each behind `ca-skeleton.persistence-mongo.advanced.<capability>.enabled`. |
| Experimental | Search, vector search, hybrid search, database-per-tenant, collection-per-tenant, reshard orchestration, provider-specific features | Same flag mechanism; promotion additionally requires the evidence in [ADR-MONGO-ADV-001](../adr/ADR-MONGO-ADV-001-capability-promotion.md). |
`MongoAdvancedCapabilityFlags.propertyFor(capability)` is the authoritative property name for any
capability; the table above is its prose form.
@@ -0,0 +1,27 @@
# NOTIF-ADR-001 — `submit()` means durable acceptance
## Status
Accepted.
## Context
The obvious API for a notification platform is `send()` returning success or failure. Every channel
this platform supports makes that return value a lie:
- SES accepts a request, returns a `MessageId`, and can still decline to send.
- Twilio separates `accepted`, `sent` and `delivered` into distinct, later events.
- APNs accepts a notification and may then deliver, store or discard it.
- Web Push separates push-service acceptance from user-agent acknowledgement at the protocol level.
## Decision
`submit()` and `schedule()` return once the logical request and its recipient jobs are committed to
the database. The receipt carries `notificationId`, `RequestStatus` and `acceptedAt`, and has no
`delivered`, `sent` or `read` component. No provider is contacted while the transaction is open.
## Consequences
Callers cannot mistake acceptance for delivery, because the type does not offer that reading.
Delivery state is a separate query against the projection built from the provider event ledger. The
cost is that "did it arrive?" is a second question — which is the honest number of questions.
@@ -0,0 +1,25 @@
# NOTIF-ADR-002 — append-only event ledger with channel projectors
## Status
Accepted.
## Context
A single linear delivery status has to be updated in place, which forces a rule for deciding whether
a new event outranks the stored one. The natural rule — compare ordinals — is wrong for real provider
traffic. Twilio does not guarantee callback ordering, so `sent` arrives after `delivered`. Email
generates complaints after deliveries. Both cases lose information under an ordinal rule.
## Decision
Provider events are appended to an immutable ledger before any projection runs. Channel-specific
projectors merge events into `SubmissionOutcome`, `DeliveryOutcome`, `EvidenceLevel`,
`EngagementFacts` and `SuppressionFacts` using explicit transition tables. Projection is idempotent
and can be replayed from the ledger.
## Consequences
Duplicate, out-of-order and late events are normal inputs rather than defects. A projector bug is
recoverable, because the events it mis-projected are still stored. Projector versions can be migrated
by replay. The cost is a second write per event and a projection that can lag its ledger.
@@ -0,0 +1,37 @@
# NOTIF-ADR-003 — ambiguous submission is a first-class state
## Status
Accepted.
## Context
The most common serious failure is not a rejection. It is a request whose body reached the provider
and whose response never came back. The platform has no provider request id, and the user may or may
not have received the notification.
Treating that as a failure produces duplicates: a retry sends a second message, and a cross-channel
fallback sends the SMS next to the push that already arrived. Treating it as a success loses real
failures.
## Decision
`AMBIGUOUS` is a stored `SubmissionOutcome` and `AttemptConfirmation`. Attempts record
`requestStarted`, `requestBodyCommitted` and `providerResponseReceived`, each with an
`EvidenceCertainty` of `PROVEN`, `INFERRED` or `UNKNOWN`, so an adapter that does not know is not
forced to answer `false`.
While an ambiguous attempt exists on a recipient delivery:
- automatic retry is blocked unless the provider proves per-request idempotency
- automatic cross-channel fallback is blocked unconditionally
- reconciliation runs where the provider supports a status query
- otherwise the delivery stops and waits for an operator
Operator redrive of an ambiguous attempt requires explicit duplicate-risk approval.
## Consequences
Some notifications stop in a state that needs a human or a reconciliation pass. That is the intended
trade: an unresolved unknown is cheaper than a guaranteed duplicate, and the state is visible rather
than silently resolved in either direction.
@@ -0,0 +1,27 @@
# NOTIF-ADR-004 — FCM installation id is the primary target
## Status
Accepted.
## Context
Firebase now recommends the installation id (FID) and treats registration-token multicast paths as
legacy. A contact point model built on a single `token` string would encode the older model as the
only one, and a later migration would be a runtime interpretation problem: the same string field
would mean different things for different rows.
## Decision
`MobilePushTarget` is a sealed hierarchy of `FcmInstallationId`, `LegacyFcmRegistrationToken` and
`ApnsDeviceToken`. The kinds are separate types, never a discriminator on one string field, and each
carries its own `ContactPointType` so the uniqueness scope and the encryption associated data differ.
APNs tokens additionally carry their environment, because sandbox and production are separate
namespaces rather than a flag.
## Consequences
Migrating a target kind is a compile-time change with an exhaustive `switch`, not a runtime guess.
The adapter maps each kind to its own wire representation, so a provider changing one path cannot
silently change the other. The cost is one more type than a string field would need.
@@ -0,0 +1,52 @@
# Callbacks and reconciliation
## Ingestion order
```text
body size limit
→ content type
→ profile lookup
→ signature verification
→ append to the ledger
→ duplicate detection
→ normalization
→ attempt resolution
→ projection
→ side effects
→ 2xx
```
Appending before projecting is what makes a fast 2xx honest. The provider is told the event is
recorded, and a projector defect becomes a replay problem rather than a lost event.
A rejected signature is recorded in the security audit, never in the provider event ledger. Writing
it to the ledger would let anyone who can reach the endpoint fill a delivery history with noise.
## Duplicates and ordering
Duplicate suppression uses `(providerProfileId, providerEventId)` where the provider supplies an
event id, and a deterministic fingerprint over profile, request id, event type, occurrence time and
payload digest where it does not. A duplicate is acknowledged and projected exactly once.
Out-of-order callbacks are normal. Ordering is resolved by event semantics, not by arrival time.
## Unknown fields
Callback parsers tolerate unknown JSON fields. Normalization only rejects a payload when a field
required to identify the attempt is missing. Providers add fields; that must not stop ingestion.
## Reconciliation
Reconciliation targets:
- attempts stuck in `DISPATCHING` past their lease
- ambiguous submissions
- accepted attempts whose callback SLA has expired
- unmatched provider events
A confirmed query result is appended to the same ledger with `source = RECONCILIATION` and projected
by the same projector, so projection replay stays possible: there is no privileged second path that
writes projections directly.
Where a provider has no status-query capability, the platform records `Unsupported` and leaves the
attempt ambiguous. It does not infer a final status.
@@ -0,0 +1,41 @@
# Configuration reference
## Dispatch
| Property | Meaning | Bound |
|---|---|---|
| `claim-batch-size` | Rows claimed per scheduler tick | 1..1000 |
| `lease-duration` | How long a claimed job stays owned | positive, finite |
| `max-global-concurrency` | Ceiling across all providers | positive |
| `max-queue-age` | Age at which a job is escalated | positive |
| `max-retry-concurrency` | Ceiling for retry work | positive |
| `scheduler-poll-interval` | Queue poll cadence | positive |
| `callback-worker-concurrency` | Callback projection workers | positive |
Every value is bounded. "Unlimited" is not an accepted configuration.
## Provider profiles
A profile pins provider type, environment, credential profile, timeouts, concurrency, rate limit,
retry policy and callback profile. Sender identity and credential profile are separate concerns.
## Startup failures
Startup fails rather than degrading when:
- a payload or queue setting is unbounded
- a timeout is negative
- a TTL-required profile has no expiry source
- a callback signing secret is missing
- a production profile enables trust-all
- an APNs profile is missing its environment or topic
- a Web Push profile is missing its VAPID key
- two provider profiles share an id
- a route points only at disabled providers
- ambiguous fallback is enabled by default
## Secrets
All key material arrives through `SecretMaterialProvider`. Nothing is read from source, from a
committed file, or from a plaintext log. Contact point encryption and lookup HMAC keys must be
distinct, and the encryption key must be exactly 256 bits.
+62
View File
@@ -0,0 +1,62 @@
# Delivery evidence model
## The shape
```text
NotificationRequest
└─ RecipientDelivery
└─ DeliveryAttempt
└─ ProviderEvent (append-only)
└─ channel projector
└─ SubmissionOutcome / DeliveryOutcome / EvidenceLevel
+ EngagementFacts + SuppressionFacts
```
Four identities, four lifecycles. A logical request is not a recipient job, a recipient job is not a
provider attempt, and a provider attempt is not the event stream that describes it.
## Why not one status enum
A single linear status would have to answer "what happened?" with one value, and the real answers do
not fit on one line:
- An email can be `DELIVERED` and then generate a complaint. Both facts are true and both matter:
one for reporting, the other for suppression.
- Twilio does not guarantee callback ordering, so `sent` routinely arrives after `delivered`. Under
an ordinal rule the later, weaker event silently overwrites the stronger one.
- APNs may accept a notification and then store, replace or discard it.
So the ledger stores events and a channel projector merges them through an explicit transition table.
`StandardDeliveryProjector` holds the shared rules; provider projectors add only their own event
vocabulary.
## Merge rules
| Transition | Result |
|---|---|
| `sent``delivered` | applied |
| `delivered``sent` | ignored, event still stored |
| `delivered``complaint` | complaint fact added, delivery preserved |
| `complaint``delivered` | delivery applied, complaint preserved |
| `accepted``bounced` | applied |
| `read``displayed` | ignored |
| hard bounce → `delivered` | ignored, hard bounce is terminal |
Engagement (`opened`, `clicked`) is stored beside the delivery outcome and never changes it.
## Ambiguity
```text
platform ──── send ────▶ provider
└── accepted
✗ connection reset
```
The platform may hold no provider request id while the notification really was sent. The attempt
records `requestStarted`, `requestBodyCommitted`, `providerResponseReceived` and an
`EvidenceCertainty` for each, so a later decision can tell "we know nothing was sent" apart from "we
could not read the answer".
`ProviderSubmissionResult` enforces this: an ambiguous result may not claim `PROVIDER_ACCEPTED`, and
no submission result of any kind may carry a delivery outcome.
+31
View File
@@ -0,0 +1,31 @@
# Migration guide
## From the R0 routing seam
The pre-existing `dev.caskeleton.adapter.outbound.notification` router (`RoutingNotifier`,
`FailOpenNotificationProvider`, the Google email and Slack webhook seams) stays untouched. The
delivery platform lives beside it under `…notification.platform` and does not modify or delete any
R0 class.
Migration order per capability:
1. Register the contact points behind `ContactPointStorePort` so the platform owns protected values.
2. Publish the template version, and pin the template id, version and locale at every call site.
3. Move the call site from the router to the N1 typed facade for the channel.
4. Verify evidence in the snapshot rather than in the caller's return value: `submit()` is durable
acceptance and nothing more.
5. Remove the R0 route only after the platform route has produced provider evidence in the target
environment.
## Return-value semantics change
The R0 seam returned a send-shaped result. `NotificationReceipt` returns `notificationId`, a request
status and an acceptance time. Callers that treated the old return value as proof of delivery must be
changed; there is no compatibility shim, because a shim would have to invent the delivery claim this
platform exists to avoid.
## FCM target migration
Registration tokens keep working through `LegacyFcmRegistrationToken`. New registrations should use
`FcmInstallationId`. The two are distinct types, so a migration is a compile-time task rather than a
runtime guess.
+94
View File
@@ -0,0 +1,94 @@
# Notification Delivery Platform — module mapping
> Source design: `notification-superpowers-package/docs/superpowers/specs/2026-08-10-notification-platform-design.md`
>
> Source plan: `notification-superpowers-package/docs/superpowers/plans/2026-08-10-notification-platform-implementation-plan.md`
## Why a mapping exists
The plan was written against a hypothetical repository (`modules/notification/**`, root package
`io.backend.skeleton.notification`, 31 Gradle projects). This repository is a fail-closed
19-leaf Clean Architecture template: `src/settings.gradle` rejects any registry that does not
contain exactly the 19 modules in `src/config/architecture/modules.json`, and
`verifyCleanArchitectureDependencies` rejects any project edge outside `allowed_dependencies`.
Creating 31 new Gradle projects would violate HARD-STOP #5 of `AGENTS.md`. The package README
anticipates this and instructs the implementer to map dependency catalog and package/file paths onto
the host repository's rules while preserving the public contracts and reliability semantics.
Every logical module of the plan is therefore implemented as a **package** inside the registered leaf
that owns its responsibility. No public contract, evidence rule, or reliability semantic is dropped.
## Logical module → registered leaf
| Plan module | Registered leaf | Package |
|---|---|---|
| `notification-core-api` | `application-core` | `dev.caskeleton.application.notification.platform.api` |
| `notification-content-api` | `application-core` | `…platform.api.content` |
| `notification-contact-api` | `application-core` | `…platform.contact` |
| `notification-template-api` | `application-core` | `…platform.template` |
| `notification-policy` | `application-core` | `…platform.policy` |
| `notification-provider-spi` | `application-core` | `…platform.provider` |
| `notification-callback-api` | `application-core` | `…platform.callback` |
| `notification-email-api` | `application-core` | `…platform.email` |
| `notification-sms-api` | `application-core` | `…platform.sms` |
| `notification-push-api` | `application-core` | `…platform.push` |
| `notification-webpush` (API half) | `application-core` | `…platform.webpush` |
| `notification-inbox-api` | `application-core` | `…platform.inbox` |
| `notification-admin-api` | `application-core` | `…platform.admin` |
| `notification-security` (ports + redaction) | `application-core` | `…platform.security` |
| `notification-observability` (ports) | `application-core` | `…platform.observation` |
| `notification-dispatch-runtime` | `adapter:outbound:notification` | `dev.caskeleton.adapter.outbound.notification.platform.dispatch` |
| `notification-security` (AES-GCM/HMAC impl) | `adapter:outbound:notification` | `…platform.security` |
| `notification-template-thymeleaf` (reference renderer) | `adapter:outbound:notification` | `…platform.template` |
| `notification-email-smtp` | `adapter:outbound:notification` | `…platform.provider.smtp` |
| `notification-email-ses` | `adapter:outbound:notification` | `…platform.provider.ses` |
| `notification-sms-twilio` | `adapter:outbound:notification` | `…platform.provider.twilio` |
| `notification-push-fcm` | `adapter:outbound:notification` | `…platform.provider.fcm` |
| `notification-push-apns` | `adapter:outbound:notification` | `…platform.provider.apns` |
| `notification-webpush` (transport + crypto) | `adapter:outbound:notification` | `…platform.provider.webpush` |
| `notification-webhook-extension` | `adapter:outbound:notification` | `…platform.provider.webhook` |
| `notification-observability` (Micrometer impl) | `adapter:outbound:notification` | `…platform.observation` |
| `notification-admin-runtime` | `adapter:outbound:notification` | `…platform.admin` |
| `notification-reactor` | `adapter:outbound:notification` | `…platform.reactor` |
| `notification-spring-boot-starter` | `adapter:outbound:notification` (+ `app-bootstrap` wiring) | `…platform.autoconfigure` |
| `notification-persistence-jpa` | `adapter:outbound:persistence-jpa` | `dev.caskeleton.adapter.outbound.persistence.notification.platform` |
| `notification-inbox-jpa` | `adapter:outbound:persistence-jpa` | `…persistence.notification.platform.inbox` |
| `notification-callback-mvc` | `adapter:inbound:web` | `dev.caskeleton.adapter.inbound.web.notification.platform.callback` |
| `notification-callback-webflux` | `adapter:inbound:web` | `…callback.reactive` |
| `notification-testkit` | test source sets of the owning leaves | `…platform.testkit` |
## Dependency-direction consequences
The plan's module DAG (`*-api``provider-spi`/`policy` → runtime/adapters → starter) is preserved
by the leaf DAG that the registry already enforces:
```text
application-core (all *-api, provider SPI, policy, callback contracts)
↑ ↑ ↑
adapter:outbound:notification adapter:outbound:persistence-jpa adapter:inbound:web
↑ ↑ ↑
app-bootstrap
```
Two plan edges cannot be expressed as project edges in this repository, and are replaced by ports:
1. `notification-email-ses`, `notification-sms-twilio`, `notification-push-fcm`,
`notification-push-apns`, `notification-webpush`, `notification-webhook-extension`
`httpclient platform`.
`adapter-outbound-notification` is not allowed to depend on `adapter-outbound-httpclient`.
The provider adapters therefore call
`dev.caskeleton.adapter.outbound.notification.platform.provider.http.NotificationHttpGateway`,
an adapter-local port with a JDK `java.net.http.HttpClient` default implementation.
`app-bootstrap` sees both leaves and is the supported place to substitute an implementation backed
by the HTTP Client Platform (TLS/timeout/circuit-breaker/SSRF/dynamic-target policy reuse).
2. `notification-inbox-jpa``optional messaging outbox integration`.
`adapter-outbound-persistence-jpa` may not depend on `adapter-outbound-messaging`; the inbox
publishes through the existing persistence outbox tables plus the
`NotificationInboxSignalPort` application port, and `app-bootstrap` binds the relay.
## Commit policy
`AGENTS.md` pins commit policy to `human-only`. Step 5 (`git add` / `git commit`) of every plan task
is therefore intentionally **not** executed by the agent; the working tree carries the change and the
human owner commits.
+47
View File
@@ -0,0 +1,47 @@
# Operations
## Runtime shape
```text
durable queue (PostgreSQL, FOR UPDATE SKIP LOCKED)
→ expiry check
→ suppression and eligibility re-check
→ provider health gate
→ rate limiter
→ concurrency limiter
→ provider adapter
```
Provider calls run outside every database transaction. The attempt row is committed first, so after a
crash the row is either absent (nothing was sent) or present in `DISPATCHING` (reconciliation has
something to ask about).
## Guards that exist for specific incidents
| Guard | The incident it prevents |
|---|---|
| Credential failure opens the provider route | One expired key multiplied by a queue becomes a self-inflicted outage |
| Retry budget per provider profile | A provider outage turning every queued notification into its own retry loop |
| Ambiguous attempts block automatic fallback | A push whose response was lost arriving alongside the "just in case" SMS |
| Permits released during backoff | A slow provider pinning the whole concurrency budget on work that is only waiting |
| Bounded drain on rotation | A provider that never answers holding a credential rotation open forever |
| Fail-fast intake on capacity | An unbounded in-memory queue absorbing a burst it cannot survive |
## Scheduling
`scheduleAt` activates the job, `notBefore` is the earliest permitted provider submission, and
`expiresAt` blocks new attempts, retries and fallbacks. Suppression and expiry are re-checked
immediately before dispatch, because a scheduled notification can sit in the queue for hours and the
user may have opted out in the meantime.
## Redrive
A redrive preserves `NotificationId` and `RecipientDeliveryId`, creates a new `DeliveryAttemptId`, and
reuses the pinned template version and rendered digest. Sending different content is a new
notification, not a redrive. Redriving an ambiguous attempt requires explicit duplicate-risk approval,
because the platform genuinely cannot tell whether the first submission reached the user.
## Actuator surface
Provider runtime states and generations, queue depth and age, callback and reconciliation health.
Never addresses, never credentials.
+65
View File
@@ -0,0 +1,65 @@
# Provider runbooks
## SMTP
| Symptom | Classification | Action |
|---|---|---|
| Final `2xx` after `DATA` | `CONFIRMED_ACCEPTED` / `PROVIDER_ACCEPTED` | None; this is acceptance, not inbox delivery |
| `4yz` | `TRANSIENT_PROVIDER` | Retry under budget and deadline |
| `5yz` | `PERMANENT_PROVIDER` or `INVALID_RECIPIENT` | Stop, or invalidate the contact point |
| Connection lost after `DATA` | `AMBIGUOUS_SUBMISSION` | Reconcile or escalate; do not resend automatically |
Connection, read, write and pool-acquire timeouts are all finite. There is no unbounded timeout.
## Amazon SES
`MessageId` is acceptance evidence. SES itself documents that it can accept a request and then not
send, so `MessageId` is never mapped to `DELIVERED`.
| Event | Normalized |
|---|---|
| `Send` | reinforces `PROVIDER_ACCEPTED` |
| `Delivery` | `DELIVERY_CONFIRMED` / `NETWORK_OR_CARRIER_ACCEPTED` |
| `DeliveryDelay` | delay fact |
| `Bounce` (permanent) | `BOUNCED_HARD` plus hard-bounce suppression |
| `Bounce` (transient) | `BOUNCED_SOFT`; retry policy input, not a suppression reason |
| `Complaint` | complaint fact plus suppression |
| `Reject` | `PROVIDER_REJECTED` |
| `RenderingFailure` | `TEMPLATE_FAILURE` |
## Twilio
`accepted`/`queued` is acceptance only. `sent` is carrier acceptance. `delivered` is device delivery.
Callbacks are not ordered. A `sent` arriving after `delivered` is stored and ignored by the
projection. Missing callbacks are corrected by status polling under the provider rate limit.
Signature verification uses the canonical external URL from the profile, not the URL the servlet
container reconstructed behind a proxy.
## FCM
| Error | Classification |
|---|---|
| `UNREGISTERED` | `INVALID_RECIPIENT`; invalidate the contact point, never retry |
| `INVALID_ARGUMENT` | `INVALID_PAYLOAD` |
| `QUOTA_EXCEEDED` | `THROTTLED`, exponential backoff |
| `UNAVAILABLE` | `TRANSIENT_PROVIDER`, honour `Retry-After`, add jitter |
| Credential failure | `AUTHENTICATION`; opens the provider route |
A batch is one transport call and many attempts. Partial results map back by input index; one
transport failure does not become one shared outcome unless the adapter can prove it.
## APNs
2xx is acceptance. Environment and topic mismatches are configuration failures, not delivery
failures. Sandbox and production tokens are separate namespaces.
## Web Push
`TTL` is mandatory by protocol. `201` is acceptance. `404` is an expired subscription per RFC 8030;
provider-documented `410` maps the same way. Payloads use `aes128gcm` per RFC 8291 and VAPID JWTs are
signed per RFC 8292 with the audience taken from the endpoint origin.
VAPID key rotation is not ordinary credential rotation: a restricted subscription may need to be
re-created, so it is a migration operation.
+56
View File
@@ -0,0 +1,56 @@
# Security and privacy
## Protected values
Email addresses, phone numbers, FCM installation ids and legacy tokens, APNs device tokens, Web Push
endpoints and keys, VAPID private keys, provider credentials, callback signing secrets, template
variables, rendered bodies, attachment references and unsubscribe tokens.
## At rest
Contact points are encrypted with AES-256-GCM. Equality lookup uses a separate HMAC-SHA-256
fingerprint.
Two keys, not one, because the requirements are opposite: the ciphertext must be non-deterministic so
two records of the same address are not visibly identical, while equality lookup must be
deterministic. The fingerprint is keyed rather than a plain digest because phone numbers and email
addresses come from a small, enumerable space — an unkeyed hash of a phone number is recoverable in
seconds.
The contact point kind is bound into the GCM associated data, so a ciphertext cannot be moved between
contact kinds without failing the authentication tag.
An unknown key id is refused rather than silently falling back to the current key: a silent fallback
would turn every historical row into a tag failure at read time.
## Never logged, never a metric tag
Addresses, tokens, Web Push endpoints and keys, message bodies, template variables, provider
credentials, unsubscribe tokens, attachment URLs, raw callback payloads and raw provider request ids.
Two mechanisms enforce this rather than convention:
- `CardinalityGuard` validates every metric tag against a closed allowlist.
- `SafeDiagnosticContext` rejects any structured-diagnostic field outside its allowlist.
An allowlist rather than a denylist, because the failure mode of a denylist is that the one field
nobody thought of is the one that leaks.
Every contact point value type overrides `toString()` to print `[redacted]`. That covers the case a
central redactor cannot: a value interpolated into a log line by accident.
## Web Push endpoints
RFC 8030 defines the push URI as a capability URL — knowing it is sufficient to push to the
subscriber. It is handled as a secret, not as a URL.
## Callbacks
TLS, provider signature verification over the exact received bytes and external URL, replay defence
where a timestamp or nonce is available, body-size and content-type limits, profile binding, rate
limiting, idempotent ingestion and a security audit trail for rejections.
## Tenant isolation
Every store port carries the tenant boundary in its signature. Administrative operations require an
explicit tenant or a global authority.
+68
View File
@@ -0,0 +1,68 @@
# Notification support matrix
What each channel can actually prove, and what the platform refuses to claim.
## Channels
| Channel | Reference implementation | Grade | Strongest evidence the platform records by default |
|---|---|---|---|
| Email | SMTP, Amazon SES API | Stable | Provider acceptance; recipient mail-server delivery, bounce and complaint when the provider publishes events |
| SMS | Twilio Programmable Messaging | Stable | `accepted`/`queued`, `sent`, and carrier-DLR `delivered`/`undelivered` |
| Mobile push (Android and cross-platform) | FCM, FID-first with legacy registration token compatibility | Stable | FCM acceptance and explicit failures |
| Mobile push (Apple) | APNs HTTP/2 provider API | Stable | APNs acceptance |
| Web Push | RFC 8030, RFC 8291, RFC 8292 | Stable | Push-service acceptance; user-agent acknowledgement only where the service offers receipts |
| In-app inbox | Own database | Optional stable | `PERSISTED`, `SEEN`, `READ` |
| Webhook | HTTP client platform | Extension | Whatever the receiving HTTP contract states |
## Evidence levels
`NONE``PLATFORM_QUEUED``PROVIDER_ACCEPTED``NETWORK_OR_CARRIER_ACCEPTED`
`DEVICE_DELIVERED``USER_AGENT_DISPLAYED``USER_READ`
| Provider signal | Highest evidence it may produce |
|---|---|
| Internal queue commit | `PLATFORM_QUEUED` |
| SES `MessageId` | `PROVIDER_ACCEPTED` |
| SES `Delivery` | `NETWORK_OR_CARRIER_ACCEPTED` |
| Twilio `accepted` / `queued` | `PROVIDER_ACCEPTED` |
| Twilio `sent` | `NETWORK_OR_CARRIER_ACCEPTED` |
| Twilio `delivered` | `DEVICE_DELIVERED` |
| FCM send success | `PROVIDER_ACCEPTED` |
| APNs 2xx | `PROVIDER_ACCEPTED` |
| Web Push `201` | `PROVIDER_ACCEPTED` |
| Web Push receipt capability | `DEVICE_DELIVERED` |
| In-app row commit | `PROVIDER_ACCEPTED` |
| In-app `seen` endpoint | `USER_AGENT_DISPLAYED` |
| In-app `read` endpoint, authenticated app receipt | `USER_READ` |
Promotions the platform will not make, in code or in configuration:
- FCM send success is not `DEVICE_DELIVERED`.
- An APNs 2xx is not `DELIVERED`.
- An SES `MessageId` is not `DELIVERED`.
- An SMTP `250` is not inbox delivery.
## Submission outcomes
`NOT_SUBMITTED`, `CONFIRMED_ACCEPTED`, `CONFIRMED_REJECTED`, `AMBIGUOUS`.
`AMBIGUOUS` is a first-class stored state, not an error path. It means the request body was committed
to the provider and the outcome could not be read. While an ambiguous attempt exists on a recipient
delivery, automatic retry and automatic cross-channel fallback are both blocked.
## Not supported
The platform will not claim any of the following, because no channel above can support them:
- guaranteed delivery
- guaranteed read
- exactly-once human notification
- unconditional multi-provider failover after an unread response
- provider SDK types in the public API
- audience selection, campaign segmentation or jurisdiction rulings
## Target model
`FCM_FID` is the primary mobile push target. `FCM_REGISTRATION_TOKEN_LEGACY` and
`APNS_DEVICE_TOKEN` are separate types with separate lifecycles; they are never flattened into one
string field.
@@ -0,0 +1,771 @@
# JPA Experimental Expansion Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Stable JPA 플랫폼을 변경하지 않고 Multi-tenancy, PostgreSQL RLS, schema/database tenant 분리, consistency-aware Read Replica, Jakarta Persistence 4.0, Hibernate ORM 8, PostgreSQL 19 호환성을 독립 Experimental 모듈과 승격 Gate로 검증한다.
**Architecture:** Experimental module은 Stable `jpa-core-api` 계약만 소비하며 Stable starter에 자동 포함되지 않는다. 각 기능은 명시적 feature flag와 별도 compatibility/failure suite를 요구한다. 실험 결과가 Stable 의미론과 충돌하면 Core를 왜곡하지 않고 capability 또는 별도 profile로 유지한다.
**Tech Stack:** Stable 계획의 Java 21·Spring Boot 4.1·PostgreSQL Testcontainers 기반, PostgreSQL RLS, AbstractRoutingDataSource, tenant-specific DataSource registry, Jakarta Persistence 4.0 preview/final compatibility lane, Hibernate ORM 8 compatibility lane, PostgreSQL 19 compatibility lane.
## Global Constraints
- Stable 계획 Task 1~53이 완료되고 Release Gate가 통과한 뒤 시작한다.
- 모듈 루트는 `modules/jpa-experimental`이다.
- Experimental module은 `jpa-spring-boot-starter`의 기본 dependency가 아니다.
- 모든 기능은 `backend.jpa.experimental.*` feature flag를 요구한다.
- Tenant ID와 consistency token은 metric label에 기록하지 않는다.
- Tenant context 누락은 fail-closed다.
- `readOnly=true`만으로 replica routing하지 않는다.
- Lock query, write transaction, read-after-write pin은 primary를 사용한다.
- JPA4/Hibernate8/PG19 결과로 Stable 3.2/7.4/PG16~18 contract를 수정하지 않는다.
- 승격 전 별도 security, failure, migration and compatibility evidence가 필요하다.
---
## 1. Experimental 파일 구조
```text
modules/jpa-experimental/
├── jpa-experimental-core/
├── jpa-multitenancy-column/
├── jpa-multitenancy-rls/
├── jpa-multitenancy-schema/
├── jpa-multitenancy-database/
├── jpa-read-replica/
└── jpa-next-compatibility/
```
---
### Task 1: Experimental Module·Feature Gate·Dependency Isolation 구성
**Files:**
- Create: `modules/jpa-experimental/jpa-experimental-core/build.gradle.kts`
- Create: `modules/jpa-experimental/jpa-multitenancy-column/build.gradle.kts`
- Create: `modules/jpa-experimental/jpa-multitenancy-rls/build.gradle.kts`
- Create: `modules/jpa-experimental/jpa-multitenancy-schema/build.gradle.kts`
- Create: `modules/jpa-experimental/jpa-multitenancy-database/build.gradle.kts`
- Create: `modules/jpa-experimental/jpa-read-replica/build.gradle.kts`
- Create: `modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts`
- Create: `modules/jpa-experimental/jpa-experimental-core/src/main/java/io/backend/skeleton/jpa/experimental/ExperimentalFeature.java`
- Create: `modules/jpa-experimental/jpa-experimental-core/src/main/java/io/backend/skeleton/jpa/experimental/ExperimentalFeatureGate.java`
- Modify: `settings.gradle.kts`
- Test: `modules/jpa-experimental/jpa-experimental-core/src/test/java/io/backend/skeleton/jpa/experimental/ExperimentalFeatureGateTest.java`
**Interfaces:**
- Consumes: Stable `jpa-core-api` and explicit environment feature flags.
- Produces: Isolated experimental projects that cannot enter the Stable starter transitively.
**Implementation requirements:**
- Every module depends only on Stable public contracts, never on Stable internal packages.
- Feature gate fails startup when module is present but flag is absent.
- Add a dependency graph test proving the Stable starter has no experimental dependency.
- [ ] **Step 1: Write the failing test**
```java
package io.backend.skeleton.jpa.experimental;
class ExperimentalFeatureGateTest {
@Test
void featureIsDisabledUnlessExplicitlyEnabled() {
assertThatThrownBy(() -> gate.requireEnabled(MULTITENANCY_COLUMN, Map.of()))
.hasMessageContaining("backend.jpa.experimental.multitenancy-column=true");
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-experimental-core:test --tests 'io.backend.skeleton.jpa.experimental.ExperimentalFeatureGateTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```java
package io.backend.skeleton.jpa.experimental;
public final class ExperimentalFeatureGate {
public void requireEnabled(
ExperimentalFeature feature,
Map<String, Boolean> flags) {
if (!Boolean.TRUE.equals(flags.get(feature.property()))) {
throw new IllegalStateException(feature.property() + "=true is required");
}
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-experimental-core:test --tests 'io.backend.skeleton.jpa.experimental.ExperimentalFeatureGateTest'
./gradlew :modules:jpa-experimental:jpa-experimental-core:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-experimental-core/build.gradle.kts' 'modules/jpa-experimental/jpa-multitenancy-column/build.gradle.kts' 'modules/jpa-experimental/jpa-multitenancy-rls/build.gradle.kts' 'modules/jpa-experimental/jpa-multitenancy-schema/build.gradle.kts' 'modules/jpa-experimental/jpa-multitenancy-database/build.gradle.kts' 'modules/jpa-experimental/jpa-read-replica/build.gradle.kts' 'modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts' 'modules/jpa-experimental/jpa-experimental-core/src/main/java/io/backend/skeleton/jpa/experimental/ExperimentalFeature.java' 'modules/jpa-experimental/jpa-experimental-core/src/main/java/io/backend/skeleton/jpa/experimental/ExperimentalFeatureGate.java' 'settings.gradle.kts' 'modules/jpa-experimental/jpa-experimental-core/src/test/java/io/backend/skeleton/jpa/experimental/ExperimentalFeatureGateTest.java'
git commit -m "build: isolate jpa experimental modules"
```
### Task 2: Shared-schema Tenant Context와 Column Guard 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantId.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantContext.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantAwareRepositoryGuard.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantEntityListenerGuard.java`
- Test: `modules/jpa-experimental/jpa-multitenancy-column/src/integrationTest/java/io/backend/skeleton/jpa/experimental/tenant/TenantColumnIsolationTest.java`
**Interfaces:**
- Consumes: Explicit request/job tenant context and domain Entity tenant-column contracts.
- Produces: Fail-closed tenant context propagation and query/write isolation evidence.
**Implementation requirements:**
- Reject Repository access when tenant context is absent outside an audited admin scope.
- Require tenant column in unique/index requirements where isolation depends on it.
- Test async job context propagation and cleanup.
- Do not rely on Hibernate filter alone as the final security boundary.
- [ ] **Step 1: Write the failing test**
```java
package io.backend.skeleton.jpa.experimental.tenant;
class TenantColumnIsolationTest {
@Test
void tenantARepositoryCannotReadTenantBRows() {
insertFor(TENANT_A, "a");
insertFor(TENANT_B, "b");
assertThat(withTenant(TENANT_A, repository::findAll))
.extracting(Item::value)
.containsExactly("a");
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-column:integrationTest --tests 'io.backend.skeleton.jpa.experimental.tenant.TenantColumnIsolationTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```java
package io.backend.skeleton.jpa.experimental.tenant;
public final class TenantContext {
private static final ThreadLocal<TenantId> CURRENT = new ThreadLocal<>();
public static TenantId require() {
TenantId tenant = CURRENT.get();
if (tenant == null) throw new IllegalStateException("tenant context is required");
return tenant;
}
public static void clear() { CURRENT.remove(); }
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-column:integrationTest --tests 'io.backend.skeleton.jpa.experimental.tenant.TenantColumnIsolationTest'
./gradlew :modules:jpa-experimental:jpa-multitenancy-column:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantId.java' 'modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantContext.java' 'modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantAwareRepositoryGuard.java' 'modules/jpa-experimental/jpa-multitenancy-column/src/main/java/io/backend/skeleton/jpa/experimental/tenant/TenantEntityListenerGuard.java' 'modules/jpa-experimental/jpa-multitenancy-column/src/integrationTest/java/io/backend/skeleton/jpa/experimental/tenant/TenantColumnIsolationTest.java'
git commit -m "feat: add experimental tenant column isolation"
```
### Task 3: PostgreSQL RLS Tenant Policy와 Connection Reuse Guard 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-multitenancy-rls/src/main/java/io/backend/skeleton/jpa/experimental/rls/RlsTenantSessionBinder.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-rls/src/main/java/io/backend/skeleton/jpa/experimental/rls/RlsPolicyVerifier.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-rls/src/main/java/io/backend/skeleton/jpa/experimental/rls/RlsAdminBypassToken.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-rls/src/main/resources/db/experimental-rls/V1__tenant_rls.sql`
- Test: `modules/jpa-experimental/jpa-multitenancy-rls/src/failureTest/java/io/backend/skeleton/jpa/experimental/rls/RlsIsolationFailureTest.java`
**Interfaces:**
- Consumes: TenantContext, PostgreSQL transaction-local settings and restricted runtime role.
- Produces: Database-enforced tenant isolation that resets safely across pooled connections.
**Implementation requirements:**
- Set tenant context with transaction-local `set_config` before tenant queries.
- Prove a pooled connection cannot leak the prior tenant into the next transaction.
- Runtime role must not own tables or bypass RLS.
- Admin bypass requires a separate DataSource and audit token.
- [ ] **Step 1: Write the failing test**
```java
package io.backend.skeleton.jpa.experimental.rls;
class RlsIsolationFailureTest {
@Test
void pooledConnectionDoesNotLeakPriorTenantSetting() {
withTenant(TENANT_A, () -> assertThat(repository.count()).isEqualTo(1));
withTenant(TENANT_B, () -> assertThat(repository.count()).isEqualTo(1));
withoutTenant(() -> assertThatThrownBy(repository::count).isInstanceOf(DataAccessException.class));
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-rls:failureTest --tests 'io.backend.skeleton.jpa.experimental.rls.RlsIsolationFailureTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```java
package io.backend.skeleton.jpa.experimental.rls;
public final class RlsTenantSessionBinder {
public void bind(EntityManager entityManager, TenantId tenant) {
entityManager.createNativeQuery(
"select set_config('app.tenant_id', :tenant, true)")
.setParameter("tenant", tenant.value())
.getSingleResult();
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-rls:failureTest --tests 'io.backend.skeleton.jpa.experimental.rls.RlsIsolationFailureTest'
./gradlew :modules:jpa-experimental:jpa-multitenancy-rls:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-multitenancy-rls/src/main/java/io/backend/skeleton/jpa/experimental/rls/RlsTenantSessionBinder.java' 'modules/jpa-experimental/jpa-multitenancy-rls/src/main/java/io/backend/skeleton/jpa/experimental/rls/RlsPolicyVerifier.java' 'modules/jpa-experimental/jpa-multitenancy-rls/src/main/java/io/backend/skeleton/jpa/experimental/rls/RlsAdminBypassToken.java' 'modules/jpa-experimental/jpa-multitenancy-rls/src/main/resources/db/experimental-rls/V1__tenant_rls.sql' 'modules/jpa-experimental/jpa-multitenancy-rls/src/failureTest/java/io/backend/skeleton/jpa/experimental/rls/RlsIsolationFailureTest.java'
git commit -m "feat: add experimental postgresql rls isolation"
```
### Task 4: Schema-per-tenant Connection Provider와 Migration Orchestrator 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-multitenancy-schema/src/main/java/io/backend/skeleton/jpa/experimental/schema/SchemaTenantRegistry.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-schema/src/main/java/io/backend/skeleton/jpa/experimental/schema/SchemaMultiTenantConnectionProvider.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-schema/src/main/java/io/backend/skeleton/jpa/experimental/schema/SchemaTenantMigrationOrchestrator.java`
- Test: `modules/jpa-experimental/jpa-multitenancy-schema/src/migrationTest/java/io/backend/skeleton/jpa/experimental/schema/SchemaTenantMigrationContractTest.java`
**Interfaces:**
- Consumes: Validated tenant→schema catalog and Flyway migration gate.
- Produces: Bounded schema selection and per-tenant migration status without accepting raw schema names.
**Implementation requirements:**
- Map TenantId to a pre-registered schema identifier; no user-provided SQL identifier.
- Reset schema/search_path when returning pooled connections.
- Track migration version and failure per tenant.
- Rate-limit tenant migrations and support resume without auto-repair.
- [ ] **Step 1: Write the failing test**
```java
package io.backend.skeleton.jpa.experimental.schema;
class SchemaTenantMigrationContractTest {
@Test
void migratesOnlyRegisteredSchemasAndResumesAfterFailure() {
orchestrator.migrateAll(List.of(TENANT_A, TENANT_B));
assertThat(status(TENANT_A).version()).isEqualTo(LATEST);
assertThatThrownBy(() -> orchestrator.migrate(new TenantId("../public")))
.isInstanceOf(IllegalArgumentException.class);
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-schema:migrationTest --tests 'io.backend.skeleton.jpa.experimental.schema.SchemaTenantMigrationContractTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```java
package io.backend.skeleton.jpa.experimental.schema;
public final class SchemaTenantRegistry {
public String requireSchema(TenantId tenant) {
return Optional.ofNullable(schemaByTenant.get(tenant))
.orElseThrow(() -> new IllegalArgumentException("unregistered tenant schema"));
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-schema:migrationTest --tests 'io.backend.skeleton.jpa.experimental.schema.SchemaTenantMigrationContractTest'
./gradlew :modules:jpa-experimental:jpa-multitenancy-schema:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-multitenancy-schema/src/main/java/io/backend/skeleton/jpa/experimental/schema/SchemaTenantRegistry.java' 'modules/jpa-experimental/jpa-multitenancy-schema/src/main/java/io/backend/skeleton/jpa/experimental/schema/SchemaMultiTenantConnectionProvider.java' 'modules/jpa-experimental/jpa-multitenancy-schema/src/main/java/io/backend/skeleton/jpa/experimental/schema/SchemaTenantMigrationOrchestrator.java' 'modules/jpa-experimental/jpa-multitenancy-schema/src/migrationTest/java/io/backend/skeleton/jpa/experimental/schema/SchemaTenantMigrationContractTest.java'
git commit -m "feat: add experimental schema per tenant persistence"
```
### Task 5: Database-per-tenant DataSource Registry와 Capacity Guard 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantDataSourceRegistry.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantEntityManagerFactoryRegistry.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantPoolBudget.java`
- Create: `modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantDataSourceLifecycle.java`
- Test: `modules/jpa-experimental/jpa-multitenancy-database/src/performanceTest/java/io/backend/skeleton/jpa/experimental/database/TenantPoolCapacityContractTest.java`
**Interfaces:**
- Consumes: Secret-backed tenant connection profiles and global DB connection budget.
- Produces: Lazy bounded per-tenant pools with eviction, credential rotation and migration status.
**Implementation requirements:**
- Never create an unbounded Hikari pool per tenant.
- Enforce global maximum pools and connections before creating a DataSource.
- Drain and close pools on tenant removal or credential rotation.
- Do not expose tenant JDBC URLs or credentials in diagnostics.
- [ ] **Step 1: Write the failing test**
```java
package io.backend.skeleton.jpa.experimental.database;
class TenantPoolCapacityContractTest {
@Test
void refusesNewTenantPoolWhenGlobalConnectionBudgetIsExhausted() {
registry.openTenants(globalBudget().maxTenants());
assertThatThrownBy(() -> registry.require(ANOTHER_TENANT))
.isInstanceOf(IllegalStateException.class)
.hasMessageContaining("tenant pool budget");
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-database:performanceTest --tests 'io.backend.skeleton.jpa.experimental.database.TenantPoolCapacityContractTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```java
package io.backend.skeleton.jpa.experimental.database;
public record TenantPoolBudget(
int maxOpenPools,
int maxConnectionsAcrossPools) {
public void requireCapacity(int openPools, int allocatedConnections) {
if (openPools >= maxOpenPools || allocatedConnections >= maxConnectionsAcrossPools) {
throw new IllegalStateException("tenant pool budget exhausted");
}
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-multitenancy-database:performanceTest --tests 'io.backend.skeleton.jpa.experimental.database.TenantPoolCapacityContractTest'
./gradlew :modules:jpa-experimental:jpa-multitenancy-database:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantDataSourceRegistry.java' 'modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantEntityManagerFactoryRegistry.java' 'modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantPoolBudget.java' 'modules/jpa-experimental/jpa-multitenancy-database/src/main/java/io/backend/skeleton/jpa/experimental/database/TenantDataSourceLifecycle.java' 'modules/jpa-experimental/jpa-multitenancy-database/src/performanceTest/java/io/backend/skeleton/jpa/experimental/database/TenantPoolCapacityContractTest.java'
git commit -m "feat: add experimental database per tenant registry"
```
### Task 6: Consistency-aware Read Replica Routing 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ReadConsistency.java`
- Create: `modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ConsistencyToken.java`
- Create: `modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ReplicaRoutingDecision.java`
- Create: `modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ConsistencyAwareDataSourceRouter.java`
- Create: `modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ReplicaLagMonitor.java`
- Test: `modules/jpa-experimental/jpa-read-replica/src/failureTest/java/io/backend/skeleton/jpa/experimental/replica/ReadAfterWriteRoutingContractTest.java`
**Interfaces:**
- Consumes: Primary/replica DataSources, transaction state, lock intent and replica lag evidence.
- Produces: Routing decisions for PRIMARY_REQUIRED, BOUNDED_STALENESS and EVENTUAL reads.
**Implementation requirements:**
- Writes, lock queries, REQUIRES_NEW writes and active write transactions always use primary.
- Read-after-write uses a consistency token or primary pin, not `readOnly=true` alone.
- Fallback to primary when replica lag exceeds policy or evidence is unavailable.
- Keep routing fixed for the life of one transaction.
- [ ] **Step 1: Write the failing test**
```java
package io.backend.skeleton.jpa.experimental.replica;
class ReadAfterWriteRoutingContractTest {
@Test
void immediateReadAfterWriteUsesPrimaryUntilConsistencyTokenIsSatisfied() {
var token = service.writeAndReturnConsistencyToken();
var decision = router.route(readOnlyTransaction(), ReadConsistency.after(token));
assertThat(decision.target()).isEqualTo(PRIMARY);
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-read-replica:failureTest --tests 'io.backend.skeleton.jpa.experimental.replica.ReadAfterWriteRoutingContractTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```java
package io.backend.skeleton.jpa.experimental.replica;
public final class ConsistencyAwareDataSourceRouter {
public ReplicaRoutingDecision route(
TransactionContext transaction,
ReadConsistency consistency) {
if (transaction.write() || transaction.locking() ||
!lagMonitor.satisfies(consistency)) {
return ReplicaRoutingDecision.primary();
}
return ReplicaRoutingDecision.replica();
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-read-replica:failureTest --tests 'io.backend.skeleton.jpa.experimental.replica.ReadAfterWriteRoutingContractTest'
./gradlew :modules:jpa-experimental:jpa-read-replica:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ReadConsistency.java' 'modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ConsistencyToken.java' 'modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ReplicaRoutingDecision.java' 'modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ConsistencyAwareDataSourceRouter.java' 'modules/jpa-experimental/jpa-read-replica/src/main/java/io/backend/skeleton/jpa/experimental/replica/ReplicaLagMonitor.java' 'modules/jpa-experimental/jpa-read-replica/src/failureTest/java/io/backend/skeleton/jpa/experimental/replica/ReadAfterWriteRoutingContractTest.java'
git commit -m "feat: add experimental consistency aware replica routing"
```
### Task 7: Jakarta Persistence 4.0 Compatibility Lane 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-next-compatibility/src/compatibilityJpa4/java/io/backend/skeleton/jpa/experimental/next/Jpa4CompatibilityTest.java`
- Create: `.github/workflows/jpa-next-jpa4.yml`
- Modify: `modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts`
- Test: `modules/jpa-experimental/jpa-next-compatibility/src/test/java/io/backend/skeleton/jpa/experimental/next/CompatibilityLaneDefinitionTest.java`
**Interfaces:**
- Consumes: Published Jakarta Persistence 4.0 milestone/final artifact when available and the Stable contract suite.
- Produces: A non-blocking compatibility report that does not alter Stable JPA 3.2 APIs.
**Implementation requirements:**
- Run the Stable public API compilation and selected mapping contracts against JPA 4.
- Record removed/changed APIs and provider support separately.
- Do not publish JPA4 compiled artifacts under Stable coordinates.
- [ ] **Step 1: Write the failing test**
```kotlin
package io.backend.skeleton.jpa.experimental.next;
class CompatibilityLaneDefinitionTest {
@Test
void jpaFourLaneIsExperimentalAndSeparateFromStablePublication() {
assertThat(lane("jpa4").publicationEnabled()).isFalse();
assertThat(lane("jpa4").supportLevel()).isEqualTo(EXPERIMENTAL);
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test --tests 'io.backend.skeleton.jpa.experimental.next.CompatibilityLaneDefinitionTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```kotlin
testing {
suites {
register<JvmTestSuite>("compatibilityJpa4") {
useJUnitJupiter()
dependencies {
implementation(project(":modules:jpa:jpa-core-api"))
implementation(libs.jakarta.persistence.next)
}
}
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test --tests 'io.backend.skeleton.jpa.experimental.next.CompatibilityLaneDefinitionTest'
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-next-compatibility/src/compatibilityJpa4/java/io/backend/skeleton/jpa/experimental/next/Jpa4CompatibilityTest.java' '.github/workflows/jpa-next-jpa4.yml' 'modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts' 'modules/jpa-experimental/jpa-next-compatibility/src/test/java/io/backend/skeleton/jpa/experimental/next/CompatibilityLaneDefinitionTest.java'
git commit -m "test: add jakarta persistence four compatibility lane"
```
### Task 8: Hibernate ORM 8 Compatibility Lane 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-next-compatibility/src/compatibilityHibernate8/java/io/backend/skeleton/jpa/experimental/next/Hibernate8CompatibilityTest.java`
- Create: `.github/workflows/jpa-next-hibernate8.yml`
- Modify: `modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts`
- Test: `modules/jpa-experimental/jpa-next-compatibility/src/test/java/io/backend/skeleton/jpa/experimental/next/HibernateCompatibilityPolicyTest.java`
**Interfaces:**
- Consumes: Hibernate ORM 8 milestone/final artifact and Stable Hibernate 7.4 regression suites.
- Produces: Generated SQL, fetch pagination, statistics, batch and extension compatibility evidence.
**Implementation requirements:**
- Re-run collection fetch pagination, StatementInspector, Statistics, JSONB, Batch and StatelessSession contracts.
- Record SQL and performance differences without weakening the 7.4 Stable gate.
- Do not allow Hibernate 8 dependencies in Stable published modules.
- [ ] **Step 1: Write the failing test**
```kotlin
package io.backend.skeleton.jpa.experimental.next;
class HibernateCompatibilityPolicyTest {
@Test
void hibernateEightCannotReplaceStableProviderWithoutPromotion() {
assertThat(policy.stableProvider()).isEqualTo("7.4");
assertThat(policy.experimentalProviders()).contains("8");
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test --tests 'io.backend.skeleton.jpa.experimental.next.HibernateCompatibilityPolicyTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```kotlin
testing {
suites {
register<JvmTestSuite>("compatibilityHibernate8") {
useJUnitJupiter()
dependencies {
implementation(project(":modules:jpa:jpa-testkit-postgresql"))
implementation(libs.hibernate.orm.next)
}
}
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test --tests 'io.backend.skeleton.jpa.experimental.next.HibernateCompatibilityPolicyTest'
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-next-compatibility/src/compatibilityHibernate8/java/io/backend/skeleton/jpa/experimental/next/Hibernate8CompatibilityTest.java' '.github/workflows/jpa-next-hibernate8.yml' 'modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts' 'modules/jpa-experimental/jpa-next-compatibility/src/test/java/io/backend/skeleton/jpa/experimental/next/HibernateCompatibilityPolicyTest.java'
git commit -m "test: add hibernate eight compatibility lane"
```
### Task 9: PostgreSQL 19 Compatibility와 Stable 승격 Gate 구현
**Files:**
- Create: `modules/jpa-experimental/jpa-next-compatibility/src/compatibilityPostgresql19/java/io/backend/skeleton/jpa/experimental/next/PostgreSql19CompatibilityTest.java`
- Create: `docs/jpa/experimental-support-matrix.md`
- Create: `docs/jpa/experimental-promotion-checklist.md`
- Create: `.github/workflows/jpa-next-postgresql19.yml`
- Modify: `modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts`
- Test: `modules/jpa-experimental/jpa-next-compatibility/src/test/java/io/backend/skeleton/jpa/experimental/next/ExperimentalPromotionGateTest.java`
**Interfaces:**
- Consumes: PG19 image when GA, all Stable contracts, experimental security/failure/migration/performance reports.
- Produces: A promotion decision that requires evidence rather than version availability alone.
**Implementation requirements:**
- Run mapping, SQLSTATE, lock, batch, Flyway, plan and native extension contracts on PG19.
- Promotion requires two supported patch runs and no unresolved semantic regression.
- Multi-tenancy/replica promotion requires tenant leakage, failover, lag and pool-capacity evidence.
- Update Stable support matrix only through a reviewed ADR.
- [ ] **Step 1: Write the failing test**
```java
package io.backend.skeleton.jpa.experimental.next;
class ExperimentalPromotionGateTest {
@Test
void promotionRequiresAllEvidenceAndReviewedAdr() {
var evidence = evidence().withCompatibility(true).withSecurity(true).withFailure(true)
.withMigration(true).withPerformance(true).withReviewedAdr(false);
assertThat(gate.evaluate(evidence)).isEqualTo(BLOCKED_MISSING_ADR);
}
}
```
- [ ] **Step 2: Run the focused test and verify the failure**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test --tests 'io.backend.skeleton.jpa.experimental.next.ExperimentalPromotionGateTest'
```
Expected: FAIL because the production type or behavior does not exist yet.
- [ ] **Step 3: Implement the smallest complete production contract**
```java
package io.backend.skeleton.jpa.experimental.next;
public final class ExperimentalPromotionGate {
public PromotionDecision evaluate(PromotionEvidence evidence) {
if (!evidence.allTechnicalGatesPassed()) return BLOCKED_TECHNICAL;
if (!evidence.reviewedAdr()) return BLOCKED_MISSING_ADR;
return ELIGIBLE_FOR_STABLE_REVIEW;
}
}
```
Implement every file and invariant listed under **Implementation requirements**; the snippet fixes the public names and central behavior rather than replacing those requirements.
- [ ] **Step 4: Run the focused test and the module test suite**
Run:
```bash
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test --tests 'io.backend.skeleton.jpa.experimental.next.ExperimentalPromotionGateTest'
./gradlew :modules:jpa-experimental:jpa-next-compatibility:test
```
Expected: PASS with all assertions green.
- [ ] **Step 5: Commit the independently reviewable change**
```bash
git add 'modules/jpa-experimental/jpa-next-compatibility/src/compatibilityPostgresql19/java/io/backend/skeleton/jpa/experimental/next/PostgreSql19CompatibilityTest.java' 'docs/jpa/experimental-support-matrix.md' 'docs/jpa/experimental-promotion-checklist.md' '.github/workflows/jpa-next-postgresql19.yml' 'modules/jpa-experimental/jpa-next-compatibility/build.gradle.kts' 'modules/jpa-experimental/jpa-next-compatibility/src/test/java/io/backend/skeleton/jpa/experimental/next/ExperimentalPromotionGateTest.java'
git commit -m "docs: add jpa experimental promotion gates"
```
## 2. Experimental 완료 조건
```text
Stable starter가 Experimental module에 의존하지 않는다.
Tenant context 누락과 connection reuse에서 fail-closed다.
RLS runtime role이 policy를 bypass하지 못한다.
Schema/database tenant migration과 pool capacity가 bounded다.
Replica routing이 read-after-write와 lock query를 primary에 고정한다.
JPA4/Hibernate8/PG19 lane이 Stable artifacts를 변경하지 않는다.
승격은 ADR와 compatibility/security/failure/migration/performance 증거를 요구한다.
```
File diff suppressed because it is too large Load Diff