feat(mongodb): implement the MongoDB document persistence platform
Implements the mongodb-superpowers-package design: Stable Tasks 1-50 and Advanced Tasks 1-15. The design assumes 19 Stable + 12 Advanced Gradle projects under modules/mongodb*. This repository's fail-closed registry declares exactly 19 leaf identities, so those modules become package boundaries inside the registered leaf :adapter:outbound:persistence-mongo, with the design's module dependency table enforced by ten ArchUnit rules. The mapping and every deviation are recorded in docs/mongodb/repository-adaptation.md. Contract highlights, all enforced by tests rather than convention: - Transaction body retry and commit retry are separate loops. A new session per body attempt; commit-only retry on an unknown commit. The body is never replayed after a commit ambiguity, so a failover cannot become a duplicate. - MongoExecutionOutcome keeps both ambiguous outcomes distinct from success and failure, and MongoFailureContext records only the design-permitted fields. - Failure classification reads server error labels before numeric codes. - BSON representations come from a pinned manifest, never a library default, and a golden type-signature gate fails on any drift. - Index and validator changes go through the manifest and the admin plane; metadata ownership gates every drop. - Every Advanced capability refuses construction unless its flag is enabled. Verified against real servers, not only unit tests. Running the lanes for the first time exposed four defects that a green `check` had hidden: - Four release lanes passed while executing zero tests; the gate now counts executed tests per lane and fails on zero. - The "single replica set" fixture was a standalone, because Testcontainers 2.x needs withReplicaSet(); its test only asserted a connection string. - The three-node fixture was three independent clusters, so no election could occur, and awaitNewPrimary() compared against the post-stop primary. - The migration lease checked modifiedCount, so a same-millisecond refresh read as a lost lease. scripts/verify-mongodb-platform.sh now reports: 9 lanes, 0 skipped, 0 failed, every evidence category produced. scripts/verify-mongodb-advanced.sh reports NOT PROMOTABLE: actual-topology evidence (real sharded cluster, real KMS, real target deployment) is unobtainable here, so it is named rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
co-authored by
Claude Opus 5
Claude Opus 5
parent
3b5aee50e3
commit
d57d2f62a0
@@ -0,0 +1,63 @@
|
||||
# ADR-MONGO-001 — MongoDB platform boundary
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** `mongodb-superpowers-package/.../2026-08-11-mongodb-document-persistence-platform-design.md` §1, §2 (D-01, D-04, D-05), §5, §6
|
||||
|
||||
## Context
|
||||
|
||||
Two failure modes are common when a team wraps MongoDB.
|
||||
|
||||
The first is flattening: a shared `CommonMongoRepository<T, ID>` and a generic CRUD facade, which
|
||||
forces every collection to share an id strategy, a consistency profile and a query surface. MongoDB's
|
||||
single-document atomicity, aggregation model and change streams stop being reachable, and the first
|
||||
collection that needs something different gets a cast or a leaky generic.
|
||||
|
||||
The second is unrestricted exposure: the driver and `runCommand` available everywhere. Then any
|
||||
service can drop a collection, run an unbounded pipeline, or issue an admin command from a request
|
||||
thread, and no review catches it because there is nothing structural to catch.
|
||||
|
||||
## Decision
|
||||
|
||||
The domain owns its documents; the platform owns the cross-cutting decisions. Four exposure planes:
|
||||
|
||||
| Plane | Contents | Client |
|
||||
|---|---|---|
|
||||
| D1 Standard document persistence | Spring Data repositories, typed queries, mapping manifest, atomic update primitives, optimistic revision | Stable API V1, `apiStrict=true` |
|
||||
| D2 Advanced document operations | `MongoTemplate`, transactions/sessions, bulk, aggregation, keyset cursors, change streams | Stable API V1, `apiStrict=true` |
|
||||
| D3 Explicit Mongo capability | Native BSON, time series, search/vector, CSFLE/QE, shard-aware operations | Separate capability client |
|
||||
| D4 Admin plane | Collection, validator, index, migration, shard, repair | Separate admin client and credential |
|
||||
|
||||
Specifically:
|
||||
|
||||
1. **No `CommonMongoRepository<T, ID>`.** Each aggregate declares its own repository.
|
||||
2. **D1/D2 run on Stable API V1 with `apiStrict=true`,** so a command outside the versioned API fails
|
||||
at development time instead of on the next server upgrade.
|
||||
3. **D3 is not a raw-client escape.** Every call passes a fixed admission order: capability registered
|
||||
→ database profile → collection allowlist → operation name → timeout → consistency profile →
|
||||
result limit → trace → redaction → command category → admin-command refusal → execute.
|
||||
4. **D4 is a separate client with a separate credential.** No application-plane path reaches it;
|
||||
`PolicyAwareMongoNativeGateway` refuses admin-category commands regardless of capability.
|
||||
5. **Advanced and Experimental capabilities are opt-in modules**, never transitive dependencies of the
|
||||
Stable surface.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** MongoDB's semantics stay reachable. Misuse is refused structurally rather than reviewed
|
||||
for. A server upgrade cannot silently change D1/D2 behaviour. Admin operations have their own audit
|
||||
trail and credential.
|
||||
|
||||
**Negative.** Every operation needs a registered name and profile, so a new query is a small amount of
|
||||
configuration rather than zero. A genuinely new capability requires a registration before it can be
|
||||
used. Both are deliberate: the cost is paid once per operation, at review time.
|
||||
|
||||
**Rejected alternative — "expose the driver, rely on code review."** Review does not scale to every
|
||||
query in every service, and the operations that matter (unbounded pipeline, `dropCollection`,
|
||||
unanchored regex on user input) look unremarkable in a diff.
|
||||
|
||||
## Repository adaptation
|
||||
|
||||
The design assumes 19 Gradle modules under `modules/mongodb/`. This repository's fail-closed registry
|
||||
declares exactly 19 leaf identities, so the modules became package boundaries inside
|
||||
`:adapter:outbound:persistence-mongo`, enforced by ArchUnit. See
|
||||
[docs/mongodb/repository-adaptation.md](../mongodb/repository-adaptation.md).
|
||||
@@ -0,0 +1,58 @@
|
||||
# ADR-MONGO-002 — BSON representation is a pinned manifest
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §10, decision D-06
|
||||
|
||||
## Context
|
||||
|
||||
How a Java value is represented in BSON is a data contract, but nothing in the default toolchain
|
||||
treats it as one. Spring Data and the MongoDB driver both have defaults, and those defaults have
|
||||
changed across versions. A `BigDecimal` can land as a `Double`, a `String` or a `Decimal128`; a `UUID`
|
||||
can land as `Binary` subtype 3 or subtype 4; an `Instant` can land as a `Date` or a `String`.
|
||||
|
||||
The consequences are asymmetric. A representation change is invisible in a value-equality test —
|
||||
`12.30` looks like `12.30` whether it is a double or a `Decimal128` — but once a collection holds
|
||||
production data, changing it is a full migration. And the UUID case is worse than a migration: legacy
|
||||
Java representation byte-swaps two halves of the UUID, so a document written under one representation
|
||||
and read under the other yields a *different, valid-looking* UUID. Nothing errors. You get the wrong
|
||||
record.
|
||||
|
||||
## Decision
|
||||
|
||||
`MongoTypeRepresentationManifest` pins the representation for every type the platform maps, and
|
||||
`MongoMappingConfiguration` builds the Spring Data converters from it. Nothing relies on a library
|
||||
default.
|
||||
|
||||
| Java | BSON | Rationale |
|
||||
|---|---|---|
|
||||
| `UUID` | `Binary` subtype 4 (`STANDARD`) | Subtype 3 byte-swaps; cross-representation reads are silently wrong. |
|
||||
| `BigDecimal` | `Decimal128` | A double cannot represent `12.30`; money compared as a double is eventually wrong by a cent. |
|
||||
| `BigInteger` | `Decimal128`, or declared `String` when out of range | 34 significant digits; out of range fails on write instead of rounding. |
|
||||
| `Instant` / `OffsetDateTime` / `ZonedDateTime` | UTC `Date` | One instant, one representation. |
|
||||
| `LocalDate` | declared per field | A calendar day is not an instant. |
|
||||
| `LocalDateTime` | **refused** | No offset: the stored value depends on the writing JVM's default zone. |
|
||||
| `enum` | `String` name | Ordinals renumber when someone inserts a constant. |
|
||||
|
||||
Type metadata follows `MongoTypeMetadataPolicy` — `NONE`, `ALIAS` or `CLASS_NAME`. A
|
||||
`@LongLivedMongoDocument` type may not use `CLASS_NAME`: writing a FQCN into a million documents makes
|
||||
a package rename a data migration.
|
||||
|
||||
The manifest is enforced by a golden gate. `MongoBsonSnapshot` canonicalises a stored document,
|
||||
preserving BSON types and keeping missing distinct from null, and
|
||||
`MongoBsonSnapshotAssert.hasTypeSignature(...)` fails on any representation change. The registry
|
||||
pins `UuidCodec(STANDARD)` explicitly rather than inheriting a default, since inheriting the default
|
||||
is the exact drift the gate exists to catch.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** A library upgrade cannot move a representation without failing a test. Money is exact.
|
||||
UUIDs read back as themselves. Class moves stay refactors.
|
||||
|
||||
**Negative.** Every representation-affecting change requires updating a snapshot *and* writing a
|
||||
migration. A new mapped type needs a manifest entry before it can be used. This is the intended
|
||||
friction: the alternative is discovering the change in production.
|
||||
|
||||
**Rejected alternative — "snapshot the JSON."** JSON destroys exactly the distinctions the gate
|
||||
protects: `Decimal128` and `String` both render as text, `Binary` UUID and `ObjectId` both render as
|
||||
hex, and missing and null both disappear.
|
||||
@@ -0,0 +1,68 @@
|
||||
# ADR-MONGO-003 — Transaction body retry and commit retry are separate loops
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §14–§16, decisions D-07 through D-10
|
||||
|
||||
## Context
|
||||
|
||||
MongoDB reports two transaction failures that look similar and must be handled in opposite ways.
|
||||
|
||||
`TransientTransactionError` means the transaction definitively did not commit. The correct response is
|
||||
to run the whole thing again.
|
||||
|
||||
`UnknownTransactionCommitResult` means the commit **may already have applied** — typically because the
|
||||
primary changed while the commit was in flight. The correct response is to retry *the commit*, which
|
||||
is a no-op if it already succeeded.
|
||||
|
||||
The common implementation wraps everything in one retry loop. That loop replays the body after an
|
||||
unknown commit, and if the commit did apply, the body applies twice. In a payment or notification path
|
||||
that is a duplicate charge or a duplicate message, produced by the error handler.
|
||||
|
||||
The related trap is session reuse: retrying on the same session after an abort carries the aborted
|
||||
transaction's state into the retry.
|
||||
|
||||
## Decision
|
||||
|
||||
`MongoTransactionRetryCoordinator` implements two loops with different scopes.
|
||||
|
||||
```
|
||||
for each body attempt within the budget:
|
||||
open a NEW session
|
||||
run the body
|
||||
TransientTransactionError -> abort, continue to next body attempt
|
||||
commitWithRetry(session):
|
||||
UnknownTransactionCommitResult -> retry the COMMIT ONLY, same session
|
||||
```
|
||||
|
||||
Rules that follow, all of them load-bearing:
|
||||
|
||||
1. **A new session per body attempt.** No aborted state leaks into a retry.
|
||||
2. **The body is never replayed after a commit ambiguity.** `MongoRetryScope.COMMIT_ONLY` is a
|
||||
distinct value from `BODY` precisely so this cannot be collapsed by accident.
|
||||
3. **One budget bounds both loops.** `MongoRetryBudget` limits attempts *and* elapsed time, with
|
||||
jittered backoff, so a struggling primary is not retried into the ground by every instance at once.
|
||||
4. **An exhausted commit retry surfaces `TRANSACTION_COMMIT_UNKNOWN`,** never a generic failure. An
|
||||
ambiguous outcome reported as a failure invites the caller to retry — the one thing that must not
|
||||
happen. See [docs/mongodb/runbooks/unknown-commit.md](../mongodb/runbooks/unknown-commit.md).
|
||||
5. **Transaction bodies write a deterministic marker** so `MongoCommitReconciler` can establish what
|
||||
actually happened. A transaction that cannot be reconciled has no recovery path.
|
||||
6. **Classification reads labels before codes.** Server error labels are the authoritative statement
|
||||
about retryability; error codes vary by version.
|
||||
|
||||
Surrounding decisions that reduce how often this path is reached at all: single-document atomic
|
||||
operations are preferred over transactions (D-09), partial changes use update operators rather than
|
||||
`save()` (D-07), and whole-document replacement requires an optimistic revision (D-08).
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** A commit ambiguity cannot become a duplicate effect. The ambiguity reaches the caller as
|
||||
an ambiguity. The retry budget is bounded in both attempts and time.
|
||||
|
||||
**Negative.** Callers must handle a third outcome beyond success and failure. Transaction bodies must
|
||||
write a marker they would not otherwise need. Both costs are small compared with reconciling
|
||||
duplicated financial effects after the fact.
|
||||
|
||||
**Rejected alternative — "one retry loop, at-least-once everywhere."** It requires every transaction
|
||||
body to be fully idempotent, which is a much stronger and much less checkable property than writing
|
||||
one marker, and it is silently violated the first time someone adds a non-idempotent step.
|
||||
@@ -0,0 +1,62 @@
|
||||
# ADR-MONGO-004 — Index and schema changes belong to the admin plane
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §21–§25, decisions D-11, D-13
|
||||
|
||||
## Context
|
||||
|
||||
Spring Data can create indexes automatically from annotations. On a laptop this is convenient. On a
|
||||
collection with a hundred million documents, an index build is a capacity event: it consumes CPU, IO
|
||||
and memory on the primary for minutes to hours, and it starts because a pod restarted.
|
||||
|
||||
Worse, it starts N times when N pods restart, and there is no approval step, no ordering relative to
|
||||
the code that needs the index, and no record afterwards of what was created.
|
||||
|
||||
Schema validators have the same shape with a sharper edge: tightening a validator on a collection with
|
||||
existing data rejects writes to documents that were legal when they were written.
|
||||
|
||||
TTL has a third shape. It looks like a scheduler and is not one: the TTL monitor runs about once a
|
||||
minute and deletes in batches, so an expired document routinely remains readable for minutes or hours.
|
||||
|
||||
## Decision
|
||||
|
||||
**Indexes and validators are declared in a manifest and applied by the admin plane (D4).** Automatic
|
||||
index creation in production is disabled.
|
||||
|
||||
1. `MongoManifestRegistry` holds the declared indexes (`MongoIndexManifest`) and validator
|
||||
(`MongoSchemaManifest`) per collection. The manifest is the source of truth, reviewed in a pull
|
||||
request.
|
||||
2. `MongoIndexDiffEngine` compares manifest against observed state and reports missing, extra and
|
||||
*changed* indexes. Changed ones are reported rather than re-issued: MongoDB will not silently
|
||||
rebuild an index whose definition moved.
|
||||
3. `MongoIndexApplyPolicy` sets what an environment may do — `APPLY` (local), `APPLY_WITH_DIFF`
|
||||
(staging), `DIFF_WITH_APPROVED_APPLY` (production), `REPORT_ONLY` (audit).
|
||||
4. **Ownership gates every drop.** `MongoMetadataOwnership` distinguishes `APPLICATION_MANAGED` from
|
||||
`SEARCH_MANAGED`, `ENCRYPTION_MANAGED` and `EXTERNAL`. Only application-managed objects are
|
||||
droppable on drift. A diff engine without ownership eventually proposes dropping
|
||||
`enxcol_.customers.esc`, and "the drift tool cleaned it up" is a very bad incident summary.
|
||||
5. **Retirement is staged.** `MongoIndexRetirementState` moves an index declared → hidden →
|
||||
observed-unused → droppable, one deployment per transition. Hiding is instantly reversible;
|
||||
dropping is a rebuild.
|
||||
6. **Stable validation actions are `error` and `warn` only.** `errorAndLog` is not part of the Stable
|
||||
contract on 7.0 or 8.0 and is refused. Tightening goes `warn`+`MODERATE` → confirm zero warnings →
|
||||
`error`+`STRICT`, in two deployments.
|
||||
7. **TTL is physical cleanup only** (D-13). `MongoExpirationAccessPolicy` states the rule: a
|
||||
document's presence is not authorization and its absence is not a deadline. Access control checks
|
||||
the expiry field; scheduling uses a scheduler.
|
||||
8. **Migrations are checksummed, locked, precondition-checked and resumable.**
|
||||
`MongoMigrationRunner` fails hard when an applied id's checksum changed — two environments running
|
||||
different code under one id is worse than a failed deploy.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** Index builds are scheduled by people who know the capacity. Rollback is possible at
|
||||
every step. Drift is visible without being dangerous. Nothing drops what it does not own.
|
||||
|
||||
**Negative.** Adding an index is a manifest change plus an apply, not an annotation. Local development
|
||||
uses `APPLY` so the friction is confined to environments where it is warranted.
|
||||
|
||||
**Rejected alternative — "auto-create with a feature flag."** The flag is either on in production,
|
||||
which is the problem, or off, in which case the manifest is the real mechanism and the annotation is a
|
||||
second, divergent source of truth.
|
||||
@@ -0,0 +1,79 @@
|
||||
# ADR-MONGO-ADV-001 — Advanced capability promotion
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §2 (D-15), §3.2–§3.3; Advanced expansion plan Task 15
|
||||
|
||||
## Context
|
||||
|
||||
Sharding, time series, CSFLE, Queryable Encryption, search, vector search and multi-tenancy each work
|
||||
in a demo within an afternoon. What they do not do is behave the same way in production, and the
|
||||
differences are not discovered by functional tests:
|
||||
|
||||
- Sharding changes which queries are efficient. A query that misses the shard key becomes
|
||||
scatter-gather, which passes every test on a one-shard cluster.
|
||||
- Encryption's failure modes are KMS failure modes — wrong key, revoked permission, mid-rotation —
|
||||
none of which occur against a local key provider.
|
||||
- Search and vector search can be functionally correct and useless: the index returns results, and
|
||||
the results are not relevant. Recall is not visible in a pass/fail assertion.
|
||||
- Database-per-tenant works until the tenant count crosses what the connection and file-handle
|
||||
budget supports, which is an operational property, not a code property.
|
||||
|
||||
The failure mode this ADR prevents is a capability marked "done" on the strength of a green test that
|
||||
never touched the environment where it will run.
|
||||
|
||||
## Decision
|
||||
|
||||
Every Advanced and Experimental capability is an **opt-in module behind its own flag**, and promotion
|
||||
requires evidence, not confidence.
|
||||
|
||||
### Enablement
|
||||
|
||||
`MongoAdvancedCapabilityFlags` gates construction of every Advanced entry point. A disabled capability
|
||||
does not produce a runtime warning — the type refuses to be constructed, naming the property that
|
||||
enables it (`MongoAdvancedCapabilityFlags.propertyFor(capability)`). Being on the classpath is not
|
||||
being enabled, and `stableNeverDependsOnAdvanced` (ArchUnit) keeps the Stable surface free of them.
|
||||
|
||||
### Promotion evidence
|
||||
|
||||
`MongoAdvancedPromotionGate.verify(evidence)` requires every category:
|
||||
|
||||
| Category | Means |
|
||||
|---|---|
|
||||
| `stable-platform` | The Stable release gate passed on the same revision. |
|
||||
| `actual-topology` | The capability ran on the real topology — a real sharded cluster, the real KMS, the actual target deployment. Atlas Local is a pull-request convenience and explicitly not release evidence (`MongoAtlasCapabilityContractSuite.Environment.ATLAS_LOCAL`). |
|
||||
| `security` | Privileges reviewed; the capability's admin role is separate from the application role. |
|
||||
| `migration` | A documented path in and, where the capability is irreversible, an explicit statement that there is no path back. |
|
||||
| `failure` | Negative cases fail closed: wrong key, missing permission, rotation, non-ready index, unrouted query. |
|
||||
| `runbook` | A runbook exists for the capability's characteristic incident. |
|
||||
|
||||
### Additional per-capability requirements
|
||||
|
||||
- **Search / vector search:** relevance and performance evidence, not functional success alone.
|
||||
`MongoVectorSearchBenchmarkGate` requires recall alongside latency and index size; a gate that
|
||||
measures only latency certifies a fast wrong answer.
|
||||
- **Database-per-tenant and reshard orchestration remain Experimental** until operational scale
|
||||
evidence exists. Both are correct in the small and unbounded in the large.
|
||||
- **Reshard requires an explicit `ReshardApproval`** — a named approver and a stated window. It
|
||||
rewrites the collection.
|
||||
|
||||
### Promotion does not change the dependency boundary
|
||||
|
||||
A capability promoted to Stable **remains an opt-in module** unless a later starter ADR changes the
|
||||
dependency boundary. Promotion is a statement about evidence, not an invitation to add a transitive
|
||||
dependency to every service.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** No capability reaches production on the strength of a container-only test. The evidence
|
||||
list is the same for every capability, so promotion is reviewable rather than negotiated.
|
||||
|
||||
**Negative.** Promotion requires access to real infrastructure — a sharded cluster, a real KMS, the
|
||||
target deployment. That is the cost of the guarantee: the alternative is finding out in production,
|
||||
where encryption and sharding are both expensive to reverse.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
bash scripts/verify-mongodb-advanced.sh
|
||||
```
|
||||
Reference in New Issue
Block a user