Merge branch 'main' into worktree-jpa-persistence-platform
This commit is contained in:
DongHyeonka
@@ -0,0 +1,63 @@
|
||||
# ADR-MONGO-001 — MongoDB platform boundary
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** `mongodb-superpowers-package/.../2026-08-11-mongodb-document-persistence-platform-design.md` §1, §2 (D-01, D-04, D-05), §5, §6
|
||||
|
||||
## Context
|
||||
|
||||
Two failure modes are common when a team wraps MongoDB.
|
||||
|
||||
The first is flattening: a shared `CommonMongoRepository<T, ID>` and a generic CRUD facade, which
|
||||
forces every collection to share an id strategy, a consistency profile and a query surface. MongoDB's
|
||||
single-document atomicity, aggregation model and change streams stop being reachable, and the first
|
||||
collection that needs something different gets a cast or a leaky generic.
|
||||
|
||||
The second is unrestricted exposure: the driver and `runCommand` available everywhere. Then any
|
||||
service can drop a collection, run an unbounded pipeline, or issue an admin command from a request
|
||||
thread, and no review catches it because there is nothing structural to catch.
|
||||
|
||||
## Decision
|
||||
|
||||
The domain owns its documents; the platform owns the cross-cutting decisions. Four exposure planes:
|
||||
|
||||
| Plane | Contents | Client |
|
||||
|---|---|---|
|
||||
| D1 Standard document persistence | Spring Data repositories, typed queries, mapping manifest, atomic update primitives, optimistic revision | Stable API V1, `apiStrict=true` |
|
||||
| D2 Advanced document operations | `MongoTemplate`, transactions/sessions, bulk, aggregation, keyset cursors, change streams | Stable API V1, `apiStrict=true` |
|
||||
| D3 Explicit Mongo capability | Native BSON, time series, search/vector, CSFLE/QE, shard-aware operations | Separate capability client |
|
||||
| D4 Admin plane | Collection, validator, index, migration, shard, repair | Separate admin client and credential |
|
||||
|
||||
Specifically:
|
||||
|
||||
1. **No `CommonMongoRepository<T, ID>`.** Each aggregate declares its own repository.
|
||||
2. **D1/D2 run on Stable API V1 with `apiStrict=true`,** so a command outside the versioned API fails
|
||||
at development time instead of on the next server upgrade.
|
||||
3. **D3 is not a raw-client escape.** Every call passes a fixed admission order: capability registered
|
||||
→ database profile → collection allowlist → operation name → timeout → consistency profile →
|
||||
result limit → trace → redaction → command category → admin-command refusal → execute.
|
||||
4. **D4 is a separate client with a separate credential.** No application-plane path reaches it;
|
||||
`PolicyAwareMongoNativeGateway` refuses admin-category commands regardless of capability.
|
||||
5. **Advanced and Experimental capabilities are opt-in modules**, never transitive dependencies of the
|
||||
Stable surface.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** MongoDB's semantics stay reachable. Misuse is refused structurally rather than reviewed
|
||||
for. A server upgrade cannot silently change D1/D2 behaviour. Admin operations have their own audit
|
||||
trail and credential.
|
||||
|
||||
**Negative.** Every operation needs a registered name and profile, so a new query is a small amount of
|
||||
configuration rather than zero. A genuinely new capability requires a registration before it can be
|
||||
used. Both are deliberate: the cost is paid once per operation, at review time.
|
||||
|
||||
**Rejected alternative — "expose the driver, rely on code review."** Review does not scale to every
|
||||
query in every service, and the operations that matter (unbounded pipeline, `dropCollection`,
|
||||
unanchored regex on user input) look unremarkable in a diff.
|
||||
|
||||
## Repository adaptation
|
||||
|
||||
The design assumes 19 Gradle modules under `modules/mongodb/`. This repository's fail-closed registry
|
||||
declares exactly 19 leaf identities, so the modules became package boundaries inside
|
||||
`:adapter:outbound:persistence-mongo`, enforced by ArchUnit. See
|
||||
[docs/mongodb/repository-adaptation.md](../mongodb/repository-adaptation.md).
|
||||
@@ -0,0 +1,58 @@
|
||||
# ADR-MONGO-002 — BSON representation is a pinned manifest
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §10, decision D-06
|
||||
|
||||
## Context
|
||||
|
||||
How a Java value is represented in BSON is a data contract, but nothing in the default toolchain
|
||||
treats it as one. Spring Data and the MongoDB driver both have defaults, and those defaults have
|
||||
changed across versions. A `BigDecimal` can land as a `Double`, a `String` or a `Decimal128`; a `UUID`
|
||||
can land as `Binary` subtype 3 or subtype 4; an `Instant` can land as a `Date` or a `String`.
|
||||
|
||||
The consequences are asymmetric. A representation change is invisible in a value-equality test —
|
||||
`12.30` looks like `12.30` whether it is a double or a `Decimal128` — but once a collection holds
|
||||
production data, changing it is a full migration. And the UUID case is worse than a migration: legacy
|
||||
Java representation byte-swaps two halves of the UUID, so a document written under one representation
|
||||
and read under the other yields a *different, valid-looking* UUID. Nothing errors. You get the wrong
|
||||
record.
|
||||
|
||||
## Decision
|
||||
|
||||
`MongoTypeRepresentationManifest` pins the representation for every type the platform maps, and
|
||||
`MongoMappingConfiguration` builds the Spring Data converters from it. Nothing relies on a library
|
||||
default.
|
||||
|
||||
| Java | BSON | Rationale |
|
||||
|---|---|---|
|
||||
| `UUID` | `Binary` subtype 4 (`STANDARD`) | Subtype 3 byte-swaps; cross-representation reads are silently wrong. |
|
||||
| `BigDecimal` | `Decimal128` | A double cannot represent `12.30`; money compared as a double is eventually wrong by a cent. |
|
||||
| `BigInteger` | `Decimal128`, or declared `String` when out of range | 34 significant digits; out of range fails on write instead of rounding. |
|
||||
| `Instant` / `OffsetDateTime` / `ZonedDateTime` | UTC `Date` | One instant, one representation. |
|
||||
| `LocalDate` | declared per field | A calendar day is not an instant. |
|
||||
| `LocalDateTime` | **refused** | No offset: the stored value depends on the writing JVM's default zone. |
|
||||
| `enum` | `String` name | Ordinals renumber when someone inserts a constant. |
|
||||
|
||||
Type metadata follows `MongoTypeMetadataPolicy` — `NONE`, `ALIAS` or `CLASS_NAME`. A
|
||||
`@LongLivedMongoDocument` type may not use `CLASS_NAME`: writing a FQCN into a million documents makes
|
||||
a package rename a data migration.
|
||||
|
||||
The manifest is enforced by a golden gate. `MongoBsonSnapshot` canonicalises a stored document,
|
||||
preserving BSON types and keeping missing distinct from null, and
|
||||
`MongoBsonSnapshotAssert.hasTypeSignature(...)` fails on any representation change. The registry
|
||||
pins `UuidCodec(STANDARD)` explicitly rather than inheriting a default, since inheriting the default
|
||||
is the exact drift the gate exists to catch.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** A library upgrade cannot move a representation without failing a test. Money is exact.
|
||||
UUIDs read back as themselves. Class moves stay refactors.
|
||||
|
||||
**Negative.** Every representation-affecting change requires updating a snapshot *and* writing a
|
||||
migration. A new mapped type needs a manifest entry before it can be used. This is the intended
|
||||
friction: the alternative is discovering the change in production.
|
||||
|
||||
**Rejected alternative — "snapshot the JSON."** JSON destroys exactly the distinctions the gate
|
||||
protects: `Decimal128` and `String` both render as text, `Binary` UUID and `ObjectId` both render as
|
||||
hex, and missing and null both disappear.
|
||||
@@ -0,0 +1,68 @@
|
||||
# ADR-MONGO-003 — Transaction body retry and commit retry are separate loops
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §14–§16, decisions D-07 through D-10
|
||||
|
||||
## Context
|
||||
|
||||
MongoDB reports two transaction failures that look similar and must be handled in opposite ways.
|
||||
|
||||
`TransientTransactionError` means the transaction definitively did not commit. The correct response is
|
||||
to run the whole thing again.
|
||||
|
||||
`UnknownTransactionCommitResult` means the commit **may already have applied** — typically because the
|
||||
primary changed while the commit was in flight. The correct response is to retry *the commit*, which
|
||||
is a no-op if it already succeeded.
|
||||
|
||||
The common implementation wraps everything in one retry loop. That loop replays the body after an
|
||||
unknown commit, and if the commit did apply, the body applies twice. In a payment or notification path
|
||||
that is a duplicate charge or a duplicate message, produced by the error handler.
|
||||
|
||||
The related trap is session reuse: retrying on the same session after an abort carries the aborted
|
||||
transaction's state into the retry.
|
||||
|
||||
## Decision
|
||||
|
||||
`MongoTransactionRetryCoordinator` implements two loops with different scopes.
|
||||
|
||||
```
|
||||
for each body attempt within the budget:
|
||||
open a NEW session
|
||||
run the body
|
||||
TransientTransactionError -> abort, continue to next body attempt
|
||||
commitWithRetry(session):
|
||||
UnknownTransactionCommitResult -> retry the COMMIT ONLY, same session
|
||||
```
|
||||
|
||||
Rules that follow, all of them load-bearing:
|
||||
|
||||
1. **A new session per body attempt.** No aborted state leaks into a retry.
|
||||
2. **The body is never replayed after a commit ambiguity.** `MongoRetryScope.COMMIT_ONLY` is a
|
||||
distinct value from `BODY` precisely so this cannot be collapsed by accident.
|
||||
3. **One budget bounds both loops.** `MongoRetryBudget` limits attempts *and* elapsed time, with
|
||||
jittered backoff, so a struggling primary is not retried into the ground by every instance at once.
|
||||
4. **An exhausted commit retry surfaces `TRANSACTION_COMMIT_UNKNOWN`,** never a generic failure. An
|
||||
ambiguous outcome reported as a failure invites the caller to retry — the one thing that must not
|
||||
happen. See [docs/mongodb/runbooks/unknown-commit.md](../mongodb/runbooks/unknown-commit.md).
|
||||
5. **Transaction bodies write a deterministic marker** so `MongoCommitReconciler` can establish what
|
||||
actually happened. A transaction that cannot be reconciled has no recovery path.
|
||||
6. **Classification reads labels before codes.** Server error labels are the authoritative statement
|
||||
about retryability; error codes vary by version.
|
||||
|
||||
Surrounding decisions that reduce how often this path is reached at all: single-document atomic
|
||||
operations are preferred over transactions (D-09), partial changes use update operators rather than
|
||||
`save()` (D-07), and whole-document replacement requires an optimistic revision (D-08).
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** A commit ambiguity cannot become a duplicate effect. The ambiguity reaches the caller as
|
||||
an ambiguity. The retry budget is bounded in both attempts and time.
|
||||
|
||||
**Negative.** Callers must handle a third outcome beyond success and failure. Transaction bodies must
|
||||
write a marker they would not otherwise need. Both costs are small compared with reconciling
|
||||
duplicated financial effects after the fact.
|
||||
|
||||
**Rejected alternative — "one retry loop, at-least-once everywhere."** It requires every transaction
|
||||
body to be fully idempotent, which is a much stronger and much less checkable property than writing
|
||||
one marker, and it is silently violated the first time someone adds a non-idempotent step.
|
||||
@@ -0,0 +1,62 @@
|
||||
# ADR-MONGO-004 — Index and schema changes belong to the admin plane
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §21–§25, decisions D-11, D-13
|
||||
|
||||
## Context
|
||||
|
||||
Spring Data can create indexes automatically from annotations. On a laptop this is convenient. On a
|
||||
collection with a hundred million documents, an index build is a capacity event: it consumes CPU, IO
|
||||
and memory on the primary for minutes to hours, and it starts because a pod restarted.
|
||||
|
||||
Worse, it starts N times when N pods restart, and there is no approval step, no ordering relative to
|
||||
the code that needs the index, and no record afterwards of what was created.
|
||||
|
||||
Schema validators have the same shape with a sharper edge: tightening a validator on a collection with
|
||||
existing data rejects writes to documents that were legal when they were written.
|
||||
|
||||
TTL has a third shape. It looks like a scheduler and is not one: the TTL monitor runs about once a
|
||||
minute and deletes in batches, so an expired document routinely remains readable for minutes or hours.
|
||||
|
||||
## Decision
|
||||
|
||||
**Indexes and validators are declared in a manifest and applied by the admin plane (D4).** Automatic
|
||||
index creation in production is disabled.
|
||||
|
||||
1. `MongoManifestRegistry` holds the declared indexes (`MongoIndexManifest`) and validator
|
||||
(`MongoSchemaManifest`) per collection. The manifest is the source of truth, reviewed in a pull
|
||||
request.
|
||||
2. `MongoIndexDiffEngine` compares manifest against observed state and reports missing, extra and
|
||||
*changed* indexes. Changed ones are reported rather than re-issued: MongoDB will not silently
|
||||
rebuild an index whose definition moved.
|
||||
3. `MongoIndexApplyPolicy` sets what an environment may do — `APPLY` (local), `APPLY_WITH_DIFF`
|
||||
(staging), `DIFF_WITH_APPROVED_APPLY` (production), `REPORT_ONLY` (audit).
|
||||
4. **Ownership gates every drop.** `MongoMetadataOwnership` distinguishes `APPLICATION_MANAGED` from
|
||||
`SEARCH_MANAGED`, `ENCRYPTION_MANAGED` and `EXTERNAL`. Only application-managed objects are
|
||||
droppable on drift. A diff engine without ownership eventually proposes dropping
|
||||
`enxcol_.customers.esc`, and "the drift tool cleaned it up" is a very bad incident summary.
|
||||
5. **Retirement is staged.** `MongoIndexRetirementState` moves an index declared → hidden →
|
||||
observed-unused → droppable, one deployment per transition. Hiding is instantly reversible;
|
||||
dropping is a rebuild.
|
||||
6. **Stable validation actions are `error` and `warn` only.** `errorAndLog` is not part of the Stable
|
||||
contract on 7.0 or 8.0 and is refused. Tightening goes `warn`+`MODERATE` → confirm zero warnings →
|
||||
`error`+`STRICT`, in two deployments.
|
||||
7. **TTL is physical cleanup only** (D-13). `MongoExpirationAccessPolicy` states the rule: a
|
||||
document's presence is not authorization and its absence is not a deadline. Access control checks
|
||||
the expiry field; scheduling uses a scheduler.
|
||||
8. **Migrations are checksummed, locked, precondition-checked and resumable.**
|
||||
`MongoMigrationRunner` fails hard when an applied id's checksum changed — two environments running
|
||||
different code under one id is worse than a failed deploy.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** Index builds are scheduled by people who know the capacity. Rollback is possible at
|
||||
every step. Drift is visible without being dangerous. Nothing drops what it does not own.
|
||||
|
||||
**Negative.** Adding an index is a manifest change plus an apply, not an annotation. Local development
|
||||
uses `APPLY` so the friction is confined to environments where it is warranted.
|
||||
|
||||
**Rejected alternative — "auto-create with a feature flag."** The flag is either on in production,
|
||||
which is the problem, or off, in which case the manifest is the real mechanism and the annotation is a
|
||||
second, divergent source of truth.
|
||||
@@ -0,0 +1,79 @@
|
||||
# ADR-MONGO-ADV-001 — Advanced capability promotion
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-08-13
|
||||
- **Design source:** design §2 (D-15), §3.2–§3.3; Advanced expansion plan Task 15
|
||||
|
||||
## Context
|
||||
|
||||
Sharding, time series, CSFLE, Queryable Encryption, search, vector search and multi-tenancy each work
|
||||
in a demo within an afternoon. What they do not do is behave the same way in production, and the
|
||||
differences are not discovered by functional tests:
|
||||
|
||||
- Sharding changes which queries are efficient. A query that misses the shard key becomes
|
||||
scatter-gather, which passes every test on a one-shard cluster.
|
||||
- Encryption's failure modes are KMS failure modes — wrong key, revoked permission, mid-rotation —
|
||||
none of which occur against a local key provider.
|
||||
- Search and vector search can be functionally correct and useless: the index returns results, and
|
||||
the results are not relevant. Recall is not visible in a pass/fail assertion.
|
||||
- Database-per-tenant works until the tenant count crosses what the connection and file-handle
|
||||
budget supports, which is an operational property, not a code property.
|
||||
|
||||
The failure mode this ADR prevents is a capability marked "done" on the strength of a green test that
|
||||
never touched the environment where it will run.
|
||||
|
||||
## Decision
|
||||
|
||||
Every Advanced and Experimental capability is an **opt-in module behind its own flag**, and promotion
|
||||
requires evidence, not confidence.
|
||||
|
||||
### Enablement
|
||||
|
||||
`MongoAdvancedCapabilityFlags` gates construction of every Advanced entry point. A disabled capability
|
||||
does not produce a runtime warning — the type refuses to be constructed, naming the property that
|
||||
enables it (`MongoAdvancedCapabilityFlags.propertyFor(capability)`). Being on the classpath is not
|
||||
being enabled, and `stableNeverDependsOnAdvanced` (ArchUnit) keeps the Stable surface free of them.
|
||||
|
||||
### Promotion evidence
|
||||
|
||||
`MongoAdvancedPromotionGate.verify(evidence)` requires every category:
|
||||
|
||||
| Category | Means |
|
||||
|---|---|
|
||||
| `stable-platform` | The Stable release gate passed on the same revision. |
|
||||
| `actual-topology` | The capability ran on the real topology — a real sharded cluster, the real KMS, the actual target deployment. Atlas Local is a pull-request convenience and explicitly not release evidence (`MongoAtlasCapabilityContractSuite.Environment.ATLAS_LOCAL`). |
|
||||
| `security` | Privileges reviewed; the capability's admin role is separate from the application role. |
|
||||
| `migration` | A documented path in and, where the capability is irreversible, an explicit statement that there is no path back. |
|
||||
| `failure` | Negative cases fail closed: wrong key, missing permission, rotation, non-ready index, unrouted query. |
|
||||
| `runbook` | A runbook exists for the capability's characteristic incident. |
|
||||
|
||||
### Additional per-capability requirements
|
||||
|
||||
- **Search / vector search:** relevance and performance evidence, not functional success alone.
|
||||
`MongoVectorSearchBenchmarkGate` requires recall alongside latency and index size; a gate that
|
||||
measures only latency certifies a fast wrong answer.
|
||||
- **Database-per-tenant and reshard orchestration remain Experimental** until operational scale
|
||||
evidence exists. Both are correct in the small and unbounded in the large.
|
||||
- **Reshard requires an explicit `ReshardApproval`** — a named approver and a stated window. It
|
||||
rewrites the collection.
|
||||
|
||||
### Promotion does not change the dependency boundary
|
||||
|
||||
A capability promoted to Stable **remains an opt-in module** unless a later starter ADR changes the
|
||||
dependency boundary. Promotion is a statement about evidence, not an invitation to add a transitive
|
||||
dependency to every service.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** No capability reaches production on the strength of a container-only test. The evidence
|
||||
list is the same for every capability, so promotion is reviewable rather than negotiated.
|
||||
|
||||
**Negative.** Promotion requires access to real infrastructure — a sharded cluster, a real KMS, the
|
||||
target deployment. That is the cost of the guarantee: the alternative is finding out in production,
|
||||
where encryption and sharding are both expensive to reverse.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
bash scripts/verify-mongodb-advanced.sh
|
||||
```
|
||||
@@ -0,0 +1,91 @@
|
||||
# Advanced — CSFLE and Queryable Encryption
|
||||
|
||||
**Capabilities:** `MongoCapability.CSFLE`, `MongoCapability.QUERYABLE_ENCRYPTION`
|
||||
**Properties:** `ca-skeleton.persistence-mongo.advanced.csfle.enabled`,
|
||||
`ca-skeleton.persistence-mongo.advanced.queryable-encryption.enabled`
|
||||
**Status:** Advanced.
|
||||
|
||||
## Requirements
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Topology | Replica set or sharded cluster. |
|
||||
| Server | MongoDB 7.0 or 8.0 (see §4 for the 8.0 query-type limits). |
|
||||
| Privilege | `MongoPrincipalRole.ENCRYPTION_ADMIN` for the key vault; the application role never holds it. |
|
||||
| Environment | A real KMS and key vault. A local key provider does not exercise any of the failure modes that matter. |
|
||||
|
||||
## 1. CSFLE
|
||||
|
||||
`MongoCsfleProfile` binds a collection to its `MongoCsfleFieldPolicy` list, a key vault
|
||||
`MongoCredentialReference` and the key vault namespace. `MongoCsfleClientFactory` builds the encrypted
|
||||
client; `MongoDataKeyResolver` resolves data keys.
|
||||
|
||||
`MongoCsfleMode`:
|
||||
|
||||
| Mode | Queryable | Trade-off |
|
||||
|---|---|---|
|
||||
| `RANDOMIZED` | no | Same plaintext encrypts differently each time. The safe default. |
|
||||
| `DETERMINISTIC` | equality only | Same plaintext always yields the same ciphertext, so equality works — and so does frequency analysis. |
|
||||
| `UNINDEXED` | no | Stored encrypted, excluded from any index. |
|
||||
|
||||
`MongoCsfleFieldPolicy.forPii(field, queryable)` defaults to `RANDOMIZED` when the field is not
|
||||
queried. Deterministic encryption requires a written `equalityQueryJustification`; the constructor
|
||||
refuses a blank one, naming frequency analysis. A low-cardinality deterministic field (a status, a
|
||||
country, a boolean) leaks its distribution to anyone who can read the collection, which is the party
|
||||
encryption was protecting against.
|
||||
|
||||
## 2. Queryable Encryption
|
||||
|
||||
`MongoQueryableEncryptionProfile` binds a collection to `MongoEncryptedFieldDescriptor` entries.
|
||||
`MongoQueryableEncryptionQueryType` has exactly two values:
|
||||
|
||||
- `EQUALITY`
|
||||
- `RANGE` — must declare its domain (`min`, `max`). The constructor refuses a range field without one,
|
||||
because changing the domain later means re-encrypting the field.
|
||||
|
||||
`MongoQueryableEncryptionCollectionManager` owns the collection's lifecycle, because a QE collection
|
||||
is not just a collection: it carries metadata collections.
|
||||
|
||||
## 3. Metadata ownership
|
||||
|
||||
`MongoEncryptionMetadataOwnership` maps `customers` to `enxcol_.customers.esc` and
|
||||
`enxcol_.customers.ecoc`, and reports `__safeContent__`-prefixed indexes as
|
||||
`MongoMetadataOwnership.ENCRYPTION_MANAGED`.
|
||||
|
||||
These are never application-owned and never droppable by drift reconciliation. A drift tool that
|
||||
drops `enxcol_.customers.ecoc` corrupts the collection's queryability. This is the single most
|
||||
important integration point between encryption and
|
||||
[ADR-MONGO-004](../../adr/ADR-MONGO-004-index-schema-admin-plane.md).
|
||||
|
||||
## 4. Unsupported combinations
|
||||
|
||||
Refused at declaration, not discovered at runtime:
|
||||
|
||||
| Combination | Why |
|
||||
|---|---|
|
||||
| CSFLE **and** QE on the same collection | Two incompatible encryption schemes over one namespace. Both profile constructors refuse it. |
|
||||
| CSFLE on a time series collection | `requireNotTimeSeries(true)` raises `MongoOperationRejectedException`. |
|
||||
| QE `prefix` / `suffix` / `substring` | Not available on the platform's 8.0 baseline. The factory methods throw `UnsupportedOperationException` rather than returning a profile that fails later. |
|
||||
| Deterministic CSFLE without a justification | `IllegalArgumentException` naming frequency analysis. |
|
||||
| Range QE without a declared domain | `IllegalArgumentException` naming re-encryption. |
|
||||
|
||||
## 5. Failure recovery
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| `MongoEncryptionException` on read | Wrong data key, or the key vault is unreachable | Check KMS reachability and the key vault credential. Data is intact; the client cannot decrypt it. |
|
||||
| `MongoEncryptionException` on write | KMS permission revoked mid-operation | Restore the grant. Writes fail closed — nothing was written in plaintext. |
|
||||
| Queries return nothing on a deterministic field | The field was re-keyed | Equality matching is over ciphertext; a new key produces different ciphertext. Re-encrypt the field. |
|
||||
| QE queries fail after a drift reconciliation | A metadata collection was dropped | Restore from backup. This is why ownership gates drops. |
|
||||
|
||||
**Key rotation.** Rotating the customer master key re-wraps the data keys and does not require
|
||||
re-encrypting documents. Rotating a *data* key does require re-encrypting every document that used
|
||||
it. These are different operations with different costs, and confusing them is how a rotation becomes
|
||||
an outage.
|
||||
|
||||
## 6. Promotion evidence
|
||||
|
||||
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md), promotion requires the
|
||||
real KMS and key vault, plus negative cases that fail closed: wrong key, missing permission, rotation
|
||||
mid-operation (`MongoAtlasCapabilityContractSuite.kmsFailureModes`). A local key provider certifies
|
||||
none of these — it never rejects anything.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Advanced — GridFS compatibility and migration
|
||||
|
||||
**Capability:** `MongoCapability.GRIDFS_COMPATIBILITY`
|
||||
**Property:** `ca-skeleton.persistence-mongo.advanced.gridfs-compatibility.enabled`
|
||||
**Status:** Advanced, compatibility only. Decision D-14.
|
||||
|
||||
## Position
|
||||
|
||||
GridFS is a **compatibility adapter for files that already exist there**. New files use the existing
|
||||
Fileserver / Object Storage adapter, which is the source of truth for binary content.
|
||||
|
||||
The reason is not preference. GridFS stores file chunks in the same collections, on the same replica
|
||||
set, competing for the same working set as your documents. A large file read evicts document pages
|
||||
from cache, and file storage growth becomes replica-set growth — which means it becomes oplog
|
||||
pressure, backup duration and failover time. Object storage was built for this and MongoDB was not.
|
||||
|
||||
## Reading legacy files
|
||||
|
||||
`MongoGridFsCompatibilityReader` reads existing GridFS content as
|
||||
`GridFsLegacyContent(legacyId, filename, sizeBytes, checksum, stream)`. It reads; it does not write.
|
||||
|
||||
## Migration
|
||||
|
||||
`MongoGridFsMigrationJob` moves a file to object storage in a fixed order:
|
||||
|
||||
```
|
||||
read legacy content
|
||||
→ write to object storage
|
||||
→ verify the target checksum matches the source
|
||||
→ switch the reference
|
||||
→ (later, separately) delete the source
|
||||
```
|
||||
|
||||
Three properties, each of which exists because of a specific way this goes wrong:
|
||||
|
||||
1. **Verify before switching.** `MongoGridFsObjectReference` requires a non-blank checksum, and the
|
||||
job returns empty and writes no reference when the target checksum does not match the source. A
|
||||
migration that switches the reference on a successful *write* rather than a verified *copy*
|
||||
silently points at a truncated object.
|
||||
2. **The source is never deleted here.** Deletion is a separate, later decision after the new
|
||||
location has been serving reads long enough to be trusted. A migration that deletes as it goes has
|
||||
no rollback.
|
||||
3. **The checkpoint separates migrated from failed.** `MongoGridFsMigrationCheckpoint` tracks
|
||||
`migratedCount()`, `failedCount()`, `clean()` and `lastMigratedLegacyId()`, so a restart continues
|
||||
from the last completed file rather than starting over, and a partially failed run is visible as
|
||||
partial rather than as "done".
|
||||
|
||||
## Failure recovery
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| `migrate` returns empty | Checksum mismatch | The copy is bad. Investigate before retrying; do not force the reference. |
|
||||
| `IllegalArgumentException` on the reference | Missing checksum | A reference without a checksum cannot be verified and is refused. |
|
||||
| Checkpoint not `clean()` | Some files failed | Re-run for the failed ids only; the checkpoint names the last successful one. |
|
||||
| Reference switched but content missing | Source deleted too early | Restore from backup. This is what rule 2 prevents. |
|
||||
|
||||
## Promotion evidence
|
||||
|
||||
Actual-topology evidence against the real object storage backend, a security review of the storage
|
||||
credential, the migration path above, failure cases (checksum mismatch refused, missing checksum
|
||||
refused, restart resumes), and this document as the runbook.
|
||||
|
||||
New file storage does not go through here at all — see the fileserver adapter.
|
||||
@@ -0,0 +1,90 @@
|
||||
# Advanced — Multi-tenancy
|
||||
|
||||
**Capabilities:** `MongoCapability.SHARED_COLLECTION_TENANCY` (Advanced),
|
||||
`MongoCapability.DATABASE_PER_TENANT` (Experimental)
|
||||
**Properties:** `ca-skeleton.persistence-mongo.advanced.shared-collection-tenancy.enabled`,
|
||||
`ca-skeleton.persistence-mongo.advanced.database-per-tenant.enabled`
|
||||
|
||||
## 1. Shared collection
|
||||
|
||||
Every tenant's documents live in one collection, discriminated by a tenant field.
|
||||
|
||||
`MongoTenantContext` carries the tenant. `MongoTenantPredicateInjector` adds the tenant predicate to
|
||||
every query, every atomic filter and the **first** aggregation stage. `TenantScopedMongoOperations`
|
||||
is the entry point, so a caller cannot construct an unscoped operation by forgetting.
|
||||
|
||||
Three details are load-bearing:
|
||||
|
||||
- **Injection, not convention.** A tenant predicate that each query is expected to add itself is a
|
||||
cross-tenant leak waiting for one missed `where(...)`. The injector adds it structurally.
|
||||
- **First aggregation stage.** `firstStageMatch(...)` places the tenant `$match` before anything else.
|
||||
A `$lookup` or `$group` that runs before the tenant filter has already crossed the boundary, even if
|
||||
a later stage filters the output.
|
||||
- **An absent tenant is not "all tenants".** The injector takes an `Optional<MongoTenantContext>` so
|
||||
the missing case is a decision the policy makes explicitly, not a predicate that quietly disappears.
|
||||
|
||||
`MongoTenantManifestValidator.validate(manifest, tenantScopedUniqueIndexes)` checks that every unique
|
||||
index that should be per-tenant actually includes the tenant field. A unique index on `email` alone in
|
||||
a shared collection makes an email globally unique across tenants — tenant B cannot register an
|
||||
address tenant A already used, which is both a bug and an information leak.
|
||||
|
||||
`requireShardKeyAnalysed(...)` requires a shard-key readiness report before a shared-collection tenant
|
||||
model is sharded: tenant id as a shard key prefix concentrates the largest tenant on one shard.
|
||||
|
||||
### Observability
|
||||
|
||||
`tenantId` and `rawTenantId` are on `MongoObservationConvention`'s forbidden tag list. Cardinality
|
||||
grows with the customer list, and the tag ships tenant identity into the metrics backend.
|
||||
|
||||
## 2. Database per tenant (Experimental)
|
||||
|
||||
`MongoTenantDatabaseResolver` maps a tenant to its database; `MongoTenantClientRegistry` holds the
|
||||
clients.
|
||||
|
||||
Experimental for a specific reason: it is correct in the small and unbounded in the large. Each tenant
|
||||
database costs connections, file handles and monitoring cardinality. It works beautifully at 20
|
||||
tenants and falls over at 2,000, and nothing in a functional test distinguishes the two. Promotion
|
||||
requires operational scale evidence.
|
||||
|
||||
`MongoTenantLifecyclePolicy`:
|
||||
|
||||
- `requireActivationReady(tenantKey, schemaAndIndexesValidated)` — a tenant is not activated until its
|
||||
schema and indexes are validated. Activating first means the first customer request is the migration
|
||||
test.
|
||||
- `requireDeleteAllowed(...)` — deletion requires an explicit retention decision. Dropping a tenant
|
||||
database is irreversible and takes the backup surface with it.
|
||||
|
||||
`MongoTenantMigrationCoordinator` runs a migration across tenant databases with per-tenant results.
|
||||
Partial failure is normal and must be reported per tenant: "migration failed" across 500 databases is
|
||||
not a report anyone can act on.
|
||||
|
||||
## 3. Choosing
|
||||
|
||||
| | Shared collection | Database per tenant |
|
||||
|---|---|---|
|
||||
| Isolation | Logical, enforced by injection | Physical |
|
||||
| Tenant count | Unbounded | Bounded by connections and file handles |
|
||||
| Per-tenant restore | Hard | Natural |
|
||||
| Noisy neighbour | Shared resources | Isolated |
|
||||
| Migration | One collection | N databases, partial failures |
|
||||
| Cross-tenant query | Possible (and must be forbidden) | Structurally impossible |
|
||||
|
||||
Shared collection is the default. Database-per-tenant is for a small number of tenants with a
|
||||
contractual isolation or per-tenant-restore requirement.
|
||||
|
||||
## 4. Failure recovery
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| Cross-tenant data visible | An operation bypassed `TenantScopedMongoOperations` | Treat as a security incident. Find the path, close it, audit access. |
|
||||
| Unique constraint fires across tenants | Unique index missing the tenant field | Rebuild the index with the tenant field as prefix; the validator catches this before it ships. |
|
||||
| One shard holds most data | Tenant id as shard-key prefix with a dominant tenant | Refine the shard key with a high-cardinality suffix. |
|
||||
| Connection exhaustion | Database-per-tenant beyond the connection budget | The scale limit. Consolidate or move to shared collections. |
|
||||
| Migration partially applied across tenants | Normal | `MongoTenantMigrationCoordinator` reports per tenant; re-run for the failures only. |
|
||||
|
||||
## 5. Promotion evidence
|
||||
|
||||
Shared-collection tenancy: actual-topology evidence, a security review covering cross-tenant access,
|
||||
a migration path, failure cases (injection proven on query, atomic filter and first aggregation
|
||||
stage), this runbook. Database-per-tenant additionally requires **operational scale evidence** and
|
||||
stays Experimental until it exists.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Advanced — Search and Vector Search
|
||||
|
||||
**Capabilities:** `MongoCapability.SEARCH`, `MongoCapability.VECTOR_SEARCH`
|
||||
**Properties:** `ca-skeleton.persistence-mongo.advanced.search.enabled`,
|
||||
`ca-skeleton.persistence-mongo.advanced.vector-search.enabled`
|
||||
**Status:** Experimental (design §3.3). Hybrid search likewise.
|
||||
|
||||
## Requirements
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Topology | A deployment with the search service. Atlas Local in a container is a pull-request convenience and is **not** release evidence. |
|
||||
| Privilege | `MongoPrincipalRole.SEARCH_ADMIN` for index management; the application role queries only. |
|
||||
| Gate | `MongoAtlasCapabilityContractSuite` on the actual target deployment. |
|
||||
|
||||
## 1. Created is not ready
|
||||
|
||||
`MongoSearchIndexState`: `CREATED` → `BUILDING` → `READY`, plus `FAILED` and `DELETING`.
|
||||
|
||||
`MongoSearchReadinessGate.requireReady(state)` refuses anything but `READY`. A search index is built
|
||||
asynchronously: the create call returns immediately and the index answers queries with *partial*
|
||||
results while building. Not an error, not empty — partial. A deployment that creates an index and
|
||||
starts querying serves incomplete results for as long as the build takes, and nothing reports it.
|
||||
|
||||
## 2. Search indexes are not application-owned
|
||||
|
||||
`MongoSearchIndexDescriptor.metadataOwnership()` is `MongoMetadataOwnership.SEARCH_MANAGED`, and
|
||||
`droppableByApplicationDrift()` is false. The index reconciliation described in
|
||||
[ADR-MONGO-004](../../adr/ADR-MONGO-004-index-schema-admin-plane.md) must not drop it.
|
||||
|
||||
## 3. Query guardrails
|
||||
|
||||
`MongoSearchQuery` binds an index, an allowlist of paths, the search text and a result limit.
|
||||
|
||||
- `requireAllowedPaths(allowed)` raises `MongoOperationRejectedException` on a path outside the
|
||||
allowlist. Without it, a caller can search any indexed field, including ones indexed for a
|
||||
different purpose.
|
||||
- Search text length and result count are bounded at construction. An unbounded search text is a
|
||||
cost multiplier on someone else's service.
|
||||
|
||||
## 4. Vector search
|
||||
|
||||
`MongoVectorIndexDescriptor.cosine(path, dimensions)` declares the index.
|
||||
`MongoEmbedding.forIndex(index, values)` binds an embedding to it and **rejects a dimension
|
||||
mismatch** — a 1536-dimension embedding against a 768-dimension index is not a runtime degradation,
|
||||
it is a category error, and catching it at construction beats catching it as a confusing server
|
||||
message.
|
||||
|
||||
`MongoEmbedding` copies its backing array in and out. A vector that shares an array with its caller
|
||||
can be mutated after the query is built, which produces a query nobody wrote.
|
||||
|
||||
`MongoVectorQuery` requires `numCandidates > limit` — searching 10 candidates to return 10 results is
|
||||
an exhaustive scan wearing an ANN index's name. `MongoVectorQuery.nearest(embedding, 10)` uses the
|
||||
standard 20× ratio (200 candidates for 10 results).
|
||||
|
||||
## 5. Relevance is the gate, not functionality
|
||||
|
||||
`MongoVectorSearchBenchmarkGate.standard()` requires **recall** alongside latency and index size.
|
||||
`requiredEvidence()` names recall explicitly.
|
||||
|
||||
This is the difference between search and everything else in the platform. A vector index can be
|
||||
functionally perfect — it accepts the index, accepts the query, returns k results, within the latency
|
||||
budget — and return the wrong k. A gate that measures only latency certifies a fast wrong answer.
|
||||
`failures(recall, latencyMs, indexMb)` reports which dimension failed so the finding is actionable.
|
||||
|
||||
## 6. Failure recovery
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| Incomplete results after a deploy | Queried a `BUILDING` index | Wait for `READY`. The gate prevents this; if it fired, something bypassed it. |
|
||||
| `MongoOperationRejectedException` on a path | Path not in the allowlist | Add it deliberately, or fix the caller. |
|
||||
| Dimension mismatch | Model changed | A new model means a new index. Build alongside, cut over, then retire. |
|
||||
| Recall dropped without a code change | The index was rebuilt with different parameters, or the data distribution shifted | Re-run the benchmark gate; treat a recall regression like a failing test. |
|
||||
| Index `FAILED` | Build error on the search service | Search-side diagnosis; the application must not fall back to a scan silently. |
|
||||
|
||||
## 7. Promotion evidence
|
||||
|
||||
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md): the actual target
|
||||
deployment (not Atlas Local), security review of `SEARCH_ADMIN`, a rebuild path, failure cases
|
||||
(non-ready index refused, disallowed path refused, dimension mismatch refused), this document as the
|
||||
runbook, **and** relevance evidence. Search and vector search do not promote on functional success.
|
||||
@@ -0,0 +1,82 @@
|
||||
# Advanced — Sharding
|
||||
|
||||
**Capability:** `MongoCapability.SHARDING`
|
||||
**Property:** `ca-skeleton.persistence-mongo.advanced.sharding.enabled`
|
||||
**Status:** Advanced. Reshard orchestration remains Experimental.
|
||||
|
||||
## Requirements
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Topology | A real sharded cluster. A replica set cannot exercise routing. |
|
||||
| Server | MongoDB 7.0 or 8.0. |
|
||||
| Privilege | `MongoPrincipalRole.SHARD_ADMIN` for the admin plane; the application role is unchanged. |
|
||||
| Gate | `mongoShardedTest` lane with `MongoShardingContractSuite`. |
|
||||
|
||||
## Shard key
|
||||
|
||||
`ShardKeyDescriptor` declares the key as an ordered list of `ShardKeyPart` plus a `ShardStrategy`:
|
||||
|
||||
| Strategy | Distributes | Cost |
|
||||
|---|---|---|
|
||||
| `RANGE` | by value ranges | Range queries stay targeted; a monotonic key (a timestamp, an `ObjectId`) sends every insert to one shard. |
|
||||
| `HASHED` | by hash of the key | Inserts spread evenly; every range query becomes scatter-gather. |
|
||||
|
||||
There is no strategy that is good at both, which is why the choice is a declaration rather than a
|
||||
default.
|
||||
|
||||
## Routing classification
|
||||
|
||||
`ShardAwareQueryValidator` classifies each query before execution:
|
||||
|
||||
| `MongoRoutingClassification` | Meaning |
|
||||
|---|---|
|
||||
| `TARGETED` | The full shard key is present. One shard answers. |
|
||||
| `PREFIX_TARGETED` | A prefix of a compound key is present. A subset of shards answers. |
|
||||
| `SCATTER_GATHER` | No shard-key predicate. Every shard answers. |
|
||||
| `REJECTED` | Scatter-gather where the profile forbids it. |
|
||||
|
||||
A scatter-gather query is not an error — some queries legitimately need every shard — but it must be
|
||||
declared. Undeclared scatter-gather raises `MongoShardRoutingException`. The reason is that
|
||||
scatter-gather passes every test on a single-shard development cluster and only degrades once the
|
||||
cluster grows, at which point the query is already in production and the fix is a schema change.
|
||||
|
||||
## Unsupported combinations
|
||||
|
||||
- Unique index on a field that is not a prefix of the shard key. MongoDB cannot enforce it across
|
||||
shards, and it fails at index creation, not at query time.
|
||||
- Transactions that touch documents on multiple shards remain supported but cost a cross-shard
|
||||
two-phase commit. Prefer a shard key that keeps a transaction's documents co-located.
|
||||
- CSFLE on a sharded collection: see [encryption.md](encryption.md) for the combinations that are
|
||||
refused.
|
||||
|
||||
## Admin plane
|
||||
|
||||
`MongoShardingAdminGateway` (D4, `SHARD_ADMIN` credential) covers shard-collection, refine-shard-key
|
||||
and reshard.
|
||||
|
||||
`ShardKeyAnalyzer` produces a `ShardKeyReadinessReport` before sharding a collection: cardinality,
|
||||
frequency skew and monotonicity. A key with low cardinality creates jumbo chunks that cannot be split;
|
||||
a monotonic key creates a hot shard. Both are visible in the report and invisible in a functional
|
||||
test.
|
||||
|
||||
`ReshardApproval` is required for a reshard — a named approver and a stated window. Resharding
|
||||
rewrites the collection: it duplicates the data during the operation and saturates IO. It is not a
|
||||
runtime operation and the type refuses to pretend otherwise.
|
||||
|
||||
## Failure recovery
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| `MongoShardRoutingException` | Undeclared scatter-gather | Add the shard key to the predicate, or declare the query as scatter-gather in its profile after review. |
|
||||
| Jumbo chunks | Low-cardinality shard key | Refine the shard key (adds a suffix, non-destructive) before considering a reshard. |
|
||||
| One hot shard | Monotonic range key | Refine with a high-cardinality prefix, or reshard to hashed if range queries are not needed. |
|
||||
| Balancer never converges | Chunk migration blocked by long-running operations | Check for long transactions and cursors; the balancer waits on them. |
|
||||
|
||||
## Promotion evidence
|
||||
|
||||
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md): actual sharded-cluster
|
||||
evidence, security review of the `SHARD_ADMIN` role, a migration path for an existing unsharded
|
||||
collection, failure cases (undeclared scatter-gather refused, jumbo chunk detected), and this
|
||||
document as the runbook. Reshard orchestration stays Experimental until operational scale evidence
|
||||
exists.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Advanced — Time Series
|
||||
|
||||
**Capability:** `MongoCapability.TIME_SERIES`
|
||||
**Property:** `ca-skeleton.persistence-mongo.advanced.time-series.enabled`
|
||||
**Status:** Advanced.
|
||||
|
||||
## Requirements
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Topology | Replica set or sharded cluster. |
|
||||
| Server | MongoDB 7.0 or 8.0. |
|
||||
| Privilege | Standard application role; collection creation goes through the admin plane. |
|
||||
|
||||
## Descriptor
|
||||
|
||||
`MongoTimeSeriesDescriptor` declares:
|
||||
|
||||
- **timeField** — required, a BSON date. This is the bucketing axis.
|
||||
- **metaField** — optional but nearly always wanted: the series identity (device id, tenant, sensor).
|
||||
Documents sharing a `metaField` value bucket together, which is where the compression comes from.
|
||||
- **granularity** — `MongoTimeSeriesGranularity`:
|
||||
|
||||
| Granularity | Bucket span | Use for |
|
||||
|---|---|---|
|
||||
| `SECONDS` | 1 hour | Sub-second to per-second ingest. |
|
||||
| `MINUTES` | 24 hours | Per-minute metrics. |
|
||||
| `HOURS` | 30 days | Hourly rollups. |
|
||||
|
||||
Granularity that is too fine produces many small buckets and loses the compression; too coarse
|
||||
produces oversized buckets that must be read whole to answer a narrow query.
|
||||
|
||||
## What a time series collection is not
|
||||
|
||||
`MongoTimeSeriesCapabilityValidator` refuses the operations the collection type does not support, at
|
||||
declaration time rather than at first use:
|
||||
|
||||
- **No arbitrary updates.** Time series data is append-mostly. Delete and limited update support
|
||||
exists on recent servers but is not part of this platform's contract.
|
||||
- **No unique index on the measurement.** There is no `_id` to be unique on in the usual sense.
|
||||
- **No CSFLE.** Refused — see [encryption.md](encryption.md).
|
||||
- **No change stream on the raw buckets** as a business event source. The bucket documents are a
|
||||
storage representation, not your measurements.
|
||||
|
||||
Converting an existing regular collection to a time series collection is a copy, not an alter. Plan
|
||||
it as a migration with a dual-write window.
|
||||
|
||||
## TTL
|
||||
|
||||
Time series collections use `expireAfterSeconds` on the collection rather than a TTL index on a
|
||||
field. The [TTL rules](../schema-index-migration-guide.md#4-ttl) still apply: expiry is physical
|
||||
cleanup on a bucket boundary, so a measurement can outlive its expiry by up to a bucket span plus the
|
||||
monitor interval. Do not treat absence as a deadline.
|
||||
|
||||
## Operations
|
||||
|
||||
`MongoTimeSeriesOperations` is the port for insert and windowed read. Reads are bounded by the same
|
||||
`MongoOperationBudget` as everything else: an unbounded time-range query on a time series collection
|
||||
is the fastest way to read a year of data into heap.
|
||||
|
||||
## Failure recovery
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| Writes rejected with an unsupported-operation error | An update or unique-index expectation | The collection type does not support it; change the access pattern. |
|
||||
| Poor compression / large storage | Missing `metaField`, or granularity too fine | Both require a rebuild; measure on a copy before committing. |
|
||||
| Slow range queries | Granularity too coarse for the query window | Same: rebuild with the granularity matched to the dominant query. |
|
||||
|
||||
## Promotion evidence
|
||||
|
||||
Actual-topology evidence on the target deployment, a migration path from the existing collection,
|
||||
failure cases (unsupported update refused, CSFLE combination refused), and this document as the
|
||||
runbook.
|
||||
@@ -0,0 +1,92 @@
|
||||
# BSON Mapping Guide
|
||||
|
||||
Design §10 and decision D-06. The representation of a value in BSON is a data contract, not an
|
||||
implementation detail: once a collection holds a million documents, changing how a `BigDecimal` is
|
||||
stored is a migration with downtime, not a code change. `MongoTypeRepresentationManifest` pins the
|
||||
representation so a library upgrade or a different default cannot move it.
|
||||
|
||||
## 1. The manifest
|
||||
|
||||
`MongoTypeRepresentationManifest.standard()` fixes:
|
||||
|
||||
| Java type | BSON | Representation type |
|
||||
|---|---|---|
|
||||
| `UUID` | `Binary` subtype 4 | `MongoUuidRepresentation.STANDARD` |
|
||||
| `BigDecimal` | `Decimal128` | `MongoDecimalRepresentation.DECIMAL_128` |
|
||||
| `BigInteger` | `Decimal128` (or `String` when out of range, declared) | `MongoBigIntegerRepresentation` |
|
||||
| `Instant` / `OffsetDateTime` / `ZonedDateTime` | UTC `Date` | `MongoTemporalRepresentation.UTC_DATE` |
|
||||
| `LocalDate` | `String` (ISO-8601) or UTC `Date`, declared per field | `MongoTemporalRepresentation` |
|
||||
| `enum` | `String` name | `MongoEnumRepresentation.NAME` |
|
||||
|
||||
`MongoMappingConfiguration` and `MongoCustomConversionsFactory` build the Spring Data converters from
|
||||
the manifest, so there is one place to read and one place to change.
|
||||
|
||||
## 2. UUID
|
||||
|
||||
`UuidRepresentation.STANDARD` (subtype 4), always. The driver's legacy Java representation
|
||||
(subtype 3) byte-swaps two halves of the UUID, so a document written by one representation and read
|
||||
by the other yields a different — and valid-looking — UUID. Nothing errors; you just get the wrong
|
||||
row. The golden snapshot kit pins the codec explicitly for this reason
|
||||
(`MongoBsonSnapshot.defaultRegistry()`).
|
||||
|
||||
## 3. Decimal
|
||||
|
||||
`BigDecimal` → `Decimal128`, never `Double`. `12.30` stored as a double is `12.299999999999999`, and
|
||||
a monetary comparison written against it will one day be wrong by a cent for a customer who notices.
|
||||
`BigDecimalToDecimal128Converter` / `Decimal128ToBigDecimalConverter` are registered from the
|
||||
manifest.
|
||||
|
||||
`Decimal128` has 34 significant digits; a `BigDecimal` beyond that range fails on write rather than
|
||||
rounding silently.
|
||||
|
||||
## 4. Time
|
||||
|
||||
Store instants, not local times. `LocalDateTimeMappingGuard` refuses `LocalDateTime` fields on a
|
||||
mapped document: a `LocalDateTime` has no offset, so the value that goes in depends on the JVM
|
||||
default zone of whichever instance wrote it, and the two instances in a rolling deploy can disagree.
|
||||
Use `Instant` when the moment matters and `LocalDate` when the calendar day matters.
|
||||
|
||||
## 5. Type metadata
|
||||
|
||||
`MongoTypeMetadataPolicy` decides what goes in `_class`:
|
||||
|
||||
| Policy | Stored | Use when |
|
||||
|---|---|---|
|
||||
| `NONE` | nothing | The collection holds exactly one type and never will hold a subtype. |
|
||||
| `ALIAS` | a registered short alias | A polymorphic hierarchy in a long-lived collection. |
|
||||
| `CLASS_NAME` | the FQCN | Short-lived or internal collections only. |
|
||||
|
||||
`PolicyAwareMongoTypeMapper` enforces it, and `MongoTypeMetadataRegistry` holds alias → class.
|
||||
A `@LongLivedMongoDocument` type with `CLASS_NAME` is refused: writing `com.example.OrderV2` into a
|
||||
million documents means that renaming the package is a data migration.
|
||||
|
||||
## 6. Missing versus null
|
||||
|
||||
The golden kit keeps these apart deliberately. `MongoBsonSnapshotAssert.hasNoField(...)` and
|
||||
`hasExplicitNull(...)` are different assertions, because in MongoDB they are different documents:
|
||||
`{"a": null}` matches `{a: null}` and `{a: {$exists: true}}`, while `{}` matches only the first.
|
||||
A mapper change that starts writing explicit nulls silently changes what your queries return.
|
||||
|
||||
## 7. Golden representation tests
|
||||
|
||||
Every collection with a fixed representation should have a snapshot test:
|
||||
|
||||
```java
|
||||
MongoBsonSnapshot snapshot = MongoBsonSnapshot.of(storedDocument);
|
||||
MongoBsonSnapshotAssert.assertThat(snapshot)
|
||||
.hasBsonType("amount", "DECIMAL128")
|
||||
.hasBsonType("externalId", "BINARY")
|
||||
.hasNoJavaClassName("dev.caskeleton")
|
||||
.hasTypeSignature("_id:OBJECT_ID,amount:DECIMAL128,createdAt:DATE_TIME,externalId:BINARY");
|
||||
```
|
||||
|
||||
`hasTypeSignature` is the regression gate: it fails on *any* representation change, including ones a
|
||||
value-equality assertion would pass. When it fails, the question is whether the change was intended
|
||||
and has a migration — not whether to update the string.
|
||||
|
||||
## 8. Round trips
|
||||
|
||||
`MongoRoundTripContract` asserts that `write → read` returns an equal domain object *and* that
|
||||
`write → read → write` produces an identical BSON document. The second half is what catches an
|
||||
asymmetric converter: a value that reads back equal but re-serialises differently makes every
|
||||
subsequent `save()` a spurious update, and turns change streams into a noise generator.
|
||||
@@ -0,0 +1,105 @@
|
||||
# Change Stream Guide
|
||||
|
||||
Design §20, decision D-12. A change stream is an **at-least-once projector**, not an event bus.
|
||||
|
||||
## 1. What a change stream is not
|
||||
|
||||
D-12 is explicit: a physical change event is not a business integration event. The two differ in
|
||||
ways that matter to every consumer:
|
||||
|
||||
| Change event | Integration event |
|
||||
|---|---|
|
||||
| Emitted per document write | Emitted per business fact |
|
||||
| Shape follows the storage schema | Shape is a published contract |
|
||||
| A refactor of the document changes it | A refactor of the document does not change it |
|
||||
| Replayed on resume, duplicated on retry | Versioned and deliberately evolved |
|
||||
|
||||
Publishing raw change events externally makes your storage schema a public API, and the first time
|
||||
someone renames a field the downstream consumers break. If you need to bridge to messaging, use the
|
||||
Advanced bridge, which maps to an owned envelope
|
||||
([advanced/multi-tenancy.md](advanced/multi-tenancy.md) is separate;
|
||||
the bridge is described in §7 below).
|
||||
|
||||
## 2. Subscription and resume
|
||||
|
||||
`MongoChangeStreamSubscription` declares the collection, pipeline and consistency. `MongoResumePosition`
|
||||
is either a resume token or a cluster time; `MongoResumeCheckpoint` is what gets persisted and
|
||||
`MongoResumeCheckpointStore` persists it.
|
||||
|
||||
The checkpoint stores the token as a Base64 `encodedToken` string rather than a byte array — a record
|
||||
with an array component has broken equality, and a checkpoint that does not compare correctly is a
|
||||
checkpoint that silently fails its own dedup test.
|
||||
|
||||
## 3. Checkpoint after processing, not after receiving
|
||||
|
||||
The ordering rule that makes at-least-once actually hold:
|
||||
|
||||
```
|
||||
receive event
|
||||
→ process it (idempotently)
|
||||
→ persist the checkpoint
|
||||
```
|
||||
|
||||
Checkpointing on receipt turns the delivery guarantee into at-most-once, and the events lost are
|
||||
exactly the ones the process died while handling.
|
||||
|
||||
## 4. Idempotency
|
||||
|
||||
`MongoChangeEventIdentity` is the dedup key: `(resumeToken, documentKey, clusterTime, operationType)`.
|
||||
`MongoChangeDeduplicationStore` records what has been applied. Duplicates are not an edge case — every
|
||||
resume after any interruption replays at least one event, so a projector that is not idempotent is
|
||||
wrong on its first restart, not on some rare day.
|
||||
|
||||
`MongoChangeProjector` returns a `MongoChangeProjectionResult` so the runner can distinguish applied
|
||||
from skipped-as-duplicate, and the skip count is worth a metric: a sudden rise means something is
|
||||
looping.
|
||||
|
||||
## 5. States and recovery
|
||||
|
||||
`MongoChangeStreamState`: `STARTING`, `RUNNING`, `RESUMING`, `STOPPED`, `HISTORY_LOST`.
|
||||
|
||||
`MongoChangeStreamRecoveryPolicy` returns a `MongoChangeStreamRecoveryDecision`, which is either
|
||||
`resume()` (auto-resume from the checkpoint) or `halt(state, runbook)`. A halting decision **must**
|
||||
name a runbook — a decision that only says "stopped" leaves the on-call engineer to work out from
|
||||
scratch whether the projection can be rebuilt and from what.
|
||||
|
||||
| Situation | Decision |
|
||||
|---|---|
|
||||
| Transient network error, token still valid | `resume()` |
|
||||
| Primary failover | `resume()` — the token survives an election |
|
||||
| `invalidate` (collection dropped/renamed) | `halt(STOPPED, …)` → `MongoInvalidateRecovery` |
|
||||
| Token no longer in the oplog | `halt(HISTORY_LOST, "history-lost")` → `MongoChangeHistoryLostException` |
|
||||
|
||||
## 6. History lost
|
||||
|
||||
`MongoChangeHistoryLostException` is raised when the resume token predates the oldest oplog entry.
|
||||
The stream **cannot** be resumed: the events between the checkpoint and now are gone from the server,
|
||||
and no amount of retrying brings them back.
|
||||
|
||||
What the platform will not do is silently restart from "now". That looks like a recovery and is
|
||||
actually a silent gap in the projection — the worst possible outcome, because nothing reports it. The
|
||||
runner halts and requires an operator decision. See
|
||||
[runbooks/history-lost.md](runbooks/history-lost.md).
|
||||
|
||||
## 7. Bridging to messaging (Advanced)
|
||||
|
||||
`MongoChangeMessagingBridge` is opt-in behind `MongoCapability.CHANGE_STREAM` plus the bridge's own
|
||||
flag. It maps a change event to a platform-owned `MongoIntegrationEventEnvelope` through
|
||||
`MongoChangeToIntegrationEventMapper` and hands it to a `MongoIntegrationEventPublisher` port.
|
||||
|
||||
The port is defined in the bridge package rather than imported from the messaging adapter because the
|
||||
architecture registry forbids adapter-to-adapter dependencies; the composition root supplies the
|
||||
implementation.
|
||||
|
||||
`MongoBridgeOutboxPolicy` and `MongoBridgeCheckpointPolicy` state the delivery contract: publish then
|
||||
checkpoint, at-least-once, consumers must dedup on the envelope's event id.
|
||||
|
||||
## 8. Operating notes
|
||||
|
||||
- Change streams require a replica set. `MongoStartupValidator` refuses a change-stream profile on
|
||||
`STANDALONE`.
|
||||
- The change-stream principal is its own role (`MongoPrincipalRole.CHANGE_STREAM`) with
|
||||
`changeStream` and `find` — not the application write credential.
|
||||
- Oplog window is the recovery budget. If the oplog holds four hours, a consumer that is down for five
|
||||
hours needs a rebuild, not a resume. Alert on consumer lag against the oplog window, not against
|
||||
wall-clock.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Consistency and Transaction Guide
|
||||
|
||||
Design §12–§16, decisions D-07 through D-10. This is the part of the platform where the wrong
|
||||
default is most expensive and the least visible in testing, because every failure mode here needs a
|
||||
primary change to reproduce.
|
||||
|
||||
## 1. Prefer a single-document atomic operation
|
||||
|
||||
D-09: a transaction is for a **multi-document invariant**, nothing else. A single document is already
|
||||
atomic in MongoDB, so wrapping a one-document update in a transaction buys nothing and costs a
|
||||
session, a two-phase commit and a new ambiguous outcome.
|
||||
|
||||
D-07: partial change uses update operators, not `save()`. `MongoAtomicOperations` /
|
||||
`MongoAtomicOperationsTemplate` expose the operator set through `MongoUpdateOperator`
|
||||
(`$set`, `$inc`, `$push`, `$pull`, `$addToSet`, `$min`, `$max`, `$currentDate`, …) with an
|
||||
`AtomicFilter` precondition and a `ReturnDocumentMode`. Read-modify-write through `save()` replaces
|
||||
the whole document and silently discards any field another writer changed in between — a lost update
|
||||
with no error.
|
||||
|
||||
## 2. Whole-document replacement needs a revision
|
||||
|
||||
D-08. `VersionedMongoUpdater` requires a `MongoRevision`: either a Spring Data `@Version` field or an
|
||||
explicit expected-revision predicate in `VersionedUpdateCommand`. A replacement whose filter matched
|
||||
zero documents is not "nothing to do" — `MongoOptimisticConflictTranslator` distinguishes:
|
||||
|
||||
- filter matched nothing and the id does not exist → `MongoDocumentNotFoundException`
|
||||
- filter matched nothing and the id exists → `MongoOptimisticConflictException`
|
||||
|
||||
Collapsing these two into one is how a concurrent overwrite becomes a 404.
|
||||
|
||||
## 3. Consistency profiles
|
||||
|
||||
`MongoConsistencyProfile` names the read/write concern pair; `MongoConsistencyRegistry` binds a
|
||||
profile to an operation or collection, and `MongoConsistencyBinder` /
|
||||
`ReactiveMongoConsistencyBinder` apply it at execution.
|
||||
|
||||
| Profile | Meaning | Use for |
|
||||
|---|---|---|
|
||||
| `PRIMARY_LOCAL` | primary read, local concern | Throughput-sensitive reads that tolerate a rollback window. |
|
||||
| `PRIMARY_MAJORITY` | primary read, majority write | The default for anything a user will see again immediately. |
|
||||
| `CAUSAL_MAJORITY` | majority inside a causal session | Read-your-writes across separate operations. |
|
||||
| `STALE_READ_ALLOWED` | secondary reads permitted | Reporting and analytics that state their staleness. |
|
||||
| `SNAPSHOT_TRANSACTION` | snapshot isolation | Multi-document reads inside a transaction. |
|
||||
|
||||
A profile is a declaration, not a hint: the registry is consulted per operation and an operation
|
||||
without a registered profile is rejected rather than defaulting.
|
||||
|
||||
## 4. Causal sessions
|
||||
|
||||
`MongoCausalSessionContext` plus `SpringMongoCausalSessionExecutor` /
|
||||
`ReactiveMongoCausalSessionExecutor` carry the cluster time and operation time between operations, so
|
||||
"write then read" returns the write even when the read lands on a different node. Without a causal
|
||||
session, `PRIMARY_MAJORITY` gives you durability but not read-your-writes across two calls.
|
||||
|
||||
In the reactive path the session travels in the Reactor context (`ReactiveMongoContextKeys`), not in
|
||||
a thread local — a thread local is empty on the next operator in the chain.
|
||||
|
||||
## 5. Transactions
|
||||
|
||||
`MongoTransactionExecutor` / `ReactiveMongoTransactionExecutor` open a session through the session
|
||||
factory, run the body, and commit. `MongoTransactionProfile` carries the consistency profile, the
|
||||
`maxCommitTime` and the retry budget. Topology matters: a transaction requires a replica set, and
|
||||
`MongoStartupValidator` refuses a transaction-declaring profile on `STANDALONE` at startup rather
|
||||
than at the first call.
|
||||
|
||||
## 6. Retry: body and commit are different loops
|
||||
|
||||
D-10, and the single most consequential rule in the design.
|
||||
|
||||
```
|
||||
for each body attempt:
|
||||
open a NEW session
|
||||
run the body
|
||||
TransientTransactionError -> abort, next body attempt
|
||||
commit
|
||||
UnknownTransactionCommitResult -> retry COMMIT ONLY, same session
|
||||
```
|
||||
|
||||
`MongoTransactionRetryCoordinator` implements exactly this:
|
||||
|
||||
- **A new session per body attempt.** Reusing the session after an abort carries the aborted
|
||||
transaction's state into the retry.
|
||||
- **The body is never replayed after a commit ambiguity.** An unknown commit means the commit may
|
||||
already have applied. Re-running the body would apply it a second time. Only the commit is retried,
|
||||
and a commit retry on an already-committed transaction is a no-op by design.
|
||||
- **A budget bounds both loops.** `MongoRetryBudget` limits attempts *and* elapsed time, with jittered
|
||||
backoff (`delayBefore(attempt, random)`), so a struggling primary is not retried into the ground.
|
||||
|
||||
`MongoRetryDecision` and `MongoRetryScope` (in `…api.error`) say what may be retried:
|
||||
`MongoRetryScope.BODY`, `COMMIT_ONLY`, or `NONE`.
|
||||
|
||||
## 7. Ambiguous outcomes
|
||||
|
||||
`MongoExecutionOutcome` has six values, two of which are ambiguous and must not be collapsed:
|
||||
|
||||
| Outcome | Did the write happen? |
|
||||
|---|---|
|
||||
| `NOT_SENT` | No. Safe to retry. |
|
||||
| `NO_WRITE_PERFORMED` | No — the server answered and did nothing. |
|
||||
| `WRITE_CONFIRMED` | Yes. |
|
||||
| `PARTIAL_BULK_WRITE` | Some of it. See `MongoBulkResult`. |
|
||||
| `WRITE_RESULT_UNKNOWN` | **Unknown.** |
|
||||
| `TRANSACTION_COMMIT_UNKNOWN` | **Unknown.** |
|
||||
|
||||
An unknown outcome is not a failure and must not be reported to a caller as one. The caller either
|
||||
reconciles (`MongoCommitReconciler` re-reads a deterministic marker the body wrote) or surfaces the
|
||||
ambiguity. See [runbooks/unknown-commit.md](runbooks/unknown-commit.md).
|
||||
|
||||
`MongoFailureContext` records only the design-permitted fields — outcome, category, operation name,
|
||||
collection profile, retry scope, attempt — never the query, the document, or the values.
|
||||
|
||||
## 8. Failure translation
|
||||
|
||||
`DefaultMongoFailureClassifier` classifies **labels before codes**. The server's error labels
|
||||
(`TransientTransactionError`, `UnknownTransactionCommitResult`, `RetryableWriteError`) are the
|
||||
authoritative statement about retryability; an error code is a secondary signal whose meaning varies
|
||||
by server version. `DefaultMongoFailureTranslator` maps a classification onto the stable exception
|
||||
hierarchy, and anything unmatched becomes `MongoUnclassifiedFailureException` rather than leaking a
|
||||
driver type.
|
||||
|
||||
## 9. Bulk writes
|
||||
|
||||
`MongoBulkExecutor` returns a `MongoBulkResult` with per-item `MongoBulkItemFailure` entries. An
|
||||
unordered bulk write that partially fails is `PARTIAL_BULK_WRITE`, not a failure: some documents were
|
||||
written. `MongoBulkPartialFailureException` carries the succeeded and failed indexes so a caller can
|
||||
resume rather than replay.
|
||||
@@ -0,0 +1,85 @@
|
||||
# Document Modeling Guide
|
||||
|
||||
Design §7–§9. The platform does not own your documents — D-01 is explicit that the domain owns
|
||||
`@Document`, repositories, queries, index requirements and schema version. What the platform owns is
|
||||
the set of modeling decisions that are expensive to reverse once a collection holds production data.
|
||||
|
||||
## 1. There is no `CommonMongoRepository`
|
||||
|
||||
A generic `CommonMongoRepository<T, ID>` is listed under explicitly unsupported (§3.4), and the
|
||||
reason is not purity. A shared supertype forces every collection to share an id strategy, a
|
||||
consistency profile and a query surface, and the first collection that needs a different one either
|
||||
gets a cast or a leaky generic parameter. Declare a Spring Data repository per aggregate.
|
||||
|
||||
## 2. Embed or reference
|
||||
|
||||
`MongoDocumentModelManifest` records the decision per collection so it is reviewable, and
|
||||
`MongoDocumentModelValidator` refuses the combinations that do not survive growth.
|
||||
|
||||
| Descriptor | Use when |
|
||||
|---|---|
|
||||
| `EmbeddedCollectionDescriptor` | The child is read with the parent, is bounded, and has no independent lifecycle. Declare `maxElements`; an unbounded array is the single most common way a document reaches the size limit. |
|
||||
| `MongoReferenceDescriptor` | The child is queried independently, is unbounded, or outlives the parent. Declare `MongoReferenceLifecycle` so the deletion story is written down rather than discovered. |
|
||||
|
||||
The validator rejects an embedded collection without a bound, and a reference whose lifecycle says
|
||||
the child is owned by the parent but which is also referenced from elsewhere.
|
||||
|
||||
## 3. Size budget
|
||||
|
||||
`MongoDocumentSizeBudget`:
|
||||
|
||||
| Constant | Bytes | Meaning |
|
||||
|---|---|---|
|
||||
| `MONGODB_HARD_LIMIT_BYTES` | 16 MiB | MongoDB's own limit. |
|
||||
| `PLATFORM_CEILING_BYTES` | 4 MiB | The largest budget the platform will accept. |
|
||||
| `DEFAULT_BYTES` | 2 MiB | `MongoDocumentSizeBudget.standard()`. |
|
||||
|
||||
A budget above the ceiling is refused at construction. Budgeting to 16 MiB means the failing write
|
||||
is the first symptom, and by then the collection is already full of near-limit documents.
|
||||
|
||||
## 4. Identity
|
||||
|
||||
`DomainDocumentId` and `MongoIdRepresentation` fix how a domain identifier becomes `_id`. Pick the
|
||||
representation once per collection and record it in the manifest:
|
||||
|
||||
- `OBJECT_ID` — server-generated, monotonic, 12 bytes. Good default when the domain has no natural id.
|
||||
- `UUID_BINARY` — a domain UUID stored as `Binary` subtype 4 (`STANDARD`). Never store a UUID as a
|
||||
string "because it is easier to read"; it doubles the index size and loses the type.
|
||||
- `STRING` — a natural key that is genuinely a string (a slug, an external system's id).
|
||||
|
||||
An `_id` choice is effectively permanent: it is the shard key candidate, the resume-token join key
|
||||
and the pagination tie-breaker.
|
||||
|
||||
## 5. Schema version
|
||||
|
||||
Every long-lived collection carries `DocumentSchemaVersion`. `MongoSchemaVersionPolicy` and
|
||||
`MongoSchemaVersionRange` say which versions the running code can read; a document outside the range
|
||||
raises `MongoDataSchemaUnsupportedException` rather than being silently mapped with missing fields.
|
||||
|
||||
Write the range down before the migration, not after: the range is what lets old and new instances
|
||||
run at once during a rolling deploy.
|
||||
|
||||
## 6. Type metadata
|
||||
|
||||
`@LongLivedMongoDocument` marks a document whose stored type alias must not be a Java class name.
|
||||
`MongoTypeMetadataRegistry` maps alias → class. Storing the FQCN means moving or renaming the class
|
||||
becomes a data migration; storing an alias keeps it a refactor. See
|
||||
[bson-mapping-guide.md](bson-mapping-guide.md) §4.
|
||||
|
||||
## 7. Collection profiles
|
||||
|
||||
`MongoCollectionProfileRegistry` binds a `CollectionProfileName` to its consistency profile, budget
|
||||
and allowlist. A collection that is not registered cannot be reached through
|
||||
`MongoImperativeExecutor` or `ReactiveMongoExecutor` — the allowlist is the mechanism that keeps an
|
||||
unreviewed collection from appearing in production by accident.
|
||||
|
||||
## 8. What to write down before the first insert
|
||||
|
||||
1. Embed/reference decision per child collection, with bounds.
|
||||
2. Size budget.
|
||||
3. `_id` representation.
|
||||
4. Schema version range.
|
||||
5. Index manifest (see [schema-index-migration-guide.md](schema-index-migration-guide.md)).
|
||||
6. Consistency profile (see [consistency-transaction-guide.md](consistency-transaction-guide.md)).
|
||||
|
||||
Each of these is cheap now and a migration later.
|
||||
@@ -0,0 +1,140 @@
|
||||
# Query and Aggregation Guide
|
||||
|
||||
Design §17–§19, decision D-11. Every query and every pipeline is a registered, bounded thing. Free-form
|
||||
JSON queries and unbounded pipelines are explicitly unsupported (§3.4).
|
||||
|
||||
## 1. Registered operations
|
||||
|
||||
Every execution carries a `MongoOperationContext`: a `MongoOperationName`, a `DatabaseProfileName`, a
|
||||
`CollectionProfileName`, a `MongoOperationType` and a `MongoOperationScope`.
|
||||
|
||||
`MongoOperationName` matches `[a-z][a-z0-9.-]{2,95}`. It is the join key for the budget registry, the
|
||||
consistency registry, the metric tag and the log line — a free-form or interpolated name breaks all
|
||||
four at once, which is why the pattern is enforced at construction.
|
||||
|
||||
`MongoOperationScope` uses an `UNSPECIFIED` sentinel rather than `null`, so "the caller did not say"
|
||||
is a value the policy layer can reject rather than an NPE further down.
|
||||
|
||||
## 2. Query guardrails
|
||||
|
||||
`PolicyAwareMongoQueryBuilder` builds a query from `MongoFieldDescriptor` + `MongoOperator` pairs
|
||||
against a `MongoQueryPolicy`. The policy refuses:
|
||||
|
||||
- a field not in the collection's allowlist
|
||||
- an operator not allowed for that field
|
||||
- a sort on an unindexed field
|
||||
- `$where`, `$expr` with arbitrary JavaScript, and server-side evaluation generally
|
||||
- an unbounded `$regex`
|
||||
|
||||
`MongoRegexPolicy` requires an anchored prefix pattern and bounds the pattern length. An unanchored
|
||||
regex is a collection scan wearing an index's clothes, and a user-supplied one is a denial-of-service
|
||||
primitive.
|
||||
|
||||
`MongoSortDescriptor` pairs a field with a direction and is validated against the index manifest, so
|
||||
a sort that would spill to disk fails review rather than production.
|
||||
|
||||
## 3. Operation budgets
|
||||
|
||||
`MongoOperationBudget` bounds four things at once:
|
||||
|
||||
| Bound | Why |
|
||||
|---|---|
|
||||
| `maxTimeMS` | The server stops working on a query nobody is waiting for. |
|
||||
| result limit | An unbounded result set is an OOM with extra steps. |
|
||||
| batch size | Bounds the per-round-trip memory. |
|
||||
| examined-document ceiling | Catches an index regression that a time limit alone would hide on a fast day. |
|
||||
|
||||
`MongoBudgetPolicyRegistry` binds a budget to an operation name; `MongoBudgetEnforcer` applies it and
|
||||
raises `MongoOperationRejectedException` before execution when a request exceeds it, and
|
||||
`MongoTimeoutException` when the server enforces it.
|
||||
|
||||
## 4. Keyset pagination
|
||||
|
||||
Unbounded `skip` is unsupported: `skip(1_000_000)` makes the server walk a million documents to throw
|
||||
them away, so page 1000 costs a thousand times page 1.
|
||||
|
||||
`MongoKeysetQueryBuilder` builds the resume predicate lexicographically. For a sort on `(a DESC, _id
|
||||
DESC)` resuming after `(A, I)`:
|
||||
|
||||
```
|
||||
(a < A) OR (a = A AND _id < I)
|
||||
```
|
||||
|
||||
`validate()` rejects a `MongoKeysetSort` without a unique tie-breaker. Without one, two documents with
|
||||
the same sort value straddle the page boundary and one of them is skipped or repeated — invisibly,
|
||||
and only under concurrency.
|
||||
|
||||
`MongoNullSortOrdering` makes null placement explicit, because MongoDB's own ordering of missing
|
||||
versus null versus present is not what most people assume.
|
||||
|
||||
### Cursors are authenticated
|
||||
|
||||
`MongoKeysetCursorCodec` signs the cursor with HMAC-SHA256 and compares with
|
||||
`MessageDigest.isEqual` (constant time). An unsigned cursor is a client-controlled query predicate: a
|
||||
caller can edit it to read a range they were never offered. A tampered or truncated cursor yields
|
||||
`MongoCursorException`, never a partially-decoded resume position.
|
||||
|
||||
## 5. Aggregation guardrails
|
||||
|
||||
`MongoAggregationPlan` is a registered pipeline: an ordered list of `MongoAggregationStageDescriptor`
|
||||
validated against a `MongoAggregationProfile`. `PolicyAwareMongoAggregationExecutor` runs only a
|
||||
registered plan.
|
||||
|
||||
`MongoAggregationRisk` grades each stage, and the profile sets the ceiling:
|
||||
|
||||
| Risk | Stages | Policy |
|
||||
|---|---|---|
|
||||
| low | `$match` on an indexed prefix, `$limit`, `$project` | Always allowed. |
|
||||
| moderate | `$group`, `$sort` with an index, `$unwind` with a bound | Allowed within budget. |
|
||||
| high | `$lookup`, `$graphLookup`, `$facet`, unindexed `$sort` | Requires explicit approval in the profile. |
|
||||
| forbidden | `$out`, `$merge` outside the admin plane, `$function`, `$accumulator` | Refused. |
|
||||
|
||||
`allowDiskUse` is a declared property of the plan, not a runtime flag. A pipeline that needs disk is a
|
||||
pipeline whose shape should be reviewed.
|
||||
|
||||
## 6. Reactive execution and cursors
|
||||
|
||||
`ReactiveMongoExecutor` / `DefaultReactiveMongoExecutor` carry the operation context in the Reactor
|
||||
context. `MongoCursorGuard` and `MongoCursorLease` bound cursor lifetime:
|
||||
|
||||
- a cursor has a lease with a deadline
|
||||
- cancellation closes the server-side cursor (`MongoCursorTermination`)
|
||||
- an abandoned cursor is a server-side resource, so the lease is released on cancel, error *and*
|
||||
completion — `MongoReactiveCursorPublisher` uses `Flux.using` so all three paths run the same
|
||||
release
|
||||
|
||||
A leaked cursor does not fail anything locally; it consumes a connection and a snapshot on the server
|
||||
until the server's own timeout, which is why the guard is not optional.
|
||||
|
||||
## 7. Geospatial
|
||||
|
||||
`MongoGeoQuery` + `MongoGeoPoint` + `MongoGeoDistance` over a `2dsphere` index. Distances are metres
|
||||
on a sphere (`nearSphere` with `maxDistance`), never degrees — a degree of longitude is a different
|
||||
distance in Oslo than in Nairobi, and a radius expressed in degrees is a bug that only shows up away
|
||||
from the equator. `SpringMongoGeospatialOperations` is the Spring Data binding;
|
||||
`MongoGeospatialOperations` is the port.
|
||||
|
||||
## 8. Native capability gateway
|
||||
|
||||
When a registered operation genuinely needs something outside the Stable API, it goes through
|
||||
`MongoNativeCapabilityGateway` (`PolicyAwareMongoNativeGateway`), never through the driver directly.
|
||||
The admission order is fixed:
|
||||
|
||||
```
|
||||
capability registered
|
||||
→ database profile
|
||||
→ collection allowlist
|
||||
→ operation name present
|
||||
→ timeout / maxTimeMS
|
||||
→ consistency profile
|
||||
→ result / batch limit
|
||||
→ trace
|
||||
→ log redaction
|
||||
→ command category (MongoNativeCommandCategory)
|
||||
→ D4 admin command refused
|
||||
→ execute
|
||||
```
|
||||
|
||||
`ApprovedMongoNativeOperation` is the registration record; `MongoNativeOperationPolicy` is the policy.
|
||||
An admin-plane command reaching this gateway is refused regardless of capability — the admin plane has
|
||||
its own credential and its own client (see [security-observability.md](security-observability.md)).
|
||||
@@ -0,0 +1,124 @@
|
||||
# MongoDB Document Persistence Platform — Repository Adaptation Contract
|
||||
|
||||
**Design source:** `mongodb-superpowers-package/docs/superpowers/specs/2026-08-11-mongodb-document-persistence-platform-design.md`
|
||||
**Stable plan:** `mongodb-superpowers-package/docs/superpowers/plans/2026-08-11-mongodb-document-persistence-platform-implementation-plan.md`
|
||||
**Advanced plan:** `mongodb-superpowers-package/docs/superpowers/plans/2026-08-11-mongodb-advanced-capabilities-expansion-plan.md`
|
||||
|
||||
The design package declares its own module root (`modules/mongodb`) and root package
|
||||
(`io.backend.skeleton.mongodb`) as *implementation assumptions*, not as contract. This file is the
|
||||
single record of how that assumed layout was mapped onto this repository. Only paths, build DSL,
|
||||
and composition-root ownership changed. Public contracts, policy order, and error semantics are
|
||||
implemented exactly as specified.
|
||||
|
||||
## 1. Why the module layout differs
|
||||
|
||||
The design assumes 19 Stable Gradle projects under `modules/mongodb/` and 12 Advanced projects
|
||||
under `modules/mongodb-advanced/`. This repository is a Clean Architecture template whose
|
||||
**fail-closed registry** (`src/config/architecture/modules.json`, enforced by `src/settings.gradle`
|
||||
and `verifyCleanArchitectureDependencies`) declares **exactly 19 leaf identities**. Creating 31 more
|
||||
Gradle projects would violate HARD-STOP #5 in `AGENTS.md`.
|
||||
|
||||
Therefore the design's 31 modules become **package boundaries inside the registered leaf**
|
||||
`:adapter:outbound:persistence-mongo`, following the precedent already set by
|
||||
[docs/httpclient/repository-adaptation.md](../httpclient/repository-adaptation.md). The design's
|
||||
module dependency table (§6.3) is reproduced as ten ArchUnit rules in `MongoModuleBoundaryTest`, so
|
||||
a forbidden edge fails the build the same way a missing Gradle dependency would.
|
||||
|
||||
## 2. Package mapping
|
||||
|
||||
Root package: `io.backend.skeleton.mongodb` → `dev.caskeleton.adapter.outbound.mongo`.
|
||||
|
||||
### 2.1 Stable modules
|
||||
|
||||
| Design module | Repository package |
|
||||
|---|---|
|
||||
| `mongodb-core-api` | `…outbound.mongo.api` (+ `.capability`, `.consistency`, `.error`, `.mapping`, `.observation`, `.profile`, `.schema`) |
|
||||
| `mongodb-spring-data` | `…outbound.mongo.mapping` (+ `.type`), `…outbound.mongo.failure` |
|
||||
| `mongodb-imperative` | `…outbound.mongo.imperative` (+ `.atomic`, `.bulk`, `.revision`) |
|
||||
| `mongodb-reactive` | `…outbound.mongo.reactive` (+ `.cursor`) |
|
||||
| `mongodb-query` | `…outbound.mongo.query` (+ `.budget`, `.pagination`) |
|
||||
| `mongodb-aggregation` | `…outbound.mongo.aggregation` |
|
||||
| `mongodb-transaction` | `…outbound.mongo.transaction` (+ `.retry`, `.session`) |
|
||||
| `mongodb-index-schema` | `…outbound.mongo.schema` (+ `.index`, `.manifest`, `.model`, `.ttl`, `.validation`) |
|
||||
| `mongodb-change-stream` | `…outbound.mongo.changestream` (+ `.projector`, `.recovery`) |
|
||||
| `mongodb-geospatial` | `…outbound.mongo.geo` |
|
||||
| `mongodb-migration-core` | `…outbound.mongo.migration` |
|
||||
| `mongodb-migration-flamingock` | `…outbound.mongo.migration.flamingock` |
|
||||
| `mongodb-observability` | `…outbound.mongo.observation` |
|
||||
| `mongodb-security` | `…outbound.mongo.security` (+ `.admin`), `…outbound.mongo.nativecap` |
|
||||
| `mongodb-spring-boot-starter` | `…outbound.mongo.autoconfigure` |
|
||||
| `mongodb-testkit-core` | `…outbound.mongo.testkit.mapping`, `.compat`, `.performance` (`testkit` source set) |
|
||||
| `mongodb-testkit-replicaset` | `…outbound.mongo.testkit.rs` (`testkit` source set) |
|
||||
| `mongodb-testkit-failover` | `…outbound.mongo.testkit.failover` (`testkit` source set) |
|
||||
| `mongodb-testkit-migration` | `…outbound.mongo.testkit.migration` (`testkit` source set) |
|
||||
|
||||
`…outbound.mongo.architecture` has no design counterpart: it holds the `@MongoOperation` marker and
|
||||
the reusable ArchUnit rule set a fork applies to its own document/repository code.
|
||||
|
||||
### 2.2 Advanced modules
|
||||
|
||||
| Design module | Repository package |
|
||||
|---|---|
|
||||
| `mongodb-sharding` | `…outbound.mongo.advanced.sharding` (+ `.admin` for the D4 shard plane) |
|
||||
| `mongodb-timeseries` | `…outbound.mongo.advanced.timeseries` |
|
||||
| `mongodb-csfle` | `…outbound.mongo.advanced.encryption.csfle` |
|
||||
| `mongodb-queryable-encryption` | `…outbound.mongo.advanced.encryption.qe` |
|
||||
| `mongodb-search` | `…outbound.mongo.advanced.search` |
|
||||
| `mongodb-vector-search` | `…outbound.mongo.advanced.vector` |
|
||||
| `mongodb-tenancy-shared` | `…outbound.mongo.advanced.tenancy.shared` |
|
||||
| `mongodb-tenancy-database` | `…outbound.mongo.advanced.tenancy.database` |
|
||||
| `mongodb-change-stream-messaging-bridge` | `…outbound.mongo.advanced.bridge` |
|
||||
| `mongodb-gridfs-compat` | `…outbound.mongo.advanced.gridfs` |
|
||||
| `mongodb-testkit-sharded` | `…outbound.mongo.testkit.sharded` (`testkit` source set) |
|
||||
| `mongodb-testkit-atlas` | `…outbound.mongo.testkit.atlas` (`testkit` source set) |
|
||||
|
||||
The design's rule that a Stable module never depends on an Advanced one survives as an ArchUnit rule
|
||||
(`stableNeverDependsOnAdvanced`) plus the opt-in flag: every Advanced entry point requires
|
||||
`MongoAdvancedCapabilityFlags` to have the matching capability enabled and refuses construction
|
||||
otherwise. Being on the classpath is not being enabled.
|
||||
|
||||
## 3. Other deliberate substitutions
|
||||
|
||||
| Design assumption | Repository reality | Adaptation |
|
||||
|---|---|---|
|
||||
| Gradle Kotlin DSL under `modules/mongodb*` | Groovy DSL, root `build.gradle` conventions, `LockMode.STRICT` locking | Dependencies declared in `src/adapter/outbound/persistence-mongo/build.gradle`; `gradle.lockfile` regenerated. |
|
||||
| `mongodb-spring-boot-starter` is a separate module the app depends on | `modules.json` gives `adapter-outbound-persistence-mongo` `runtime_memberships: []` and does **not** list it among `app-bootstrap`'s allowed dependencies | The `autoconfigure` package stays inside the leaf and registers through the leaf's own `META-INF/spring/…AutoConfiguration.imports`. This differs from the httpclient precedent, where the starter moved to `:app-bootstrap`; here the registry forbids that edge. |
|
||||
| Spring Boot 4.1 / Spring Data MongoDB 5.1 baseline | Repository baseline is Spring Boot 4.0.0 / Spring Data MongoDB 5.0.0 | The platform targets the Spring Data MongoDB **API surface** common to both; no 5.1-only type is referenced. The support matrix records the actual pinned versions. |
|
||||
| `MongoRetryScope` lives in `mongodb-transaction` | The `mongodb-spring-data` failure translator must classify retry scope, and it cannot depend on `mongodb-transaction` | `MongoRetryScope` lives in `…api.error` (core-api), which both packages already depend on. Same values, same meaning, one legal position in the DAG. |
|
||||
| `mongodb-migration-flamingock` depends on Flamingock | Adding an unvetted external dependency is out of scope for this task, and the design itself requires the public contract not to depend on Flamingock types | The adapter is provider-neutral: it consumes a platform-owned `FlamingockChangeUnitView`. Wiring an actual Flamingock distribution is a one-file change behind that view. |
|
||||
| Testkit as its own Gradle module | The design forbids production modules depending on the testkit | A dedicated `testkit` source set whose output is on the test compile/runtime classpaths only. ArchUnit rule `productionNeverDependsOnTestkit` enforces the direction. |
|
||||
| Per-task `git commit` | `AGENTS.md`: commit policy is `human-only` | Implementation is delivered unstaged; commits are the human's action. This is the only plan step intentionally not executed, and it is recorded here. |
|
||||
| `docs/mongodb/**`, `scripts/verify-mongodb-*.sh` | Repository already owns `docs/` and `scripts/` | Created at the same repository-relative paths. |
|
||||
|
||||
## 4. What is unchanged from the design
|
||||
|
||||
- D1 / D2 / D3 / D4 exposure planes and the ordered D3 admission sequence (§5).
|
||||
- Stable API V1 with `apiStrict=true` on the D1/D2 client generation; D3/D4 on separate generations.
|
||||
- `MongoExecutionOutcome`, including both ambiguous outcomes (`WRITE_RESULT_UNKNOWN`,
|
||||
`TRANSACTION_COMMIT_UNKNOWN`), and `MongoFailureContext`'s permitted-field list.
|
||||
- The complete stable exception hierarchy and the label-before-code classification order.
|
||||
- The BSON representation manifest (UUID `STANDARD`, `Decimal128`, UTC instants, alias type metadata)
|
||||
and the document-size budget.
|
||||
- Update-operator-first writes, and optimistic revision as the precondition for whole-document
|
||||
replacement.
|
||||
- Transaction body retry and commit retry as separate loops: a new session per body attempt, and
|
||||
commit-only retry on unknown commit. The body is never replayed after a commit ambiguity.
|
||||
- Registered operation names and manifests for query, aggregation and index; no free-form JSON query
|
||||
and no unbounded pipeline.
|
||||
- Keyset pagination with an authenticated cursor and a unique tie-breaker requirement.
|
||||
- Change stream as an at-least-once projector with resume-token checkpointing and explicit
|
||||
history-lost handling.
|
||||
- TTL as physical cleanup only, never the sole basis for access denial or business scheduling.
|
||||
- Manifest-owned index/validator state with an apply policy that never drops what it does not own.
|
||||
- Low-cardinality observation tags, command redaction, and the credential reference indirection.
|
||||
- The Stable release gate's evidence categories, and the Advanced promotion gate's requirement for
|
||||
actual-topology evidence.
|
||||
|
||||
## 5. Verification
|
||||
|
||||
```bash
|
||||
bash scripts/verify-mongodb-platform.sh # Stable gate
|
||||
bash scripts/verify-mongodb-advanced.sh # Advanced gate (opt-in lanes)
|
||||
```
|
||||
|
||||
Both scripts run from the repository root and delegate to `src/gradlew`.
|
||||
@@ -0,0 +1,98 @@
|
||||
---
|
||||
title: Runbook — MongoDB primary failover
|
||||
category: mongodb
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-08-13
|
||||
status: active
|
||||
---
|
||||
|
||||
# Runbook: MongoDB primary failover
|
||||
|
||||
Design §29, scenarios `PRIMARY_KILL`, `NETWORK_PARTITION`, `SERVER_SELECTION_TIMEOUT`,
|
||||
`WRITE_RESPONSE_LOSS`.
|
||||
|
||||
## Symptoms
|
||||
|
||||
- `MongoServerSelectionException` / `MongoConnectionException` spike, then recovery within seconds.
|
||||
- `MongoSdamObservationListener` reports a topology change (primary removed, new primary elected).
|
||||
- `MongoPoolObservationListener` shows checkout wait times rising while server-side command duration
|
||||
stays flat — the wait is topology, not query cost.
|
||||
- Latency spike on writes with no corresponding rise in read latency.
|
||||
|
||||
A failover that resolves in under ~15 s and produces no `WRITE_RESULT_UNKNOWN` is normal replica-set
|
||||
behaviour and needs no action beyond confirming it self-healed.
|
||||
|
||||
## Diagnosis
|
||||
|
||||
1. Confirm an election actually happened. SDAM events distinguish an election from "the database got
|
||||
slow"; without them the two are indistinguishable in application metrics.
|
||||
2. Split the failure categories. Metric tag `failureCategory`:
|
||||
- `SERVER_SELECTION` / `CONNECTION` → the driver could not reach a primary. `NOT_SENT`; safe.
|
||||
- `TIMEOUT` with outcome `WRITE_RESULT_UNKNOWN` → a write may have applied. Not safe; see below.
|
||||
- `TRANSACTION_COMMIT_UNKNOWN` → go to [unknown-commit.md](unknown-commit.md) instead.
|
||||
3. Check the election duration against `MongoRetryBudget`. If the election outlasted the budget, the
|
||||
retries were exhausted before a primary existed and callers saw errors that a longer budget would
|
||||
have absorbed.
|
||||
4. Check whether the new primary is in the expected region/AZ. A failover to a distant node changes
|
||||
write latency permanently, not transiently.
|
||||
|
||||
## Action
|
||||
|
||||
**Self-healed (the common case).**
|
||||
Confirm outcome distribution contains no `WRITE_RESULT_UNKNOWN`, record the election in the incident
|
||||
log, and close. Nothing to replay.
|
||||
|
||||
**Writes with `WRITE_RESULT_UNKNOWN`.**
|
||||
These writes may or may not have applied. Do not blind-retry.
|
||||
- Idempotent operation (registered `MongoUpdateOperator` with an `AtomicFilter` precondition): retry.
|
||||
The precondition makes the second application a no-op.
|
||||
- Non-idempotent operation: reconcile by reading the target document and comparing against the
|
||||
intended post-state. Retry only if it does not reflect the write.
|
||||
|
||||
**Server selection never recovers.**
|
||||
The set has lost quorum — two of three nodes are down or partitioned. No client-side action fixes
|
||||
this; escalate to the database owner to restore a majority. The application should be failing closed,
|
||||
not queueing.
|
||||
|
||||
**Elections are frequent (more than one a day, unprompted).**
|
||||
This is an infrastructure symptom, not an application one: check node resource saturation, disk
|
||||
latency on the primary, and network stability between members. Repeated elections cause repeated
|
||||
unknown-outcome windows.
|
||||
|
||||
## Escalation
|
||||
|
||||
- P2 → P1 if server selection has failed for more than 2 minutes, or if any non-idempotent write
|
||||
returned `WRITE_RESULT_UNKNOWN` and cannot be reconciled.
|
||||
- Page the database owner for quorum loss, and the service owner for reconciliation of ambiguous
|
||||
writes.
|
||||
|
||||
## Verification
|
||||
|
||||
The failover lane reproduces this deliberately:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:persistence-mongo:mongoFailoverTest --console=plain
|
||||
```
|
||||
|
||||
It starts a real three-node set (`MongoThreeNodeReplicaSet`), stops the primary
|
||||
(`MongoPrimaryController`), and injects network faults through Toxiproxy
|
||||
(`ToxiproxyMongoNetworkFaultController`). A single-node set is not sufficient for the election: it
|
||||
never holds one, so every guarantee that depends on a primary change goes untested.
|
||||
|
||||
The network faults need their own fixture (`MongoProxiedReplicaSetNode`) because a stopped container
|
||||
cannot produce them. Stopping a node tells the client the write did not happen; cutting the *path*
|
||||
while the server keeps running produces a client that cannot tell. `MongoNetworkFaultLaneTest`
|
||||
asserts the difference by reaching the same server twice — once through the proxy, once directly:
|
||||
|
||||
- **Partition**: the proxied client fails, the direct client finds the server healthy and the earlier
|
||||
write intact. The path was cut, not the server.
|
||||
- **Response loss**: the proxied client fails, and the direct client then finds the document
|
||||
*present*. The write applied and only the acknowledgement was lost —
|
||||
`DefaultMongoFailureClassifier` returns `WRITE_RESULT_UNKNOWN`, and a retry would have inserted a
|
||||
second document.
|
||||
|
||||
One detail the lane depends on: the connection is warmed before the toxic is applied. On a cold
|
||||
connection it is the driver's handshake whose response is dropped, so the write is never transmitted
|
||||
— `NOT_SENT`, the opposite of the ambiguity being tested.
|
||||
@@ -0,0 +1,90 @@
|
||||
---
|
||||
title: Runbook — MongoDB change stream history lost
|
||||
category: mongodb
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-08-13
|
||||
status: active
|
||||
---
|
||||
|
||||
# Runbook: change stream history lost
|
||||
|
||||
Design §20.3, scenarios `OPLOG_HISTORY_LOSS`, `RESUME_TOKEN_LOSS`.
|
||||
|
||||
The stored resume token predates the oldest entry in the oplog. The events between the checkpoint and
|
||||
now are gone from the server; no retry recovers them. `MongoChangeStreamRecoveryPolicy` returns
|
||||
`halt(HISTORY_LOST, "history-lost")` and the runner stops.
|
||||
|
||||
**The platform will not silently restart from "now".** That looks like a recovery and is actually a
|
||||
permanent, unreported gap in the projection.
|
||||
|
||||
## Symptoms
|
||||
|
||||
- `MongoChangeHistoryLostException`.
|
||||
- `MongoChangeStreamState.HISTORY_LOST`; the consumer is stopped, not looping.
|
||||
- Precedes it: consumer lag approaching the oplog window, or a consumer that was down for a long
|
||||
period (a deploy that failed, a scaled-to-zero worker, a long outage).
|
||||
|
||||
## Diagnosis
|
||||
|
||||
1. **Determine the gap.** The checkpoint's cluster time is the start; the oldest oplog entry is the
|
||||
end of what is unrecoverable. Everything in between was never processed.
|
||||
2. **Determine the oplog window.** `rs.printReplicationInfo()` on the primary gives the first and last
|
||||
oplog timestamps. If the window is materially smaller than it was, the write rate rose or the
|
||||
oplog was resized — the consumer may be fine and the server changed.
|
||||
3. **Determine what the projection is missing.** Which collections and which operations does this
|
||||
projector consume? The gap is bounded by that, not by everything that happened.
|
||||
4. **Check for a second consumer.** If another projector on the same collection is healthy, its
|
||||
checkpoint tells you whether the problem is this consumer or the oplog.
|
||||
|
||||
## Action
|
||||
|
||||
Resuming is not an option. The choices are:
|
||||
|
||||
**Rebuild from source.** If the projection is derivable from the current state of the source
|
||||
collections, rebuild it: stop the consumer, rebuild the projection, then start the stream from the
|
||||
cluster time at which the rebuild snapshot was taken. This is the correct answer whenever the
|
||||
projection is a materialised view rather than an event log, and it is the reason a projection should
|
||||
be derivable.
|
||||
|
||||
**Backfill the gap.** If the source documents carry a timestamp covering the gap, run a bounded
|
||||
backfill for that window through the migration runner (checkpointed, resumable — see
|
||||
[schema-index-migration-guide.md](../schema-index-migration-guide.md) §5), then resume from the
|
||||
current cluster time.
|
||||
|
||||
**Accept the gap explicitly.** Only when the projection is advisory and the business owner says so.
|
||||
Record the window in the incident log and reset the checkpoint. This is a decision someone signs, not
|
||||
a default.
|
||||
|
||||
Never: reset the checkpoint to "now" and restart quietly. That converts a visible P1 into an
|
||||
invisible data-quality defect that surfaces months later as "the report has been wrong since March".
|
||||
|
||||
## Prevention
|
||||
|
||||
- **Alert on lag against the oplog window, not wall-clock.** "Consumer is 30 minutes behind" is fine
|
||||
with a 24-hour oplog and an emergency with a 45-minute one. The threshold that matters is
|
||||
`lag / oplogWindow`.
|
||||
- **Size the oplog for the longest tolerable consumer outage**, including a failed deploy discovered
|
||||
the next morning.
|
||||
- **Checkpoint after processing, never on receipt** — see
|
||||
[change-stream-guide.md](../change-stream-guide.md) §3.
|
||||
- **Back up the checkpoint store.** `RESUME_TOKEN_LOSS` is the same incident reached from the other
|
||||
direction: the oplog is fine, the checkpoint is gone.
|
||||
- **Make the projection rebuildable.** A projection that can only be built by replaying every event
|
||||
has no recovery path once the oplog rolls.
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 on detection. The consumer is stopped, so lag grows for as long as this is unresolved.
|
||||
- Page the service owner for the rebuild decision, and the database owner if the oplog window shrank
|
||||
unexpectedly.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:persistence-mongo:mongoFailoverTest --console=plain
|
||||
```
|
||||
|
||||
`MongoFailoverScenario.OPLOG_HISTORY_LOSS` and `RESUME_TOKEN_LOSS` assert the runner halts and names
|
||||
this runbook rather than restarting from the current position.
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
title: Runbook — MongoDB unknown transaction commit result
|
||||
category: mongodb
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-08-13
|
||||
status: active
|
||||
---
|
||||
|
||||
# Runbook: unknown transaction commit result
|
||||
|
||||
Design §16, decision D-10, scenario `UNKNOWN_TRANSACTION_COMMIT_RESULT`.
|
||||
|
||||
`MongoExecutionOutcome.TRANSACTION_COMMIT_UNKNOWN` means the commit **may have applied**. It is not a
|
||||
failure and must never be reported to a caller as one. The single worst response is to re-run the
|
||||
transaction body: if the commit did apply, the body applies a second time.
|
||||
|
||||
## Symptoms
|
||||
|
||||
- `MongoTransactionCommitUnknownException` in logs.
|
||||
- Metric `failureCategory=TRANSACTION_COMMIT_UNKNOWN`.
|
||||
- Usually accompanies a primary election — see [failover.md](failover.md).
|
||||
- Downstream reports of duplicated effects (double charge, double increment) are the symptom of this
|
||||
being handled wrongly, not of the condition itself.
|
||||
|
||||
## Diagnosis
|
||||
|
||||
1. **Confirm the platform did the right thing automatically.**
|
||||
`MongoTransactionRetryCoordinator` retries the *commit only*, on the same session, within
|
||||
`MongoRetryBudget`. A commit retry against an already-committed transaction is a no-op by design.
|
||||
Most occurrences resolve here and never reach a human.
|
||||
|
||||
2. **If the budget was exhausted, determine the actual state.** The commit either applied or it did
|
||||
not; you must find out which, not guess.
|
||||
- If the transaction body wrote a deterministic marker (an idempotency key, a business id, a
|
||||
revision), read it back. That is exactly what `MongoCommitReconciler` does, and it is the
|
||||
reason the design requires transactions to write one.
|
||||
- If there is no marker: reconstruct from a downstream artefact — an outbox row, an audit record,
|
||||
an external side effect. If nothing exists to compare against, the transaction was not
|
||||
designed to be reconcilable and that is the finding to record.
|
||||
|
||||
3. **Check whether the body was replayed.** Grep for a second execution with the same operation name
|
||||
and correlation id. If the body ran twice, the effects need reversing, and the code path that
|
||||
replayed it is a defect: an ambiguous commit is `COMMIT_ONLY` scope
|
||||
(`MongoRetryScope.COMMIT_ONLY`), never `BODY`.
|
||||
|
||||
## Action
|
||||
|
||||
**Commit applied.** Nothing to do. Record the reconciliation.
|
||||
|
||||
**Commit did not apply.** Re-run the whole operation from the top — a new session, a new body
|
||||
attempt. This is safe precisely because you established the previous attempt left no trace.
|
||||
|
||||
**Cannot determine.** Do not retry. Escalate. A blind retry here is a coin flip between "no effect"
|
||||
and "duplicate effect", and duplicates in a financial or notification path are worse than a delay.
|
||||
Freeze the affected entity if the domain supports it, and hand off with: operation name, correlation
|
||||
id, document id, the time window, and what you checked.
|
||||
|
||||
**Recurring.** More than one an hour means the commit path is racing something structural — a
|
||||
`maxCommitTime` shorter than the observed election duration, an oversized transaction, or an
|
||||
undersized retry budget. Fix the budget or the transaction shape; do not raise the retry count and
|
||||
call it resolved.
|
||||
|
||||
## Prevention
|
||||
|
||||
- Every transaction body writes a deterministic marker that identifies its own commit.
|
||||
- `maxCommitTime` exceeds the observed p99 election duration.
|
||||
- Callers surface the ambiguity to their own callers rather than mapping it to a generic 500 — an
|
||||
ambiguous outcome reported as a failure invites the caller to retry, which is the one thing that
|
||||
must not happen.
|
||||
- Prefer a single-document atomic operation (D-09). A transaction that exists only to wrap one
|
||||
document write has invented this failure mode for nothing.
|
||||
|
||||
## Escalation
|
||||
|
||||
- Always P1 when the state cannot be determined and the operation has an external effect.
|
||||
- Page the service owner immediately; the database owner only if elections are the trigger.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:persistence-mongo:mongoFailoverTest --console=plain
|
||||
```
|
||||
|
||||
`MongoFailoverScenario.UNKNOWN_TRANSACTION_COMMIT_RESULT` runs this path against a real three-node
|
||||
set, and the coordinator test asserts the body is never replayed after a commit ambiguity.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Schema, Index and Migration Guide
|
||||
|
||||
Design §21–§25, decision D-13. Indexes and validators are **declared** in a manifest and **applied**
|
||||
by an explicit plane. Automatic index creation in production is explicitly unsupported (§3.4): an
|
||||
index build on a large collection is a capacity event, and discovering it because a deployment
|
||||
started one is not an operating model.
|
||||
|
||||
## 1. The manifest is the source of truth
|
||||
|
||||
`MongoManifestRegistry` holds one `MongoCollectionManifest` per collection, containing:
|
||||
|
||||
- `MongoIndexManifest` — the declared indexes (`MongoIndexKey`, `MongoIndexDirection`, uniqueness,
|
||||
partial filter, collation)
|
||||
- `MongoSchemaManifest` — the declared `$jsonSchema` validator
|
||||
- `MongoMetadataOwnership` — who owns each observed object
|
||||
|
||||
Ownership is the field that makes drift handling safe:
|
||||
|
||||
| Ownership | Owner | Droppable on drift |
|
||||
|---|---|---|
|
||||
| `APPLICATION_MANAGED` | this manifest | yes |
|
||||
| `SEARCH_MANAGED` | the search service | no |
|
||||
| `ENCRYPTION_MANAGED` | Queryable Encryption | no |
|
||||
| `EXTERNAL` | someone else (a DBA, another service) | no |
|
||||
|
||||
A diff engine that does not know about ownership eventually proposes dropping
|
||||
`enxcol_.customers.esc` or a search index, and "the drift tool cleaned it up" is a very bad incident
|
||||
summary.
|
||||
|
||||
## 2. Index diff and apply
|
||||
|
||||
`MongoIndexDiffEngine` compares the manifest against `MongoIndexDescriptorView` observations and
|
||||
produces a `MongoIndexDiff`: missing, extra, and *changed* (same name, different definition —
|
||||
MongoDB will not silently rebuild these, so they must be reported rather than re-issued).
|
||||
|
||||
`MongoIndexApplyPolicy` decides what happens with a diff:
|
||||
|
||||
| Policy | Behaviour | Environment |
|
||||
|---|---|---|
|
||||
| `APPLY` | create what is missing | local / test |
|
||||
| `APPLY_WITH_DIFF` | create what is missing and report the rest | staging |
|
||||
| `DIFF_WITH_APPROVED_APPLY` | apply only what a human approved | production |
|
||||
| `REPORT_ONLY` | never write | audit |
|
||||
|
||||
Dropping is never implicit. `MongoIndexRetirementPlan` moves an index through
|
||||
`MongoIndexRetirementState` — declared → hidden → observed-unused → droppable — and each transition
|
||||
is a separate deployment. Hiding an index makes the planner ignore it while keeping it maintained, so
|
||||
an unexpected regression is one command to undo. Dropping it is not.
|
||||
|
||||
## 3. Validators
|
||||
|
||||
`MongoValidatorDescriptor` carries the `$jsonSchema`, a `MongoValidationLevel`
|
||||
(`OFF` / `MODERATE` / `STRICT`) and a `MongoValidationAction`.
|
||||
|
||||
**Stable validation actions are `error` and `warn` only.** `errorAndLog` is not part of the Stable
|
||||
contract on MongoDB 7.0 or 8.0 and the descriptor refuses it.
|
||||
|
||||
`MongoValidatorDiffEngine` produces a `MongoValidatorDiff`; `MongoValidatorApplyPolicy` gates the
|
||||
apply. Tightening a validator on a collection with existing data is the dangerous direction: introduce
|
||||
it as `warn` + `MODERATE`, confirm the warning count is zero, then promote to `error` + `STRICT` in a
|
||||
second deployment.
|
||||
|
||||
## 4. TTL
|
||||
|
||||
D-13: TTL is **physical cleanup**, nothing else.
|
||||
|
||||
`MongoTtlIndexDescriptor` declares the field and `expireAfterSeconds`. `MongoTtlPolicyValidator`
|
||||
enforces what `MongoTtlPolicy` allows, and `MongoExpirationAccessPolicy` states the rule that matters:
|
||||
|
||||
> A document's presence is not authorization, and its absence is not a deadline.
|
||||
|
||||
The TTL monitor runs about once a minute and deletes in batches, so a document can outlive its
|
||||
expiry by minutes to hours under load. Consequences:
|
||||
|
||||
- Access control must check the expiry field, not the document's existence. A still-present expired
|
||||
session is a valid document and an invalid session.
|
||||
- Business scheduling must not be built on TTL. If something must happen at a time, schedule it.
|
||||
- A TTL field must be a BSON date. A TTL index on a string silently never deletes anything.
|
||||
|
||||
## 5. Migrations
|
||||
|
||||
`MongoMigrationRunner` executes `MongoMigration` units with:
|
||||
|
||||
- `MongoMigrationId` — ordered, unique
|
||||
- `MongoMigrationChecksum` — content hash; a changed checksum for an applied id is a hard failure, not
|
||||
a re-run. Editing an applied migration means two environments ran different code under the same id.
|
||||
- `MongoMigrationLedger` — what has been applied
|
||||
- `MongoMigrationLock` — one runner at a time; a rolling deploy starts several instances at once
|
||||
- `MongoMigrationPrecondition` / `MongoMigrationPostcondition` — checked before and after; a migration
|
||||
that cannot verify its own result is a migration whose failure is discovered by a customer
|
||||
- `MongoMigrationCheckpoint` — a resumable position for a backfill
|
||||
|
||||
`MongoMigrationResult` reports applied / incomplete / dry-run with the reason. `INCOMPLETE` is not a
|
||||
failure: a rate-limited backfill that ran out of its time budget has done real work and stored a
|
||||
checkpoint, and reporting it as failed would send the next run back to the beginning.
|
||||
|
||||
`MongoCollectionMigrationLedger` and `MongoCollectionMigrationLock` are the MongoDB-backed
|
||||
implementations. Two details are load-bearing and only exist on a server:
|
||||
|
||||
- The ledger's **unique index** on the migration id, created by `ensureIndexes()`. Without it, two
|
||||
runners that both pass the "not applied yet" read both insert, and the ledger then reports one
|
||||
migration applied twice with two checksums — indistinguishable from tampering.
|
||||
- The lease is taken with **one conditional update**, not read-then-write. A filter matching only a
|
||||
free or expired lease lets the server pick the winner; two runners that each read "free" and then
|
||||
write would both believe they hold it.
|
||||
|
||||
The lease expires so a runner killed mid-migration does not block every future deployment, and
|
||||
`refresh` between batches is what proves the holder is still alive.
|
||||
|
||||
### Backfills restart, they do not restart-from-zero
|
||||
|
||||
A long backfill will be interrupted — a deploy, an OOM, a node replacement. The checkpoint records
|
||||
the last completed key so the restart continues rather than re-processing from the beginning.
|
||||
`MongoBackfillRestartFixture` in the testkit asserts exactly this: kill mid-run, restart, and the
|
||||
result is identical to the uninterrupted run and does not re-apply completed work.
|
||||
|
||||
### Flamingock
|
||||
|
||||
`FlamingockMongoMigrationAdapter` bridges to Flamingock through the platform-owned
|
||||
`FlamingockChangeUnitView`, with `FlamingockLedgerAdapter` and `FlamingockLockAdapter` mapping the
|
||||
ledger and lock. The public contract does not reference Flamingock types, so the provider can be
|
||||
replaced without touching a migration.
|
||||
|
||||
## 6. Ordering with deployments
|
||||
|
||||
```
|
||||
1. Add the index (hidden if it is large) -> deployment N
|
||||
2. Unhide / verify usage -> deployment N+1
|
||||
3. Ship code that depends on the index -> deployment N+1
|
||||
4. Backfill data -> migration, resumable
|
||||
5. Tighten the validator from warn to error -> deployment N+2
|
||||
6. Retire the old index through the retirement states -> deployments N+3…
|
||||
```
|
||||
|
||||
Each step is independently reversible. A deployment that adds an index and the code that requires it
|
||||
at the same time has no safe rollback: rolling back the code leaves the index build running, and
|
||||
rolling back the index breaks the code that is still live on half the fleet.
|
||||
@@ -0,0 +1,147 @@
|
||||
# Security and Observability
|
||||
|
||||
Design §26–§28, decision D-05. The application plane and the admin plane are different credentials on
|
||||
different clients, and telemetry never becomes an exfiltration path.
|
||||
|
||||
## 1. Roles
|
||||
|
||||
`MongoPrincipalRole` — one credential per role, least privilege:
|
||||
|
||||
| Role | Grants |
|
||||
|---|---|
|
||||
| `APP_READ` | `find` on allowlisted collections |
|
||||
| `APP_WRITE` | `insert`, `update`, `delete` on allowlisted collections |
|
||||
| `CHANGE_STREAM` | `changeStream`, `find` |
|
||||
| `MIGRATION` | index and validator management on the target collections |
|
||||
| `SEARCH_ADMIN` | search index management |
|
||||
| `SHARD_ADMIN` | shard key operations |
|
||||
| `ENCRYPTION_ADMIN` | key vault access |
|
||||
| `DBA` | the human plane; never used by an application |
|
||||
|
||||
`MongoSecurityProfileValidator` checks the profile at startup. `forbiddenPrivilegesHeld()` names the
|
||||
privileges the profile holds and must not — the validator reports *which* one, because "your
|
||||
credential is over-privileged" without a name is an unactionable finding.
|
||||
|
||||
The privileges that must never appear on an application credential: `dropDatabase`,
|
||||
`dropCollection`, `shutdown`, `killop`, `root`, `__system`, `dbOwner`, `userAdminAnyDatabase`.
|
||||
|
||||
## 2. Credentials are references, not values
|
||||
|
||||
`MongoCredentialReference` holds a `secret://…` reference plus the role. The reference is resolved at
|
||||
connection time by the secret provider; the password is never a property value, a log field, or a
|
||||
constructor argument that could end up in a stack trace.
|
||||
|
||||
`MongoCredentialRotationPolicy` states the rotation contract: overlapping validity, a drain window,
|
||||
and a rotation that never requires a restart. `MongoClientGenerationRegistry` implements the swap —
|
||||
a new `MongoClientGeneration` starts serving new operations while the previous generation is
|
||||
`markDraining()` until its in-flight operations finish. Killing the old client immediately fails every
|
||||
in-flight request, which is why rotation without generations is an outage.
|
||||
|
||||
Rotation is a failover scenario in the release gate (`MongoFailoverScenario.CREDENTIAL_ROTATION`),
|
||||
not a runbook step people hope works.
|
||||
|
||||
## 3. TLS and connection policy
|
||||
|
||||
`MongoSecurityProfile.production(...)` requires TLS and refuses `tlsAllowInvalidCertificates` /
|
||||
`tlsAllowInvalidHostnames`. `MongoSecurityProfile.local(...)` exists so a developer does not have to
|
||||
weaken the production factory to get a container to connect; the startup validator refuses a local
|
||||
profile on a production runtime profile.
|
||||
|
||||
## 4. Admin plane (D4)
|
||||
|
||||
`MongoAdminGateway` is the only path to `MongoAdminOperation`, and it runs on the D4 client with the
|
||||
DBA-scoped credential — not the application's.
|
||||
|
||||
- `MongoAdminAuthorization` checks the caller's role against the operation.
|
||||
- `MongoAdminRuntimeGuard` refuses high-risk operations (`highRisk()`) unless the runtime profile
|
||||
explicitly permits them; a `dropCollection` reachable from a running application is a data-loss
|
||||
vector regardless of how well-reviewed the calling code is.
|
||||
- `MongoAdminAuditRecord` records who ran what, when and against which collection profile — before
|
||||
execution, so a failed attempt is recorded too.
|
||||
|
||||
The native capability gateway (D3) refuses any admin-category command, so there is no path from the
|
||||
application plane into the admin plane.
|
||||
|
||||
## 5. Observability tags
|
||||
|
||||
`MongoObservationConvention` allowlists exactly eight tag names:
|
||||
|
||||
```
|
||||
mongoProfile, databaseProfile, collectionProfile, operationName,
|
||||
operationType, result, failureCategory, consistencyProfile
|
||||
```
|
||||
|
||||
and explicitly forbids:
|
||||
|
||||
```
|
||||
documentId, rawTenantId, tenantId, dynamicCollectionName, queryParameter,
|
||||
query, fullBson, resumeToken, shardKeyValue, plaintextPII, credential
|
||||
```
|
||||
|
||||
Two reasons, and both matter. Cardinality: a tag whose values are document ids produces one time
|
||||
series per document, which is how a metrics backend falls over. Confidentiality: a metric label is
|
||||
stored, shipped and retained by systems with a different access model than the database.
|
||||
`requireAllowed(tagName)` throws on anything outside the list, so a new tag is a deliberate change to
|
||||
the convention rather than a line in a service.
|
||||
|
||||
`MicrometerMongoOperationObserver` implements the `MongoOperationObserver` port;
|
||||
`NoOpMongoOperationObserver` is the default so observation is opt-in and never a hard dependency.
|
||||
|
||||
## 6. Driver-native listeners
|
||||
|
||||
`MongoDriverObservabilityConfiguration` registers three driver listeners, because they answer
|
||||
questions the application-level timer cannot:
|
||||
|
||||
| Listener | Answers |
|
||||
|---|---|
|
||||
| `MongoCommandObservationListener` | How long did the *server* take, versus how long the caller waited? |
|
||||
| `MongoPoolObservationListener` | Was the wait time connection checkout rather than query execution? |
|
||||
| `MongoSdamObservationListener` | Did the topology change — an election, a node removed — during the window? |
|
||||
|
||||
Without pool and SDAM events, every failover looks like "the database got slow", and the difference
|
||||
between "we need a bigger pool" and "we lost a primary" is invisible.
|
||||
|
||||
## 7. Command redaction
|
||||
|
||||
`MongoObservationRedactor.describe(commandName)`:
|
||||
|
||||
- Authentication and user-management commands (`authenticate`, `saslStart`, `saslContinue`,
|
||||
`getnonce`, `createUser`, `updateUser`, `copydb*`) render as `<redacted>` — their arguments carry
|
||||
credentials and key material.
|
||||
- Structural commands (`ping`, `hello`, `buildInfo`, `listCollections`, `listIndexes`, `collStats`)
|
||||
render by name; their arguments are not data-bearing.
|
||||
- Everything else renders as `name(...)`: you get the command, never the filter or the document.
|
||||
|
||||
`isAlwaysRedacted(...)` is the assertion hook so a test can prove no logging path can render an auth
|
||||
command's arguments.
|
||||
|
||||
## 8. Startup validation
|
||||
|
||||
`MongoStartupValidator` runs at context refresh, before the first request:
|
||||
|
||||
1. `MongoTopologyProbe` reports the actual `MongoTopology`.
|
||||
2. Each declared `MongoTopologyRequirement` is checked against it — a transaction, causal-session or
|
||||
change-stream requirement fails closed on `STANDALONE`.
|
||||
3. `MongoSecurityProfileValidator` checks credentials and TLS.
|
||||
4. `MongoCapabilitySupport` checks declared capabilities against the server version, with
|
||||
`MongoSupportLevel` distinguishing `STABLE` / `ADVANCED` / `EXPERIMENTAL` / `UNSUPPORTED`.
|
||||
5. `MongoPlatformHealthIndicator` reports the outcome for the readiness probe.
|
||||
|
||||
A misconfiguration found at startup costs a failed deploy. The same misconfiguration found at runtime
|
||||
costs an incident, and the failing operation is rarely the one that reveals the cause.
|
||||
|
||||
## 9. How the security lane proves any of this
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:persistence-mongo:mongoSecurityIntegrationTest --console=plain
|
||||
```
|
||||
|
||||
The lane runs against `MongoAuthenticatedReplicaSetContainer`, which starts mongod with `--auth` and a
|
||||
generated keyfile. That detail is the whole lane: Testcontainers' `MongoDBContainer` starts mongod
|
||||
*without* `--auth`, so users created on it all have every privilege and a least-privilege assertion
|
||||
passes no matter how wrong the roles are. A security test that cannot fail is not a security test.
|
||||
|
||||
What the lane asserts is the refusal: the `read` role's insert is rejected, and the application role's
|
||||
`dropDatabase` is rejected. Then it checks that `MongoSecurityProfileValidator` names the same
|
||||
privilege the server just refused.
|
||||
@@ -0,0 +1,91 @@
|
||||
# MongoDB Platform — Support Matrix
|
||||
|
||||
Design §4. This file records what the platform is *certified* on, not what it happens to run on.
|
||||
A configuration absent from this table is unsupported until someone runs the gate against it and
|
||||
adds a row.
|
||||
|
||||
## 1. Runtime baseline
|
||||
|
||||
| Component | Version | Policy |
|
||||
|---|---|---|
|
||||
| Java | 21 | Repository runtime baseline. |
|
||||
| Spring Boot | 4.0.0 | BOM-managed. Individual driver overrides are forbidden. |
|
||||
| Spring Data MongoDB | 5.0.0 | Repository and `MongoTemplate` integration. Version comes from the Boot BOM. |
|
||||
| MongoDB Java Driver | 5.6.1 | BOM-managed. Never pinned directly in the module. |
|
||||
| Reactor | 3.8.0 | Reactive execution path. |
|
||||
| Micrometer | 1.16.0 | Driver-native observability. |
|
||||
| Testcontainers | 2.0.2 | Replica-set, failover, migration and compatibility lanes. |
|
||||
|
||||
The design's baseline is Spring Boot 4.1.x / Spring Data MongoDB 5.1.x. This repository is on
|
||||
4.0.0 / 5.0.0, so the platform targets only the API surface common to both. See
|
||||
[repository-adaptation.md](repository-adaptation.md) §3.
|
||||
|
||||
## 2. Server versions
|
||||
|
||||
| Lane | Version | Pinned image | Gradle task |
|
||||
|---|---|---|---|
|
||||
| Primary certification | MongoDB 8.0 | `mongo:8.0.16` | `mongoReplicaSetTest`, `mongoFailoverTest` |
|
||||
| Compatibility | MongoDB 7.0 | `mongo:7.0.28` | `mongoCompatibilityTest` |
|
||||
| Network fault injection | — | `ghcr.io/shopify/toxiproxy:2.12.0` | `mongoFailoverTest` |
|
||||
|
||||
Images are pinned, never `latest`: a mutable tag means the certification result describes whatever
|
||||
was pulled that morning, not the version in the row. Override with
|
||||
`-PmongoPrimaryImage=…` / `-PmongoCompatibilityImage=…` when testing a new patch level, and update
|
||||
the row once the gate passes.
|
||||
|
||||
`MongoVersionMatrix.standard()` is the machine-readable form of this table; a version outside it
|
||||
fails `certifies()`.
|
||||
|
||||
## 3. Topologies
|
||||
|
||||
| Topology | Status | What is certified | What is not |
|
||||
|---|---|---|---|
|
||||
| Standalone | **Smoke only** | Basic CRUD and mapping. | Not a production profile and never counts as Stable release evidence (D-03). Transactions, retryable writes and change streams are refused at startup by `MongoStartupValidator`. |
|
||||
| Single-node replica set | **Local default** (D-02) | Transactions, retryable writes, change streams — the same semantics as production. | Elections. A single-node set never holds one, so failover behaviour is untested here. |
|
||||
| 3-node replica set | **Stable production gate** | Everything above plus primary failover, unknown-commit handling and change-stream resume across an election. | Shard routing. |
|
||||
| Sharded cluster | **Advanced gate** | Shard-key routing classification, scatter-gather refusal, `admin` plane operations. | Not included in the Stable gate. |
|
||||
| Atlas / provider-managed | **Per-capability gate** | Search, vector search and encryption against the actual target deployment. | Atlas Local in a container is a pull-request convenience, explicitly **not** release evidence (`MongoAtlasCapabilityContractSuite.Environment`). |
|
||||
|
||||
## 4. Stable API and client generations
|
||||
|
||||
| Plane | Stable API | Purpose |
|
||||
|---|---|---|
|
||||
| D1 Standard document persistence | V1, `apiStrict=true` | Repositories, typed queries, atomic updates, optimistic revision. |
|
||||
| D2 Advanced document operations | V1, `apiStrict=true` | `MongoTemplate`, transactions, bulk, aggregation, keyset cursors, change streams. |
|
||||
| D3 Explicit Mongo capability | Not strict | Native BSON, time series, search/vector, CSFLE/QE, shard-aware operations — each behind a registered capability. |
|
||||
| D4 Admin plane | Not strict | Collection, validator, index, migration, shard and repair commands. Separate credential, separate client. |
|
||||
|
||||
D3 is not a raw-client escape. Every call passes capability registration → database profile →
|
||||
collection allowlist → operation name → timeout → consistency profile → result limit → trace →
|
||||
redaction → command category → D4 refusal, in that order.
|
||||
|
||||
## 5. Validation actions
|
||||
|
||||
Stable validation actions are `error` and `warn`. `errorAndLog` is **not** part of the Stable
|
||||
contract on 7.0 or 8.0 and `MongoValidatorDescriptor` refuses it.
|
||||
|
||||
## 6. Explicitly unsupported
|
||||
|
||||
Per design §3.4, none of the following is provided, and adding one is a design change rather than a
|
||||
feature request:
|
||||
|
||||
- A generic `CommonMongoRepository<T, ID>`.
|
||||
- Arbitrary runtime `runCommand`.
|
||||
- Automatic index creation in production.
|
||||
- A Standalone production contract.
|
||||
- TTL as an exact business scheduler or as the only access control.
|
||||
- Publishing raw change events as external business integration events.
|
||||
- GridFS as the source of truth for new files.
|
||||
- Java fully-qualified class names as a long-lived BSON schema.
|
||||
- Unbounded skip pagination, unbounded aggregation pipelines, unbounded regex, unbounded results.
|
||||
|
||||
## 7. Capability tiers
|
||||
|
||||
| Tier | Capabilities | Enablement |
|
||||
|---|---|---|
|
||||
| Stable | Mapping, imperative/reactive execution, atomic update, optimistic lock, transactions, consistency profiles, retry/translation, query and aggregation guardrails, schema/index manifests, keyset pagination, bulk partial results, change streams, TTL contract, GeoJSON, security, observability | On when `ca-skeleton.persistence-mongo.enabled=true`. |
|
||||
| Advanced | Sharding-aware query, time series, CSFLE, Queryable Encryption (equality/range), change-stream→messaging bridge, shared-collection multi-tenancy | Each behind `ca-skeleton.persistence-mongo.advanced.<capability>.enabled`. |
|
||||
| Experimental | Search, vector search, hybrid search, database-per-tenant, collection-per-tenant, reshard orchestration, provider-specific features | Same flag mechanism; promotion additionally requires the evidence in [ADR-MONGO-ADV-001](../adr/ADR-MONGO-ADV-001-capability-promotion.md). |
|
||||
|
||||
`MongoAdvancedCapabilityFlags.propertyFor(capability)` is the authoritative property name for any
|
||||
capability; the table above is its prose form.
|
||||
@@ -0,0 +1,27 @@
|
||||
# NOTIF-ADR-001 — `submit()` means durable acceptance
|
||||
|
||||
## Status
|
||||
|
||||
Accepted.
|
||||
|
||||
## Context
|
||||
|
||||
The obvious API for a notification platform is `send()` returning success or failure. Every channel
|
||||
this platform supports makes that return value a lie:
|
||||
|
||||
- SES accepts a request, returns a `MessageId`, and can still decline to send.
|
||||
- Twilio separates `accepted`, `sent` and `delivered` into distinct, later events.
|
||||
- APNs accepts a notification and may then deliver, store or discard it.
|
||||
- Web Push separates push-service acceptance from user-agent acknowledgement at the protocol level.
|
||||
|
||||
## Decision
|
||||
|
||||
`submit()` and `schedule()` return once the logical request and its recipient jobs are committed to
|
||||
the database. The receipt carries `notificationId`, `RequestStatus` and `acceptedAt`, and has no
|
||||
`delivered`, `sent` or `read` component. No provider is contacted while the transaction is open.
|
||||
|
||||
## Consequences
|
||||
|
||||
Callers cannot mistake acceptance for delivery, because the type does not offer that reading.
|
||||
Delivery state is a separate query against the projection built from the provider event ledger. The
|
||||
cost is that "did it arrive?" is a second question — which is the honest number of questions.
|
||||
@@ -0,0 +1,25 @@
|
||||
# NOTIF-ADR-002 — append-only event ledger with channel projectors
|
||||
|
||||
## Status
|
||||
|
||||
Accepted.
|
||||
|
||||
## Context
|
||||
|
||||
A single linear delivery status has to be updated in place, which forces a rule for deciding whether
|
||||
a new event outranks the stored one. The natural rule — compare ordinals — is wrong for real provider
|
||||
traffic. Twilio does not guarantee callback ordering, so `sent` arrives after `delivered`. Email
|
||||
generates complaints after deliveries. Both cases lose information under an ordinal rule.
|
||||
|
||||
## Decision
|
||||
|
||||
Provider events are appended to an immutable ledger before any projection runs. Channel-specific
|
||||
projectors merge events into `SubmissionOutcome`, `DeliveryOutcome`, `EvidenceLevel`,
|
||||
`EngagementFacts` and `SuppressionFacts` using explicit transition tables. Projection is idempotent
|
||||
and can be replayed from the ledger.
|
||||
|
||||
## Consequences
|
||||
|
||||
Duplicate, out-of-order and late events are normal inputs rather than defects. A projector bug is
|
||||
recoverable, because the events it mis-projected are still stored. Projector versions can be migrated
|
||||
by replay. The cost is a second write per event and a projection that can lag its ledger.
|
||||
@@ -0,0 +1,37 @@
|
||||
# NOTIF-ADR-003 — ambiguous submission is a first-class state
|
||||
|
||||
## Status
|
||||
|
||||
Accepted.
|
||||
|
||||
## Context
|
||||
|
||||
The most common serious failure is not a rejection. It is a request whose body reached the provider
|
||||
and whose response never came back. The platform has no provider request id, and the user may or may
|
||||
not have received the notification.
|
||||
|
||||
Treating that as a failure produces duplicates: a retry sends a second message, and a cross-channel
|
||||
fallback sends the SMS next to the push that already arrived. Treating it as a success loses real
|
||||
failures.
|
||||
|
||||
## Decision
|
||||
|
||||
`AMBIGUOUS` is a stored `SubmissionOutcome` and `AttemptConfirmation`. Attempts record
|
||||
`requestStarted`, `requestBodyCommitted` and `providerResponseReceived`, each with an
|
||||
`EvidenceCertainty` of `PROVEN`, `INFERRED` or `UNKNOWN`, so an adapter that does not know is not
|
||||
forced to answer `false`.
|
||||
|
||||
While an ambiguous attempt exists on a recipient delivery:
|
||||
|
||||
- automatic retry is blocked unless the provider proves per-request idempotency
|
||||
- automatic cross-channel fallback is blocked unconditionally
|
||||
- reconciliation runs where the provider supports a status query
|
||||
- otherwise the delivery stops and waits for an operator
|
||||
|
||||
Operator redrive of an ambiguous attempt requires explicit duplicate-risk approval.
|
||||
|
||||
## Consequences
|
||||
|
||||
Some notifications stop in a state that needs a human or a reconciliation pass. That is the intended
|
||||
trade: an unresolved unknown is cheaper than a guaranteed duplicate, and the state is visible rather
|
||||
than silently resolved in either direction.
|
||||
@@ -0,0 +1,27 @@
|
||||
# NOTIF-ADR-004 — FCM installation id is the primary target
|
||||
|
||||
## Status
|
||||
|
||||
Accepted.
|
||||
|
||||
## Context
|
||||
|
||||
Firebase now recommends the installation id (FID) and treats registration-token multicast paths as
|
||||
legacy. A contact point model built on a single `token` string would encode the older model as the
|
||||
only one, and a later migration would be a runtime interpretation problem: the same string field
|
||||
would mean different things for different rows.
|
||||
|
||||
## Decision
|
||||
|
||||
`MobilePushTarget` is a sealed hierarchy of `FcmInstallationId`, `LegacyFcmRegistrationToken` and
|
||||
`ApnsDeviceToken`. The kinds are separate types, never a discriminator on one string field, and each
|
||||
carries its own `ContactPointType` so the uniqueness scope and the encryption associated data differ.
|
||||
|
||||
APNs tokens additionally carry their environment, because sandbox and production are separate
|
||||
namespaces rather than a flag.
|
||||
|
||||
## Consequences
|
||||
|
||||
Migrating a target kind is a compile-time change with an exhaustive `switch`, not a runtime guess.
|
||||
The adapter maps each kind to its own wire representation, so a provider changing one path cannot
|
||||
silently change the other. The cost is one more type than a string field would need.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Callbacks and reconciliation
|
||||
|
||||
## Ingestion order
|
||||
|
||||
```text
|
||||
body size limit
|
||||
→ content type
|
||||
→ profile lookup
|
||||
→ signature verification
|
||||
→ append to the ledger
|
||||
→ duplicate detection
|
||||
→ normalization
|
||||
→ attempt resolution
|
||||
→ projection
|
||||
→ side effects
|
||||
→ 2xx
|
||||
```
|
||||
|
||||
Appending before projecting is what makes a fast 2xx honest. The provider is told the event is
|
||||
recorded, and a projector defect becomes a replay problem rather than a lost event.
|
||||
|
||||
A rejected signature is recorded in the security audit, never in the provider event ledger. Writing
|
||||
it to the ledger would let anyone who can reach the endpoint fill a delivery history with noise.
|
||||
|
||||
## Duplicates and ordering
|
||||
|
||||
Duplicate suppression uses `(providerProfileId, providerEventId)` where the provider supplies an
|
||||
event id, and a deterministic fingerprint over profile, request id, event type, occurrence time and
|
||||
payload digest where it does not. A duplicate is acknowledged and projected exactly once.
|
||||
|
||||
Out-of-order callbacks are normal. Ordering is resolved by event semantics, not by arrival time.
|
||||
|
||||
## Unknown fields
|
||||
|
||||
Callback parsers tolerate unknown JSON fields. Normalization only rejects a payload when a field
|
||||
required to identify the attempt is missing. Providers add fields; that must not stop ingestion.
|
||||
|
||||
## Reconciliation
|
||||
|
||||
Reconciliation targets:
|
||||
|
||||
- attempts stuck in `DISPATCHING` past their lease
|
||||
- ambiguous submissions
|
||||
- accepted attempts whose callback SLA has expired
|
||||
- unmatched provider events
|
||||
|
||||
A confirmed query result is appended to the same ledger with `source = RECONCILIATION` and projected
|
||||
by the same projector, so projection replay stays possible: there is no privileged second path that
|
||||
writes projections directly.
|
||||
|
||||
Where a provider has no status-query capability, the platform records `Unsupported` and leaves the
|
||||
attempt ambiguous. It does not infer a final status.
|
||||
@@ -0,0 +1,41 @@
|
||||
# Configuration reference
|
||||
|
||||
## Dispatch
|
||||
|
||||
| Property | Meaning | Bound |
|
||||
|---|---|---|
|
||||
| `claim-batch-size` | Rows claimed per scheduler tick | 1..1000 |
|
||||
| `lease-duration` | How long a claimed job stays owned | positive, finite |
|
||||
| `max-global-concurrency` | Ceiling across all providers | positive |
|
||||
| `max-queue-age` | Age at which a job is escalated | positive |
|
||||
| `max-retry-concurrency` | Ceiling for retry work | positive |
|
||||
| `scheduler-poll-interval` | Queue poll cadence | positive |
|
||||
| `callback-worker-concurrency` | Callback projection workers | positive |
|
||||
|
||||
Every value is bounded. "Unlimited" is not an accepted configuration.
|
||||
|
||||
## Provider profiles
|
||||
|
||||
A profile pins provider type, environment, credential profile, timeouts, concurrency, rate limit,
|
||||
retry policy and callback profile. Sender identity and credential profile are separate concerns.
|
||||
|
||||
## Startup failures
|
||||
|
||||
Startup fails rather than degrading when:
|
||||
|
||||
- a payload or queue setting is unbounded
|
||||
- a timeout is negative
|
||||
- a TTL-required profile has no expiry source
|
||||
- a callback signing secret is missing
|
||||
- a production profile enables trust-all
|
||||
- an APNs profile is missing its environment or topic
|
||||
- a Web Push profile is missing its VAPID key
|
||||
- two provider profiles share an id
|
||||
- a route points only at disabled providers
|
||||
- ambiguous fallback is enabled by default
|
||||
|
||||
## Secrets
|
||||
|
||||
All key material arrives through `SecretMaterialProvider`. Nothing is read from source, from a
|
||||
committed file, or from a plaintext log. Contact point encryption and lookup HMAC keys must be
|
||||
distinct, and the encryption key must be exactly 256 bits.
|
||||
@@ -0,0 +1,62 @@
|
||||
# Delivery evidence model
|
||||
|
||||
## The shape
|
||||
|
||||
```text
|
||||
NotificationRequest
|
||||
└─ RecipientDelivery
|
||||
└─ DeliveryAttempt
|
||||
└─ ProviderEvent (append-only)
|
||||
└─ channel projector
|
||||
└─ SubmissionOutcome / DeliveryOutcome / EvidenceLevel
|
||||
+ EngagementFacts + SuppressionFacts
|
||||
```
|
||||
|
||||
Four identities, four lifecycles. A logical request is not a recipient job, a recipient job is not a
|
||||
provider attempt, and a provider attempt is not the event stream that describes it.
|
||||
|
||||
## Why not one status enum
|
||||
|
||||
A single linear status would have to answer "what happened?" with one value, and the real answers do
|
||||
not fit on one line:
|
||||
|
||||
- An email can be `DELIVERED` and then generate a complaint. Both facts are true and both matter:
|
||||
one for reporting, the other for suppression.
|
||||
- Twilio does not guarantee callback ordering, so `sent` routinely arrives after `delivered`. Under
|
||||
an ordinal rule the later, weaker event silently overwrites the stronger one.
|
||||
- APNs may accept a notification and then store, replace or discard it.
|
||||
|
||||
So the ledger stores events and a channel projector merges them through an explicit transition table.
|
||||
`StandardDeliveryProjector` holds the shared rules; provider projectors add only their own event
|
||||
vocabulary.
|
||||
|
||||
## Merge rules
|
||||
|
||||
| Transition | Result |
|
||||
|---|---|
|
||||
| `sent` → `delivered` | applied |
|
||||
| `delivered` → `sent` | ignored, event still stored |
|
||||
| `delivered` → `complaint` | complaint fact added, delivery preserved |
|
||||
| `complaint` → `delivered` | delivery applied, complaint preserved |
|
||||
| `accepted` → `bounced` | applied |
|
||||
| `read` → `displayed` | ignored |
|
||||
| hard bounce → `delivered` | ignored, hard bounce is terminal |
|
||||
|
||||
Engagement (`opened`, `clicked`) is stored beside the delivery outcome and never changes it.
|
||||
|
||||
## Ambiguity
|
||||
|
||||
```text
|
||||
platform ──── send ────▶ provider
|
||||
│
|
||||
└── accepted
|
||||
✗ connection reset
|
||||
```
|
||||
|
||||
The platform may hold no provider request id while the notification really was sent. The attempt
|
||||
records `requestStarted`, `requestBodyCommitted`, `providerResponseReceived` and an
|
||||
`EvidenceCertainty` for each, so a later decision can tell "we know nothing was sent" apart from "we
|
||||
could not read the answer".
|
||||
|
||||
`ProviderSubmissionResult` enforces this: an ambiguous result may not claim `PROVIDER_ACCEPTED`, and
|
||||
no submission result of any kind may carry a delivery outcome.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Migration guide
|
||||
|
||||
## From the R0 routing seam
|
||||
|
||||
The pre-existing `dev.caskeleton.adapter.outbound.notification` router (`RoutingNotifier`,
|
||||
`FailOpenNotificationProvider`, the Google email and Slack webhook seams) stays untouched. The
|
||||
delivery platform lives beside it under `…notification.platform` and does not modify or delete any
|
||||
R0 class.
|
||||
|
||||
Migration order per capability:
|
||||
|
||||
1. Register the contact points behind `ContactPointStorePort` so the platform owns protected values.
|
||||
2. Publish the template version, and pin the template id, version and locale at every call site.
|
||||
3. Move the call site from the router to the N1 typed facade for the channel.
|
||||
4. Verify evidence in the snapshot rather than in the caller's return value: `submit()` is durable
|
||||
acceptance and nothing more.
|
||||
5. Remove the R0 route only after the platform route has produced provider evidence in the target
|
||||
environment.
|
||||
|
||||
## Return-value semantics change
|
||||
|
||||
The R0 seam returned a send-shaped result. `NotificationReceipt` returns `notificationId`, a request
|
||||
status and an acceptance time. Callers that treated the old return value as proof of delivery must be
|
||||
changed; there is no compatibility shim, because a shim would have to invent the delivery claim this
|
||||
platform exists to avoid.
|
||||
|
||||
## FCM target migration
|
||||
|
||||
Registration tokens keep working through `LegacyFcmRegistrationToken`. New registrations should use
|
||||
`FcmInstallationId`. The two are distinct types, so a migration is a compile-time task rather than a
|
||||
runtime guess.
|
||||
@@ -0,0 +1,94 @@
|
||||
# Notification Delivery Platform — module mapping
|
||||
|
||||
> Source design: `notification-superpowers-package/docs/superpowers/specs/2026-08-10-notification-platform-design.md`
|
||||
>
|
||||
> Source plan: `notification-superpowers-package/docs/superpowers/plans/2026-08-10-notification-platform-implementation-plan.md`
|
||||
|
||||
## Why a mapping exists
|
||||
|
||||
The plan was written against a hypothetical repository (`modules/notification/**`, root package
|
||||
`io.backend.skeleton.notification`, 31 Gradle projects). This repository is a fail-closed
|
||||
19-leaf Clean Architecture template: `src/settings.gradle` rejects any registry that does not
|
||||
contain exactly the 19 modules in `src/config/architecture/modules.json`, and
|
||||
`verifyCleanArchitectureDependencies` rejects any project edge outside `allowed_dependencies`.
|
||||
|
||||
Creating 31 new Gradle projects would violate HARD-STOP #5 of `AGENTS.md`. The package README
|
||||
anticipates this and instructs the implementer to map dependency catalog and package/file paths onto
|
||||
the host repository's rules while preserving the public contracts and reliability semantics.
|
||||
|
||||
Every logical module of the plan is therefore implemented as a **package** inside the registered leaf
|
||||
that owns its responsibility. No public contract, evidence rule, or reliability semantic is dropped.
|
||||
|
||||
## Logical module → registered leaf
|
||||
|
||||
| Plan module | Registered leaf | Package |
|
||||
|---|---|---|
|
||||
| `notification-core-api` | `application-core` | `dev.caskeleton.application.notification.platform.api` |
|
||||
| `notification-content-api` | `application-core` | `…platform.api.content` |
|
||||
| `notification-contact-api` | `application-core` | `…platform.contact` |
|
||||
| `notification-template-api` | `application-core` | `…platform.template` |
|
||||
| `notification-policy` | `application-core` | `…platform.policy` |
|
||||
| `notification-provider-spi` | `application-core` | `…platform.provider` |
|
||||
| `notification-callback-api` | `application-core` | `…platform.callback` |
|
||||
| `notification-email-api` | `application-core` | `…platform.email` |
|
||||
| `notification-sms-api` | `application-core` | `…platform.sms` |
|
||||
| `notification-push-api` | `application-core` | `…platform.push` |
|
||||
| `notification-webpush` (API half) | `application-core` | `…platform.webpush` |
|
||||
| `notification-inbox-api` | `application-core` | `…platform.inbox` |
|
||||
| `notification-admin-api` | `application-core` | `…platform.admin` |
|
||||
| `notification-security` (ports + redaction) | `application-core` | `…platform.security` |
|
||||
| `notification-observability` (ports) | `application-core` | `…platform.observation` |
|
||||
| `notification-dispatch-runtime` | `adapter:outbound:notification` | `dev.caskeleton.adapter.outbound.notification.platform.dispatch` |
|
||||
| `notification-security` (AES-GCM/HMAC impl) | `adapter:outbound:notification` | `…platform.security` |
|
||||
| `notification-template-thymeleaf` (reference renderer) | `adapter:outbound:notification` | `…platform.template` |
|
||||
| `notification-email-smtp` | `adapter:outbound:notification` | `…platform.provider.smtp` |
|
||||
| `notification-email-ses` | `adapter:outbound:notification` | `…platform.provider.ses` |
|
||||
| `notification-sms-twilio` | `adapter:outbound:notification` | `…platform.provider.twilio` |
|
||||
| `notification-push-fcm` | `adapter:outbound:notification` | `…platform.provider.fcm` |
|
||||
| `notification-push-apns` | `adapter:outbound:notification` | `…platform.provider.apns` |
|
||||
| `notification-webpush` (transport + crypto) | `adapter:outbound:notification` | `…platform.provider.webpush` |
|
||||
| `notification-webhook-extension` | `adapter:outbound:notification` | `…platform.provider.webhook` |
|
||||
| `notification-observability` (Micrometer impl) | `adapter:outbound:notification` | `…platform.observation` |
|
||||
| `notification-admin-runtime` | `adapter:outbound:notification` | `…platform.admin` |
|
||||
| `notification-reactor` | `adapter:outbound:notification` | `…platform.reactor` |
|
||||
| `notification-spring-boot-starter` | `adapter:outbound:notification` (+ `app-bootstrap` wiring) | `…platform.autoconfigure` |
|
||||
| `notification-persistence-jpa` | `adapter:outbound:persistence-jpa` | `dev.caskeleton.adapter.outbound.persistence.notification.platform` |
|
||||
| `notification-inbox-jpa` | `adapter:outbound:persistence-jpa` | `…persistence.notification.platform.inbox` |
|
||||
| `notification-callback-mvc` | `adapter:inbound:web` | `dev.caskeleton.adapter.inbound.web.notification.platform.callback` |
|
||||
| `notification-callback-webflux` | `adapter:inbound:web` | `…callback.reactive` |
|
||||
| `notification-testkit` | test source sets of the owning leaves | `…platform.testkit` |
|
||||
|
||||
## Dependency-direction consequences
|
||||
|
||||
The plan's module DAG (`*-api` → `provider-spi`/`policy` → runtime/adapters → starter) is preserved
|
||||
by the leaf DAG that the registry already enforces:
|
||||
|
||||
```text
|
||||
application-core (all *-api, provider SPI, policy, callback contracts)
|
||||
↑ ↑ ↑
|
||||
adapter:outbound:notification adapter:outbound:persistence-jpa adapter:inbound:web
|
||||
↑ ↑ ↑
|
||||
app-bootstrap
|
||||
```
|
||||
|
||||
Two plan edges cannot be expressed as project edges in this repository, and are replaced by ports:
|
||||
|
||||
1. `notification-email-ses`, `notification-sms-twilio`, `notification-push-fcm`,
|
||||
`notification-push-apns`, `notification-webpush`, `notification-webhook-extension`
|
||||
→ `httpclient platform`.
|
||||
`adapter-outbound-notification` is not allowed to depend on `adapter-outbound-httpclient`.
|
||||
The provider adapters therefore call
|
||||
`dev.caskeleton.adapter.outbound.notification.platform.provider.http.NotificationHttpGateway`,
|
||||
an adapter-local port with a JDK `java.net.http.HttpClient` default implementation.
|
||||
`app-bootstrap` sees both leaves and is the supported place to substitute an implementation backed
|
||||
by the HTTP Client Platform (TLS/timeout/circuit-breaker/SSRF/dynamic-target policy reuse).
|
||||
2. `notification-inbox-jpa` → `optional messaging outbox integration`.
|
||||
`adapter-outbound-persistence-jpa` may not depend on `adapter-outbound-messaging`; the inbox
|
||||
publishes through the existing persistence outbox tables plus the
|
||||
`NotificationInboxSignalPort` application port, and `app-bootstrap` binds the relay.
|
||||
|
||||
## Commit policy
|
||||
|
||||
`AGENTS.md` pins commit policy to `human-only`. Step 5 (`git add` / `git commit`) of every plan task
|
||||
is therefore intentionally **not** executed by the agent; the working tree carries the change and the
|
||||
human owner commits.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Operations
|
||||
|
||||
## Runtime shape
|
||||
|
||||
```text
|
||||
durable queue (PostgreSQL, FOR UPDATE SKIP LOCKED)
|
||||
→ expiry check
|
||||
→ suppression and eligibility re-check
|
||||
→ provider health gate
|
||||
→ rate limiter
|
||||
→ concurrency limiter
|
||||
→ provider adapter
|
||||
```
|
||||
|
||||
Provider calls run outside every database transaction. The attempt row is committed first, so after a
|
||||
crash the row is either absent (nothing was sent) or present in `DISPATCHING` (reconciliation has
|
||||
something to ask about).
|
||||
|
||||
## Guards that exist for specific incidents
|
||||
|
||||
| Guard | The incident it prevents |
|
||||
|---|---|
|
||||
| Credential failure opens the provider route | One expired key multiplied by a queue becomes a self-inflicted outage |
|
||||
| Retry budget per provider profile | A provider outage turning every queued notification into its own retry loop |
|
||||
| Ambiguous attempts block automatic fallback | A push whose response was lost arriving alongside the "just in case" SMS |
|
||||
| Permits released during backoff | A slow provider pinning the whole concurrency budget on work that is only waiting |
|
||||
| Bounded drain on rotation | A provider that never answers holding a credential rotation open forever |
|
||||
| Fail-fast intake on capacity | An unbounded in-memory queue absorbing a burst it cannot survive |
|
||||
|
||||
## Scheduling
|
||||
|
||||
`scheduleAt` activates the job, `notBefore` is the earliest permitted provider submission, and
|
||||
`expiresAt` blocks new attempts, retries and fallbacks. Suppression and expiry are re-checked
|
||||
immediately before dispatch, because a scheduled notification can sit in the queue for hours and the
|
||||
user may have opted out in the meantime.
|
||||
|
||||
## Redrive
|
||||
|
||||
A redrive preserves `NotificationId` and `RecipientDeliveryId`, creates a new `DeliveryAttemptId`, and
|
||||
reuses the pinned template version and rendered digest. Sending different content is a new
|
||||
notification, not a redrive. Redriving an ambiguous attempt requires explicit duplicate-risk approval,
|
||||
because the platform genuinely cannot tell whether the first submission reached the user.
|
||||
|
||||
## Actuator surface
|
||||
|
||||
Provider runtime states and generations, queue depth and age, callback and reconciliation health.
|
||||
Never addresses, never credentials.
|
||||
@@ -0,0 +1,65 @@
|
||||
# Provider runbooks
|
||||
|
||||
## SMTP
|
||||
|
||||
| Symptom | Classification | Action |
|
||||
|---|---|---|
|
||||
| Final `2xx` after `DATA` | `CONFIRMED_ACCEPTED` / `PROVIDER_ACCEPTED` | None; this is acceptance, not inbox delivery |
|
||||
| `4yz` | `TRANSIENT_PROVIDER` | Retry under budget and deadline |
|
||||
| `5yz` | `PERMANENT_PROVIDER` or `INVALID_RECIPIENT` | Stop, or invalidate the contact point |
|
||||
| Connection lost after `DATA` | `AMBIGUOUS_SUBMISSION` | Reconcile or escalate; do not resend automatically |
|
||||
|
||||
Connection, read, write and pool-acquire timeouts are all finite. There is no unbounded timeout.
|
||||
|
||||
## Amazon SES
|
||||
|
||||
`MessageId` is acceptance evidence. SES itself documents that it can accept a request and then not
|
||||
send, so `MessageId` is never mapped to `DELIVERED`.
|
||||
|
||||
| Event | Normalized |
|
||||
|---|---|
|
||||
| `Send` | reinforces `PROVIDER_ACCEPTED` |
|
||||
| `Delivery` | `DELIVERY_CONFIRMED` / `NETWORK_OR_CARRIER_ACCEPTED` |
|
||||
| `DeliveryDelay` | delay fact |
|
||||
| `Bounce` (permanent) | `BOUNCED_HARD` plus hard-bounce suppression |
|
||||
| `Bounce` (transient) | `BOUNCED_SOFT`; retry policy input, not a suppression reason |
|
||||
| `Complaint` | complaint fact plus suppression |
|
||||
| `Reject` | `PROVIDER_REJECTED` |
|
||||
| `RenderingFailure` | `TEMPLATE_FAILURE` |
|
||||
|
||||
## Twilio
|
||||
|
||||
`accepted`/`queued` is acceptance only. `sent` is carrier acceptance. `delivered` is device delivery.
|
||||
|
||||
Callbacks are not ordered. A `sent` arriving after `delivered` is stored and ignored by the
|
||||
projection. Missing callbacks are corrected by status polling under the provider rate limit.
|
||||
|
||||
Signature verification uses the canonical external URL from the profile, not the URL the servlet
|
||||
container reconstructed behind a proxy.
|
||||
|
||||
## FCM
|
||||
|
||||
| Error | Classification |
|
||||
|---|---|
|
||||
| `UNREGISTERED` | `INVALID_RECIPIENT`; invalidate the contact point, never retry |
|
||||
| `INVALID_ARGUMENT` | `INVALID_PAYLOAD` |
|
||||
| `QUOTA_EXCEEDED` | `THROTTLED`, exponential backoff |
|
||||
| `UNAVAILABLE` | `TRANSIENT_PROVIDER`, honour `Retry-After`, add jitter |
|
||||
| Credential failure | `AUTHENTICATION`; opens the provider route |
|
||||
|
||||
A batch is one transport call and many attempts. Partial results map back by input index; one
|
||||
transport failure does not become one shared outcome unless the adapter can prove it.
|
||||
|
||||
## APNs
|
||||
|
||||
2xx is acceptance. Environment and topic mismatches are configuration failures, not delivery
|
||||
failures. Sandbox and production tokens are separate namespaces.
|
||||
|
||||
## Web Push
|
||||
|
||||
`TTL` is mandatory by protocol. `201` is acceptance. `404` is an expired subscription per RFC 8030;
|
||||
provider-documented `410` maps the same way. Payloads use `aes128gcm` per RFC 8291 and VAPID JWTs are
|
||||
signed per RFC 8292 with the audience taken from the endpoint origin.
|
||||
|
||||
VAPID key rotation is not ordinary credential rotation: a restricted subscription may need to be
|
||||
re-created, so it is a migration operation.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Security and privacy
|
||||
|
||||
## Protected values
|
||||
|
||||
Email addresses, phone numbers, FCM installation ids and legacy tokens, APNs device tokens, Web Push
|
||||
endpoints and keys, VAPID private keys, provider credentials, callback signing secrets, template
|
||||
variables, rendered bodies, attachment references and unsubscribe tokens.
|
||||
|
||||
## At rest
|
||||
|
||||
Contact points are encrypted with AES-256-GCM. Equality lookup uses a separate HMAC-SHA-256
|
||||
fingerprint.
|
||||
|
||||
Two keys, not one, because the requirements are opposite: the ciphertext must be non-deterministic so
|
||||
two records of the same address are not visibly identical, while equality lookup must be
|
||||
deterministic. The fingerprint is keyed rather than a plain digest because phone numbers and email
|
||||
addresses come from a small, enumerable space — an unkeyed hash of a phone number is recoverable in
|
||||
seconds.
|
||||
|
||||
The contact point kind is bound into the GCM associated data, so a ciphertext cannot be moved between
|
||||
contact kinds without failing the authentication tag.
|
||||
|
||||
An unknown key id is refused rather than silently falling back to the current key: a silent fallback
|
||||
would turn every historical row into a tag failure at read time.
|
||||
|
||||
## Never logged, never a metric tag
|
||||
|
||||
Addresses, tokens, Web Push endpoints and keys, message bodies, template variables, provider
|
||||
credentials, unsubscribe tokens, attachment URLs, raw callback payloads and raw provider request ids.
|
||||
|
||||
Two mechanisms enforce this rather than convention:
|
||||
|
||||
- `CardinalityGuard` validates every metric tag against a closed allowlist.
|
||||
- `SafeDiagnosticContext` rejects any structured-diagnostic field outside its allowlist.
|
||||
|
||||
An allowlist rather than a denylist, because the failure mode of a denylist is that the one field
|
||||
nobody thought of is the one that leaks.
|
||||
|
||||
Every contact point value type overrides `toString()` to print `[redacted]`. That covers the case a
|
||||
central redactor cannot: a value interpolated into a log line by accident.
|
||||
|
||||
## Web Push endpoints
|
||||
|
||||
RFC 8030 defines the push URI as a capability URL — knowing it is sufficient to push to the
|
||||
subscriber. It is handled as a secret, not as a URL.
|
||||
|
||||
## Callbacks
|
||||
|
||||
TLS, provider signature verification over the exact received bytes and external URL, replay defence
|
||||
where a timestamp or nonce is available, body-size and content-type limits, profile binding, rate
|
||||
limiting, idempotent ingestion and a security audit trail for rejections.
|
||||
|
||||
## Tenant isolation
|
||||
|
||||
Every store port carries the tenant boundary in its signature. Administrative operations require an
|
||||
explicit tenant or a global authority.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Notification support matrix
|
||||
|
||||
What each channel can actually prove, and what the platform refuses to claim.
|
||||
|
||||
## Channels
|
||||
|
||||
| Channel | Reference implementation | Grade | Strongest evidence the platform records by default |
|
||||
|---|---|---|---|
|
||||
| Email | SMTP, Amazon SES API | Stable | Provider acceptance; recipient mail-server delivery, bounce and complaint when the provider publishes events |
|
||||
| SMS | Twilio Programmable Messaging | Stable | `accepted`/`queued`, `sent`, and carrier-DLR `delivered`/`undelivered` |
|
||||
| Mobile push (Android and cross-platform) | FCM, FID-first with legacy registration token compatibility | Stable | FCM acceptance and explicit failures |
|
||||
| Mobile push (Apple) | APNs HTTP/2 provider API | Stable | APNs acceptance |
|
||||
| Web Push | RFC 8030, RFC 8291, RFC 8292 | Stable | Push-service acceptance; user-agent acknowledgement only where the service offers receipts |
|
||||
| In-app inbox | Own database | Optional stable | `PERSISTED`, `SEEN`, `READ` |
|
||||
| Webhook | HTTP client platform | Extension | Whatever the receiving HTTP contract states |
|
||||
|
||||
## Evidence levels
|
||||
|
||||
`NONE` → `PLATFORM_QUEUED` → `PROVIDER_ACCEPTED` → `NETWORK_OR_CARRIER_ACCEPTED` →
|
||||
`DEVICE_DELIVERED` → `USER_AGENT_DISPLAYED` → `USER_READ`
|
||||
|
||||
| Provider signal | Highest evidence it may produce |
|
||||
|---|---|
|
||||
| Internal queue commit | `PLATFORM_QUEUED` |
|
||||
| SES `MessageId` | `PROVIDER_ACCEPTED` |
|
||||
| SES `Delivery` | `NETWORK_OR_CARRIER_ACCEPTED` |
|
||||
| Twilio `accepted` / `queued` | `PROVIDER_ACCEPTED` |
|
||||
| Twilio `sent` | `NETWORK_OR_CARRIER_ACCEPTED` |
|
||||
| Twilio `delivered` | `DEVICE_DELIVERED` |
|
||||
| FCM send success | `PROVIDER_ACCEPTED` |
|
||||
| APNs 2xx | `PROVIDER_ACCEPTED` |
|
||||
| Web Push `201` | `PROVIDER_ACCEPTED` |
|
||||
| Web Push receipt capability | `DEVICE_DELIVERED` |
|
||||
| In-app row commit | `PROVIDER_ACCEPTED` |
|
||||
| In-app `seen` endpoint | `USER_AGENT_DISPLAYED` |
|
||||
| In-app `read` endpoint, authenticated app receipt | `USER_READ` |
|
||||
|
||||
Promotions the platform will not make, in code or in configuration:
|
||||
|
||||
- FCM send success is not `DEVICE_DELIVERED`.
|
||||
- An APNs 2xx is not `DELIVERED`.
|
||||
- An SES `MessageId` is not `DELIVERED`.
|
||||
- An SMTP `250` is not inbox delivery.
|
||||
|
||||
## Submission outcomes
|
||||
|
||||
`NOT_SUBMITTED`, `CONFIRMED_ACCEPTED`, `CONFIRMED_REJECTED`, `AMBIGUOUS`.
|
||||
|
||||
`AMBIGUOUS` is a first-class stored state, not an error path. It means the request body was committed
|
||||
to the provider and the outcome could not be read. While an ambiguous attempt exists on a recipient
|
||||
delivery, automatic retry and automatic cross-channel fallback are both blocked.
|
||||
|
||||
## Not supported
|
||||
|
||||
The platform will not claim any of the following, because no channel above can support them:
|
||||
|
||||
- guaranteed delivery
|
||||
- guaranteed read
|
||||
- exactly-once human notification
|
||||
- unconditional multi-provider failover after an unread response
|
||||
- provider SDK types in the public API
|
||||
- audience selection, campaign segmentation or jurisdiction rulings
|
||||
|
||||
## Target model
|
||||
|
||||
`FCM_FID` is the primary mobile push target. `FCM_REGISTRATION_TOKEN_LEGACY` and
|
||||
`APNS_DEVICE_TOKEN` are separate types with separate lifecycles; they are never flattened into one
|
||||
string field.
|
||||
Reference in New Issue
Block a user