DongHyeonkaandClaude Opus 5
d57d2f62a0
Implements the mongodb-superpowers-package design: Stable Tasks 1-50 and Advanced Tasks 1-15. The design assumes 19 Stable + 12 Advanced Gradle projects under modules/mongodb*. This repository's fail-closed registry declares exactly 19 leaf identities, so those modules become package boundaries inside the registered leaf :adapter:outbound:persistence-mongo, with the design's module dependency table enforced by ten ArchUnit rules. The mapping and every deviation are recorded in docs/mongodb/repository-adaptation.md. Contract highlights, all enforced by tests rather than convention: - Transaction body retry and commit retry are separate loops. A new session per body attempt; commit-only retry on an unknown commit. The body is never replayed after a commit ambiguity, so a failover cannot become a duplicate. - MongoExecutionOutcome keeps both ambiguous outcomes distinct from success and failure, and MongoFailureContext records only the design-permitted fields. - Failure classification reads server error labels before numeric codes. - BSON representations come from a pinned manifest, never a library default, and a golden type-signature gate fails on any drift. - Index and validator changes go through the manifest and the admin plane; metadata ownership gates every drop. - Every Advanced capability refuses construction unless its flag is enabled. Verified against real servers, not only unit tests. Running the lanes for the first time exposed four defects that a green `check` had hidden: - Four release lanes passed while executing zero tests; the gate now counts executed tests per lane and fails on zero. - The "single replica set" fixture was a standalone, because Testcontainers 2.x needs withReplicaSet(); its test only asserted a connection string. - The three-node fixture was three independent clusters, so no election could occur, and awaitNewPrimary() compared against the post-stop primary. - The migration lease checked modifiedCount, so a same-millisecond refresh read as a lost lease. scripts/verify-mongodb-platform.sh now reports: 9 lanes, 0 skipped, 0 failed, every evidence category produced. scripts/verify-mongodb-advanced.sh reports NOT PROMOTABLE: actual-topology evidence (real sharded cluster, real KMS, real target deployment) is unobtainable here, so it is named rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
141 lines
6.2 KiB
Markdown
141 lines
6.2 KiB
Markdown
# Query and Aggregation Guide
|
|
|
|
Design §17–§19, decision D-11. Every query and every pipeline is a registered, bounded thing. Free-form
|
|
JSON queries and unbounded pipelines are explicitly unsupported (§3.4).
|
|
|
|
## 1. Registered operations
|
|
|
|
Every execution carries a `MongoOperationContext`: a `MongoOperationName`, a `DatabaseProfileName`, a
|
|
`CollectionProfileName`, a `MongoOperationType` and a `MongoOperationScope`.
|
|
|
|
`MongoOperationName` matches `[a-z][a-z0-9.-]{2,95}`. It is the join key for the budget registry, the
|
|
consistency registry, the metric tag and the log line — a free-form or interpolated name breaks all
|
|
four at once, which is why the pattern is enforced at construction.
|
|
|
|
`MongoOperationScope` uses an `UNSPECIFIED` sentinel rather than `null`, so "the caller did not say"
|
|
is a value the policy layer can reject rather than an NPE further down.
|
|
|
|
## 2. Query guardrails
|
|
|
|
`PolicyAwareMongoQueryBuilder` builds a query from `MongoFieldDescriptor` + `MongoOperator` pairs
|
|
against a `MongoQueryPolicy`. The policy refuses:
|
|
|
|
- a field not in the collection's allowlist
|
|
- an operator not allowed for that field
|
|
- a sort on an unindexed field
|
|
- `$where`, `$expr` with arbitrary JavaScript, and server-side evaluation generally
|
|
- an unbounded `$regex`
|
|
|
|
`MongoRegexPolicy` requires an anchored prefix pattern and bounds the pattern length. An unanchored
|
|
regex is a collection scan wearing an index's clothes, and a user-supplied one is a denial-of-service
|
|
primitive.
|
|
|
|
`MongoSortDescriptor` pairs a field with a direction and is validated against the index manifest, so
|
|
a sort that would spill to disk fails review rather than production.
|
|
|
|
## 3. Operation budgets
|
|
|
|
`MongoOperationBudget` bounds four things at once:
|
|
|
|
| Bound | Why |
|
|
|---|---|
|
|
| `maxTimeMS` | The server stops working on a query nobody is waiting for. |
|
|
| result limit | An unbounded result set is an OOM with extra steps. |
|
|
| batch size | Bounds the per-round-trip memory. |
|
|
| examined-document ceiling | Catches an index regression that a time limit alone would hide on a fast day. |
|
|
|
|
`MongoBudgetPolicyRegistry` binds a budget to an operation name; `MongoBudgetEnforcer` applies it and
|
|
raises `MongoOperationRejectedException` before execution when a request exceeds it, and
|
|
`MongoTimeoutException` when the server enforces it.
|
|
|
|
## 4. Keyset pagination
|
|
|
|
Unbounded `skip` is unsupported: `skip(1_000_000)` makes the server walk a million documents to throw
|
|
them away, so page 1000 costs a thousand times page 1.
|
|
|
|
`MongoKeysetQueryBuilder` builds the resume predicate lexicographically. For a sort on `(a DESC, _id
|
|
DESC)` resuming after `(A, I)`:
|
|
|
|
```
|
|
(a < A) OR (a = A AND _id < I)
|
|
```
|
|
|
|
`validate()` rejects a `MongoKeysetSort` without a unique tie-breaker. Without one, two documents with
|
|
the same sort value straddle the page boundary and one of them is skipped or repeated — invisibly,
|
|
and only under concurrency.
|
|
|
|
`MongoNullSortOrdering` makes null placement explicit, because MongoDB's own ordering of missing
|
|
versus null versus present is not what most people assume.
|
|
|
|
### Cursors are authenticated
|
|
|
|
`MongoKeysetCursorCodec` signs the cursor with HMAC-SHA256 and compares with
|
|
`MessageDigest.isEqual` (constant time). An unsigned cursor is a client-controlled query predicate: a
|
|
caller can edit it to read a range they were never offered. A tampered or truncated cursor yields
|
|
`MongoCursorException`, never a partially-decoded resume position.
|
|
|
|
## 5. Aggregation guardrails
|
|
|
|
`MongoAggregationPlan` is a registered pipeline: an ordered list of `MongoAggregationStageDescriptor`
|
|
validated against a `MongoAggregationProfile`. `PolicyAwareMongoAggregationExecutor` runs only a
|
|
registered plan.
|
|
|
|
`MongoAggregationRisk` grades each stage, and the profile sets the ceiling:
|
|
|
|
| Risk | Stages | Policy |
|
|
|---|---|---|
|
|
| low | `$match` on an indexed prefix, `$limit`, `$project` | Always allowed. |
|
|
| moderate | `$group`, `$sort` with an index, `$unwind` with a bound | Allowed within budget. |
|
|
| high | `$lookup`, `$graphLookup`, `$facet`, unindexed `$sort` | Requires explicit approval in the profile. |
|
|
| forbidden | `$out`, `$merge` outside the admin plane, `$function`, `$accumulator` | Refused. |
|
|
|
|
`allowDiskUse` is a declared property of the plan, not a runtime flag. A pipeline that needs disk is a
|
|
pipeline whose shape should be reviewed.
|
|
|
|
## 6. Reactive execution and cursors
|
|
|
|
`ReactiveMongoExecutor` / `DefaultReactiveMongoExecutor` carry the operation context in the Reactor
|
|
context. `MongoCursorGuard` and `MongoCursorLease` bound cursor lifetime:
|
|
|
|
- a cursor has a lease with a deadline
|
|
- cancellation closes the server-side cursor (`MongoCursorTermination`)
|
|
- an abandoned cursor is a server-side resource, so the lease is released on cancel, error *and*
|
|
completion — `MongoReactiveCursorPublisher` uses `Flux.using` so all three paths run the same
|
|
release
|
|
|
|
A leaked cursor does not fail anything locally; it consumes a connection and a snapshot on the server
|
|
until the server's own timeout, which is why the guard is not optional.
|
|
|
|
## 7. Geospatial
|
|
|
|
`MongoGeoQuery` + `MongoGeoPoint` + `MongoGeoDistance` over a `2dsphere` index. Distances are metres
|
|
on a sphere (`nearSphere` with `maxDistance`), never degrees — a degree of longitude is a different
|
|
distance in Oslo than in Nairobi, and a radius expressed in degrees is a bug that only shows up away
|
|
from the equator. `SpringMongoGeospatialOperations` is the Spring Data binding;
|
|
`MongoGeospatialOperations` is the port.
|
|
|
|
## 8. Native capability gateway
|
|
|
|
When a registered operation genuinely needs something outside the Stable API, it goes through
|
|
`MongoNativeCapabilityGateway` (`PolicyAwareMongoNativeGateway`), never through the driver directly.
|
|
The admission order is fixed:
|
|
|
|
```
|
|
capability registered
|
|
→ database profile
|
|
→ collection allowlist
|
|
→ operation name present
|
|
→ timeout / maxTimeMS
|
|
→ consistency profile
|
|
→ result / batch limit
|
|
→ trace
|
|
→ log redaction
|
|
→ command category (MongoNativeCommandCategory)
|
|
→ D4 admin command refused
|
|
→ execute
|
|
```
|
|
|
|
`ApprovedMongoNativeOperation` is the registration record; `MongoNativeOperationPolicy` is the policy.
|
|
An admin-plane command reaching this gateway is refused regardless of capability — the admin plane has
|
|
its own credential and its own client (see [security-observability.md](security-observability.md)).
|