DongHyeonkaandClaude Opus 5
d57d2f62a0
Implements the mongodb-superpowers-package design: Stable Tasks 1-50 and Advanced Tasks 1-15. The design assumes 19 Stable + 12 Advanced Gradle projects under modules/mongodb*. This repository's fail-closed registry declares exactly 19 leaf identities, so those modules become package boundaries inside the registered leaf :adapter:outbound:persistence-mongo, with the design's module dependency table enforced by ten ArchUnit rules. The mapping and every deviation are recorded in docs/mongodb/repository-adaptation.md. Contract highlights, all enforced by tests rather than convention: - Transaction body retry and commit retry are separate loops. A new session per body attempt; commit-only retry on an unknown commit. The body is never replayed after a commit ambiguity, so a failover cannot become a duplicate. - MongoExecutionOutcome keeps both ambiguous outcomes distinct from success and failure, and MongoFailureContext records only the design-permitted fields. - Failure classification reads server error labels before numeric codes. - BSON representations come from a pinned manifest, never a library default, and a golden type-signature gate fails on any drift. - Index and validator changes go through the manifest and the admin plane; metadata ownership gates every drop. - Every Advanced capability refuses construction unless its flag is enabled. Verified against real servers, not only unit tests. Running the lanes for the first time exposed four defects that a green `check` had hidden: - Four release lanes passed while executing zero tests; the gate now counts executed tests per lane and fails on zero. - The "single replica set" fixture was a standalone, because Testcontainers 2.x needs withReplicaSet(); its test only asserted a connection string. - The three-node fixture was three independent clusters, so no election could occur, and awaitNewPrimary() compared against the post-stop primary. - The migration lease checked modifiedCount, so a same-millisecond refresh read as a lost lease. scripts/verify-mongodb-platform.sh now reports: 9 lanes, 0 skipped, 0 failed, every evidence category produced. scripts/verify-mongodb-advanced.sh reports NOT PROMOTABLE: actual-topology evidence (real sharded cluster, real KMS, real target deployment) is unobtainable here, so it is named rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
83 lines
4.0 KiB
Markdown
83 lines
4.0 KiB
Markdown
# Advanced — Sharding
|
|
|
|
**Capability:** `MongoCapability.SHARDING`
|
|
**Property:** `ca-skeleton.persistence-mongo.advanced.sharding.enabled`
|
|
**Status:** Advanced. Reshard orchestration remains Experimental.
|
|
|
|
## Requirements
|
|
|
|
| | |
|
|
|---|---|
|
|
| Topology | A real sharded cluster. A replica set cannot exercise routing. |
|
|
| Server | MongoDB 7.0 or 8.0. |
|
|
| Privilege | `MongoPrincipalRole.SHARD_ADMIN` for the admin plane; the application role is unchanged. |
|
|
| Gate | `mongoShardedTest` lane with `MongoShardingContractSuite`. |
|
|
|
|
## Shard key
|
|
|
|
`ShardKeyDescriptor` declares the key as an ordered list of `ShardKeyPart` plus a `ShardStrategy`:
|
|
|
|
| Strategy | Distributes | Cost |
|
|
|---|---|---|
|
|
| `RANGE` | by value ranges | Range queries stay targeted; a monotonic key (a timestamp, an `ObjectId`) sends every insert to one shard. |
|
|
| `HASHED` | by hash of the key | Inserts spread evenly; every range query becomes scatter-gather. |
|
|
|
|
There is no strategy that is good at both, which is why the choice is a declaration rather than a
|
|
default.
|
|
|
|
## Routing classification
|
|
|
|
`ShardAwareQueryValidator` classifies each query before execution:
|
|
|
|
| `MongoRoutingClassification` | Meaning |
|
|
|---|---|
|
|
| `TARGETED` | The full shard key is present. One shard answers. |
|
|
| `PREFIX_TARGETED` | A prefix of a compound key is present. A subset of shards answers. |
|
|
| `SCATTER_GATHER` | No shard-key predicate. Every shard answers. |
|
|
| `REJECTED` | Scatter-gather where the profile forbids it. |
|
|
|
|
A scatter-gather query is not an error — some queries legitimately need every shard — but it must be
|
|
declared. Undeclared scatter-gather raises `MongoShardRoutingException`. The reason is that
|
|
scatter-gather passes every test on a single-shard development cluster and only degrades once the
|
|
cluster grows, at which point the query is already in production and the fix is a schema change.
|
|
|
|
## Unsupported combinations
|
|
|
|
- Unique index on a field that is not a prefix of the shard key. MongoDB cannot enforce it across
|
|
shards, and it fails at index creation, not at query time.
|
|
- Transactions that touch documents on multiple shards remain supported but cost a cross-shard
|
|
two-phase commit. Prefer a shard key that keeps a transaction's documents co-located.
|
|
- CSFLE on a sharded collection: see [encryption.md](encryption.md) for the combinations that are
|
|
refused.
|
|
|
|
## Admin plane
|
|
|
|
`MongoShardingAdminGateway` (D4, `SHARD_ADMIN` credential) covers shard-collection, refine-shard-key
|
|
and reshard.
|
|
|
|
`ShardKeyAnalyzer` produces a `ShardKeyReadinessReport` before sharding a collection: cardinality,
|
|
frequency skew and monotonicity. A key with low cardinality creates jumbo chunks that cannot be split;
|
|
a monotonic key creates a hot shard. Both are visible in the report and invisible in a functional
|
|
test.
|
|
|
|
`ReshardApproval` is required for a reshard — a named approver and a stated window. Resharding
|
|
rewrites the collection: it duplicates the data during the operation and saturates IO. It is not a
|
|
runtime operation and the type refuses to pretend otherwise.
|
|
|
|
## Failure recovery
|
|
|
|
| Symptom | Cause | Action |
|
|
|---|---|---|
|
|
| `MongoShardRoutingException` | Undeclared scatter-gather | Add the shard key to the predicate, or declare the query as scatter-gather in its profile after review. |
|
|
| Jumbo chunks | Low-cardinality shard key | Refine the shard key (adds a suffix, non-destructive) before considering a reshard. |
|
|
| One hot shard | Monotonic range key | Refine with a high-cardinality prefix, or reshard to hashed if range queries are not needed. |
|
|
| Balancer never converges | Chunk migration blocked by long-running operations | Check for long transactions and cursors; the balancer waits on them. |
|
|
|
|
## Promotion evidence
|
|
|
|
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md): actual sharded-cluster
|
|
evidence, security review of the `SHARD_ADMIN` role, a migration path for an existing unsharded
|
|
collection, failure cases (undeclared scatter-gather refused, jumbo chunk detected), and this
|
|
document as the runbook. Reshard orchestration stays Experimental until operational scale evidence
|
|
exists.
|