83 lines
4.3 KiB
Markdown
83 lines
4.3 KiB
Markdown
# Advanced — Sharding
|
|
|
|
**Capability:** `MongoCapability.SHARDING`
|
|
**Property:** `ca-skeleton.persistence-mongo.advanced.sharding.enabled`
|
|
**Status:** Advanced. Reshard orchestration remains Experimental.
|
|
|
|
## Requirements
|
|
|
|
| | |
|
|
|---|---|
|
|
| Topology | A real sharded cluster. A replica set cannot exercise routing. |
|
|
| Server | MongoDB 7.0 or 8.0. |
|
|
| Privilege | `MongoPrincipalRole.SHARD_ADMIN` for the admin plane; the application role is unchanged. |
|
|
| Gate | **Not promoted.** No `mongoShardedTest` lane is registered, and a sharded cluster is not an environment this repository stands up. Listed under `experimental_contracts` in `src/config/mongodb/release-contracts.json`; promoting it needs the lane, its required class, and protected-environment evidence to exist first. |
|
|
|
|
## Shard key
|
|
|
|
`ShardKeyDescriptor` declares the key as an ordered list of `ShardKeyPart` plus a `ShardStrategy`:
|
|
|
|
| Strategy | Distributes | Cost |
|
|
|---|---|---|
|
|
| `RANGE` | by value ranges | Range queries stay targeted; a monotonic key (a timestamp, an `ObjectId`) sends every insert to one shard. |
|
|
| `HASHED` | by hash of the key | Inserts spread evenly; every range query becomes scatter-gather. |
|
|
|
|
There is no strategy that is good at both, which is why the choice is a declaration rather than a
|
|
default.
|
|
|
|
## Routing classification
|
|
|
|
`ShardAwareQueryValidator` classifies each query before execution:
|
|
|
|
| `MongoRoutingClassification` | Meaning |
|
|
|---|---|
|
|
| `TARGETED` | The full shard key is present. One shard answers. |
|
|
| `PREFIX_TARGETED` | A prefix of a compound key is present. A subset of shards answers. |
|
|
| `SCATTER_GATHER` | No shard-key predicate. Every shard answers. |
|
|
| `REJECTED` | Scatter-gather where the profile forbids it. |
|
|
|
|
A scatter-gather query is not an error — some queries legitimately need every shard — but it must be
|
|
declared. Undeclared scatter-gather raises `MongoShardRoutingException`. The reason is that
|
|
scatter-gather passes every test on a single-shard development cluster and only degrades once the
|
|
cluster grows, at which point the query is already in production and the fix is a schema change.
|
|
|
|
## Unsupported combinations
|
|
|
|
- Unique index on a field that is not a prefix of the shard key. MongoDB cannot enforce it across
|
|
shards, and it fails at index creation, not at query time.
|
|
- Transactions that touch documents on multiple shards remain supported but cost a cross-shard
|
|
two-phase commit. Prefer a shard key that keeps a transaction's documents co-located.
|
|
- CSFLE on a sharded collection: see [encryption.md](encryption.md) for the combinations that are
|
|
refused.
|
|
|
|
## Admin plane
|
|
|
|
`MongoShardingAdminGateway` (D4, `SHARD_ADMIN` credential) covers shard-collection, refine-shard-key
|
|
and reshard.
|
|
|
|
`ShardKeyAnalyzer` produces a `ShardKeyReadinessReport` before sharding a collection: cardinality,
|
|
frequency skew and monotonicity. A key with low cardinality creates jumbo chunks that cannot be split;
|
|
a monotonic key creates a hot shard. Both are visible in the report and invisible in a functional
|
|
test.
|
|
|
|
`ReshardApproval` is required for a reshard — a named approver and a stated window. Resharding
|
|
rewrites the collection: it duplicates the data during the operation and saturates IO. It is not a
|
|
runtime operation and the type refuses to pretend otherwise.
|
|
|
|
## Failure recovery
|
|
|
|
| Symptom | Cause | Action |
|
|
|---|---|---|
|
|
| `MongoShardRoutingException` | Undeclared scatter-gather | Add the shard key to the predicate, or declare the query as scatter-gather in its profile after review. |
|
|
| Jumbo chunks | Low-cardinality shard key | Refine the shard key (adds a suffix, non-destructive) before considering a reshard. |
|
|
| One hot shard | Monotonic range key | Refine with a high-cardinality prefix, or reshard to hashed if range queries are not needed. |
|
|
| Balancer never converges | Chunk migration blocked by long-running operations | Check for long transactions and cursors; the balancer waits on them. |
|
|
|
|
## Promotion evidence
|
|
|
|
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md): actual sharded-cluster
|
|
evidence, security review of the `SHARD_ADMIN` role, a migration path for an existing unsharded
|
|
collection, failure cases (undeclared scatter-gather refused, jumbo chunk detected), and this
|
|
document as the runbook. Reshard orchestration stays Experimental until operational scale evidence
|
|
exists.
|