feat(mongodb): implement the MongoDB document persistence platform
Implements the mongodb-superpowers-package design: Stable Tasks 1-50 and Advanced Tasks 1-15. The design assumes 19 Stable + 12 Advanced Gradle projects under modules/mongodb*. This repository's fail-closed registry declares exactly 19 leaf identities, so those modules become package boundaries inside the registered leaf :adapter:outbound:persistence-mongo, with the design's module dependency table enforced by ten ArchUnit rules. The mapping and every deviation are recorded in docs/mongodb/repository-adaptation.md. Contract highlights, all enforced by tests rather than convention: - Transaction body retry and commit retry are separate loops. A new session per body attempt; commit-only retry on an unknown commit. The body is never replayed after a commit ambiguity, so a failover cannot become a duplicate. - MongoExecutionOutcome keeps both ambiguous outcomes distinct from success and failure, and MongoFailureContext records only the design-permitted fields. - Failure classification reads server error labels before numeric codes. - BSON representations come from a pinned manifest, never a library default, and a golden type-signature gate fails on any drift. - Index and validator changes go through the manifest and the admin plane; metadata ownership gates every drop. - Every Advanced capability refuses construction unless its flag is enabled. Verified against real servers, not only unit tests. Running the lanes for the first time exposed four defects that a green `check` had hidden: - Four release lanes passed while executing zero tests; the gate now counts executed tests per lane and fails on zero. - The "single replica set" fixture was a standalone, because Testcontainers 2.x needs withReplicaSet(); its test only asserted a connection string. - The three-node fixture was three independent clusters, so no election could occur, and awaitNewPrimary() compared against the post-stop primary. - The migration lease checked modifiedCount, so a same-millisecond refresh read as a lost lease. scripts/verify-mongodb-platform.sh now reports: 9 lanes, 0 skipped, 0 failed, every evidence category produced. scripts/verify-mongodb-advanced.sh reports NOT PROMOTABLE: actual-topology evidence (real sharded cluster, real KMS, real target deployment) is unobtainable here, so it is named rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
3b5aee50e3
commit
d57d2f62a0
@@ -0,0 +1,63 @@
|
||||
# Advanced — GridFS compatibility and migration
|
||||
|
||||
**Capability:** `MongoCapability.GRIDFS_COMPATIBILITY`
|
||||
**Property:** `ca-skeleton.persistence-mongo.advanced.gridfs-compatibility.enabled`
|
||||
**Status:** Advanced, compatibility only. Decision D-14.
|
||||
|
||||
## Position
|
||||
|
||||
GridFS is a **compatibility adapter for files that already exist there**. New files use the existing
|
||||
Fileserver / Object Storage adapter, which is the source of truth for binary content.
|
||||
|
||||
The reason is not preference. GridFS stores file chunks in the same collections, on the same replica
|
||||
set, competing for the same working set as your documents. A large file read evicts document pages
|
||||
from cache, and file storage growth becomes replica-set growth — which means it becomes oplog
|
||||
pressure, backup duration and failover time. Object storage was built for this and MongoDB was not.
|
||||
|
||||
## Reading legacy files
|
||||
|
||||
`MongoGridFsCompatibilityReader` reads existing GridFS content as
|
||||
`GridFsLegacyContent(legacyId, filename, sizeBytes, checksum, stream)`. It reads; it does not write.
|
||||
|
||||
## Migration
|
||||
|
||||
`MongoGridFsMigrationJob` moves a file to object storage in a fixed order:
|
||||
|
||||
```
|
||||
read legacy content
|
||||
→ write to object storage
|
||||
→ verify the target checksum matches the source
|
||||
→ switch the reference
|
||||
→ (later, separately) delete the source
|
||||
```
|
||||
|
||||
Three properties, each of which exists because of a specific way this goes wrong:
|
||||
|
||||
1. **Verify before switching.** `MongoGridFsObjectReference` requires a non-blank checksum, and the
|
||||
job returns empty and writes no reference when the target checksum does not match the source. A
|
||||
migration that switches the reference on a successful *write* rather than a verified *copy*
|
||||
silently points at a truncated object.
|
||||
2. **The source is never deleted here.** Deletion is a separate, later decision after the new
|
||||
location has been serving reads long enough to be trusted. A migration that deletes as it goes has
|
||||
no rollback.
|
||||
3. **The checkpoint separates migrated from failed.** `MongoGridFsMigrationCheckpoint` tracks
|
||||
`migratedCount()`, `failedCount()`, `clean()` and `lastMigratedLegacyId()`, so a restart continues
|
||||
from the last completed file rather than starting over, and a partially failed run is visible as
|
||||
partial rather than as "done".
|
||||
|
||||
## Failure recovery
|
||||
|
||||
| Symptom | Cause | Action |
|
||||
|---|---|---|
|
||||
| `migrate` returns empty | Checksum mismatch | The copy is bad. Investigate before retrying; do not force the reference. |
|
||||
| `IllegalArgumentException` on the reference | Missing checksum | A reference without a checksum cannot be verified and is refused. |
|
||||
| Checkpoint not `clean()` | Some files failed | Re-run for the failed ids only; the checkpoint names the last successful one. |
|
||||
| Reference switched but content missing | Source deleted too early | Restore from backup. This is what rule 2 prevents. |
|
||||
|
||||
## Promotion evidence
|
||||
|
||||
Actual-topology evidence against the real object storage backend, a security review of the storage
|
||||
credential, the migration path above, failure cases (checksum mismatch refused, missing checksum
|
||||
refused, restart resumes), and this document as the runbook.
|
||||
|
||||
New file storage does not go through here at all — see the fileserver adapter.
|
||||
Reference in New Issue
Block a user