DongHyeonkaandClaude Opus 5
d57d2f62a0
Implements the mongodb-superpowers-package design: Stable Tasks 1-50 and Advanced Tasks 1-15. The design assumes 19 Stable + 12 Advanced Gradle projects under modules/mongodb*. This repository's fail-closed registry declares exactly 19 leaf identities, so those modules become package boundaries inside the registered leaf :adapter:outbound:persistence-mongo, with the design's module dependency table enforced by ten ArchUnit rules. The mapping and every deviation are recorded in docs/mongodb/repository-adaptation.md. Contract highlights, all enforced by tests rather than convention: - Transaction body retry and commit retry are separate loops. A new session per body attempt; commit-only retry on an unknown commit. The body is never replayed after a commit ambiguity, so a failover cannot become a duplicate. - MongoExecutionOutcome keeps both ambiguous outcomes distinct from success and failure, and MongoFailureContext records only the design-permitted fields. - Failure classification reads server error labels before numeric codes. - BSON representations come from a pinned manifest, never a library default, and a golden type-signature gate fails on any drift. - Index and validator changes go through the manifest and the admin plane; metadata ownership gates every drop. - Every Advanced capability refuses construction unless its flag is enabled. Verified against real servers, not only unit tests. Running the lanes for the first time exposed four defects that a green `check` had hidden: - Four release lanes passed while executing zero tests; the gate now counts executed tests per lane and fails on zero. - The "single replica set" fixture was a standalone, because Testcontainers 2.x needs withReplicaSet(); its test only asserted a connection string. - The three-node fixture was three independent clusters, so no election could occur, and awaitNewPrimary() compared against the post-stop primary. - The migration lease checked modifiedCount, so a same-millisecond refresh read as a lost lease. scripts/verify-mongodb-platform.sh now reports: 9 lanes, 0 skipped, 0 failed, every evidence category produced. scripts/verify-mongodb-advanced.sh reports NOT PROMOTABLE: actual-topology evidence (real sharded cluster, real KMS, real target deployment) is unobtainable here, so it is named rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
64 lines
3.1 KiB
Markdown
64 lines
3.1 KiB
Markdown
# Advanced — GridFS compatibility and migration
|
|
|
|
**Capability:** `MongoCapability.GRIDFS_COMPATIBILITY`
|
|
**Property:** `ca-skeleton.persistence-mongo.advanced.gridfs-compatibility.enabled`
|
|
**Status:** Advanced, compatibility only. Decision D-14.
|
|
|
|
## Position
|
|
|
|
GridFS is a **compatibility adapter for files that already exist there**. New files use the existing
|
|
Fileserver / Object Storage adapter, which is the source of truth for binary content.
|
|
|
|
The reason is not preference. GridFS stores file chunks in the same collections, on the same replica
|
|
set, competing for the same working set as your documents. A large file read evicts document pages
|
|
from cache, and file storage growth becomes replica-set growth — which means it becomes oplog
|
|
pressure, backup duration and failover time. Object storage was built for this and MongoDB was not.
|
|
|
|
## Reading legacy files
|
|
|
|
`MongoGridFsCompatibilityReader` reads existing GridFS content as
|
|
`GridFsLegacyContent(legacyId, filename, sizeBytes, checksum, stream)`. It reads; it does not write.
|
|
|
|
## Migration
|
|
|
|
`MongoGridFsMigrationJob` moves a file to object storage in a fixed order:
|
|
|
|
```
|
|
read legacy content
|
|
→ write to object storage
|
|
→ verify the target checksum matches the source
|
|
→ switch the reference
|
|
→ (later, separately) delete the source
|
|
```
|
|
|
|
Three properties, each of which exists because of a specific way this goes wrong:
|
|
|
|
1. **Verify before switching.** `MongoGridFsObjectReference` requires a non-blank checksum, and the
|
|
job returns empty and writes no reference when the target checksum does not match the source. A
|
|
migration that switches the reference on a successful *write* rather than a verified *copy*
|
|
silently points at a truncated object.
|
|
2. **The source is never deleted here.** Deletion is a separate, later decision after the new
|
|
location has been serving reads long enough to be trusted. A migration that deletes as it goes has
|
|
no rollback.
|
|
3. **The checkpoint separates migrated from failed.** `MongoGridFsMigrationCheckpoint` tracks
|
|
`migratedCount()`, `failedCount()`, `clean()` and `lastMigratedLegacyId()`, so a restart continues
|
|
from the last completed file rather than starting over, and a partially failed run is visible as
|
|
partial rather than as "done".
|
|
|
|
## Failure recovery
|
|
|
|
| Symptom | Cause | Action |
|
|
|---|---|---|
|
|
| `migrate` returns empty | Checksum mismatch | The copy is bad. Investigate before retrying; do not force the reference. |
|
|
| `IllegalArgumentException` on the reference | Missing checksum | A reference without a checksum cannot be verified and is refused. |
|
|
| Checkpoint not `clean()` | Some files failed | Re-run for the failed ids only; the checkpoint names the last successful one. |
|
|
| Reference switched but content missing | Source deleted too early | Restore from backup. This is what rule 2 prevents. |
|
|
|
|
## Promotion evidence
|
|
|
|
Actual-topology evidence against the real object storage backend, a security review of the storage
|
|
credential, the migration path above, failure cases (checksum mismatch refused, missing checksum
|
|
refused, restart resumes), and this document as the runbook.
|
|
|
|
New file storage does not go through here at all — see the fileserver adapter.
|