DongHyeonkaandClaude Opus 5
d57d2f62a0
Implements the mongodb-superpowers-package design: Stable Tasks 1-50 and Advanced Tasks 1-15. The design assumes 19 Stable + 12 Advanced Gradle projects under modules/mongodb*. This repository's fail-closed registry declares exactly 19 leaf identities, so those modules become package boundaries inside the registered leaf :adapter:outbound:persistence-mongo, with the design's module dependency table enforced by ten ArchUnit rules. The mapping and every deviation are recorded in docs/mongodb/repository-adaptation.md. Contract highlights, all enforced by tests rather than convention: - Transaction body retry and commit retry are separate loops. A new session per body attempt; commit-only retry on an unknown commit. The body is never replayed after a commit ambiguity, so a failover cannot become a duplicate. - MongoExecutionOutcome keeps both ambiguous outcomes distinct from success and failure, and MongoFailureContext records only the design-permitted fields. - Failure classification reads server error labels before numeric codes. - BSON representations come from a pinned manifest, never a library default, and a golden type-signature gate fails on any drift. - Index and validator changes go through the manifest and the admin plane; metadata ownership gates every drop. - Every Advanced capability refuses construction unless its flag is enabled. Verified against real servers, not only unit tests. Running the lanes for the first time exposed four defects that a green `check` had hidden: - Four release lanes passed while executing zero tests; the gate now counts executed tests per lane and fails on zero. - The "single replica set" fixture was a standalone, because Testcontainers 2.x needs withReplicaSet(); its test only asserted a connection string. - The three-node fixture was three independent clusters, so no election could occur, and awaitNewPrimary() compared against the post-stop primary. - The migration lease checked modifiedCount, so a same-millisecond refresh read as a lost lease. scripts/verify-mongodb-platform.sh now reports: 9 lanes, 0 skipped, 0 failed, every evidence category produced. scripts/verify-mongodb-advanced.sh reports NOT PROMOTABLE: actual-topology evidence (real sharded cluster, real KMS, real target deployment) is unobtainable here, so it is named rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
82 lines
4.5 KiB
Markdown
82 lines
4.5 KiB
Markdown
# Advanced — Search and Vector Search
|
||
|
||
**Capabilities:** `MongoCapability.SEARCH`, `MongoCapability.VECTOR_SEARCH`
|
||
**Properties:** `ca-skeleton.persistence-mongo.advanced.search.enabled`,
|
||
`ca-skeleton.persistence-mongo.advanced.vector-search.enabled`
|
||
**Status:** Experimental (design §3.3). Hybrid search likewise.
|
||
|
||
## Requirements
|
||
|
||
| | |
|
||
|---|---|
|
||
| Topology | A deployment with the search service. Atlas Local in a container is a pull-request convenience and is **not** release evidence. |
|
||
| Privilege | `MongoPrincipalRole.SEARCH_ADMIN` for index management; the application role queries only. |
|
||
| Gate | `MongoAtlasCapabilityContractSuite` on the actual target deployment. |
|
||
|
||
## 1. Created is not ready
|
||
|
||
`MongoSearchIndexState`: `CREATED` → `BUILDING` → `READY`, plus `FAILED` and `DELETING`.
|
||
|
||
`MongoSearchReadinessGate.requireReady(state)` refuses anything but `READY`. A search index is built
|
||
asynchronously: the create call returns immediately and the index answers queries with *partial*
|
||
results while building. Not an error, not empty — partial. A deployment that creates an index and
|
||
starts querying serves incomplete results for as long as the build takes, and nothing reports it.
|
||
|
||
## 2. Search indexes are not application-owned
|
||
|
||
`MongoSearchIndexDescriptor.metadataOwnership()` is `MongoMetadataOwnership.SEARCH_MANAGED`, and
|
||
`droppableByApplicationDrift()` is false. The index reconciliation described in
|
||
[ADR-MONGO-004](../../adr/ADR-MONGO-004-index-schema-admin-plane.md) must not drop it.
|
||
|
||
## 3. Query guardrails
|
||
|
||
`MongoSearchQuery` binds an index, an allowlist of paths, the search text and a result limit.
|
||
|
||
- `requireAllowedPaths(allowed)` raises `MongoOperationRejectedException` on a path outside the
|
||
allowlist. Without it, a caller can search any indexed field, including ones indexed for a
|
||
different purpose.
|
||
- Search text length and result count are bounded at construction. An unbounded search text is a
|
||
cost multiplier on someone else's service.
|
||
|
||
## 4. Vector search
|
||
|
||
`MongoVectorIndexDescriptor.cosine(path, dimensions)` declares the index.
|
||
`MongoEmbedding.forIndex(index, values)` binds an embedding to it and **rejects a dimension
|
||
mismatch** — a 1536-dimension embedding against a 768-dimension index is not a runtime degradation,
|
||
it is a category error, and catching it at construction beats catching it as a confusing server
|
||
message.
|
||
|
||
`MongoEmbedding` copies its backing array in and out. A vector that shares an array with its caller
|
||
can be mutated after the query is built, which produces a query nobody wrote.
|
||
|
||
`MongoVectorQuery` requires `numCandidates > limit` — searching 10 candidates to return 10 results is
|
||
an exhaustive scan wearing an ANN index's name. `MongoVectorQuery.nearest(embedding, 10)` uses the
|
||
standard 20× ratio (200 candidates for 10 results).
|
||
|
||
## 5. Relevance is the gate, not functionality
|
||
|
||
`MongoVectorSearchBenchmarkGate.standard()` requires **recall** alongside latency and index size.
|
||
`requiredEvidence()` names recall explicitly.
|
||
|
||
This is the difference between search and everything else in the platform. A vector index can be
|
||
functionally perfect — it accepts the index, accepts the query, returns k results, within the latency
|
||
budget — and return the wrong k. A gate that measures only latency certifies a fast wrong answer.
|
||
`failures(recall, latencyMs, indexMb)` reports which dimension failed so the finding is actionable.
|
||
|
||
## 6. Failure recovery
|
||
|
||
| Symptom | Cause | Action |
|
||
|---|---|---|
|
||
| Incomplete results after a deploy | Queried a `BUILDING` index | Wait for `READY`. The gate prevents this; if it fired, something bypassed it. |
|
||
| `MongoOperationRejectedException` on a path | Path not in the allowlist | Add it deliberately, or fix the caller. |
|
||
| Dimension mismatch | Model changed | A new model means a new index. Build alongside, cut over, then retire. |
|
||
| Recall dropped without a code change | The index was rebuilt with different parameters, or the data distribution shifted | Re-run the benchmark gate; treat a recall regression like a failing test. |
|
||
| Index `FAILED` | Build error on the search service | Search-side diagnosis; the application must not fall back to a scan silently. |
|
||
|
||
## 7. Promotion evidence
|
||
|
||
Per [ADR-MONGO-ADV-001](../../adr/ADR-MONGO-ADV-001-capability-promotion.md): the actual target
|
||
deployment (not Atlas Local), security review of `SEARCH_ADMIN`, a rebuild path, failure cases
|
||
(non-ready index refused, disallowed path refused, dimension mismatch refused), this document as the
|
||
runbook, **and** relevance evidence. Search and vector search do not promote on functional success.
|