From 16abd920a7003a31e876ca776ce3cbe6acbe50ff Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Thu, 24 Sep 2026 13:15:23 +0900 Subject: [PATCH] refactor(storage): keep IndexedDB stored-row rules in one module (Q-6.5-7) The runtime and maintenance each carried their own Stored* row types, isStored* validators, record size formula, countBucket and failureOf, and the dataset binding check had a third, hand-written budget validator. A field added to one copy would make the other judge a healthy row corrupt. indexeddb-records.ts now holds the row types, the validators, the budget key and measureRecordBytes (payload + 512-byte envelope + UTF-16 key). Each call site keeps its own failure code for a bad measurement (runtime INVALID_INPUT, maintenance MIGRATION_FAILED), and only the binding check still bounds a stored budget by hardBudgetBytes. The one validator difference (maintenance's retention check did not re-validate the key it compared against) is unobservable: it always passes a validated record key. Verified: characterization tests added first in tests/unit/indexeddb-maintenance.test.ts pin the size formula in both writers, the per-site failure codes for NaN/negative/oversized sizes, and the hardBudgetBytes bound at open (runtime and maintenance fail with POLICY_REJECTED one byte over, open at the limit). They passed before and after the refactor. INVENTORY.md (137/137) and the storage detail table list the new file. All gates pass. Co-Authored-By: Claude Opus 5.5 --- .../adapters/03-storage-and-browser-files.md | 1 + docs/reviews/adapters/INVENTORY.md | 53 ++--- .../storage/indexeddb/indexeddb-governance.ts | 28 +-- .../indexeddb/indexeddb-maintenance.ts | 212 ++---------------- .../storage/indexeddb/indexeddb-records.ts | 188 ++++++++++++++++ .../storage/indexeddb/indexeddb-runtime.ts | 195 ++-------------- tests/unit/indexeddb-maintenance.test.ts | 121 +++++++++- 7 files changed, 386 insertions(+), 412 deletions(-) create mode 100644 src/adapters/storage/indexeddb/indexeddb-records.ts diff --git a/docs/reviews/adapters/03-storage-and-browser-files.md b/docs/reviews/adapters/03-storage-and-browser-files.md index b9dea4f1..6d9959ae 100644 --- a/docs/reviews/adapters/03-storage-and-browser-files.md +++ b/docs/reviews/adapters/03-storage-and-browser-files.md @@ -67,6 +67,7 @@ | `src/adapters/storage/indexeddb/indexeddb-governance.ts` | opaque dataset scope/physical DB identity 및 frozen policy binding | indexeddb/shared ports. account/business ID를 physical name에 쓰지 않는 양방향 binding 유지. | | `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | post-open codec migration 및 idempotency receipt prune, keyset checkpoint, budget/revision fencing | IndexedDB port/types/failure/governance. async transform outside tx, row+sidecar+budget+checkpoint atomic commit은 좋다. STO-06 및 temporal drain lease 개선 후보. | | `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | additive-only contiguous DDL planner/validator | indexeddb types. destructive DDL 거절 유지. | +| `src/adapters/storage/indexeddb/indexeddb-records.ts` | runtime·maintenance·dataset binding이 함께 쓰는 저장 행 타입, 검증기, record 바이트 측정, `countBucket`/`failureOf` | IndexedDB port/shared/types. Q-6.5-7: 복사본 3벌을 하나로 합쳤다. 실패 코드는 호출부가 정한다(runtime `INVALID_INPUT`, maintenance `MIGRATION_FAILED`). budget의 `hardBudgetBytes` 상한은 binding 검증에서만 본다. | | `src/adapters/storage/indexeddb/indexeddb-runtime.ts` | generic repository open/read/query/CAS/delete, idempotency, retention, lifecycle purge, connection lifecycle | indexeddb ports/types/governance/failure/migrations. transaction `complete` truth, versionchange close, shared open/abort isolation, exact budgets 유지. lifecycle proof는 현재 문서 계약(형식 검증 후 폐기)과 일치하므로 결함으로 분류하지 않았다. | | `src/adapters/storage/indexeddb/indexeddb-types.ts` | adapter-local codec/query/schema/dependency contracts | application indexeddb/shared ports. `isOldWriterDrainConfirmed()` boolean은 provider가 전체 window를 보장한다는 문서 전제; lease형으로 강화 권고. | diff --git a/docs/reviews/adapters/INVENTORY.md b/docs/reviews/adapters/INVENTORY.md index 87666385..39868d1d 100644 --- a/docs/reviews/adapters/INVENTORY.md +++ b/docs/reviews/adapters/INVENTORY.md @@ -119,31 +119,32 @@ | 109 | `src/adapters/storage/indexeddb/indexeddb-governance.ts` | [Storage/files](./03-storage-and-browser-files.md) | | 110 | `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | [Storage/files](./03-storage-and-browser-files.md) | | 111 | `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 112 | `src/adapters/storage/indexeddb/indexeddb-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 113 | `src/adapters/storage/indexeddb/indexeddb-types.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 114 | `src/adapters/storage/opfs/browser-opfs-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 115 | `src/adapters/storage/opfs/index.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 116 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 117 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 118 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 119 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 120 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 121 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 122 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) | -| 123 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) | -| 124 | `src/adapters/telemetry/feature-telemetry.ts` | [Network/state](./01-network-and-state.md) | -| 125 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) | -| 126 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 127 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 128 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 129 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 130 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 131 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 132 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 133 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 134 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 135 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -| 136 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 112 | `src/adapters/storage/indexeddb/indexeddb-records.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 113 | `src/adapters/storage/indexeddb/indexeddb-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 114 | `src/adapters/storage/indexeddb/indexeddb-types.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 115 | `src/adapters/storage/opfs/browser-opfs-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 116 | `src/adapters/storage/opfs/index.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 117 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 118 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 119 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 120 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 121 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 122 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 123 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) | +| 124 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) | +| 125 | `src/adapters/telemetry/feature-telemetry.ts` | [Network/state](./01-network-and-state.md) | +| 126 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) | +| 127 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 128 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 129 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 130 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 131 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 132 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 133 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 134 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 135 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 136 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) | +| 137 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) | -합계: **136/136**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다. +합계: **137/137**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다. diff --git a/src/adapters/storage/indexeddb/indexeddb-governance.ts b/src/adapters/storage/indexeddb/indexeddb-governance.ts index d1659e08..1fc0c6c5 100644 --- a/src/adapters/storage/indexeddb/indexeddb-governance.ts +++ b/src/adapters/storage/indexeddb/indexeddb-governance.ts @@ -3,9 +3,12 @@ import { assertValidStoragePolicy, type BrowserStoragePolicy, } from "../../../application/ports/browser-file-storage/shared.ts"; +import { + INDEXEDDB_DATASET_BUDGET_KEY, + isStoredDatasetBudget, +} from "./indexeddb-records.ts"; export const INDEXEDDB_DATASET_BINDING_KEY = "dataset-binding"; -export const INDEXEDDB_DATASET_BUDGET_KEY = "dataset-budget"; const OPAQUE_SCOPE_TOKEN = /^[A-Za-z0-9_-]{16,48}$/u; @@ -280,27 +283,8 @@ export function verifyIndexedDbDatasetBinding( } else if (!sameIndexedDbDatasetBinding(observed, expected)) { finish({ ok: false, reason: "MISMATCH" }); } else if ( - !observedBudget || - typeof observedBudget !== "object" || - (observedBudget as { bindingKey?: unknown }).bindingKey !== - INDEXEDDB_DATASET_BUDGET_KEY || - (observedBudget as { budgetVersion?: unknown }).budgetVersion !== - 1 || - !Number.isSafeInteger( - (observedBudget as { usedBytes?: unknown }).usedBytes, - ) || - typeof (observedBudget as { usedBytes?: unknown }).usedBytes !== - "number" || - (observedBudget as { usedBytes: number }).usedBytes < 0 || - (observedBudget as { usedBytes: number }).usedBytes > - expected.storagePolicy.hardBudgetBytes - || - !Number.isSafeInteger( - (observedBudget as { receiptCount?: unknown }).receiptCount, - ) || - typeof (observedBudget as { receiptCount?: unknown }) - .receiptCount !== "number" || - (observedBudget as { receiptCount: number }).receiptCount < 0 + !isStoredDatasetBudget(observedBudget) || + observedBudget.usedBytes > expected.storagePolicy.hardBudgetBytes ) { finish({ ok: false, reason: "CORRUPT" }); } else { diff --git a/src/adapters/storage/indexeddb/indexeddb-maintenance.ts b/src/adapters/storage/indexeddb/indexeddb-maintenance.ts index deada7a9..3bedea47 100644 --- a/src/adapters/storage/indexeddb/indexeddb-maintenance.ts +++ b/src/adapters/storage/indexeddb/indexeddb-maintenance.ts @@ -27,23 +27,29 @@ import { import { mapIndexedDbException } from "./indexeddb-failure.ts"; import { createIndexedDbDatasetBinding, - INDEXEDDB_DATASET_BUDGET_KEY, indexedDbPhysicalDatabaseName, verifyIndexedDbDatasetBinding, } from "./indexeddb-governance.ts"; +import { + countBucket, + failureOf, + INDEXEDDB_DATASET_BUDGET_KEY, + isPositiveInteger, + isStoredDatasetBudget, + isStoredReceipt, + isStoredRecord, + isStoredRetentionRecord, + isValidBoundaryIdentifier, + measureRecordBytes, + type StoredDatasetBudget, + type StoredRecord, + type StoredRetentionRecord, +} from "./indexeddb-records.ts"; import type { - IndexedDbCountBucket, IndexedDbMaintenanceDependencies, IndexedDbObservation, } from "./indexeddb-types.ts"; -type StoredRecord = Readonly<{ - key: string; - codecVersion: number; - revision: number; - payload: unknown; -}>; - type StoredCheckpoint = Readonly<{ checkpointKey: string; migrationId: string; @@ -52,32 +58,6 @@ type StoredCheckpoint = Readonly<{ state: "MORE" | "COMPLETE"; }>; -type StoredIdempotencyReceipt = Readonly<{ - idempotencyKey: string; - operation: "PUT" | "DELETE"; - recordKey: string; - expectedRevision: number | null; - fingerprint: string; - synchronization: "NONE" | "PENDING" | "CONFIRMED"; - revision: number; - expiresAtEpochMs: number; -}>; - -type StoredRetentionRecord = Readonly<{ - recordKey: string; - writtenAtEpochMs: number; - synchronization: "NONE" | "PENDING" | "CONFIRMED"; - measuredBytes: number; - eligibleAtEpochMs?: number; -}>; - -type StoredDatasetBudget = Readonly<{ - bindingKey: typeof INDEXEDDB_DATASET_BUDGET_KEY; - budgetVersion: 1; - usedBytes: number; - receiptCount: number; -}>; - type CheckpointState = Readonly<{ persisted: StoredCheckpoint | null; effective: Readonly<{ @@ -86,57 +66,22 @@ type CheckpointState = Readonly<{ }>; }>; -type ScannedRecord = Readonly<{ - key: string; - codecVersion: number; - revision: number; - payload: unknown; -}>; - type ScanBatch = Readonly<{ - rows: readonly ScannedRecord[]; + rows: readonly StoredRecord[]; reachedEnd: boolean; budgetExhausted: boolean; }>; type PreparedRecord = Readonly<{ - source: ScannedRecord; + source: StoredRecord; needsMigration: boolean; migratedPayload: WireValue | undefined; measuredBytes: number | undefined; }>; const SAFE_IDENTIFIER = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/u; -const OPAQUE_SHA256_FINGERPRINT = /^[a-f0-9]{64}$/u; const MAX_BATCH_ROWS = 500; const MAX_BATCH_DURATION_MS = 30_000; -const RECORD_ENVELOPE_RESERVATION_BYTES = 512; -const MAX_MEASURED_RECORD_BYTES = 2_147_483_647; - -function isPositiveInteger(value: unknown): value is number { - return ( - typeof value === "number" && - Number.isSafeInteger(value) && - value >= 1 - ); -} - -function isStoredRecord( - value: unknown, - expectedKey?: string, -): value is StoredRecord { - if (!value || typeof value !== "object") return false; - const record = value as Partial; - return ( - typeof record.key === "string" && - record.key.length > 0 && - record.key.length <= 200 && - (expectedKey === undefined || record.key === expectedKey) && - isPositiveInteger(record.codecVersion) && - isPositiveInteger(record.revision) && - Object.hasOwn(record, "payload") - ); -} function isStoredCheckpoint( value: unknown, @@ -150,84 +95,12 @@ function isStoredCheckpoint( SAFE_IDENTIFIER.test(checkpoint.migrationId) && isPositiveInteger(checkpoint.targetCodecVersion) && (checkpoint.lastKey === null || - (typeof checkpoint.lastKey === "string" && - checkpoint.lastKey.length > 0 && - checkpoint.lastKey.length <= 200)) && + isValidBoundaryIdentifier(checkpoint.lastKey)) && (checkpoint.state === "MORE" || checkpoint.state === "COMPLETE") ); } -function isStoredReceipt( - value: unknown, -): value is StoredIdempotencyReceipt { - if (!value || typeof value !== "object") return false; - const receipt = value as Partial; - return ( - typeof receipt.idempotencyKey === "string" && - receipt.idempotencyKey.length > 0 && - receipt.idempotencyKey.length <= 200 && - (receipt.operation === "PUT" || - receipt.operation === "DELETE") && - typeof receipt.recordKey === "string" && - receipt.recordKey.length > 0 && - receipt.recordKey.length <= 200 && - (receipt.expectedRevision === null || - isPositiveInteger(receipt.expectedRevision)) && - typeof receipt.fingerprint === "string" && - OPAQUE_SHA256_FINGERPRINT.test(receipt.fingerprint) && - (receipt.synchronization === "NONE" || - receipt.synchronization === "PENDING" || - receipt.synchronization === "CONFIRMED") && - isPositiveInteger(receipt.revision) && - typeof receipt.expiresAtEpochMs === "number" && - Number.isSafeInteger(receipt.expiresAtEpochMs) && - receipt.expiresAtEpochMs >= 0 - ); -} - -function isStoredRetentionRecord( - value: unknown, - expectedKey: string, -): value is StoredRetentionRecord { - if (!value || typeof value !== "object") return false; - const record = value as Partial; - return ( - record.recordKey === expectedKey && - typeof record.writtenAtEpochMs === "number" && - Number.isSafeInteger(record.writtenAtEpochMs) && - record.writtenAtEpochMs >= 0 && - (record.synchronization === "NONE" || - record.synchronization === "PENDING" || - record.synchronization === "CONFIRMED") && - typeof record.measuredBytes === "number" && - Number.isSafeInteger(record.measuredBytes) && - record.measuredBytes >= 1 && - record.measuredBytes <= MAX_MEASURED_RECORD_BYTES && - (record.eligibleAtEpochMs === undefined || - (typeof record.eligibleAtEpochMs === "number" && - Number.isSafeInteger(record.eligibleAtEpochMs) && - record.eligibleAtEpochMs >= record.writtenAtEpochMs)) - ); -} - -function isStoredDatasetBudget( - value: unknown, -): value is StoredDatasetBudget { - if (!value || typeof value !== "object") return false; - const budget = value as Partial; - return ( - budget.bindingKey === INDEXEDDB_DATASET_BUDGET_KEY && - budget.budgetVersion === 1 && - typeof budget.usedBytes === "number" && - Number.isSafeInteger(budget.usedBytes) && - budget.usedBytes >= 0 && - typeof budget.receiptCount === "number" && - Number.isSafeInteger(budget.receiptCount) && - budget.receiptCount >= 0 - ); -} - function sameCheckpoint( left: StoredCheckpoint | null, right: StoredCheckpoint | null, @@ -242,14 +115,6 @@ function sameCheckpoint( ); } -function countBucket(count: number): IndexedDbCountBucket { - if (count <= 0) return "0"; - if (count === 1) return "1"; - if (count <= 10) return "2-10"; - if (count <= 100) return "11-100"; - return "101+"; -} - function invalidInput(): BrowserDataResult { return browserDataFailure("INVALID_INPUT", "INDEXEDDB_MIGRATE"); } @@ -273,19 +138,6 @@ function unsupported(): BrowserDataResult { }); } -/** - * `browserDataFailure` and `mapIndexedDbException` build a `Result`, while the - * kernel's `translate` and `context.fail` want the failure on its own. Both - * only ever build the failure arm, so the branch below is a narrowing rather - * than a claim. - */ -function failureOf(result: BrowserDataResult): BrowserDataFailure { - if (result.ok) { - throw new TypeError("A browser data failure was expected."); - } - return result.error; -} - function invalidInputFailure(): BrowserDataFailure { return failureOf(invalidInput()); } @@ -705,7 +557,7 @@ export function createIndexedDbMaintenance( return; } - const rows: ScannedRecord[] = []; + const rows: StoredRecord[] = []; // The scan bounds on rows it read, so the counter the kernel offers is // the right one here. The time check comes first so a batch that ran // out of both answers `budgetExhausted` the way it does today. @@ -830,28 +682,10 @@ export function createIndexedDbMaintenance( } catch { return migrationFailed(); } - let measuredPayloadBytes: number; - try { - measuredPayloadBytes = - dependencies.migrationPolicy.measureStoredBytes( - migratedPayload, - ); - } catch { - return migrationFailed(); - } - const measuredBytes = - measuredPayloadBytes + - RECORD_ENVELOPE_RESERVATION_BYTES + - source.key.length * 2; - if ( - !Number.isSafeInteger(measuredPayloadBytes) || - measuredPayloadBytes < 0 || - !Number.isSafeInteger(measuredBytes) || - measuredBytes < 1 || - measuredBytes > MAX_MEASURED_RECORD_BYTES - ) { - return migrationFailed(); - } + const measuredBytes = measureRecordBytes(source.key, () => + dependencies.migrationPolicy.measureStoredBytes(migratedPayload), + ); + if (measuredBytes === null) return migrationFailed(); const afterMigration = clock(); if (!afterMigration.ok) return afterMigration; if (afterMigration.value >= deadline) { diff --git a/src/adapters/storage/indexeddb/indexeddb-records.ts b/src/adapters/storage/indexeddb/indexeddb-records.ts new file mode 100644 index 00000000..188be397 --- /dev/null +++ b/src/adapters/storage/indexeddb/indexeddb-records.ts @@ -0,0 +1,188 @@ +import type { IndexedDbSynchronizationState } from "../../../application/ports/browser-file-storage/indexeddb-port.ts"; +import type { + BrowserDataFailure, + BrowserDataResult, +} from "../../../application/ports/browser-file-storage/shared.ts"; +import type { IndexedDbCountBucket } from "./indexeddb-types.ts"; + +/** + * Q-6.5-7. The one definition of the rows the runtime and maintenance store: + * their shapes, their validators and how a record is charged against the + * dataset budget. Each call site used to carry its own copy, so a field added + * to one copy made the other judge a healthy row corrupt. + */ + +export const INDEXEDDB_DATASET_BUDGET_KEY = "dataset-budget"; + +export const OPAQUE_SHA256_FINGERPRINT = /^[a-f0-9]{64}$/u; +const RECORD_ENVELOPE_RESERVATION_BYTES = 512; +const MAX_MEASURED_RECORD_BYTES = 2_147_483_647; + +type StoredSynchronization = IndexedDbSynchronizationState | "NONE"; + +export type StoredRecord = Readonly<{ + key: string; + codecVersion: number; + revision: number; + payload: unknown; +}>; + +export type StoredIdempotencyReceipt = Readonly<{ + idempotencyKey: string; + operation: "PUT" | "DELETE"; + recordKey: string; + expectedRevision: number | null; + fingerprint: string; + synchronization: StoredSynchronization; + revision: number; + expiresAtEpochMs: number; +}>; + +export type StoredRetentionRecord = Readonly<{ + recordKey: string; + writtenAtEpochMs: number; + synchronization: StoredSynchronization; + measuredBytes: number; + eligibleAtEpochMs?: number; +}>; + +export type StoredDatasetBudget = Readonly<{ + bindingKey: typeof INDEXEDDB_DATASET_BUDGET_KEY; + budgetVersion: 1; + usedBytes: number; + receiptCount: number; +}>; + +export function isPositiveInteger(value: unknown): value is number { + return typeof value === "number" && Number.isSafeInteger(value) && value >= 1; +} + +export function isValidBoundaryIdentifier(value: unknown): value is string { + return typeof value === "string" && value.length > 0 && value.length <= 200; +} + +function isNonNegativeInteger(value: unknown): value is number { + return typeof value === "number" && Number.isSafeInteger(value) && value >= 0; +} + +function isStoredSynchronization(value: unknown): boolean { + return value === "NONE" || value === "PENDING" || value === "CONFIRMED"; +} + +export function isStoredRecord( + value: unknown, + expectedKey?: string, +): value is StoredRecord { + if (!value || typeof value !== "object") return false; + const record = value as Partial; + return ( + isValidBoundaryIdentifier(record.key) && + (expectedKey === undefined || record.key === expectedKey) && + isPositiveInteger(record.codecVersion) && + isPositiveInteger(record.revision) && + Object.hasOwn(record, "payload") + ); +} + +export function isStoredReceipt( + value: unknown, +): value is StoredIdempotencyReceipt { + if (!value || typeof value !== "object") return false; + const receipt = value as Partial; + return ( + isValidBoundaryIdentifier(receipt.idempotencyKey) && + (receipt.operation === "PUT" || receipt.operation === "DELETE") && + isValidBoundaryIdentifier(receipt.recordKey) && + (receipt.expectedRevision === null || + isPositiveInteger(receipt.expectedRevision)) && + typeof receipt.fingerprint === "string" && + OPAQUE_SHA256_FINGERPRINT.test(receipt.fingerprint) && + isStoredSynchronization(receipt.synchronization) && + isPositiveInteger(receipt.revision) && + isNonNegativeInteger(receipt.expiresAtEpochMs) + ); +} + +export function isStoredRetentionRecord( + value: unknown, + expectedKey?: string, +): value is StoredRetentionRecord { + if (!value || typeof value !== "object") return false; + const record = value as Partial; + return ( + isValidBoundaryIdentifier(record.recordKey) && + (expectedKey === undefined || record.recordKey === expectedKey) && + isNonNegativeInteger(record.writtenAtEpochMs) && + isStoredSynchronization(record.synchronization) && + isPositiveInteger(record.measuredBytes) && + record.measuredBytes <= MAX_MEASURED_RECORD_BYTES && + (record.eligibleAtEpochMs === undefined || + (Number.isSafeInteger(record.eligibleAtEpochMs) && + record.eligibleAtEpochMs >= record.writtenAtEpochMs)) + ); +} + +/** + * A budget row's own shape. The dataset binding check additionally bounds + * `usedBytes` by the policy's `hardBudgetBytes`; transactions check the bytes + * they are about to add instead. + */ +export function isStoredDatasetBudget( + value: unknown, +): value is StoredDatasetBudget { + if (!value || typeof value !== "object") return false; + const budget = value as Partial; + return ( + budget.bindingKey === INDEXEDDB_DATASET_BUDGET_KEY && + budget.budgetVersion === 1 && + isNonNegativeInteger(budget.usedBytes) && + isNonNegativeInteger(budget.receiptCount) + ); +} + +/** + * The bytes a record is charged: the codec's payload size, a fixed envelope + * reservation and the key as UTF-16. `null` when the codec throws or the size + * is out of range; each caller maps that to its own failure code. + */ +export function measureRecordBytes( + key: string, + measurePayload: () => number, +): number | null { + let payloadBytes: number; + try { + payloadBytes = measurePayload(); + } catch { + return null; + } + const measuredBytes = + payloadBytes + RECORD_ENVELOPE_RESERVATION_BYTES + key.length * 2; + return isNonNegativeInteger(payloadBytes) && + isPositiveInteger(measuredBytes) && + measuredBytes <= MAX_MEASURED_RECORD_BYTES + ? measuredBytes + : null; +} + +export function countBucket(count: number): IndexedDbCountBucket { + if (count <= 0) return "0"; + if (count === 1) return "1"; + if (count <= 10) return "2-10"; + if (count <= 100) return "11-100"; + return "101+"; +} + +/** + * `browserDataFailure` and `mapIndexedDbException` build a `Result`, while the + * kernel's `translate` and `context.fail` want the failure on its own. Both + * only ever build the failure arm, so the branch below is a narrowing rather + * than a claim. + */ +export function failureOf( + result: BrowserDataResult, +): BrowserDataFailure { + if (result.ok) { + throw new TypeError("A browser data failure was expected."); + } + return result.error; +} diff --git a/src/adapters/storage/indexeddb/indexeddb-runtime.ts b/src/adapters/storage/indexeddb/indexeddb-runtime.ts index 60f6ca9c..ad4e4926 100644 --- a/src/adapters/storage/indexeddb/indexeddb-runtime.ts +++ b/src/adapters/storage/indexeddb/indexeddb-runtime.ts @@ -38,7 +38,6 @@ import { import { mapIndexedDbException } from "./indexeddb-failure.ts"; import { createIndexedDbDatasetBinding, - INDEXEDDB_DATASET_BUDGET_KEY, indexedDbPhysicalDatabaseName, queueIndexedDbUpgradeBinding, verifyIndexedDbDatasetBinding, @@ -48,47 +47,30 @@ import { assertIndexedDbRuntimeStores, validateIndexedDbMigrations, } from "./indexeddb-migrations.ts"; +import { + countBucket, + failureOf, + INDEXEDDB_DATASET_BUDGET_KEY, + isPositiveInteger, + isStoredDatasetBudget, + isStoredReceipt, + isStoredRecord, + isStoredRetentionRecord, + isValidBoundaryIdentifier, + measureRecordBytes, + OPAQUE_SHA256_FINGERPRINT, + type StoredDatasetBudget, + type StoredIdempotencyReceipt, + type StoredRecord, + type StoredRetentionRecord, +} from "./indexeddb-records.ts"; import type { - IndexedDbCountBucket, IndexedDbKeyRangePlan, IndexedDbObservation, IndexedDbQueryPlan, IndexedDbRuntimeDependencies, } from "./indexeddb-types.ts"; -type StoredRecord = Readonly<{ - key: string; - codecVersion: number; - revision: number; - payload: unknown; -}>; - -type StoredIdempotencyReceipt = Readonly<{ - idempotencyKey: string; - operation: "PUT" | "DELETE"; - recordKey: string; - expectedRevision: number | null; - fingerprint: string; - synchronization: IndexedDbSynchronizationState | "NONE"; - revision: number; - expiresAtEpochMs: number; -}>; - -type StoredRetentionRecord = Readonly<{ - recordKey: string; - writtenAtEpochMs: number; - synchronization: IndexedDbSynchronizationState | "NONE"; - measuredBytes: number; - eligibleAtEpochMs?: number; -}>; - -type StoredDatasetBudget = Readonly<{ - bindingKey: typeof INDEXEDDB_DATASET_BUDGET_KEY; - budgetVersion: 1; - usedBytes: number; - receiptCount: number; -}>; - type ReceiptLookup = | Readonly<{ kind: "MISSING" | "EXPIRED" }> | Readonly<{ @@ -102,7 +84,6 @@ type TransactionContext = IndexedDbTransactionContext< >; const SAFE_DATABASE_NAME = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,63}$/u; -const OPAQUE_SHA256_FINGERPRINT = /^[a-f0-9]{64}$/u; const DELETE_OPERATION_FINGERPRINT = "65daeb378bdec35d90224389f6e10917bcc0b94c90c489014ba156dd6cbf9350"; const MAX_QUERY_LIMIT = 1_000; @@ -113,31 +94,9 @@ const MAX_RECEIPT_RETENTION_MS = 31 * 24 * 60 * 60 * 1_000; const MAX_LIFECYCLE_ROWS = 500; const MAX_LIFECYCLE_DURATION_MS = 30_000; const OPAQUE_AUTHORITY_PROOF = /^[A-Za-z0-9_-]{16,128}$/u; -const RECORD_ENVELOPE_RESERVATION_BYTES = 512; -const MAX_MEASURED_RECORD_BYTES = 2_147_483_647; const MAX_QUERY_SCANNED_ROWS = 5_000; const MAX_IDEMPOTENCY_RECEIPTS = 1_000_000; -function countBucket(count: number): IndexedDbCountBucket { - if (count <= 0) return "0"; - if (count === 1) return "1"; - if (count <= 10) return "2-10"; - if (count <= 100) return "11-100"; - return "101+"; -} - -function isValidBoundaryIdentifier(value: string): boolean { - return ( - typeof value === "string" && - value.length > 0 && - value.length <= 200 - ); -} - -function isRevision(value: unknown): value is number { - return Number.isSafeInteger(value) && typeof value === "number" && value >= 1; -} - function isBoundedCursorKey(value: unknown): value is IDBValidKey { const pending: Array> = [ { value, depth: 0 }, @@ -222,86 +181,6 @@ function isBoundedRangePlan( } } -function isStoredRecord(value: unknown, expectedKey?: string): value is StoredRecord { - if (!value || typeof value !== "object") return false; - const record = value as Partial; - return ( - typeof record.key === "string" && - isValidBoundaryIdentifier(record.key) && - (expectedKey === undefined || record.key === expectedKey) && - isRevision(record.codecVersion) && - isRevision(record.revision) && - Object.hasOwn(record, "payload") - ); -} - -function isStoredReceipt(value: unknown): value is StoredIdempotencyReceipt { - if (!value || typeof value !== "object") return false; - const receipt = value as Partial; - return ( - typeof receipt.idempotencyKey === "string" && - isValidBoundaryIdentifier(receipt.idempotencyKey) && - (receipt.operation === "PUT" || receipt.operation === "DELETE") && - typeof receipt.recordKey === "string" && - isValidBoundaryIdentifier(receipt.recordKey) && - (receipt.expectedRevision === null || - isRevision(receipt.expectedRevision)) && - typeof receipt.fingerprint === "string" && - OPAQUE_SHA256_FINGERPRINT.test(receipt.fingerprint) && - (receipt.synchronization === "NONE" || - receipt.synchronization === "PENDING" || - receipt.synchronization === "CONFIRMED") && - isRevision(receipt.revision) && - typeof receipt.expiresAtEpochMs === "number" && - Number.isSafeInteger(receipt.expiresAtEpochMs) && - receipt.expiresAtEpochMs >= 0 - ); -} - -function isStoredRetentionRecord( - value: unknown, - expectedKey?: string, -): value is StoredRetentionRecord { - if (!value || typeof value !== "object") return false; - const record = value as Partial; - return ( - typeof record.recordKey === "string" && - isValidBoundaryIdentifier(record.recordKey) && - (expectedKey === undefined || record.recordKey === expectedKey) && - typeof record.writtenAtEpochMs === "number" && - Number.isSafeInteger(record.writtenAtEpochMs) && - record.writtenAtEpochMs >= 0 && - (record.synchronization === "NONE" || - record.synchronization === "PENDING" || - record.synchronization === "CONFIRMED") && - typeof record.measuredBytes === "number" && - Number.isSafeInteger(record.measuredBytes) && - record.measuredBytes >= 1 && - record.measuredBytes <= MAX_MEASURED_RECORD_BYTES && - (record.eligibleAtEpochMs === undefined || - (typeof record.eligibleAtEpochMs === "number" && - Number.isSafeInteger(record.eligibleAtEpochMs) && - record.eligibleAtEpochMs >= record.writtenAtEpochMs)) - ); -} - -function isStoredDatasetBudget( - value: unknown, -): value is StoredDatasetBudget { - if (!value || typeof value !== "object") return false; - const budget = value as Partial; - return ( - budget.bindingKey === INDEXEDDB_DATASET_BUDGET_KEY && - budget.budgetVersion === 1 && - typeof budget.usedBytes === "number" && - Number.isSafeInteger(budget.usedBytes) && - budget.usedBytes >= 0 && - typeof budget.receiptCount === "number" && - Number.isSafeInteger(budget.receiptCount) && - budget.receiptCount >= 0 - ); -} - function normalizeCursorKey(value: IDBValidKey): IndexedDbCursorKey { if (Array.isArray(value)) { return Object.freeze(value.map((entry) => normalizeCursorKey(entry))); @@ -405,18 +284,6 @@ function unavailable( }); } -/** - * The kernel's `translate` and `context.fail` take the failure on its own while - * every helper above builds a whole `Result`. All of them only ever build the - * failure arm, so the branch below is a narrowing rather than a claim. - */ -function failureOf(result: BrowserDataResult): BrowserDataFailure { - if (result.ok) { - throw new TypeError("A browser data failure was expected."); - } - return result.error; -} - function mappedFailure( error: unknown, operation: BrowserDataOperation, @@ -1475,7 +1342,7 @@ export function createIndexedDbRuntime( !isValidBoundaryIdentifier(input.key) || !isValidBoundaryIdentifier(input.idempotencyKey) || (input.expectedRevision !== null && - !isRevision(input.expectedRevision)) || + !isPositiveInteger(input.expectedRevision)) || (storagePolicySnapshot.retention.kind === "UNTIL_SYNCED" ? input.synchronization !== "PENDING" && input.synchronization !== "CONFIRMED" @@ -1499,26 +1366,10 @@ export function createIndexedDbRuntime( } catch (error) { return mapIndexedDbException(error, operation); } - let measuredPayloadBytes: number; - try { - measuredPayloadBytes = - dependencies.codec.measureStoredBytes(wireValue); - } catch { - return invalidInput(operation); - } - const measuredBytes = - measuredPayloadBytes + - RECORD_ENVELOPE_RESERVATION_BYTES + - input.key.length * 2; - if ( - !Number.isSafeInteger(measuredPayloadBytes) || - measuredPayloadBytes < 0 || - !Number.isSafeInteger(measuredBytes) || - measuredBytes < 1 || - measuredBytes > MAX_MEASURED_RECORD_BYTES - ) { - return invalidInput(operation); - } + const measuredBytes = measureRecordBytes(input.key, () => + dependencies.codec.measureStoredBytes(wireValue), + ); + if (measuredBytes === null) return invalidInput(operation); const cancelled = abortedResult(input.signal, operation); if (cancelled) return cancelled; @@ -1887,7 +1738,7 @@ export function createIndexedDbRuntime( if ( !isValidBoundaryIdentifier(sourceInput.key) || !isValidBoundaryIdentifier(sourceInput.idempotencyKey) || - !isRevision(sourceInput.expectedRevision) + !isPositiveInteger(sourceInput.expectedRevision) ) { return observeResult(operation, invalidInput(operation)); } diff --git a/tests/unit/indexeddb-maintenance.test.ts b/tests/unit/indexeddb-maintenance.test.ts index 3fe8ea2a..13fd4999 100644 --- a/tests/unit/indexeddb-maintenance.test.ts +++ b/tests/unit/indexeddb-maintenance.test.ts @@ -35,7 +35,11 @@ const TEST_STORAGE_POLICY = Object.freeze({ unavailableFallback: "ONLINE_ONLY" as const, }); -function createSchemaRuntime(memory: MemoryIndexedDbFactory) { +function createSchemaRuntime( + memory: MemoryIndexedDbFactory, + measureStoredBytes: (value: CurrentPayload) => number = (value) => + new TextEncoder().encode(JSON.stringify(value)).byteLength, +) { return createIndexedDbRuntime({ scope: TEST_SCOPE, storagePolicy: TEST_STORAGE_POLICY, @@ -98,8 +102,7 @@ function createSchemaRuntime(memory: MemoryIndexedDbFactory) { codec: { currentVersion: 2, encode: (value) => ({ ok: true, value }), - measureStoredBytes: (value) => - new TextEncoder().encode(JSON.stringify(value)).byteLength, + measureStoredBytes, decode: (version, value) => version === 2 && value !== null && @@ -1008,3 +1011,115 @@ describe("IndexedDB bounded codec maintenance", () => { ).toBeDefined(); }); }); + +/** + * Q-6.5-7 characterization. The runtime, maintenance and the dataset binding + * each carried their own copy of the stored-row rules. These pin what every + * call site does today so merging the copies cannot move any of it. + */ +describe("Q-6.5-7 stored-row rules stay the same at every call site", () => { + const encodedBytes = (value: unknown) => + new TextEncoder().encode(JSON.stringify(value)).byteLength; + + it("measures a record as payload + 512 + 2 bytes per key character in both writers", async () => { + const memory = new MemoryIndexedDbFactory(); + const runtime = createSchemaRuntime(memory); + expect(await runtime.open()).toMatchObject({ ok: true }); + expect( + await runtime.compareAndSwap({ + key: "abc", + value: { label: "x" }, + expectedRevision: null, + idempotencyKey: "abc-create", + }), + ).toMatchObject({ ok: true }); + runtime.close(); + expect(memory.readRaw("retention", "abc")).toMatchObject({ + measuredBytes: encodedBytes({ label: "x" }) + 512 + 6, + }); + + seedLegacy(memory, "k1", "legacy"); + const maintenance = createMaintenance(memory, defaultPolicy()); + expect( + await maintenance.migrateCodecBatch({ + maxRows: 10, + maxDurationMs: 10_000, + }), + ).toMatchObject({ ok: true, value: { migratedRows: 1 } }); + expect(memory.readRaw("retention", "k1")).toMatchObject({ + measuredBytes: encodedBytes({ label: "legacy" }) + 512 + 4, + }); + }); + + it.each([ + ["NaN", Number.NaN], + ["a negative size", -1], + ["a size past the stored maximum", 2_147_483_647], + ])( + "rejects %s as INVALID_INPUT in the runtime and MIGRATION_FAILED in maintenance", + async (_label, measured) => { + const memory = new MemoryIndexedDbFactory(); + const runtime = createSchemaRuntime(memory, () => measured); + expect(await runtime.open()).toMatchObject({ ok: true }); + expect( + await runtime.compareAndSwap({ + key: "measured", + value: { label: "x" }, + expectedRevision: null, + idempotencyKey: "measured-create", + }), + ).toMatchObject({ ok: false, error: { code: "INVALID_INPUT" } }); + runtime.close(); + + seedLegacy(memory, "legacy", "value"); + const maintenance = createMaintenance(memory, { + ...defaultPolicy(), + measureStoredBytes: () => measured, + }); + expect( + await maintenance.migrateCodecBatch({ + maxRows: 10, + maxDurationMs: 10_000, + }), + ).toMatchObject({ + ok: false, + error: { code: "MIGRATION_FAILED", recovery: "READ_ONLY" }, + }); + }, + ); + + it("bounds a stored budget by hardBudgetBytes when the dataset binding is verified", async () => { + const memory = new MemoryIndexedDbFactory(); + await prepareSchema(memory); + const budget = memory.readRaw("governance", "dataset-budget") as Record< + string, + unknown + >; + + memory.seed("governance", { + ...budget, + usedBytes: TEST_STORAGE_POLICY.hardBudgetBytes, + }); + const atLimit = createSchemaRuntime(memory); + expect(await atLimit.open()).toMatchObject({ ok: true }); + atLimit.close(); + + memory.seed("governance", { + ...budget, + usedBytes: TEST_STORAGE_POLICY.hardBudgetBytes + 1, + }); + const overLimit = createSchemaRuntime(memory); + const opened = await overLimit.open(); + overLimit.close(); + expect(opened).toMatchObject({ + ok: false, + error: { code: "POLICY_REJECTED" }, + }); + expect( + await createMaintenance(memory, defaultPolicy()).migrateCodecBatch({ + maxRows: 10, + maxDurationMs: 10_000, + }), + ).toMatchObject({ ok: false, error: { code: "POLICY_REJECTED" } }); + }); +});