Merge lane E: browser storage and adapter kernel fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
2026-09-25 23:05:41 +09:00
co-authored by Claude Opus 5.5
29 changed files with 2621 additions and 634 deletions
@@ -759,7 +759,7 @@ CLEANED -> journal 제거
- journal row와 budget reservation은 physical cleanup effect가
`CLEANED` 또는 `ALREADY_CLEAN`으로 확인된 뒤에만 해제한다. timeout, crash,
malformed response, `EFFECT_UNKNOWN`은 성공이 아니며 `PREPARING`/`FILES_READY`를
malformed response, worker cleanup 실패는 성공이 아니며 `PREPARING`/`FILES_READY`를
그대로 남긴다. 이때 put은 자기 실패 코드(취소면 `ABORTED`, 쿼터 초과면
`QUOTA_EXCEEDED` 등)를 그대로 반환하고 recovery만 `RECONCILE`로 바꾼다
(Q-6.5-9). journal rollback이 실패해도 같다.
@@ -16,15 +16,24 @@ primitive를 coordinator 완료 증거로 사용하지 않는다.
## OPFS 보상 실패와 reconcile (STO-01)
`put()`이 실패했는데 보상 cleanup effect가 확인되지 않으면 runtime은 실패를
`OBJECT_RECONCILE` / `CONFLICT`(recovery `RETRY`)로 보고하고 journal row를 남긴다.
이는 결함이 아니라 설계된 상태다.
`put()`이 commit 전에 실패하면 coordinator가 그 transaction의 staging을 지우고
journal row를 되돌린다(보상). 보상의 worker cleanup이나 journal rollback이 끝나지
않으면 put은 자기 실패 코드(`QUOTA_EXCEEDED`, `ABORTED` 등)와 operation
`OBJECT_WRITE`를 그대로 두고 recovery만 `RECONCILE`로 바꿔 반환한다. journal
row(`PREPARING`/`FILES_READY`)는 남는다(`opfs-byte-store-adapter.ts`
`compensatePreparedPut`, Q-6.5-9). 이는 결함이 아니라 설계된 상태다. worker가
cleanup 결과를 확인하지 못하면 성공 값이 아니라 실패로 답한다. 별도의 "결과 불명"
값은 없다(05-09).
1. journal에 `PREPARING` 또는 `FILES_READY` row가 남아 있는지 확인한다. 남아
있다면 staging bytes가 아직 존재할 수 있다는 뜻이다.
2. `maintenance.reconcile()`을 실행한다. reconcile은 같은 exact physical
generation token만 삭제하고, effect가 여전히 `EFFECT_UNKNOWN`이면 journal을
유지한 채 다시 `OBJECT_RECONCILE`을 반환한다.
2. `maintenance.reconcile()`을 실행한다. reconcile은 exact physical generation
token만 삭제한다. report에서 다음을 본다.
- `skippedLiveTransactions`: 어느 tab에서든 아직 진행 중인 put/delete의 row다.
reconcile은 transaction별 Web Lock을 `ifAvailable`로 잡지 못한 row를 건드리지
않는다(05-01, `reconcile`의 `holdTransactionLock`). 결함이 아니다.
- `failedTransactions`: 이번에 정리하지 못해 journal에 남긴 row 수다. 나머지
row와 orphan GC는 계속 돈다(05-02). 같은 row가 계속 실패하면 §7을 본다.
3. journal row를 수동으로 삭제하지 않는다. row가 사라지면 stale staging을 추적할
근거가 사라지고 quota만 누수된다.
4. OPFS root나 journal database를 통째로 삭제하거나 schema를 downgrade하지
@@ -36,6 +45,9 @@ primitive를 coordinator 완료 증거로 사용하지 않는다.
incident 중에도 다음 작업은 금지한다.
- 자동 또는 일괄 `deleteDatabase()`
- 예외는 하나다. authority가 승인한 LOGOUT/ACCOUNT_DELETION이 손상된
governance 때문에 dataset을 열 수 없을 때, runtime이 그 partition의 DB
하나만 지운다(§5 "database 전체 삭제" 참고).
- origin의 모든 `caches.keys()` 삭제
- user-authored/unsynced IndexedDB 또는 OPFS data 자동 purge
- schema version downgrade
@@ -154,6 +166,12 @@ read-only나 online-only가 사용자 작성 내용을 잃게 한다면 먼저 e
- `versionchange` 뒤 connection이 남음
- repeated reload/update loop
- open/maintenance의 `POLICY_REJECTED`: immutable dataset binding missing/mismatch
- open의 `UNSUPPORTED` / `ONLINE_ONLY`와 status `INCOMPATIBLE`: 더 새 release가
schema를 올린 DB를 이전 bundle이 열었다(05-06). migration 실패가 아니므로 §5로
가지 않는다. 이 bundle은 그 dataset을 읽지도 쓰지도 않고 online-only로
동작하며, 새 release로 reload하면 풀린다
(`indexeddb-runtime.ts` open의 `VersionError` 분기). OPFS journal도 같은 답을
낸다(`indexeddb-opfs-journal.ts` `translateFor`)
### 확인 순서
@@ -170,7 +188,8 @@ read-only나 online-only가 사용자 작성 내용을 잃게 한다면 먼저 e
6. 모든 connection에 `versionchange`/forced-close handler가 설치됐는지 확인한다.
7. BroadcastChannel prepare hint는 참고만 하고 실제 open/connection registry를
확인한다.
8. N-1 bundle이 future schema를 read-only/online-only로 처리하는지 확인한다.
8. N-1 bundle이 future schema를 `INCOMPATIBLE` / `ONLINE_ONLY`로 처리하는지
확인한다. reference runtime은 read-only가 아니라 online-only로 닫는다.
### 안전한 조치
@@ -196,6 +215,12 @@ remote tab 강제 종료, database 삭제, 무한 reload는 금지한다.
- `MIGRATION_FAILED`, unknown codec, schema postcondition failure
- `CORRUPT_DATA`, `NOT_READABLE`, forced close
- `RETENTION_SWEEP`(`INDEXEDDB_WRITE`)과 `pruneExpiredReceipts`
(`INDEXEDDB_MIGRATE`)의 `CORRUPT_DATA` 관측은 실패가 아니다. 손상 행을 지우지
않고 건너뛴 수이고, 나머지 정상 행은 정리된다(05-05,
`indexeddb-runtime.ts` `purgeEligibleRecords`,
`indexeddb-maintenance.ts` `pruneExpiredReceipts`). budget이 삭제 bytes나
receipt 수를 감당하지 못하면 그 값은 내리지 않고 둔다
- transaction request success 후 commit failure
- `LIMIT_EXCEEDED`: dataset logical byte/receipt budget 초과
- lifecycle batch가 proof 없이 실행되거나 TTL/UNTIL_SYNCED record가 잘못 노출·삭제
@@ -248,7 +273,21 @@ remote tab 강제 종료, database 삭제, 무한 reload는 금지한다.
proof를 운영자가 수동 생성·재사용하지 않음
database 전체 삭제는 data owner가 export/recovery와 영향 scope를 승인한 별도
action이다.
action이다. runtime이 스스로 DB를 지우는 경우는 하나뿐이다(05-03).
- 조건: `enforceLifecycleBatch`의 action이 `LOGOUT` 또는 `ACCOUNT_DELETION`이고,
`authorizeLifecycle`이 proof를 내준 뒤, open이 `POLICY_REJECTED` 또는
`CORRUPT_DATA`로 실패했을 때(budget/binding 행 손상, 바뀐 policy). 행 단위
purge는 open이 필요해서 이 상태에서는 영영 돌 수 없다
(`src/adapters/storage/indexeddb/indexeddb-runtime.ts` `deleteUnadmittedDataset`).
- 범위: 세 opaque token에서 만든 그 partition의 DB 이름 하나
(`indexeddb-governance.ts` `indexedDbPhysicalDatabaseName`). 다른 partition과
origin의 다른 DB는 건드리지 않는다.
- 결과: 지워지면 receipt `COMPLETE`와 `datasetDeleted: true`(행 수는 세지
않는다). 다른 context가 연결을 닫지 않아 blocked deadline이 지나면 `BLOCKED`
/ `RELOAD_OTHER_CONTEXTS`를 반환한다. 이때 삭제 요청은 브라우저에 살아 있어
나중에 적용될 수 있으므로 다른 tab을 닫고 같은 purge를 다시 호출한다.
- governance가 정상이면 지금처럼 행 단위 purge만 하고 DB는 지우지 않는다.
성공 판정:
@@ -336,6 +375,11 @@ VD-15 목표 runtime을 먼저 구현한다.
- committed logical row와 physical manifest/file 불일치
- digest/length mismatch
- worker crash, handle lock timeout, `NoModificationAllowedError`
- put의 retryable `NOT_READABLE`: source가 `putIdleTimeoutMs`(기본 2분) 동안 다음
bytes를 주지 않아 put을 끝내고 origin mutation lock을 풀었다(05-07,
`opfs-worker-client.ts` `preparePut`, `opfs-policy.ts`). OPFS 쓰기는 origin
전체에서 한 번에 하나라서, 이 bound가 없으면 멈춘 source 하나가 모든 tab의 쓰기를
`mutationLockWaitTimeoutMs` 뒤 `BLOCKED`로 만든다
- sensitive policy maintenance의 `POLICY_REJECTED`, proof replay/expiry 또는
authority provider/consumer unavailable
@@ -371,12 +415,25 @@ VD-15 목표 runtime을 먼저 구현한다.
### recovery
- `PREPARING`: partial staging 검증 후 idempotent resume 또는 purge
- `FILES_READY`: expected generation/digest 일치 시 logical commit, 아니면
quarantine
- `PREPARING`: reconcile은 resume하지 않는다. staging receipt가 가리키는 그
transaction의 physical generation과 staging을 지우고 journal row를 rollback한다.
receipt가 없거나 0바이트면(첫 receipt 쓰기가 quota나 crash로 끝나지 못함) 지울
generation이 없으므로 staging만 지운다(05-02). 읽을 수 없는 receipt는 지우지
않고 그 row를 `failedTransactions`로 남긴다
(`opfs-byte-store-adapter.ts` `reconcileTransaction`,
`opfs-worker-runtime.ts` `cleanupTransactionLocked`)
- `FILES_READY`: worker가 manifest와 모든 chunk digest를 다시 검증한다. 맞으면
logical commit 뒤 finalize한다. 맞지 않으면 quarantine하지 않고 그 generation과
staging을 지운 뒤 rollback한다. 지워지는 것은 commit 전 staging뿐이다
- `COMMITTED` + file missing/corrupt: reconstructable만 server rehydrate;
user-created private는 read-only/export/recovery
- stale staging/orphan chunk는 grace period와 bounded mark/sweep 후 제거
- `LOGOUT`/`ACCOUNT_DELETION` purge는 객체를 모두 지운 뒤 그 partition의 chunk를
grace 없이 sweep한다(05-04, `opfs-byte-store-adapter.ts` `enforcePolicies`).
journal이나 staging receipt가 참조하는 chunk는 이때도 지우지 않는다. report의
`chunkSweep`이 `COMPLETED`여야 bytes가 지워진 것이다. `DEADLINE_REACHED`면 같은
purge를 다시 호출하고, `STAGING_STATE_UNREADABLE`이면 읽을 수 없는 staging부터
처리한다
- lock/worker 문제면 OPFS write off → approved capped fallback 또는 online-only
- scope binding mismatch면 affected scope를 read-only로 격리하고 registry/
composition을 forward fix; 다른 token path로 bytes를 이동하거나 추측 복구 금지
@@ -31,7 +31,7 @@
| 파일 | 책임 | 주요 의존성 / 리뷰 결과 |
| --- | --- | --- |
| `src/adapters/browser-file-storage/index.ts` | browser data 공통 Result와 StorageManager adapter barrel export | 내부 두 모듈만 export. 경계가 작고 유지 대상. |
| `src/adapters/browser-file-storage/result.ts` | native 예외를 closed `BrowserDataFailure`로 정규화하고 안전한 observation 제공 | `application/ports/browser-file-storage/shared.ts`; raw path/name/message 비노출, observer 예외 격리가 좋다. |
| `src/adapters/browser-file-storage/result.ts` | native 예외를 closed `BrowserDataFailure`로 정규화하고 안전한 observation 제공. IndexedDB 예외 표 `mapIndexedDbException`도 여기 있다(05-11, 옛 `storage/indexeddb/indexeddb-failure.ts`) | `application/ports/browser-file-storage/shared.ts`; raw path/name/message 비노출, observer 예외 격리가 좋다. |
| `src/adapters/browser-file-storage/storage-manager-adapter.ts` | `estimate/persisted/persist` snapshot, pressure bucket, user-activation-bound persistence 요청 | storage durability port/result. estimate를 예약량으로 오인하지 않고 irreversible `persist()` truth를 보존한다. origin coordinator는 의도적으로 없음(GAP-03). |
### 1.2 `browser-files`
@@ -66,7 +66,6 @@
| `src/adapters/storage/browser-storage-adapter.ts` | registry key별 local/session/memory 저장, TTL, failure overlay/tombstone, quota fallback | `storage-keys`, `StoragePort`, codec, diagnostics. strict registry와 stale persistent suppression을 유지. adjacent physical-key migration/sweep는 문서상 미구현. |
| `src/adapters/storage/browser-storage-codec.ts` | bounded exact JSON envelope, exotic/accessor/unsafe-key/cycle/depth/node 거절 | 독립 codec. prototype pollution/JSON silent coercion 방어가 좋다. |
| `src/adapters/storage/indexeddb/index.ts` | IndexedDB runtime/maintenance/governance/migration export | native IDB type을 application port 밖으로 내보내지 않는 구조 유지. |
| `src/adapters/storage/indexeddb/indexeddb-failure.ts` | IDB/DOM failure를 closed browser failure로 변환 | common Result. raw native detail 비노출 유지. |
| `src/adapters/storage/indexeddb/indexeddb-governance.ts` | opaque dataset scope/physical DB identity 및 frozen policy binding | indexeddb/shared ports. account/business ID를 physical name에 쓰지 않는 양방향 binding 유지. |
| `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | post-open codec migration 및 idempotency receipt prune, keyset checkpoint, budget/revision fencing | IndexedDB port/types/failure/governance. async transform outside tx, row+sidecar+budget+checkpoint atomic commit은 좋다. STO-06 및 temporal drain lease 개선 후보. |
| `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | additive-only contiguous DDL planner/validator | indexeddb types. destructive DDL 거절 유지. |
+32 -33
View File
@@ -122,38 +122,37 @@
| 112 | `src/adapters/storage/browser-storage-codec.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 113 | `src/adapters/storage/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 114 | `src/adapters/storage/indexeddb/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 115 | `src/adapters/storage/indexeddb/indexeddb-failure.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 116 | `src/adapters/storage/indexeddb/indexeddb-governance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 117 | `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 118 | `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 119 | `src/adapters/storage/indexeddb/indexeddb-records.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 120 | `src/adapters/storage/indexeddb/indexeddb-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 121 | `src/adapters/storage/indexeddb/indexeddb-types.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 122 | `src/adapters/storage/opfs/browser-opfs-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 123 | `src/adapters/storage/opfs/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 124 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 125 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 126 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 127 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 128 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 129 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 130 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 131 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
| 132 | `src/adapters/telemetry/feature-telemetry.ts` | [Network/state](./01-network-and-state.md) |
| 133 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
| 134 | `src/adapters/telemetry/telemetry-collector.ts` | [Network/state](./01-network-and-state.md) |
| 135 | `src/adapters/web-push/codec-primitives.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 136 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 137 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 138 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 139 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 140 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 141 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 142 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 143 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 144 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 145 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 146 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 115 | `src/adapters/storage/indexeddb/indexeddb-governance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 116 | `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 117 | `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 118 | `src/adapters/storage/indexeddb/indexeddb-records.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 119 | `src/adapters/storage/indexeddb/indexeddb-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 120 | `src/adapters/storage/indexeddb/indexeddb-types.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 121 | `src/adapters/storage/opfs/browser-opfs-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 122 | `src/adapters/storage/opfs/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 123 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 124 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 125 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 126 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 127 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 128 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 129 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 130 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
| 131 | `src/adapters/telemetry/feature-telemetry.ts` | [Network/state](./01-network-and-state.md) |
| 132 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
| 133 | `src/adapters/telemetry/telemetry-collector.ts` | [Network/state](./01-network-and-state.md) |
| 134 | `src/adapters/web-push/codec-primitives.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 135 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 136 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 137 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 138 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 139 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 140 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 141 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 142 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 143 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 144 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 145 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
합계: **146/146**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
합계: **145/145**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
@@ -40,6 +40,12 @@ export function abortedResult(
: null;
}
/**
* The table for File, Storage Manager, Cache Storage and download DOM errors.
* IndexedDB errors use `mapIndexedDbException` below: the same exception name
* means something else there (a `NotFoundError` is a missing store, not a
* missing file).
*/
export function mapBrowserDataException(
error: unknown,
operation: BrowserDataOperation,
@@ -89,6 +95,81 @@ export function mapBrowserDataException(
}
}
function exceptionName(error: unknown): string {
if (
error &&
typeof error === "object" &&
"name" in error &&
typeof error.name === "string"
) {
return error.name;
}
return "";
}
/**
* 05-11. The one table for native IndexedDB errors. It lives in this kernel
* exception file so the OPFS journal no longer reaches into the IndexedDB
* runtime's folder for it. It maps the closed DOMException vocabulary without
* exposing an exception object, message, key or stored value across the
* adapter boundary.
*
* Differences from `mapBrowserDataException`, on purpose: a missing store or
* index, an inactive or read-only transaction and a `VersionError` are schema
* problems (`MIGRATION_FAILED`, READ_ONLY), not a missing file or a transient
* failure; quota and constraint failures are not retried blindly.
*/
export function mapIndexedDbException(
error: unknown,
operation: BrowserDataOperation,
): BrowserDataResult<never> {
switch (exceptionName(error)) {
case "AbortError":
return browserDataFailure("ABORTED", operation);
case "ConstraintError":
return browserDataFailure("CONFLICT", operation);
case "DataCloneError":
case "DataError":
return browserDataFailure("CORRUPT_DATA", operation, {
recovery: "READ_ONLY",
});
case "InvalidAccessError":
case "InvalidStateError":
case "NotFoundError":
case "ReadOnlyError":
case "TransactionInactiveError":
case "VersionError":
return browserDataFailure("MIGRATION_FAILED", operation, {
recovery: "READ_ONLY",
});
case "NotAllowedError":
case "SecurityError":
return browserDataFailure("PERMISSION_DENIED", operation, {
recovery: "ONLINE_ONLY",
});
case "NotReadableError":
return browserDataFailure("NOT_READABLE", operation, {
retryable: true,
recovery: "REOPEN",
});
case "QuotaExceededError":
case "NS_ERROR_DOM_QUOTA_REACHED":
return browserDataFailure("QUOTA_EXCEEDED", operation, {
recovery: "READ_ONLY",
});
case "UnknownError":
return browserDataFailure("UNAVAILABLE", operation, {
retryable: true,
recovery: "REOPEN",
});
default:
return browserDataFailure("UNAVAILABLE", operation, {
retryable: true,
recovery: "RETRY",
});
}
}
export function observeBrowserData(
observer: BrowserDataObserver | undefined,
observation: BrowserDataObservation,
+13 -3
View File
@@ -427,6 +427,16 @@ export function openIndexedDbDatabase<Failure>(
settle(succeeded(database));
return;
}
// 05-13. Admission is asynchronous and nothing else answers another
// context's upgrade until the caller adopts the connection, so that
// upgrade would wait blocked until its deadline. While admission runs, a
// versionchange closes this connection and ends the open as CLOSED. An
// `admit` that installs its own handler replaces this one.
database.onversionchange = () => {
if (settled) return;
closeQuietly(database);
settleFailure({ kind: "CLOSED" });
};
let admission: IndexedDbAdmission | Promise<IndexedDbAdmission>;
try {
admission = input.admit(database);
@@ -605,9 +615,9 @@ export function createIndexedDbConnection<Failure>(
/**
* In this module because it is the same `IDBOpenDBRequest` state machine as
* `openIndexedDbDatabase`, not because three subsystems need it — only
* `indexeddb-checkpoint-store.ts` deletes a database, and nothing here asks the
* other three to start. Leaving it out would leave a third hand-written copy of
* `openIndexedDbDatabase`. Two callers delete a database:
* `indexeddb-checkpoint-store.ts`, and the IndexedDB runtime's authorized
* LOGOUT/ACCOUNT_DELETION purge when its dataset fails admission. Leaving it out would leave a third hand-written copy of
* the settle-once blocked-deadline latch eight lines away from the kernel's.
*
* There is deliberately no `signal`: the request cannot be cancelled after
+5 -3
View File
@@ -2,9 +2,11 @@
* 최상위 storage 배럴은 Web Storage 어댑터만 내보낸다.
*
* `./indexeddb/index.ts`와 `./opfs/index.ts`를 여기서 재수출하지 말 것.
* `scripts/test-browser-file-storage-runtime-removal.ts:23-34`가 그 두 폴더만
* 삭제한 뒤 잔존 import를 예외로 잡는다 — 재수출하면 제거 드릴이 죽는다.
* 두 서브배럴이 각자 제거 가능한 런타임의 경계다.
* `scripts/test-browser-file-storage-runtime-removal.ts`의 `runtimePaths`는 이 두
* 폴더를 browser-file-storage 번들 전체(ports, browser-files, browser-transfer,
* cache-storage 등)와 함께 삭제한 뒤 잔존 import를 예외로 잡는다 — 재수출하면
* 제거 드릴이 죽는다. 드릴은 번들을 통째로만 지운다. 두 폴더 중 하나만 지우는
* 경우는 드릴이 검증하지 않는다.
*/
export {
createBrowserStorageAdapter,
@@ -1,72 +0,0 @@
import type {
BrowserDataOperation,
BrowserDataResult,
} from "../../../application/ports/browser-file-storage/shared.ts";
import { browserDataFailure } from "../../browser-file-storage/result.ts";
function exceptionName(error: unknown): string {
if (
error &&
typeof error === "object" &&
"name" in error &&
typeof error.name === "string"
) {
return error.name;
}
return "";
}
/**
* Maps the closed DOMException vocabulary without exposing an exception
* object, message, key or stored value across the adapter boundary.
*/
export function mapIndexedDbException(
error: unknown,
operation: BrowserDataOperation,
): BrowserDataResult<never> {
switch (exceptionName(error)) {
case "AbortError":
return browserDataFailure("ABORTED", operation);
case "ConstraintError":
return browserDataFailure("CONFLICT", operation);
case "DataCloneError":
case "DataError":
return browserDataFailure("CORRUPT_DATA", operation, {
recovery: "READ_ONLY",
});
case "InvalidAccessError":
case "InvalidStateError":
case "NotFoundError":
case "ReadOnlyError":
case "TransactionInactiveError":
case "VersionError":
return browserDataFailure("MIGRATION_FAILED", operation, {
recovery: "READ_ONLY",
});
case "NotAllowedError":
case "SecurityError":
return browserDataFailure("PERMISSION_DENIED", operation, {
recovery: "ONLINE_ONLY",
});
case "NotReadableError":
return browserDataFailure("NOT_READABLE", operation, {
retryable: true,
recovery: "REOPEN",
});
case "QuotaExceededError":
case "NS_ERROR_DOM_QUOTA_REACHED":
return browserDataFailure("QUOTA_EXCEEDED", operation, {
recovery: "READ_ONLY",
});
case "UnknownError":
return browserDataFailure("UNAVAILABLE", operation, {
retryable: true,
recovery: "REOPEN",
});
default:
return browserDataFailure("UNAVAILABLE", operation, {
retryable: true,
recovery: "RETRY",
});
}
}
@@ -3,6 +3,10 @@ import {
assertValidStoragePolicy,
type BrowserStoragePolicy,
} from "../../../application/ports/browser-file-storage/shared.ts";
import {
onIndexedDbRequest,
runIndexedDbTransaction,
} from "../../platform/indexeddb-transaction.ts";
import {
INDEXEDDB_DATASET_BUDGET_KEY,
isStoredDatasetBudget,
@@ -219,105 +223,72 @@ export function queueIndexedDbUpgradeBinding(
/**
* Post-open verification protects non-upgrade opens and maintenance callers.
* 05-12. It runs on the kernel transaction, so settle-once, caller abort and
* the abort-after-commit rule (CP-4) are the kernel's, not a local copy's.
*/
export function verifyIndexedDbDatasetBinding(
export async function verifyIndexedDbDatasetBinding(
database: IDBDatabase,
governanceStore: string,
expected: StoredDatasetBinding,
signal?: AbortSignal,
): Promise<IndexedDbBindingVerification> {
if (signal?.aborted) {
return Promise.resolve({ ok: false, reason: "ABORTED" });
}
let transaction: IDBTransaction;
try {
transaction = database.transaction(governanceStore, "readonly");
} catch (error) {
return Promise.resolve({
ok: false,
reason: "NATIVE_ERROR",
error,
});
}
return new Promise((resolve) => {
let settled = false;
let observed: unknown;
let observedBudget: unknown;
let requestError: unknown;
let callerAborted = false;
const finish = (result: IndexedDbBindingVerification) => {
if (settled) return;
settled = true;
signal?.removeEventListener("abort", onAbort);
resolve(result);
};
function onAbort(): void {
callerAborted = true;
try {
transaction.abort();
} catch {
// Completion determines the race.
}
}
signal?.addEventListener("abort", onAbort, { once: true });
transaction.onerror = () => {
requestError ??= transaction.error;
};
transaction.onabort = () =>
finish(
callerAborted
? { ok: false, reason: "ABORTED" }
: {
ok: false,
reason: "NATIVE_ERROR",
error: requestError ?? transaction.error,
},
const outcome = await runIndexedDbTransaction<
IndexedDbBindingVerification,
BindingFailure
>({
database,
stores: [governanceStore],
mode: "readonly",
signal,
translate: (cause) =>
cause.kind === "CALLER_ABORT"
? { ok: false, reason: "ABORTED" }
: {
ok: false,
reason: "NATIVE_ERROR",
error: cause.kind === "NATIVE_EXCEPTION" ? cause.error : undefined,
},
queue: (transaction, context) => {
const store = transaction.objectStore(governanceStore);
let observed: unknown;
onIndexedDbRequest<unknown, BindingFailure>(
store.get(INDEXEDDB_DATASET_BINDING_KEY),
context,
(value) => {
observed = value;
},
);
transaction.oncomplete = () => {
if (observed === undefined) {
finish({ ok: false, reason: "MISSING" });
} else if (!isStoredDatasetBinding(observed)) {
finish({ ok: false, reason: "CORRUPT" });
} else if (!sameIndexedDbDatasetBinding(observed, expected)) {
finish({ ok: false, reason: "MISMATCH" });
} else if (
!isStoredDatasetBudget(observedBudget) ||
observedBudget.usedBytes > expected.storagePolicy.hardBudgetBytes
) {
finish({ ok: false, reason: "CORRUPT" });
} else {
finish({ ok: true });
}
};
try {
const request = transaction
.objectStore(governanceStore)
.get(INDEXEDDB_DATASET_BINDING_KEY);
request.onerror = () => {
requestError ??= request.error;
};
request.onsuccess = () => {
observed = request.result;
};
const budgetRequest = transaction
.objectStore(governanceStore)
.get(INDEXEDDB_DATASET_BUDGET_KEY);
budgetRequest.onerror = () => {
requestError ??= budgetRequest.error;
};
budgetRequest.onsuccess = () => {
observedBudget = budgetRequest.result;
};
} catch (error) {
requestError = error;
try {
transaction.abort();
} catch {
finish({ ok: false, reason: "NATIVE_ERROR", error });
}
}
onIndexedDbRequest<unknown, BindingFailure>(
store.get(INDEXEDDB_DATASET_BUDGET_KEY),
context,
(budget) => {
context.succeed(bindingVerdict(observed, budget, expected));
},
);
},
});
return outcome.ok ? outcome.value : outcome.error;
}
type BindingFailure = Extract<IndexedDbBindingVerification, { ok: false }>;
function bindingVerdict(
observed: unknown,
budget: unknown,
expected: StoredDatasetBinding,
): IndexedDbBindingVerification {
if (observed === undefined) return { ok: false, reason: "MISSING" };
if (!isStoredDatasetBinding(observed)) {
return { ok: false, reason: "CORRUPT" };
}
if (!sameIndexedDbDatasetBinding(observed, expected)) {
return { ok: false, reason: "MISMATCH" };
}
if (
!isStoredDatasetBudget(budget) ||
budget.usedBytes > expected.storagePolicy.hardBudgetBytes
) {
return { ok: false, reason: "CORRUPT" };
}
return { ok: true };
}
@@ -12,6 +12,7 @@ import {
abortedResult,
browserDataFailure,
browserDataSuccess,
mapIndexedDbException,
} from "../../browser-file-storage/result.ts";
import {
openIndexedDbDatabase,
@@ -24,7 +25,6 @@ import {
type IndexedDbBudget,
type IndexedDbTransactionContext,
} from "../../platform/indexeddb-transaction.ts";
import { mapIndexedDbException } from "./indexeddb-failure.ts";
import {
createIndexedDbDatasetBinding,
indexedDbPhysicalDatabaseName,
@@ -146,6 +146,14 @@ function migrationFailure(): BrowserDataFailure {
return failureOf(migrationFailed());
}
function corruptFailure(): BrowserDataFailure {
return failureOf(
browserDataFailure("CORRUPT_DATA", "INDEXEDDB_MIGRATE", {
recovery: "READ_ONLY",
}),
);
}
function unsupportedFailure(): BrowserDataFailure {
return failureOf(unsupported());
}
@@ -314,10 +322,9 @@ export function createIndexedDbMaintenance<WireValue>(
}
/**
* STO-06. Maintenance keeps `mapIndexedDbException`, which the runtime and
* the OPFS journal share, rather than the checkpoint store's own table: the
* same native error is a different answer in the two families, and unifying
* them is a separate change from moving the mechanics onto the kernel.
* STO-06. Maintenance uses `mapIndexedDbException`, the one IndexedDB table
* in `browser-file-storage/result.ts` that the runtime and the OPFS journal
* share (05-11).
*
* One translator covers the whole adapter because the operation label is the
* constant `INDEXEDDB_MIGRATE` here — the runtime needs one per operation.
@@ -1124,6 +1131,11 @@ export function createIndexedDbMaintenance<WireValue>(
const opened = await openExactVersion(input.signal);
if (!opened.ok) return observeResult(opened);
const database = opened.value;
// 05-05. A corrupt receipt is left in place and counted, not a stop: one
// bad row used to end every later prune, so receipts piled up until every
// write failed with LIMIT_EXCEEDED. A receipt count already at zero stays
// there. Both are reported as one CORRUPT_DATA observation.
let corruptRows = 0;
try {
const pruned = await runTransaction<
@@ -1188,8 +1200,8 @@ export function createIndexedDbMaintenance<WireValue>(
String(cursor.primaryKey) ||
cursor.value.expiresAtEpochMs > cutoff.value
) {
context.fail(migrationFailure());
return { kind: "SUSPEND" };
corruptRows += 1;
return { kind: "CONTINUE" };
}
let deleteRequest: IDBRequest<undefined>;
try {
@@ -1207,11 +1219,14 @@ export function createIndexedDbMaintenance<WireValue>(
budgetRequest.onerror = () =>
context.requestFailed(budgetRequest.error);
budgetRequest.onsuccess = () => {
if (
!isStoredDatasetBudget(budgetRequest.result) ||
budgetRequest.result.receiptCount < 1
) {
context.fail(migrationFailure());
if (!isStoredDatasetBudget(budgetRequest.result)) {
context.fail(corruptFailure());
return;
}
if (budgetRequest.result.receiptCount < 1) {
corruptRows += 1;
deletedRows += 1;
resume({ kind: "CONTINUE" });
return;
}
const budgetWrite = governance.put({
@@ -1248,6 +1263,15 @@ export function createIndexedDbMaintenance<WireValue>(
});
},
);
if (corruptRows > 0) {
observe({
operation: "INDEXEDDB_MIGRATE",
outcome: "FAILED",
schemaVersion: dependencies.schemaVersion,
countBucket: countBucket(corruptRows),
failureCode: "CORRUPT_DATA",
});
}
return observeResult(
pruned,
pruned.ok ? pruned.value.deletedRows : 0,
@@ -20,10 +20,12 @@ import {
abortedResult,
browserDataFailure,
browserDataSuccess,
mapIndexedDbException,
} from "../../browser-file-storage/result.ts";
import { snapshotAbortTimers } from "../../platform/abortable-operation.ts";
import {
createIndexedDbConnection,
deleteIndexedDbDatabase,
openIndexedDbDatabase,
type IndexedDbTranslate,
} from "../../platform/indexeddb-connection.ts";
@@ -35,7 +37,6 @@ import {
type IndexedDbCursorVisit,
type IndexedDbTransactionContext,
} from "../../platform/indexeddb-transaction.ts";
import { mapIndexedDbException } from "./indexeddb-failure.ts";
import {
createIndexedDbDatasetBinding,
indexedDbPhysicalDatabaseName,
@@ -284,6 +285,14 @@ function unavailable(
});
}
function incompatible(
operation: BrowserDataOperation,
): BrowserDataResult<never> {
return browserDataFailure("UNSUPPORTED", operation, {
recovery: "ONLINE_ONLY",
});
}
function mappedFailure(
error: unknown,
operation: BrowserDataOperation,
@@ -550,10 +559,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
}
/**
* STO-06. The runtime keeps `mapIndexedDbException`, which maintenance and
* the OPFS journal share, rather than the checkpoint store's own table: the
* same native error is a different answer in the two families, and unifying
* them is a separate change from moving the mechanics onto the kernel.
* STO-06. The runtime uses `mapIndexedDbException`, the one IndexedDB table
* in `browser-file-storage/result.ts` that maintenance and the OPFS journal
* share (05-11).
*
* A translator per operation, because a read, a write and an open carry
* different labels. `CLOSED` is mapped explicitly: closing the handle while
@@ -642,6 +650,7 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
// so the migration count and the binding verdict belong to the attempt.
let appliedMigrations = 0;
let policyBindingRejected = false;
let futureSchema = false;
updateStatus({
kind: "OPENING",
targetVersion: dependencies.schemaVersion,
@@ -650,7 +659,20 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
factory,
databaseName,
version: dependencies.schemaVersion,
translate: translateFor(operation),
translate: (cause) => {
// 05-06. A VersionError on open means a newer release already
// upgraded this database. That is not a failed migration: this
// bundle can only work online until it is reloaded.
if (
cause.kind === "NATIVE_EXCEPTION" &&
(cause.error as { name?: unknown } | null)?.name ===
"VersionError"
) {
futureSchema = true;
return failureOf(incompatible(operation));
}
return translateFor(operation)(cause);
},
signal,
blockedTimeoutMs,
timers,
@@ -743,7 +765,14 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
: opened;
if (disposed) return result;
if (!result.ok) {
updateStatus({ kind: "CLOSED", reason: "NOT_OPENED" });
updateStatus(
futureSchema
? {
kind: "INCOMPATIBLE",
schemaVersion: dependencies.schemaVersion,
}
: { kind: "CLOSED", reason: "NOT_OPENED" },
);
// A failed migration is observed against INDEXEDDB_MIGRATE as well
// as against the open the caller asked for.
if (result.error.operation === "INDEXEDDB_MIGRATE") {
@@ -833,6 +862,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
const cancelled = abortedResult(signal, operation);
if (cancelled) return Promise.resolve(cancelled);
const db = connection.current();
if (!db && !disposed && status.kind === "INCOMPATIBLE") {
return Promise.resolve(incompatible(operation));
}
if (!db || disposed) return Promise.resolve(unavailable(operation));
return runIndexedDbTransaction<ResultValue, BrowserDataFailure>({
@@ -2044,12 +2076,21 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
return browserDataSuccess(undefined);
}
function purgeEligibleRecords(
/**
* 05-05. Like the partition purge (R-5.4), the sweep never stops on a row it
* cannot read: failing closed left every later expired row on the device and
* failed the next sweep the same way. A corrupt sidecar is left in place,
* counted and reported as one CORRUPT_DATA observation per batch. A budget
* that cannot account for a deleted row's bytes is left as it is: it can
* only read high, which blocks writes and never admits one.
*/
async function purgeEligibleRecords(
input: IndexedDbLifecycleBatchInput,
deadline: number,
cutoffEpochMs: number,
): Promise<BrowserDataResult<IndexedDbLifecycleBatchReceipt>> {
return runTransaction(
let corruptRows = 0;
const result = await runTransaction<IndexedDbLifecycleBatchReceipt>(
[
dependencies.recordStore,
dependencies.governanceStore,
@@ -2140,11 +2181,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
"UNTIL_SYNCED" &&
cursor.value.synchronization !== "CONFIRMED")
) {
context.fail(corruptFailure("INDEXEDDB_WRITE"));
// `fail` aborts, so the pump has nowhere to go. Suspending
// without ever resuming says that without claiming a summary
// the sweep never reached.
return { kind: "SUSPEND" };
scannedRows += 1;
corruptRows += 1;
return { kind: "CONTINUE" };
}
scannedRows += 1;
const measuredBytes = cursor.value.measuredBytes;
@@ -2176,13 +2215,16 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
budgetRequest.onerror = () =>
context.requestFailed(budgetRequest.error);
budgetRequest.onsuccess = () => {
if (
!isStoredDatasetBudget(budgetRequest.result) ||
budgetRequest.result.usedBytes < measuredBytes
) {
if (!isStoredDatasetBudget(budgetRequest.result)) {
context.fail(corruptFailure("INDEXEDDB_WRITE"));
return;
}
if (budgetRequest.result.usedBytes < measuredBytes) {
corruptRows += 1;
deletedRows += 1;
resume({ kind: "CONTINUE" });
return;
}
const storedBudgetRequest = governance.put({
bindingKey: INDEXEDDB_DATASET_BUDGET_KEY,
budgetVersion: 1,
@@ -2215,6 +2257,16 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
});
},
);
if (corruptRows > 0) {
observe({
operation: "INDEXEDDB_WRITE",
outcome: "FAILED",
schemaVersion: dependencies.schemaVersion,
countBucket: countBucket(corruptRows),
failureCode: "CORRUPT_DATA",
});
}
return result;
}
/**
@@ -2500,7 +2552,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
cutoffEpochMs,
);
} else {
result = await purgePartitionRecords(input, deadline);
result =
(await deleteUnadmittedDataset(input.signal)) ??
(await purgePartitionRecords(input, deadline));
}
return observeResult(
operation,
@@ -2509,6 +2563,54 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
);
}
/**
* 05-03. A LOGOUT or ACCOUNT_DELETION purge must not depend on the dataset
* passing admission: a corrupt governance row rejects every open, so no row
* could ever be purged one by one. The authority has already approved this
* purge, and the database name comes from this partition's own tokens, so
* deleting the whole database touches no other partition. Returns `null`
* when a connection is ready for the row-by-row purge.
*/
async function deleteUnadmittedDataset(
signal: AbortSignal | undefined,
): Promise<BrowserDataResult<IndexedDbLifecycleBatchReceipt> | null> {
if (connection.current()) return null;
const opened = await open(signal);
if (opened.ok) return null;
if (
(opened.error.code !== "POLICY_REJECTED" &&
opened.error.code !== "CORRUPT_DATA") ||
disposed ||
typeof factory?.deleteDatabase !== "function"
) {
return opened;
}
const deleted = await deleteIndexedDbDatabase<BrowserDataFailure>({
factory,
databaseName,
translate: translateFor("INDEXEDDB_WRITE"),
blockedTimeoutMs,
timers,
});
if (!deleted.ok) return deleted;
if (deleted.value.kind === "BLOCKED_DEADLINE") {
// The request is still live and may yet land; the next call finds out.
return browserDataFailure("BLOCKED", "INDEXEDDB_WRITE", {
retryable: true,
recovery: "RELOAD_OTHER_CONTEXTS",
});
}
return browserDataSuccess(
Object.freeze({
state: "COMPLETE" as const,
scannedRows: 0,
deletedRows: 0,
budgetExhausted: false,
datasetDeleted: true as const,
}),
);
}
function close(): void {
if (disposed) return;
disposed = true;
@@ -17,6 +17,7 @@ import {
import {
browserDataFailure,
browserDataSuccess,
mapIndexedDbException,
} from "../../browser-file-storage/result.ts";
import { snapshotAbortTimers } from "../../platform/abortable-operation.ts";
import {
@@ -31,9 +32,8 @@ import {
type IndexedDbRequestSink,
type IndexedDbTransactionContext,
} from "../../platform/indexeddb-transaction.ts";
import { mapIndexedDbException } from "../indexeddb/indexeddb-failure.ts";
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
import { isPreparedObject } from "./opfs-prepared-object.ts";
import { isPreparedObject, stableJson } from "./opfs-prepared-object.ts";
export type IndexedDbOpfsJournalDependencies = Readonly<{
authorityToken: string;
@@ -1608,28 +1608,6 @@ function isChunkReference(
);
}
function stableJson(value: unknown): string {
if (
value === null ||
typeof value === "string" ||
typeof value === "boolean" ||
typeof value === "number"
) {
return JSON.stringify(value);
}
if (Array.isArray(value)) {
return `[${value.map(stableJson).join(",")}]`;
}
if (typeof value === "object") {
const record = value as Record<string, unknown>;
return `{${Object.keys(record)
.sort()
.map((key) => `${JSON.stringify(key)}:${stableJson(record[key])}`)
.join(",")}}`;
}
throw new TypeError("Journal value is not JSON-safe.");
}
/**
* `browserDataFailure` and `mapIndexedDbException` build a `Result`, while the
* kernel's `translate` and `context.fail` want the failure on its own. Both
@@ -1698,10 +1676,9 @@ function limitExceeded(
}
/**
* STO-01. The journal keeps `mapIndexedDbException`, which the IndexedDB
* runtime and maintenance share, rather than the checkpoint store's own table:
* the same native error is a different answer in the two families and unifying
* them is a separate change from moving the mechanics onto the kernel.
* STO-01. The journal uses `mapIndexedDbException`, the one IndexedDB table
* in `browser-file-storage/result.ts` that the IndexedDB runtime and
* maintenance share (05-11). It imports nothing from `storage/indexeddb/`.
*
* A translator per operation, because the journal labels a read and a write
* differently. The connection always translates as `INDEXEDDB_OPEN`: an open
@@ -1714,6 +1691,17 @@ function translateFor(
return (cause) => {
switch (cause.kind) {
case "NATIVE_EXCEPTION":
// 05-06. A journal a newer release upgraded is not a failed
// migration; this bundle cannot use it and works online instead.
if (
(cause.error as { name?: unknown } | null)?.name === "VersionError"
) {
return failureOf(
browserDataFailure("UNSUPPORTED", operation, {
recovery: "ONLINE_ONLY",
}),
);
}
return failureOf(mapIndexedDbException(cause.error, operation));
case "BLOCKED":
case "BLOCKED_DEADLINE":
@@ -96,6 +96,13 @@ export type OpfsByteStoreDependencies = Readonly<{
* deliberately separate from any caller signal.
*/
compensationSignal?: AbortSignal;
/**
* 05-01 (D-3). Web Locks, `navigator.locks` by default. A put or delete
* holds an exclusive per-transaction lock from journal `begin` until it
* returns; reconcile only touches rows whose lock it can take at once, so a
* live transaction in any tab is never cleaned up underneath its owner.
*/
lockManager?: Pick<LockManager, "request">;
now?: () => number;
observer?: OpfsSafeObserver;
/**
@@ -152,6 +159,12 @@ export function createOpfsByteStoreAdapter(
*/
const compensationSignal = dependencies.compensationSignal;
const now = dependencies.now ?? Date.now;
const lockManager =
dependencies.lockManager ?? globalThis.navigator?.locks;
if (!lockManager || typeof lockManager.request !== "function") {
throw new TypeError("OPFS byte store requires Web Locks.");
}
const requestLock = lockManager.request.bind(lockManager);
const objects: DurableObjectStorePort = Object.freeze({
async capabilities() {
@@ -203,146 +216,152 @@ export function createOpfsByteStoreAdapter(
}
const targetGeneration = (current?.descriptor.generation ?? 0) + 1;
const transactionId = createTransactionId();
// STO-01. The logical generation is reused across transactions; this
// token makes the physical target unique so a late compensation can never
// delete a newer transaction's directory.
const physicalGenerationId = createPhysicalGenerationId();
notifyProgress(request, "PREPARING", 0);
const begun = await dependencies.journal.begin({
return await withTransactionLock(
transactionId,
mutation: "PUT",
scope: dependencies.scope,
objectId: request.objectId,
expectedGeneration: request.expectedGeneration,
targetGeneration,
targetByteLength: request.source.byteLength!,
targetStoragePolicy: dependencies.storagePolicy,
startedAtEpochMs: now(),
});
if (!begun.ok) {
return observeFailure(
rebaseFailure(begun.error, "OBJECT_WRITE"),
dependencies.observer,
request.source.byteLength!,
);
}
"OBJECT_WRITE",
async () => {
// STO-01. The logical generation is reused across transactions; this
// token makes the physical target unique so a late compensation can never
// delete a newer transaction's directory.
const physicalGenerationId = createPhysicalGenerationId();
notifyProgress(request, "PREPARING", 0);
const begun = await dependencies.journal.begin({
transactionId,
mutation: "PUT",
scope: dependencies.scope,
objectId: request.objectId,
expectedGeneration: request.expectedGeneration,
targetGeneration,
targetByteLength: request.source.byteLength!,
targetStoragePolicy: dependencies.storagePolicy,
startedAtEpochMs: now(),
});
if (!begun.ok) {
return observeFailure(
rebaseFailure(begun.error, "OBJECT_WRITE"),
dependencies.observer,
request.source.byteLength!,
);
}
const descriptor: Omit<DurableObjectDescriptor, "integrity"> =
Object.freeze({
objectId: request.objectId,
scope: dependencies.scope,
generation: targetGeneration,
byteLength: request.source.byteLength!,
mediaType: request.mediaType,
createdAtEpochMs: now(),
storagePolicy: dependencies.storagePolicy,
});
const prepared = await dependencies.worker.preparePut({
transactionId,
physicalGenerationId,
descriptor,
source: request.source,
signal: request.signal,
onProgress: request.onProgress,
});
if (!prepared.ok) {
return observeFailure(
await compensatePreparedPut(
begun.value,
const descriptor: Omit<DurableObjectDescriptor, "integrity"> =
Object.freeze({
objectId: request.objectId,
scope: dependencies.scope,
generation: targetGeneration,
byteLength: request.source.byteLength!,
mediaType: request.mediaType,
createdAtEpochMs: now(),
storagePolicy: dependencies.storagePolicy,
});
const prepared = await dependencies.worker.preparePut({
transactionId,
physicalGenerationId,
prepared.error,
),
dependencies.observer,
request.source.byteLength!,
);
}
descriptor,
source: request.source,
signal: request.signal,
onProgress: request.onProgress,
});
if (!prepared.ok) {
return observeFailure(
await compensatePreparedPut(
begun.value,
physicalGenerationId,
prepared.error,
),
dependencies.observer,
request.source.byteLength!,
);
}
const filesReady = await dependencies.journal.markFilesReady(
transactionId,
begun.value.fencingToken,
prepared.value,
);
if (!filesReady.ok) {
return observeFailure(
await compensatePreparedPut(
begun.value,
physicalGenerationId,
filesReady.error,
),
dependencies.observer,
request.source.byteLength!,
);
}
const filesReady = await dependencies.journal.markFilesReady(
transactionId,
begun.value.fencingToken,
prepared.value,
);
if (!filesReady.ok) {
return observeFailure(
await compensatePreparedPut(
begun.value,
physicalGenerationId,
filesReady.error,
),
dependencies.observer,
request.source.byteLength!,
);
}
const committed = await dependencies.journal.commitPut(
transactionId,
begun.value.fencingToken,
);
if (!committed.ok) {
// The commit response can be lost after IndexedDB commits. Reconciliation
// decides from the durable journal; rolling back here would be unsafe.
return observeFailure(
rebaseFailure(committed.error, "OBJECT_WRITE"),
dependencies.observer,
request.source.byteLength!,
);
}
const committed = await dependencies.journal.commitPut(
transactionId,
begun.value.fencingToken,
);
if (!committed.ok) {
// The commit response can be lost after IndexedDB commits. Reconciliation
// decides from the durable journal; rolling back here would be unsafe.
return observeFailure(
rebaseFailure(committed.error, "OBJECT_WRITE"),
dependencies.observer,
request.source.byteLength!,
);
}
notifyProgress(
request,
"FINALIZING",
prepared.value.descriptor.byteLength,
notifyProgress(
request,
"FINALIZING",
prepared.value.descriptor.byteLength,
);
const finalized = await dependencies.worker.finalizePut(
transactionId,
prepared.value,
request.signal,
);
if (!finalized.ok) {
// STO-RR-01. The commit fence already passed, so the payload is durable
// and the journal keeps its COMMITTED record for reconciliation to
// settle. What did not happen is finalization: the previous generation
// and the staging directory are still present. Reporting a plain
// success here would claim a settled state nobody observed, so the
// worker's own failure is surfaced and the record is left recoverable.
return observeFailure(
rebaseFailure(finalized.error, "OBJECT_WRITE"),
dependencies.observer,
request.source.byteLength!,
);
}
const completed = await dependencies.journal.complete(
transactionId,
begun.value.fencingToken,
);
if (!completed.ok) {
// NS-04. The payload is durable and finalized, so nothing is rolled
// back and the `COMMITTED` row stays the reconciler's authority. What
// did not happen is settling the transaction, and reporting a plain
// success for it claimed a state nobody observed while the reconcile
// backlog and its quota pressure grew unseen.
observeOpfsSafely(dependencies.observer, {
operation: "OBJECT_WRITE",
outcome: "DEGRADED",
failureCode: completed.error.code,
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
});
return {
ok: false as const,
error: Object.freeze({
...completed.error,
operation: "OBJECT_WRITE" as const,
retryable: true,
recovery: "RECONCILE" as const,
}),
};
}
observeOpfsSafely(dependencies.observer, {
operation: "OBJECT_WRITE",
outcome: "SUCCEEDED",
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
});
return browserDataSuccess(prepared.value.descriptor);
},
);
const finalized = await dependencies.worker.finalizePut(
transactionId,
prepared.value,
request.signal,
);
if (!finalized.ok) {
// STO-RR-01. The commit fence already passed, so the payload is durable
// and the journal keeps its COMMITTED record for reconciliation to
// settle. What did not happen is finalization: the previous generation
// and the staging directory are still present. Reporting a plain
// success here would claim a settled state nobody observed, so the
// worker's own failure is surfaced and the record is left recoverable.
return observeFailure(
rebaseFailure(finalized.error, "OBJECT_WRITE"),
dependencies.observer,
request.source.byteLength!,
);
}
const completed = await dependencies.journal.complete(
transactionId,
begun.value.fencingToken,
);
if (!completed.ok) {
// NS-04. The payload is durable and finalized, so nothing is rolled
// back and the `COMMITTED` row stays the reconciler's authority. What
// did not happen is settling the transaction, and reporting a plain
// success for it claimed a state nobody observed while the reconcile
// backlog and its quota pressure grew unseen.
observeOpfsSafely(dependencies.observer, {
operation: "OBJECT_WRITE",
outcome: "DEGRADED",
failureCode: completed.error.code,
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
});
return {
ok: false as const,
error: Object.freeze({
...completed.error,
operation: "OBJECT_WRITE" as const,
retryable: true,
recovery: "RECONCILE" as const,
}),
};
}
observeOpfsSafely(dependencies.observer, {
operation: "OBJECT_WRITE",
outcome: "SUCCEEDED",
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
});
return browserDataSuccess(prepared.value.descriptor);
},
async open(
@@ -469,63 +488,70 @@ export function createOpfsByteStoreAdapter(
);
}
const targetStoragePolicy = current.value.descriptor.storagePolicy;
const transactionId = createTransactionId();
const begun = await dependencies.journal.begin({
return await withTransactionLock(
transactionId,
mutation: "DELETE",
scope: dependencies.scope,
objectId: request.objectId,
expectedGeneration: request.expectedGeneration,
targetGeneration: request.expectedGeneration + 1,
targetByteLength: 0,
targetStoragePolicy: current.value.descriptor.storagePolicy,
startedAtEpochMs: now(),
});
if (!begun.ok) {
return observeFailure(
rebaseFailure(begun.error, "OBJECT_DELETE"),
dependencies.observer,
);
}
const committed = await dependencies.journal.commitDelete(
transactionId,
begun.value.fencingToken,
);
if (!committed.ok) {
return observeFailure(
rebaseFailure(committed.error, "OBJECT_DELETE"),
dependencies.observer,
);
}
const removed = await dependencies.worker.removeObject(
dependencies.scope,
request.objectId,
request.expectedGeneration,
request.signal,
);
const completed = removed.ok
? await dependencies.journal.complete(
"OBJECT_DELETE",
async () => {
const begun = await dependencies.journal.begin({
transactionId,
mutation: "DELETE",
scope: dependencies.scope,
objectId: request.objectId,
expectedGeneration: request.expectedGeneration,
targetGeneration: request.expectedGeneration + 1,
targetByteLength: 0,
targetStoragePolicy,
startedAtEpochMs: now(),
});
if (!begun.ok) {
return observeFailure(
rebaseFailure(begun.error, "OBJECT_DELETE"),
dependencies.observer,
);
}
const committed = await dependencies.journal.commitDelete(
transactionId,
begun.value.fencingToken,
)
: null;
// Logical deletion is already committed. Physical cleanup is retryable
// maintenance and must not make the caller repeat a non-idempotent delete.
// NS-04. It is still not a settled state: an unfinished physical removal
// or an unsettled journal row is maintenance debt the reconciler owns, so
// it is observed as such instead of as a clean success.
const settled = removed.ok && completed !== null && completed.ok;
const debtCode = removed.ok
? completed !== null && !completed.ok
? completed.error.code
: undefined
: removed.error.code;
observeOpfsSafely(dependencies.observer, {
operation: "OBJECT_DELETE",
outcome: settled ? "SUCCEEDED" : "DEGRADED",
...(debtCode ? { failureCode: debtCode } : {}),
});
return browserDataSuccess(undefined);
);
if (!committed.ok) {
return observeFailure(
rebaseFailure(committed.error, "OBJECT_DELETE"),
dependencies.observer,
);
}
const removed = await dependencies.worker.removeObject(
dependencies.scope,
request.objectId,
request.expectedGeneration,
request.signal,
);
const completed = removed.ok
? await dependencies.journal.complete(
transactionId,
begun.value.fencingToken,
)
: null;
// Logical deletion is already committed. Physical cleanup is retryable
// maintenance and must not make the caller repeat a non-idempotent delete.
// NS-04. It is still not a settled state: an unfinished physical removal
// or an unsettled journal row is maintenance debt the reconciler owns, so
// it is observed as such instead of as a clean success.
const settled = removed.ok && completed !== null && completed.ok;
const debtCode = removed.ok
? completed !== null && !completed.ok
? completed.error.code
: undefined
: removed.error.code;
observeOpfsSafely(dependencies.observer, {
operation: "OBJECT_DELETE",
outcome: settled ? "SUCCEEDED" : "DEGRADED",
...(debtCode ? { failureCode: debtCode } : {}),
});
return browserDataSuccess(undefined);
},
);
},
});
@@ -563,14 +589,14 @@ export function createOpfsByteStoreAdapter(
const maxTransactions =
request.maxTransactions ?? policy.reconciliationBatchSize;
const deadline = now() + budgetMs;
const page = await dependencies.journal.listIncomplete(maxTransactions);
if (!page.ok) {
return rebaseFailure(page.error, "OBJECT_RECONCILE");
const listed = await dependencies.journal.listIncomplete(maxTransactions);
if (!listed.ok) {
return rebaseFailure(listed.error, "OBJECT_RECONCILE");
}
observeOpfsSafely(dependencies.observer, {
operation: "OBJECT_RECONCILE",
outcome: "STARTED",
transactionBucket: transactionBucket(page.value.transactions.length),
transactionBucket: transactionBucket(listed.value.transactions.length),
});
const report: {
@@ -586,6 +612,8 @@ export function createOpfsByteStoreAdapter(
| "STAGING_STATE_UNREADABLE";
moreTransactionsAvailable: boolean;
deadlineReached: boolean;
skippedLiveTransactions: number;
failedTransactions: number;
} = {
inspectedTransactions: 0,
committedTransactions: 0,
@@ -594,26 +622,63 @@ export function createOpfsByteStoreAdapter(
inspectedOrphanChunks: 0,
deletedOrphanChunks: 0,
orphanGcStatus: "COMPLETED",
moreTransactionsAvailable: page.value.moreAvailable,
moreTransactionsAvailable: listed.value.moreAvailable,
deadlineReached: false,
skippedLiveTransactions: 0,
failedTransactions: 0,
};
for (const transaction of page.value.transactions) {
if (request.signal?.aborted) {
return browserDataFailure("ABORTED", "OBJECT_RECONCILE");
// 05-01 (D-3). A row is reconciled only while this call holds its
// per-transaction lock, so a put or delete still running in any tab is
// skipped. The page is read again under those locks: a row listed while
// its owner still ran may have moved on before the owner let go.
const held = new Map<string, () => void>();
try {
for (const transaction of listed.value.transactions) {
let release: (() => void) | null;
try {
release = await holdTransactionLock(transaction.transactionId);
} catch {
return browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE", {
retryable: true,
recovery: "RETRY",
});
}
if (release) held.set(transaction.transactionId, release);
else report.skippedLiveTransactions += 1;
}
if (now() >= deadline) {
report.deadlineReached = true;
report.moreTransactionsAvailable = true;
break;
const page =
held.size === 0
? listed
: await dependencies.journal.listIncomplete(maxTransactions);
if (!page.ok) {
return rebaseFailure(page.error, "OBJECT_RECONCILE");
}
report.inspectedTransactions += 1;
const reconciled = await reconcileTransaction(transaction, request.signal);
if (!reconciled.ok) {
return observeFailure(reconciled, dependencies.observer);
report.moreTransactionsAvailable = page.value.moreAvailable;
for (const transaction of page.value.transactions) {
if (!held.has(transaction.transactionId)) continue;
if (request.signal?.aborted) {
return browserDataFailure("ABORTED", "OBJECT_RECONCILE");
}
if (now() >= deadline) {
report.deadlineReached = true;
report.moreTransactionsAvailable = true;
break;
}
report.inspectedTransactions += 1;
const reconciled = await reconcileTransaction(transaction, request.signal);
if (!reconciled.ok) {
// 05-02. A row that cannot be settled now stays for a later run;
// it must not stop the rest of the page or orphan GC.
observeFailure(reconciled, dependencies.observer);
report.failedTransactions += 1;
continue;
}
report.committedTransactions += reconciled.value.committed;
report.rolledBackTransactions += reconciled.value.rolledBack;
report.cleanedTransactions += reconciled.value.cleaned;
}
report.committedTransactions += reconciled.value.committed;
report.rolledBackTransactions += reconciled.value.rolledBack;
report.cleanedTransactions += reconciled.value.cleaned;
} finally {
for (const release of held.values()) release();
}
if (now() >= deadline) {
report.deadlineReached = true;
@@ -781,12 +846,38 @@ export function createOpfsByteStoreAdapter(
afterObjectId = page.value.nextObjectId;
moreObjectsAvailable = page.value.moreAvailable;
}
// 05-04. Removing an object leaves its chunks, and GC only takes a
// chunk 24 h after it was written, so a purged user's bytes stayed on
// the device. Once every object is gone the partition's chunks are swept
// without that grace. The sweep still skips any chunk the journal or a
// staged put references.
let chunkSweep: OpfsPolicyMaintenanceReport["chunkSweep"];
if (
(request.reason === "LOGOUT" ||
request.reason === "ACCOUNT_DELETION") &&
!moreObjectsAvailable &&
!deadlineReached &&
!request.signal?.aborted
) {
const swept = await reconcileOrphanChunks(
deadline,
request.signal,
now(),
);
if (!swept.ok) return rebaseFailure(swept.error, "OBJECT_RECONCILE");
chunkSweep = swept.value.status;
if (chunkSweep === "DEADLINE_REACHED") {
deadlineReached = true;
moreObjectsAvailable = true;
}
}
const report: OpfsPolicyMaintenanceReport = Object.freeze({
inspectedObjects,
removedObjects,
releasedBytes,
moreObjectsAvailable,
deadlineReached,
...(chunkSweep ? { chunkSweep } : {}),
});
return browserDataSuccess(report);
},
@@ -905,13 +996,63 @@ export function createOpfsByteStoreAdapter(
return Object.freeze({ objects, maintenance });
/**
* Takes the exclusive per-transaction Web Lock only if nobody holds it.
* Resolves with its release, or `null` while another context holds it.
*/
async function holdTransactionLock(
transactionId: string,
): Promise<(() => void) | null> {
let release!: () => void;
const held = new Promise<void>((resolve) => {
release = resolve;
});
return await new Promise((resolve, reject) => {
requestLock(
`${policy.mutationLockName}:transaction:${scope.authorityToken}:${transactionId}`,
{ mode: "exclusive", ifAvailable: true },
async (lock) => {
if (!lock) {
resolve(null);
return;
}
resolve(release);
await held;
},
).catch(reject);
});
}
async function withTransactionLock<Value>(
transactionId: string,
operation: "OBJECT_WRITE" | "OBJECT_DELETE",
run: () => Promise<BrowserDataResult<Value>>,
): Promise<BrowserDataResult<Value>> {
let release: (() => void) | null;
try {
release = await holdTransactionLock(transactionId);
} catch {
return browserDataFailure("UNAVAILABLE", operation, {
retryable: true,
recovery: "RETRY",
});
}
// A fresh transaction id is never held; a holder means a duplicate id.
if (!release) return browserDataFailure("CONFLICT", operation);
try {
return await run();
} finally {
release();
}
}
/**
* STO-01. The compensating half of the put saga.
*
* The journal row and its budget reservation are the only durable evidence
* that a physical staging generation may still exist, so they are released
* exactly when the physical effect is confirmed `CLEANED` or
* `ALREADY_CLEAN`. A timeout, crash, malformed response or `EFFECT_UNKNOWN`
* `ALREADY_CLEAN`. A timeout, crash, malformed response or worker failure
* keeps `PREPARING`/`FILES_READY` in place and asks for reconciliation.
*/
/**
@@ -933,7 +1074,6 @@ export function createOpfsByteStoreAdapter(
});
const compensated =
cleanup.ok &&
cleanup.value.kind !== "EFFECT_UNKNOWN" &&
(
await dependencies.journal.rollback(
transaction.transactionId,
@@ -961,11 +1101,6 @@ export function createOpfsByteStoreAdapter(
signal,
);
if (!cleaned.ok) return cleaned;
if (cleaned.value.kind === "EFFECT_UNKNOWN") {
return browserDataFailure("CONFLICT", "OBJECT_RECONCILE", {
recovery: "RETRY",
});
}
const rolledBack = await dependencies.journal.rollback(
transaction.transactionId,
transaction.fencingToken,
@@ -1073,6 +1208,7 @@ export function createOpfsByteStoreAdapter(
async function reconcileOrphanChunks(
deadline: number,
signal: AbortSignal | undefined,
olderThanEpochMs = Math.max(0, now() - policy.orphanGracePeriodMs),
): Promise<
BrowserDataResult<
Readonly<{
@@ -1085,10 +1221,6 @@ export function createOpfsByteStoreAdapter(
}>
>
> {
const olderThanEpochMs = Math.max(
0,
now() - policy.orphanGracePeriodMs,
);
let inspected = 0;
let deleted = 0;
// R-5.1. Candidates include committed chunks, so a page can be all
+45 -1
View File
@@ -20,6 +20,13 @@ export type OpfsRuntimePolicy = Readonly<{
* is bounded on its own and never counted against `rpcTimeoutMs`.
*/
mutationLockWaitTimeoutMs: number;
/**
* 05-07. How long a put may wait for its source's next bytes. The put holds
* the origin mutation lock from BEGIN to FINISH, so a stalled source would
* otherwise stop every tab's writes until their lock wait ran out. Keep it
* below `mutationLockWaitTimeoutMs` so a queued writer outlasts it.
*/
putIdleTimeoutMs: number;
reconciliationBudgetMs: number;
reconciliationBatchSize: number;
orphanGracePeriodMs: number;
@@ -56,6 +63,7 @@ export const DEFAULT_OPFS_RUNTIME_POLICY: OpfsRuntimePolicy = Object.freeze({
maxChunkCount: 512,
rpcTimeoutMs: 60_000,
mutationLockWaitTimeoutMs: 10 * 60_000,
putIdleTimeoutMs: 2 * 60_000,
reconciliationBudgetMs: 5_000,
reconciliationBatchSize: 100,
orphanGracePeriodMs: 24 * 60 * 60 * 1_000,
@@ -94,6 +102,9 @@ export function assertOpfsRuntimePolicy(policy: OpfsRuntimePolicy): void {
!Number.isSafeInteger(policy.mutationLockWaitTimeoutMs) ||
policy.mutationLockWaitTimeoutMs < 1_000 ||
policy.mutationLockWaitTimeoutMs > 60 * 60_000 ||
!Number.isSafeInteger(policy.putIdleTimeoutMs) ||
policy.putIdleTimeoutMs < 1_000 ||
policy.putIdleTimeoutMs > 60 * 60_000 ||
!Number.isSafeInteger(policy.reconciliationBudgetMs) ||
policy.reconciliationBudgetMs < 1 ||
policy.reconciliationBudgetMs > 60_000 ||
@@ -109,12 +120,45 @@ export function assertOpfsRuntimePolicy(policy: OpfsRuntimePolicy): void {
policy.maxCancellationTombstones < 16 ||
policy.maxCancellationTombstones > 10_000 ||
typeof policy.isObjectIdAllowed !== "function" ||
typeof policy.isMediaTypeAllowed !== "function"
typeof policy.isMediaTypeAllowed !== "function" ||
widensObjectIdGrammar(policy.isObjectIdAllowed)
) {
throw new TypeError("OPFS runtime policy is invalid.");
}
}
/**
* 05-10. Stored prepared objects, journal rows and physical paths all use the
* opaque id grammar, so a policy may narrow it but never widen it: a wider
* policy let BEGIN through and failed every put at its last step. Ids just
* outside the grammar catch the usual widenings when the policy is resolved.
*/
const OBJECT_IDS_OUTSIDE_GRAMMAR = Object.freeze([
"",
"short_7",
"a".repeat(129),
"object 12345",
"object.12345",
"object/12345",
"object~12345",
]);
function widensObjectIdGrammar(
isObjectIdAllowed: (objectId: string) => boolean,
): boolean {
try {
return OBJECT_IDS_OUTSIDE_GRAMMAR.some((objectId) =>
isObjectIdAllowed(objectId),
);
} catch {
return true;
}
}
export function isOpaqueObjectId(value: string): boolean {
return OPAQUE_OBJECT_ID.test(value);
}
export function validateObjectWriteInput(
input: Readonly<{
scope: OpfsStorageScope;
@@ -3,9 +3,8 @@ import type {
OpfsStorageScope,
} from "../../../application/ports/browser-file-storage/opfs-ports.ts";
import { assertValidStoragePolicy } from "../../../application/ports/browser-file-storage/shared.ts";
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
import { isOpaqueObjectId, isValidOpfsStorageScope } from "./opfs-policy.ts";
const OPAQUE_OBJECT_ID = /^[A-Za-z0-9_-]{8,128}$/u;
const SHA256_HEX = /^[a-f0-9]{64}$/u;
/**
@@ -53,7 +52,7 @@ export function isPreparedObject(
: null;
if (
typeof descriptor.objectId !== "string" ||
!OPAQUE_OBJECT_ID.test(descriptor.objectId) ||
!isOpaqueObjectId(descriptor.objectId) ||
!descriptor.scope ||
typeof descriptor.scope !== "object" ||
!isValidOpfsStorageScope(descriptor.scope as OpfsStorageScope) ||
@@ -129,3 +128,36 @@ export function isPreparedObject(
Math.ceil(descriptor.byteLength / integrity.chunkSizeBytes)
);
}
/**
* 05-10. The one canonical JSON for comparing OPFS records, used by the
* worker and the journal alike. A value JSON cannot carry exactly (a
* non-finite number, `undefined`, a function) throws instead of being
* written as something else, so two different values never compare equal.
*/
export function stableJson(value: unknown): string {
if (
value === null ||
typeof value === "string" ||
typeof value === "boolean"
) {
return JSON.stringify(value);
}
if (typeof value === "number") {
if (!Number.isFinite(value)) {
throw new TypeError("OPFS value is not JSON-safe.");
}
return JSON.stringify(value);
}
if (Array.isArray(value)) {
return `[${value.map(stableJson).join(",")}]`;
}
if (typeof value === "object") {
const record = value as Record<string, unknown>;
return `{${Object.keys(record)
.sort()
.map((key) => `${JSON.stringify(key)}:${stableJson(record[key])}`)
.join(",")}}`;
}
throw new TypeError("OPFS value is not JSON-safe.");
}
+32 -10
View File
@@ -14,6 +14,7 @@ import {
browserDataFailure,
browserDataSuccess,
} from "../../browser-file-storage/result.ts";
import { createAbortableOperation } from "../../platform/abortable-operation.ts";
import type { OpfsRuntimePolicy } from "./opfs-policy.ts";
import { isPreparedObject } from "./opfs-prepared-object.ts";
import {
@@ -272,13 +273,36 @@ export function createOpfsWorkerGateway(
let sequence = 0;
let transferredBytes = 0;
notifyProgress(request, "TRANSFERRING", 0);
// 05-07. BEGIN took the origin mutation lock, so every wait for the
// source is bounded by `putIdleTimeoutMs`. A source that stops fails the
// put, and the coordinator's compensation releases the lock.
const sourceStop = new AbortController();
const stopSource = () => sourceStop.abort();
request.signal?.addEventListener("abort", stopSource, { once: true });
if (request.signal?.aborted) stopSource();
const chunks = rechunk(
request.source.stream(sourceStop.signal),
dependencies.policy.chunkSizeBytes,
dependencies.policy.maxObjectBytes,
request.signal,
);
try {
for await (const chunk of rechunk(
request.source.stream(requiredSignal(request.signal)),
dependencies.policy.chunkSizeBytes,
dependencies.policy.maxObjectBytes,
request.signal,
)) {
for (;;) {
const idle = createAbortableOperation({
signal: request.signal,
timeoutMs: dependencies.policy.putIdleTimeoutMs,
});
const next = await idle.race(chunks.next());
idle.close();
if (next.kind === "REJECTED") throw next.reason;
if (next.kind === "TERMINAL") {
stopSource();
throw new OpfsRpcError(
next.terminal === "DEADLINE" ? "NOT_READABLE" : "ABORTED",
);
}
if (next.value.done) break;
const chunk = next.value.value;
const chunkByteLength = chunk.byteLength;
const append = await invoke<void>(
"OBJECT_WRITE",
@@ -320,6 +344,8 @@ export function createOpfsWorkerGateway(
error instanceof OpfsRpcError ? error.code : "NOT_READABLE",
"OBJECT_WRITE",
);
} finally {
request.signal?.removeEventListener("abort", stopSource);
}
},
@@ -585,10 +611,6 @@ async function* rechunk(
}
}
function requiredSignal(signal: AbortSignal | undefined): AbortSignal {
return signal ?? new AbortController().signal;
}
function notifyProgress(
request: PreparePhysicalObjectRequest,
phase: "TRANSFERRING" | "VERIFYING",
@@ -16,7 +16,7 @@ import {
isValidOpfsStorageScope,
type OpfsRuntimePolicy,
} from "./opfs-policy.ts";
import { isPreparedObject } from "./opfs-prepared-object.ts";
import { isPreparedObject, stableJson } from "./opfs-prepared-object.ts";
import {
OPFS_WORKER_PROTOCOL_VERSION,
} from "./opfs-worker-protocol.ts";
@@ -890,16 +890,11 @@ export function createOpfsWorkerRuntime(
}
{
if (removePreparedGeneration) {
let receipt: unknown;
try {
receipt = await readJson(receiptPath(scope, transactionId));
} catch (error) {
if (isNotFound(error)) {
await removeEntryIfPresent(staging, transactionId, true);
return CLEANUP_ALREADY_CLEAN;
}
// Keep unreadable staging in place so orphan GC fails closed.
throw error;
// Keep unreadable staging in place so orphan GC fails closed.
const receipt = await readReceipt(scope, transactionId);
if (receipt === undefined) {
await removeEntryIfPresent(staging, transactionId, true);
return CLEANUP_ALREADY_CLEAN;
}
const target = extractReceiptPhysicalTarget(receipt, scope);
if (!target) {
@@ -1133,10 +1128,11 @@ export function createOpfsWorkerRuntime(
}
let receipt: unknown;
try {
receipt = await readJson(receiptPath(scope, transactionId));
receipt = await readReceipt(scope, transactionId);
} catch {
return { safeToSweep: false, digests };
}
if (receipt === undefined) continue;
if (!receiptBelongsToScope(receipt, scope)) {
return { safeToSweep: false, digests };
}
@@ -1303,8 +1299,31 @@ export function createOpfsWorkerRuntime(
await writeWithAtomicWritable(handle, bytes);
}
/**
* 05-02. `writeJsonAtomic` creates the file before its first write lands,
* so a first receipt write that hit the quota or a crash leaves a zero-byte
* file. BEGIN writes that receipt before any chunk or generation directory,
* so an empty receipt, like a missing one, has nothing to clean up.
*/
async function readReceipt(
scope: OpfsStorageScope,
transactionId: string,
): Promise<unknown> {
let bytes: Uint8Array;
try {
bytes = await readFile(receiptPath(scope, transactionId));
} catch (error) {
if (isNotFound(error)) return undefined;
throw error;
}
return bytes.byteLength === 0 ? undefined : parseJson(bytes);
}
async function readJson(path: readonly string[]): Promise<unknown> {
const bytes = await readFile(path);
return parseJson(await readFile(path));
}
function parseJson(bytes: Uint8Array): unknown {
if (bytes.byteLength > MAX_MANIFEST_BYTES) {
throw new OpfsRuntimeFailure("CORRUPT_DATA");
}
@@ -1720,31 +1739,6 @@ function bytesToHex(bytes: Uint8Array): string {
return hex;
}
function stableJson(value: unknown): string {
if (
value === null ||
typeof value === "string" ||
typeof value === "boolean"
) {
return JSON.stringify(value);
}
if (typeof value === "number") {
if (!Number.isFinite(value)) throw new OpfsRuntimeFailure("CORRUPT_DATA");
return JSON.stringify(value);
}
if (Array.isArray(value)) {
return `[${value.map(stableJson).join(",")}]`;
}
if (typeof value === "object") {
const record = value as Record<string, unknown>;
return `{${Object.keys(record)
.sort()
.map((key) => `${JSON.stringify(key)}:${stableJson(record[key])}`)
.join(",")}}`;
}
throw new OpfsRuntimeFailure("CORRUPT_DATA");
}
function success(
requestId: string,
kind: OpfsWorkerRequest["kind"],
@@ -43,6 +43,13 @@ export type IndexedDbConnectionStatus =
targetVersion: number;
}>
| Readonly<{ kind: "READY"; schemaVersion: number }>
/**
* 05-06. The database was upgraded past `schemaVersion` by a newer release
* (another tab, or the bundle rolled back). This runtime cannot read or
* write it: open and every operation answer `UNSUPPORTED` with recovery
* `ONLINE_ONLY`, and a reload onto the newer release is the way back.
*/
| Readonly<{ kind: "INCOMPATIBLE"; schemaVersion: number }>
| Readonly<{ kind: "DISPOSED" }>;
export type IndexedDbCursorKey =
@@ -144,6 +151,13 @@ export type IndexedDbLifecycleBatchReceipt = Readonly<{
scannedRows: number;
deletedRows: number;
budgetExhausted: boolean;
/**
* 05-03. Present when an authorized LOGOUT or ACCOUNT_DELETION found the
* dataset's governance rows unreadable, so no row could be purged one by
* one, and deleted the partition's whole database instead. Rows are then
* not counted.
*/
datasetDeleted?: true;
}>;
export type IndexedDbLifecycleAuthorityRequest = Readonly<{
@@ -76,9 +76,23 @@ export type OpfsCapabilities = Readonly<{
export interface DurableObjectStorePort {
capabilities(): Promise<BrowserDataResult<OpfsCapabilities>>;
/**
* 05-07. OPFS writes are one at a time across the whole origin: a put holds
* the origin mutation lock while it streams, and every put, remove and GC in
* every tab waits for it. A source that stops for the runtime's put idle
* bound fails the put with a retryable `NOT_READABLE` and frees the lock.
*/
put(
request: PutDurableObjectRequest,
): Promise<BrowserDataResult<DurableObjectDescriptor>>;
/**
* 05-14. `open` checks the manifest; the bytes are read and verified while
* the source streams. A put that replaces the object meanwhile leaves the
* older generation's chunks to orphan GC, whose grace counts from when a
* chunk was written, not from when it lost its last reference. A stream
* that reaches a chunk GC already took ends with `NOT_FOUND` / `REHYDRATE`.
* Nothing is lost: open again to read the generation that replaced it.
*/
open(
request: OpenDurableObjectRequest,
): Promise<BrowserDataResult<OpenedDurableObject>>;
@@ -100,6 +114,17 @@ export type OpfsReconciliationReport = Readonly<{
| "STAGING_STATE_UNREADABLE";
moreTransactionsAvailable: boolean;
deadlineReached: boolean;
/**
* 05-01. Rows whose owner still holds the per-transaction lock. They are
* a live put or delete, not debris, and are left for a later reconcile.
*/
skippedLiveTransactions: number;
/**
* 05-02. Rows that failed to reconcile this time. Each stays in the journal
* for a later run and is observed as a `FAILED` reconcile; it no longer
* stops the rest of the page or orphan GC.
*/
failedTransactions: number;
}>;
export type OpfsPolicyMaintenanceReport = Readonly<{
@@ -108,6 +133,12 @@ export type OpfsPolicyMaintenanceReport = Readonly<{
releasedBytes: number;
moreObjectsAvailable: boolean;
deadlineReached: boolean;
/**
* 05-04. LOGOUT and ACCOUNT_DELETION only, once every object is removed:
* how the sweep of the partition's now unreferenced chunks ended. Chunk
* bytes are gone only when it is `COMPLETED`.
*/
chunkSweep?: "COMPLETED" | "DEADLINE_REACHED" | "STAGING_STATE_UNREADABLE";
}>;
export type OpfsSensitiveMaintenanceReason =
@@ -210,11 +241,12 @@ export type OpfsPreparedObject = OpfsPreparedObjectV1 | OpfsPreparedObjectV2;
/**
* STO-01. Compensation is only allowed to release journal and budget state
* after the physical effect is confirmed. `EFFECT_UNKNOWN` is never a success.
* after the physical effect is confirmed. An effect the worker cannot confirm
* comes back as a failure, never as a success value (05-09).
*/
export type OpfsCleanupEffect =
| Readonly<{ kind: "CLEANED" | "ALREADY_CLEAN" }>
| Readonly<{ kind: "EFFECT_UNKNOWN" }>;
export type OpfsCleanupEffect = Readonly<{
kind: "CLEANED" | "ALREADY_CLEAN";
}>;
export type OpfsJournalMutation = "PUT" | "DELETE";
export type OpfsJournalPhase =
@@ -23,12 +23,14 @@ async function startLifecycleRuntime(
name: string,
targetVersion: 1 | 2,
blockedTimeoutMs = 1_000,
partitioned = false,
): Promise<void> {
await page.evaluate(
async ({
currentDatabaseName,
currentTargetVersion,
currentBlockedTimeoutMs,
currentPartitioned,
}) => {
const modulePath =
"/src/adapters/storage/indexeddb/indexeddb-runtime.ts";
@@ -38,24 +40,31 @@ async function startLifecycleRuntime(
type LifecycleValue = Readonly<{ value: string }>;
const [authorityToken, namespaceToken, partitionToken] =
currentDatabaseName.slice("ca-idb-v1:".length).split(".");
const accountScope = currentPartitioned
? ("OPAQUE_PARTITION" as const)
: ("ORIGIN_SHARED" as const);
const scope = {
authorityToken,
namespaceToken,
partitionToken,
accountScope: "ORIGIN_SHARED" as const,
accountScope,
};
const storagePolicy = {
owner: "platform-storage",
namespace: "indexeddb-browser-lifecycle",
classification: "INTERNAL" as const,
authority: "SERVER" as const,
accountScope: "ORIGIN_SHARED" as const,
accountScope,
retention: { kind: "EXPLICIT_DELETE" as const },
softBudgetBytes: 1_000_000,
hardBudgetBytes: 2_000_000,
evictionPriority: "SYNCED_COPY" as const,
logoutAction: "KEEP_ORIGIN_SHARED" as const,
accountDeletionAction: "KEEP_ORIGIN_SHARED" as const,
logoutAction: currentPartitioned
? ("PURGE_PARTITION" as const)
: ("KEEP_ORIGIN_SHARED" as const),
accountDeletionAction: currentPartitioned
? ("PURGE_PARTITION" as const)
: ("KEEP_ORIGIN_SHARED" as const),
pressureAction: "RETAIN" as const,
unavailableFallback: "ONLINE_ONLY" as const,
};
@@ -184,6 +193,7 @@ async function startLifecycleRuntime(
currentDatabaseName: name,
currentTargetVersion: targetVersion,
currentBlockedTimeoutMs: blockedTimeoutMs,
currentPartitioned: partitioned,
},
);
}
@@ -1067,3 +1077,155 @@ test("fails a blocked v2 upgrade closed, then closes its late connection after t
await blockerPage.close();
}
});
test("05-03 deletes a partition's dataset for an authorized ACCOUNT_DELETION when its budget row is corrupt", async ({
page,
}) => {
const name = databaseName("corrupt-governance");
await page.goto("/");
await startLifecycleRuntime(page, name, 1, 1_000, true);
expect(await settleLifecycleRuntime(page)).toMatchObject({
opened: { ok: true },
});
expect(
await page.evaluate(async () => {
const runtime = (
globalThis as unknown as {
__indexedDbLifecycleRuntime: {
compareAndSwap(input: unknown): Promise<unknown>;
close(): void;
};
}
).__indexedDbLifecycleRuntime;
const written = await runtime.compareAndSwap({
key: "personal-1",
value: { value: "mine" },
expectedRevision: null,
idempotencyKey: "personal-create",
});
runtime.close();
return written;
}),
).toMatchObject({ ok: true });
expect(
await page.evaluate(
async (currentDatabaseName) =>
await new Promise<string>((resolve) => {
const request = indexedDB.open(currentDatabaseName, 1);
request.onerror = () =>
resolve(request.error?.name ?? "UNKNOWN_ERROR");
request.onsuccess = () => {
const database = request.result;
const transaction = database.transaction(
"governance",
"readwrite",
);
transaction.objectStore("governance").put({
bindingKey: "dataset-budget",
budgetVersion: 1,
usedBytes: -5,
receiptCount: 1,
});
transaction.oncomplete = () => {
database.close();
resolve("CORRUPTED");
};
transaction.onabort = () => {
database.close();
resolve("ABORTED");
};
};
}),
name,
),
).toBe("CORRUPTED");
await startLifecycleRuntime(page, name, 1, 1_000, true);
expect(await settleLifecycleRuntime(page)).toMatchObject({
opened: { ok: false, error: { code: "POLICY_REJECTED" } },
});
const purged = await page.evaluate(async () => {
const runtime = (
globalThis as unknown as {
__indexedDbLifecycleRuntime: {
enforceLifecycleBatch(input: unknown): Promise<unknown>;
};
}
).__indexedDbLifecycleRuntime;
return await runtime.enforceLifecycleBatch({
action: "ACCOUNT_DELETION",
maxRows: 100,
maxDurationMs: 5_000,
});
});
expect(purged).toMatchObject({
ok: true,
value: { state: "COMPLETE", datasetDeleted: true },
});
const remaining = await page.evaluate(async (currentDatabaseName) => {
(
globalThis as unknown as {
__indexedDbLifecycleRuntime?: { close(): void };
}
).__indexedDbLifecycleRuntime?.close();
// A fresh open of a deleted database runs its upgrade from version 0.
return await new Promise<number>((resolve, reject) => {
const request = indexedDB.open(currentDatabaseName);
let oldVersion = -1;
request.onupgradeneeded = (event) => {
oldVersion = event.oldVersion;
};
request.onsuccess = () => {
request.result.close();
const deletion = indexedDB.deleteDatabase(currentDatabaseName);
deletion.onsuccess = () => resolve(oldVersion);
deletion.onerror = () => reject(deletion.error);
};
request.onerror = () => reject(request.error);
});
}, name);
expect(remaining).toBe(0);
});
test("05-06 an older tab meets a schema a newer tab upgraded and goes online-only", async ({
page,
}) => {
const olderPage = await page.context().newPage();
const name = databaseName("future-schema");
try {
await Promise.all([page.goto("/"), olderPage.goto("/")]);
expect(await installLifecycleRuntime(page, name, 2)).toMatchObject({
opened: { ok: true },
status: { kind: "READY", schemaVersion: 2 },
});
await closeLifecycleRuntime(page);
expect(await installLifecycleRuntime(olderPage, name, 1)).toEqual({
opened: {
ok: false,
error: {
code: "UNSUPPORTED",
operation: "INDEXEDDB_OPEN",
retryable: false,
recovery: "ONLINE_ONLY",
},
},
status: { kind: "INCOMPATIBLE", schemaVersion: 1 },
});
await closeLifecycleRuntime(olderPage);
const deletion = await page.evaluate(
async (currentDatabaseName) =>
await new Promise<string>((resolve) => {
const request = indexedDB.deleteDatabase(currentDatabaseName);
request.onsuccess = () => resolve("DELETED");
request.onerror = () =>
resolve(request.error?.name ?? "UNKNOWN_ERROR");
request.onblocked = () => resolve("BLOCKED");
}),
name,
);
expect(deletion).toBe("DELETED");
} finally {
await olderPage.close();
}
});
@@ -219,3 +219,193 @@ test("runs put, open, remove and reconcile through a native DedicatedWorker, OPF
expect(result.deletion).toBe("DELETED");
expect(["DELETED", "ABSENT"]).toContain(result.opfsCleanup);
});
type OpfsTabScope = Readonly<{
namespace: string;
authorityToken: string;
namespaceToken: string;
partitionToken: string;
}>;
/**
* Installs an OPFS runtime for `scope` in this tab and keeps it on
* `globalThis.__opfsTab` for later evaluate calls.
*/
async function installOpfsTab(
page: import("@playwright/test").Page,
scope: OpfsTabScope,
): Promise<boolean> {
return await page.evaluate(async (tabScope) => {
const runtimeModulePath =
"/src/adapters/storage/opfs/browser-opfs-runtime.ts";
const policyModulePath = "/src/adapters/storage/opfs/opfs-policy.ts";
const { createBrowserOpfsRuntime, inspectBrowserOpfsSupport } =
(await import(
/* @vite-ignore */ runtimeModulePath
)) as typeof import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts");
const { resolveOpfsRuntimePolicy } = (await import(
/* @vite-ignore */ policyModulePath
)) as typeof import("../../src/adapters/storage/opfs/opfs-policy.ts");
const policy = resolveOpfsRuntimePolicy();
if (!inspectBrowserOpfsSupport(policy).ok) return false;
(globalThis as unknown as { __opfsTab: unknown }).__opfsTab =
createBrowserOpfsRuntime({
workerUrl: new URL(
"/tests/browser-capabilities/opfs-test.worker.ts",
location.href,
),
workerName: `opfs-tab-${crypto.randomUUID()}`,
scope: tabScope,
storagePolicy: {
owner: "browser-capability",
namespace: tabScope.namespace,
classification: "PERSONAL",
authority: "LOCAL_FIRST",
accountScope: "OPAQUE_PARTITION",
retention: { kind: "EXPLICIT_DELETE" },
softBudgetBytes: 1024 * 1024,
hardBudgetBytes: 2 * 1024 * 1024,
evictionPriority: "USER_AUTHORED",
logoutAction: "EXPORT_THEN_PURGE",
accountDeletionAction: "PURGE_PARTITION",
pressureAction: "RETAIN",
unavailableFallback: "EXPORT_REQUIRED",
},
policy,
});
return true;
}, scope);
}
test("05-01 a reconcile in another tab skips a live put and the put stays readable", async ({
page,
}) => {
const secondPage = await page.context().newPage();
const suffix = crypto.randomUUID().replaceAll("-", "");
const scope: OpfsTabScope = {
namespace: "durable-objects",
authorityToken: `authority_${suffix}`,
namespaceToken: `namespace_${suffix}`,
partitionToken: `partition_${suffix}`,
};
type Tab = import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts").BrowserOpfsRuntime;
try {
await Promise.all([page.goto("/"), secondPage.goto("/")]);
if (!(await installOpfsTab(page, scope))) return;
expect(await installOpfsTab(secondPage, scope)).toBe(true);
// Tab A starts a put whose source stalls, so its journal row stays
// PREPARING and its worker holds the OPFS mutation lock.
await page.evaluate(async () => {
const state = globalThis as unknown as {
__opfsTab: Tab;
__release?: () => void;
__put?: Promise<unknown>;
};
let streaming!: () => void;
const started = new Promise<void>((resolve) => {
streaming = resolve;
});
const gate = new Promise<void>((resolve) => {
state.__release = resolve;
});
state.__put = state.__opfsTab.objects.put({
objectId: "object_tab_12345678",
expectedGeneration: null,
mediaType: "application/octet-stream",
source: {
byteLength: 4,
async *stream() {
streaming();
await gate;
yield { ok: true as const, value: new Uint8Array([2, 7, 1, 8]) };
},
},
});
await started;
});
// Tab B reconciles while tab A's put is in flight.
await secondPage.evaluate(() => {
const state = globalThis as unknown as {
__opfsTab: Tab;
__reconcile?: Promise<unknown>;
};
state.__reconcile = state.__opfsTab.maintenance.reconcile();
});
await page.waitForTimeout(250);
const put = await page.evaluate(async () => {
const state = globalThis as unknown as {
__release: () => void;
__put: Promise<unknown>;
};
state.__release();
return await state.__put;
});
const reconciled = await secondPage.evaluate(
async () =>
await (globalThis as unknown as { __reconcile: Promise<unknown> })
.__reconcile,
);
const opened = await page.evaluate(async () => {
const runtime = (globalThis as unknown as { __opfsTab: Tab }).__opfsTab;
const result = await runtime.objects.open({
objectId: "object_tab_12345678",
});
if (!result.ok) return result.error.code;
const bytes: number[] = [];
for await (const chunk of result.value.source.stream(
new AbortController().signal,
)) {
if (!chunk.ok) return chunk.error.code;
bytes.push(...chunk.value);
}
return bytes;
});
expect(put).toMatchObject({ ok: true, value: { generation: 1 } });
expect(reconciled).toMatchObject({
ok: true,
value: { skippedLiveTransactions: 1, rolledBackTransactions: 0 },
});
expect(opened).toEqual([2, 7, 1, 8]);
} finally {
for (const tab of [page, secondPage]) {
await tab.evaluate(() =>
(globalThis as unknown as { __opfsTab?: Tab }).__opfsTab?.close(),
);
}
const cleanup = await page.evaluate(async (tabScope) => {
const journalModulePath =
"/src/adapters/storage/opfs/indexeddb-opfs-journal.ts";
const policyModulePath = "/src/adapters/storage/opfs/opfs-policy.ts";
const { opfsJournalDatabaseName } = (await import(
/* @vite-ignore */ journalModulePath
)) as typeof import("../../src/adapters/storage/opfs/indexeddb-opfs-journal.ts");
const { DEFAULT_OPFS_RUNTIME_POLICY } = (await import(
/* @vite-ignore */ policyModulePath
)) as typeof import("../../src/adapters/storage/opfs/opfs-policy.ts");
const deletion = await new Promise<string>((resolve) => {
const request = indexedDB.deleteDatabase(
opfsJournalDatabaseName(tabScope.authorityToken),
);
request.onsuccess = () => resolve("DELETED");
request.onerror = () => resolve(request.error?.name ?? "UNKNOWN");
request.onblocked = () => resolve("BLOCKED");
});
try {
const root = await (
await navigator.storage.getDirectory()
).getDirectoryHandle(DEFAULT_OPFS_RUNTIME_POLICY.rootDirectoryName);
await (
await root.getDirectoryHandle("authorities")
).removeEntry(tabScope.authorityToken, { recursive: true });
} catch {
// Nothing was written for this authority.
}
return deletion;
}, scope);
expect(cleanup).toBe("DELETED");
await secondPage.close();
}
});
+258 -62
View File
@@ -24,6 +24,31 @@ type StoreState = {
indexes: Map<string, IndexState>;
};
/** One named database: its schema, rows and version. */
type DatabaseState = {
stores: Map<string, StoreState>;
version: number;
latest: FakeDatabase | null;
/** Connections handed out by a successful open, closed or not. */
connections: FakeDatabase[];
/** Transactions that have not finished, in creation order. */
transactions: FakeTransaction[];
/** A `deleteDatabase` waiting for the open connections to close. */
pendingDelete: (() => void) | null;
};
function storeOf(state: DatabaseState, name: string): StoreState {
const store = state.stores.get(name);
if (!store) {
throw new DOMException("Store does not exist.", "NotFoundError");
}
return store;
}
// The fake decides when a transaction's task has ended with a real macrotask,
// so a test that fakes timers cannot keep a transaction alive by accident.
const realSetTimeout = globalThis.setTimeout.bind(globalThis);
type FakeKeyRange =
| Readonly<{
fakeKind: "ONLY";
@@ -221,23 +246,26 @@ class FakeDatabase {
onversionchange: ((event: IDBVersionChangeEvent) => unknown) | null = null;
onclose: ((event: Event) => unknown) | null = null;
constructor(private readonly owner: MemoryIndexedDbFactory) {}
constructor(
private readonly owner: MemoryIndexedDbFactory,
readonly state: DatabaseState,
) {}
get objectStoreNames(): DOMStringList {
return stringList(this.owner.storeNames());
return stringList(this.state.stores.keys());
}
createObjectStore(
name: string,
options?: IDBObjectStoreParameters,
): IDBObjectStore {
if (this.owner.hasStore(name)) {
if (this.state.stores.has(name)) {
throw new DOMException("Store already exists.", "ConstraintError");
}
if (typeof options?.keyPath !== "string") {
throw new DOMException("The fake requires a string key path.", "DataError");
}
this.owner.createStore(name, {
this.state.stores.set(name, {
keyPath: options.keyPath,
autoIncrement: options.autoIncrement ?? false,
data: new Map(),
@@ -247,13 +275,13 @@ class FakeDatabase {
}
deleteObjectStore(name: string): void {
if (!this.owner.deleteStore(name)) {
if (!this.state.stores.delete(name)) {
throw new DOMException("Store does not exist.", "NotFoundError");
}
}
schemaObjectStore(name: string): IDBObjectStore {
const state = this.owner.store(name);
const state = storeOf(this.state, name);
return new FakeObjectStore(null, state) as unknown as IDBObjectStore;
}
@@ -261,7 +289,7 @@ class FakeDatabase {
name: string,
transaction: FakeUpgradeTransaction,
): IDBObjectStore {
const state = this.owner.store(name);
const state = storeOf(this.state, name);
return new FakeObjectStore(
null,
state,
@@ -278,16 +306,22 @@ class FakeDatabase {
}
const names =
typeof storeNames === "string" ? [storeNames] : [...storeNames];
const transaction = new FakeTransaction(this.owner, names, mode);
const transaction = new FakeTransaction(
this.owner,
this.state,
names,
mode,
);
this.owner.noteTransaction(transaction);
return transaction as unknown as IDBTransaction;
}
close(): void {
this.closed = true;
this.state.pendingDelete?.();
}
fireVersionChange(newVersion: number): void {
fireVersionChange(newVersion: number | null): void {
this.onversionchange?.({
oldVersion: this.version,
newVersion,
@@ -307,19 +341,108 @@ class FakeTransaction {
error: DOMException | null = null;
readonly mode: IDBTransactionMode;
private active = true;
/**
* Native "active" state: a request may be placed only in the task that
* created the transaction or while one of its request events (and the
* microtasks that event queued) is being handled.
*/
private accepting = true;
/** Held back until every earlier overlapping transaction finishes. */
private started = false;
private readonly deferred: Array<() => void> = [];
private completionQueued = false;
private pending = 0;
private readonly workingStores: Map<string, StoreState>;
constructor(
private readonly owner: MemoryIndexedDbFactory,
private readonly database: DatabaseState,
private readonly names: readonly string[],
mode: IDBTransactionMode,
) {
this.mode = mode;
this.workingStores = new Map(
names.map((name) => [name, cloneStore(owner.store(name))]),
names.map((name) => [name, cloneStore(storeOf(database, name))]),
);
database.transactions.push(this);
this.startIfUnblocked();
this.deactivateAfterTask();
}
/**
* Native IndexedDB runs transactions with overlapping scopes in creation
* order unless both are readonly. A later one starts from the rows the
* earlier one committed.
*/
startIfUnblocked(): void {
if (this.started || !this.active) return;
const earlier = this.database.transactions.slice(
0,
this.database.transactions.indexOf(this),
);
if (earlier.some((transaction) => transaction.blocks(this))) return;
this.started = true;
for (const name of this.names) {
const fresh = cloneStore(storeOf(this.database, name));
const working = this.workingStores.get(name)!;
working.data = fresh.data;
working.indexes = fresh.indexes;
}
for (const callback of this.deferred.splice(0)) {
this.owner.scheduleTransaction(callback, this.mode);
}
if (!this.accepting) this.commitIfIdle();
}
private blocks(later: FakeTransaction): boolean {
return (
this.active &&
(this.mode === "readwrite" || later.mode === "readwrite") &&
this.names.some((name) => later.names.includes(name))
);
}
private schedule(callback: () => void): void {
if (this.started) {
this.owner.scheduleTransaction(callback, this.mode);
} else {
this.deferred.push(callback);
}
}
private deactivateAfterTask(): void {
realSetTimeout(() => {
this.accepting = false;
this.commitIfIdle();
}, 0);
}
/** A native transaction with no request left commits once inactive. */
private commitIfIdle(): void {
if (this.started && this.pending === 0) this.queueCompletion();
}
private assertAccepting(): void {
if (!this.active || !this.accepting) {
throw new DOMException(
"The transaction is not active.",
"TransactionInactiveError",
);
}
}
private dispatch(handler: () => void): void {
this.accepting = true;
handler();
this.deactivateAfterTask();
}
private finish(): void {
const index = this.database.transactions.indexOf(this);
if (index >= 0) this.database.transactions.splice(index, 1);
for (const transaction of [...this.database.transactions]) {
transaction.startIfUnblocked();
}
}
objectStore(name: string): IDBObjectStore {
@@ -341,22 +464,23 @@ class FakeTransaction {
}
request<Value>(operation: () => Value): IDBRequest<Value> {
this.assertAccepting();
const request = createRequest<Value>();
this.pending += 1;
this.owner.scheduleTransaction(() => {
this.schedule(() => {
if (!this.active) return;
try {
request.result = structuredClone(operation());
request.onsuccess?.(new Event("success"));
this.dispatch(() => request.onsuccess?.(new Event("success")));
} catch (error) {
request.error = asException(error);
request.onerror?.(new Event("error"));
this.dispatch(() => request.onerror?.(new Event("error")));
this.abortWith(request.error);
}
if (!this.active) return;
this.pending -= 1;
this.queueCompletion();
}, this.mode);
});
return request as unknown as IDBRequest<Value>;
}
@@ -368,12 +492,13 @@ class FakeTransaction {
}>[],
direction: IDBCursorDirection,
): IDBRequest<IDBCursorWithValue | null> {
this.assertAccepting();
const request = createRequest<IDBCursorWithValue | null>();
let position = 0;
this.pending += 1;
const emit = () => {
this.owner.scheduleTransaction(() => {
this.schedule(() => {
if (!this.active) return;
let continued = false;
let eventReturned = false;
@@ -445,12 +570,12 @@ class FakeTransaction {
},
} as unknown as IDBCursorWithValue;
}
request.onsuccess?.(new Event("success"));
this.dispatch(() => request.onsuccess?.(new Event("success")));
eventReturned = true;
if (continued || !this.active) return;
this.pending -= 1;
this.queueCompletion();
}, this.mode);
});
};
emit();
@@ -472,7 +597,7 @@ class FakeTransaction {
return;
}
this.completionQueued = true;
this.owner.scheduleTransaction(() => {
this.schedule(() => {
this.completionQueued = false;
if (!this.active || this.pending !== 0) return;
const commitFailure =
@@ -485,18 +610,20 @@ class FakeTransaction {
}
if (this.mode === "readwrite") {
for (const [name, state] of this.workingStores) {
this.owner.replaceStore(name, cloneStore(state));
this.database.stores.set(name, cloneStore(state));
}
}
this.active = false;
this.finish();
this.oncomplete?.(new Event("complete"));
}, this.mode);
});
}
private abortWith(error: DOMException): void {
if (!this.active) return;
this.active = false;
this.error = error;
this.finish();
this.owner.scheduleLifecycle(() => {
this.onerror?.(new Event("error"));
this.onabort?.(new Event("abort"));
@@ -721,14 +848,36 @@ class FakeIndex {
}
/**
* Minimal deterministic IndexedDB model for adapter contract tests. It models
* request ordering, transaction-level commit/rollback and upgrade blocking;
* it is not intended to polyfill IndexedDB for application code.
* Minimal deterministic IndexedDB model for adapter contract tests. It is not
* intended to polyfill IndexedDB for application code.
*
* Modeled like a browser:
* - request ordering and transaction-level commit/rollback;
* - one database per name (`open(name)`), so two names never share rows;
* - transactions with overlapping scopes run one after another unless both
* are readonly, and a later one sees what the earlier one committed;
* - a request placed after the transaction's task ended, outside a request
* event and its microtasks, throws `TransactionInactiveError`, and a
* transaction with nothing left to do commits on its own;
* - upgrade blocking, through `blockNextOpen`/`releaseBlockedOpen`.
*
* Not modeled (05-08) — use a browser capability spec for these:
* - key comparison is by `String`/JSON, and a compound (array) index key path
* is not read, so such an index orders by primary key;
* - opening a higher version does not fire `versionchange` on open
* connections, and `onblocked` fires only for `blockNextOpen`; tests drive
* `triggerVersionChange` by hand;
* - the upgrade transaction ends synchronously, so an abort requested from an
* asynchronous step inside `onupgradeneeded` is not rolled back;
* - quota: writes never fail for lack of space unless a test injects it with
* `failNextWriteCommit`.
*
* `seed`, `readRaw` and the lifecycle triggers act on the database opened
* most recently.
*/
export class MemoryIndexedDbFactory {
private stores = new Map<string, StoreState>();
private databaseVersion = 0;
private latestDatabase = new FakeDatabase(this);
private readonly databases = new Map<string, DatabaseState>();
private current: DatabaseState = this.databaseNamed("");
private blockedRequest: Readonly<{
request: MutableOpenRequest;
targetVersion: number;
@@ -741,8 +890,10 @@ export class MemoryIndexedDbFactory {
lastTransaction: FakeTransaction | null = null;
readonly factory: IDBFactory = {
open: (_name: string, version?: number) =>
this.open(version ?? 1) as unknown as IDBOpenDBRequest,
open: (name: string, version?: number) =>
this.open(name, version ?? 1) as unknown as IDBOpenDBRequest,
deleteDatabase: (name: string) =>
this.deleteDatabase(name) as unknown as IDBOpenDBRequest,
cmp: compareKeys,
} as IDBFactory;
@@ -782,10 +933,12 @@ export class MemoryIndexedDbFactory {
}) as unknown as IDBKeyRange,
};
open(version: number): MutableOpenRequest {
const database = new FakeDatabase(this);
database.version = this.databaseVersion;
this.latestDatabase = database;
open(name: string, version: number): MutableOpenRequest {
const state = this.databaseNamed(name);
const database = new FakeDatabase(this, state);
database.version = state.version;
state.latest = database;
this.current = state;
const request: MutableOpenRequest = {
...createRequest<IDBDatabase>(),
result: database as unknown as IDBDatabase,
@@ -802,7 +955,7 @@ export class MemoryIndexedDbFactory {
database,
};
request.onblocked?.({
oldVersion: this.databaseVersion,
oldVersion: state.version,
newVersion: version,
} as IDBVersionChangeEvent);
return;
@@ -812,6 +965,51 @@ export class MemoryIndexedDbFactory {
return request;
}
/**
* Like a browser: open connections get `versionchange` (new version
* `null`); if any stays open the request reports `blocked` and completes
* once the last one closes. The name then holds an empty version-0
* database, so `hasStore` answers false.
*/
deleteDatabase(name: string): MutableOpenRequest {
const request: MutableOpenRequest = {
...createRequest<IDBDatabase>(),
transaction: null,
onblocked: null,
onupgradeneeded: null,
};
queueMicrotask(() => {
const state = this.databaseNamed(name);
const oldVersion = state.version;
const finish = () => {
if (state.connections.some((connection) => !connection.closed)) {
return;
}
state.pendingDelete = null;
state.stores = new Map();
state.version = 0;
state.connections = [];
request.onsuccess?.({
oldVersion,
newVersion: null,
} as unknown as Event);
};
for (const connection of state.connections) {
if (!connection.closed) connection.fireVersionChange(null);
}
if (state.connections.some((connection) => !connection.closed)) {
state.pendingDelete = finish;
request.onblocked?.({
oldVersion,
newVersion: null,
} as IDBVersionChangeEvent);
return;
}
finish();
});
return request;
}
blockNextOpen(): void {
this.shouldBlockNextOpen = true;
}
@@ -848,25 +1046,25 @@ export class MemoryIndexedDbFactory {
}
triggerVersionChange(newVersion: number): void {
this.latestDatabase.fireVersionChange(newVersion);
this.current.latest?.fireVersionChange(newVersion);
}
triggerForcedClose(): void {
this.latestDatabase.fireForcedClose();
this.current.latest?.fireForcedClose();
}
isConnectionClosed(): boolean {
return this.latestDatabase.closed;
return this.current.latest?.closed ?? true;
}
seed(storeName: string, value: unknown): void {
const state = this.store(storeName);
const state = storeOf(this.current, storeName);
const key = primaryKey(state, value);
state.data.set(key, structuredClone(value));
}
readRaw(storeName: string, key: string): unknown {
const value = this.store(storeName).data.get(key);
const value = storeOf(this.current, storeName).data.get(key);
return value === undefined ? undefined : structuredClone(value);
}
@@ -896,39 +1094,36 @@ export class MemoryIndexedDbFactory {
}
storeNames(): Iterable<string> {
return this.stores.keys();
return this.current.stores.keys();
}
hasStore(name: string): boolean {
return this.stores.has(name);
return this.current.stores.has(name);
}
store(name: string): StoreState {
const state = this.stores.get(name);
private databaseNamed(name: string): DatabaseState {
let state = this.databases.get(name);
if (!state) {
throw new DOMException("Store does not exist.", "NotFoundError");
state = {
stores: new Map(),
version: 0,
latest: null,
connections: [],
transactions: [],
pendingDelete: null,
};
this.databases.set(name, state);
}
return state;
}
createStore(name: string, state: StoreState): void {
this.stores.set(name, state);
}
replaceStore(name: string, state: StoreState): void {
this.stores.set(name, state);
}
deleteStore(name: string): boolean {
return this.stores.delete(name);
}
private finishOpen(
request: MutableOpenRequest,
targetVersion: number,
database: FakeDatabase,
): void {
if (targetVersion < this.databaseVersion) {
const state = database.state;
if (targetVersion < state.version) {
request.error = new DOMException(
"Requested version is older.",
"VersionError",
@@ -936,9 +1131,9 @@ export class MemoryIndexedDbFactory {
request.onerror?.(new Event("error"));
return;
}
if (targetVersion > this.databaseVersion) {
const before = cloneStores(this.stores);
const oldVersion = this.databaseVersion;
if (targetVersion > state.version) {
const before = cloneStores(state.stores);
const oldVersion = state.version;
const upgrade = new FakeUpgradeTransaction(database);
request.transaction = upgrade as unknown as IDBTransaction;
database.version = targetVersion;
@@ -948,7 +1143,7 @@ export class MemoryIndexedDbFactory {
} as IDBVersionChangeEvent);
request.transaction = null;
if (upgrade.aborted) {
this.stores = before;
state.stores = before;
database.version = oldVersion;
request.error = new DOMException(
"Upgrade transaction aborted.",
@@ -957,9 +1152,10 @@ export class MemoryIndexedDbFactory {
request.onerror?.(new Event("error"));
return;
}
this.databaseVersion = targetVersion;
state.version = targetVersion;
}
database.version = this.databaseVersion;
database.version = state.version;
state.connections.push(database);
request.result = database as unknown as IDBDatabase;
request.onsuccess?.(new Event("success"));
}
+32
View File
@@ -943,3 +943,35 @@ describe("IndexedDB database deletion", () => {
});
});
});
describe("05-13 a versionchange that arrives while admission runs", () => {
it("closes the connection and ends the open instead of blocking the upgrade", async () => {
const memory = new MemoryIndexedDbFactory();
let admitting!: () => void;
const admissionStarted = new Promise<void>((resolve) => {
admitting = resolve;
});
let finishAdmission!: () => void;
const pending = openIndexedDbDatabase<TestFailure>({
factory: memory.factory,
databaseName: "kernel",
version: 1,
translate: recordingTranslate([]),
upgrade: () => ({ kind: "APPLIED" }),
admit: async () => {
admitting();
await new Promise<void>((resolve) => {
finishAdmission = resolve;
});
return { kind: "ADMIT" };
},
});
await admissionStarted;
// Another context asks to upgrade while this one is still admitting.
memory.triggerVersionChange(2);
expect(memory.isConnectionClosed()).toBe(true);
finishAdmission();
expect(await pending).toEqual({ ok: false, error: { cause: "CLOSED" } });
});
});
+37
View File
@@ -1123,3 +1123,40 @@ describe("Q-6.5-7 stored-row rules stay the same at every call site", () => {
).toMatchObject({ ok: false, error: { code: "POLICY_REJECTED" } });
});
});
describe("05-05 a receipt prune steps over a corrupt receipt", () => {
it("deletes every expired healthy receipt, leaves the corrupt one and reports CORRUPT_DATA", async () => {
const memory = new MemoryIndexedDbFactory();
await prepareSchema(memory);
seedReceipt(memory, "expired-a", 100);
seedReceipt(memory, "expired-b", 200);
memory.seed("receipts", { idempotencyKey: "broken", expiresAtEpochMs: 0 });
const observations: IndexedDbObservation[] = [];
const maintenance = createMaintenance(memory, defaultPolicy(), {
nowEpochMilliseconds: () => 300,
observe: (event) => observations.push(event),
});
expect(
await maintenance.pruneExpiredReceipts({
maxRows: 10,
maxDurationMs: 10_000,
}),
).toMatchObject({
ok: true,
value: { state: "COMPLETE", deletedRows: 2 },
});
expect(memory.readRaw("receipts", "expired-a")).toBeUndefined();
expect(memory.readRaw("receipts", "expired-b")).toBeUndefined();
expect(memory.readRaw("receipts", "broken")).toBeDefined();
expect(
memory.readRaw("governance", "dataset-budget"),
).toMatchObject({ receiptCount: 0 });
expect(observations).toContainEqual(
expect.objectContaining({
outcome: "FAILED",
failureCode: "CORRUPT_DATA",
}),
);
});
});
+25
View File
@@ -795,3 +795,28 @@ describe("IndexedDB OPFS journal", () => {
).toMatchObject({ ok: false, error: { code: "UNSUPPORTED" } });
});
});
describe("05-06 a journal from a newer release", () => {
it("answers online-only instead of a migration failure", async () => {
const memory = new MemoryIndexedDbFactory();
const name = opfsJournalDatabaseName("authority_12345678");
await new Promise<void>((resolve, reject) => {
const request = memory.factory.open(name, 2);
request.onsuccess = () => {
request.result.close();
resolve();
};
request.onerror = () => reject(request.error);
});
const journal = createIndexedDbOpfsJournal({
authorityToken: "authority_12345678",
factory: memory.factory,
keyRange: memory.keyRange,
});
expect(await journal.listIncomplete(10)).toMatchObject({
ok: false,
error: { code: "UNSUPPORTED", recovery: "ONLINE_ONLY" },
});
});
});
+276 -13
View File
@@ -6,7 +6,7 @@ import type {
IndexedDbObservation,
IndexedDbRuntimeDependencies,
} from "../../src/adapters/storage/indexeddb/indexeddb-types.ts";
import { mapIndexedDbException } from "../../src/adapters/storage/indexeddb/indexeddb-failure.ts";
import { mapIndexedDbException } from "../../src/adapters/browser-file-storage/result.ts";
import { MemoryIndexedDbFactory } from "../helpers/memory-indexeddb.ts";
type Item = Readonly<{
@@ -1035,9 +1035,10 @@ describe("IndexedDB runtime", () => {
});
});
it("aborts a sweep whose budget row cannot account for the deleted bytes", async () => {
it("05-05 sweeps a row the budget cannot account for and leaves the budget high", async () => {
const memory = new MemoryIndexedDbFactory();
let epochMs = 1_000;
const observations: IndexedDbObservation[] = [];
const runtime = createIndexedDbRuntime(
dependencies(memory, {
storagePolicy: {
@@ -1054,6 +1055,7 @@ describe("IndexedDB runtime", () => {
authorized: true,
proofToken: "sweep_authority_proof_001",
}),
observe: (event) => observations.push(event),
}),
);
expect(await runtime.open()).toMatchObject({ ok: true });
@@ -1066,9 +1068,8 @@ describe("IndexedDB runtime", () => {
}),
).toMatchObject({ ok: true });
// A budget that cannot account for the record's bytes is corrupt, and the
// sweep only discovers it after both deletions have already succeeded
// inside the transaction.
// A budget that cannot account for the record's bytes is corrupt. The
// sweep finds it only after both deletions succeeded in the transaction.
memory.seed("governance", {
bindingKey: "dataset-budget",
budgetVersion: 1,
@@ -1085,15 +1086,20 @@ describe("IndexedDB runtime", () => {
maxDurationMs: 1_000,
}),
).toMatchObject({
ok: false,
error: { code: "CORRUPT_DATA" },
ok: true,
value: { state: "COMPLETE", deletedRows: 1 },
});
// `fail` aborts. Recording the failure without aborting would report the
// same code while letting the two deletions commit against a budget that
// never moved.
expect(aborts.count).toBe(1);
expect(memory.readRaw("records", "sweep-record")).toBeDefined();
expect(memory.readRaw("retention", "sweep-record")).toBeDefined();
expect(aborts.count).toBe(0);
expect(memory.readRaw("records", "sweep-record")).toBeUndefined();
expect(memory.readRaw("retention", "sweep-record")).toBeUndefined();
// Not lowered below what it cannot account for: a budget that reads high
// blocks writes, one that reads low would admit them.
expect(memory.readRaw("governance", "dataset-budget")).toMatchObject({
usedBytes: 0,
});
expect(observations).toContainEqual(
expect.objectContaining({ failureCode: "CORRUPT_DATA" }),
);
});
it("requires lifecycle authority and only deletes sync-confirmed rows", async () => {
@@ -1453,3 +1459,260 @@ describe("IndexedDB runtime", () => {
expect(hardMemory.readRaw("records", "x")).toBeUndefined();
});
});
describe("05-03 a LOGOUT or ACCOUNT_DELETION purge survives corrupt governance", () => {
const partitionScope = {
...TEST_SCOPE,
accountScope: "OPAQUE_PARTITION" as const,
};
const otherScope = {
...partitionScope,
partitionToken: "partitiontoken_002",
};
const partitionPolicy = {
...TEST_STORAGE_POLICY,
accountScope: "OPAQUE_PARTITION" as const,
logoutAction: "PURGE_PARTITION" as const,
accountDeletionAction: "PURGE_PARTITION" as const,
};
const batch = { maxRows: 100, maxDurationMs: 1_000 } as const;
/**
* Another partition and this one each hold a record in one factory; this
* partition's budget row is then corrupted, so a fresh runtime cannot pass
* admission (E1).
*/
async function corruptedPartition(
authorizeLifecycle: IndexedDbRuntimeDependencies<
Item,
Item,
Query
>["authorizeLifecycle"] = () => ({
authorized: true,
proofToken: "authorityproof_001",
}),
) {
const memory = new MemoryIndexedDbFactory();
const other = createIndexedDbRuntime(
dependencies(memory, {
scope: otherScope,
storagePolicy: partitionPolicy,
}),
);
expect(await other.open()).toMatchObject({ ok: true });
await other.compareAndSwap({
key: "kept",
value: { label: "other", rank: 1 },
expectedRevision: null,
idempotencyKey: "kept-create",
});
const writer = createIndexedDbRuntime(
dependencies(memory, {
scope: partitionScope,
storagePolicy: partitionPolicy,
}),
);
expect(await writer.open()).toMatchObject({ ok: true });
await writer.compareAndSwap({
key: "personal-1",
value: { label: "mine", rank: 1 },
expectedRevision: null,
idempotencyKey: "personal-create",
});
writer.close();
memory.seed("governance", {
bindingKey: "dataset-budget",
budgetVersion: 1,
usedBytes: -5,
receiptCount: 1,
});
const deleteDatabase = vi.spyOn(memory.factory, "deleteDatabase");
const fresh = createIndexedDbRuntime(
dependencies(memory, {
scope: partitionScope,
storagePolicy: partitionPolicy,
authorizeLifecycle,
}),
);
expect(await fresh.open()).toMatchObject({
ok: false,
error: { code: "POLICY_REJECTED" },
});
return { memory, other, fresh, deleteDatabase };
}
it.each(["LOGOUT", "ACCOUNT_DELETION"] as const)(
"deletes the partition's whole dataset for an authorized %s and leaves other partitions alone",
async (action) => {
const { memory, other, fresh, deleteDatabase } =
await corruptedPartition();
expect(
await fresh.enforceLifecycleBatch({ action, ...batch }),
).toMatchObject({
ok: true,
value: { state: "COMPLETE", datasetDeleted: true },
});
expect(deleteDatabase).toHaveBeenCalledTimes(1);
expect(deleteDatabase).toHaveBeenCalledWith(
indexedDbPhysicalDatabaseName(partitionScope),
);
expect(memory.hasStore("records")).toBe(false);
expect(await other.read("kept")).toMatchObject({
ok: true,
value: { value: { label: "other", rank: 1 } },
});
},
);
it("keeps the dataset when the authority does not approve", async () => {
const { memory, fresh, deleteDatabase } = await corruptedPartition(
() => ({ authorized: false }),
);
expect(
await fresh.enforceLifecycleBatch({ action: "ACCOUNT_DELETION", ...batch }),
).toMatchObject({ ok: false, error: { code: "POLICY_REJECTED" } });
expect(deleteDatabase).not.toHaveBeenCalled();
expect(memory.readRaw("records", "personal-1")).toBeDefined();
});
it("purges row by row, never the database, when governance is healthy", async () => {
const memory = new MemoryIndexedDbFactory();
const deleteDatabase = vi.spyOn(memory.factory, "deleteDatabase");
const runtime = createIndexedDbRuntime(
dependencies(memory, {
scope: partitionScope,
storagePolicy: partitionPolicy,
}),
);
expect(await runtime.open()).toMatchObject({ ok: true });
await runtime.compareAndSwap({
key: "personal-1",
value: { label: "mine", rank: 1 },
expectedRevision: null,
idempotencyKey: "personal-create",
});
runtime.close();
const fresh = createIndexedDbRuntime(
dependencies(memory, {
scope: partitionScope,
storagePolicy: partitionPolicy,
}),
);
const purged = await fresh.enforceLifecycleBatch({
action: "LOGOUT",
...batch,
});
expect(purged).toMatchObject({ ok: true, value: { state: "COMPLETE" } });
expect(purged.ok && purged.value).not.toHaveProperty("datasetDeleted");
expect(deleteDatabase).not.toHaveBeenCalled();
expect(memory.hasStore("records")).toBe(true);
expect(memory.readRaw("records", "personal-1")).toBeUndefined();
});
});
describe("05-05 a retention sweep steps over a corrupt row", () => {
it("deletes every expired healthy row, leaves the corrupt sidecar and reports it", async () => {
const memory = new MemoryIndexedDbFactory();
let epochMs = 1_000;
const observations: IndexedDbObservation[] = [];
const runtime = createIndexedDbRuntime(
dependencies(memory, {
storagePolicy: {
...TEST_STORAGE_POLICY,
retention: { kind: "TTL", maxAgeMs: 10 },
},
keyRange:
memory.keyRange as NonNullable<
IndexedDbRuntimeDependencies<Item, Item, Query>["keyRange"]
>,
nowEpochMilliseconds: () => epochMs,
nowMonotonicMilliseconds: () => 0,
observe: (event) => observations.push(event),
}),
);
expect(await runtime.open()).toMatchObject({ ok: true });
for (const key of ["a", "b", "c"]) {
await runtime.compareAndSwap({
key,
value: { label: key, rank: 1 },
expectedRevision: null,
idempotencyKey: `create-${key}`,
});
}
// Sorts first on the eligibility index and names a record it is not keyed by.
memory.seed("retention", {
recordKey: "zzz",
writtenAtEpochMs: 0,
synchronization: "NONE",
measuredBytes: 0,
eligibleAtEpochMs: 0,
});
epochMs = 5_000;
for (let attempt = 0; attempt < 2; attempt += 1) {
expect(
await runtime.enforceLifecycleBatch({
action: "RETENTION_SWEEP",
maxRows: 100,
maxDurationMs: 1_000,
}),
).toMatchObject({ ok: true, value: { state: "COMPLETE" } });
}
expect(
["a", "b", "c"].filter(
(key) => memory.readRaw("records", key) !== undefined,
),
).toEqual([]);
expect(memory.readRaw("retention", "zzz")).toBeDefined();
expect(observations).toContainEqual(
expect.objectContaining({
operation: "INDEXEDDB_WRITE",
outcome: "FAILED",
failureCode: "CORRUPT_DATA",
}),
);
});
});
describe("05-06 an older tab that meets a newer schema", () => {
it("reports INCOMPATIBLE and online-only, not a migration failure", async () => {
const memory = new MemoryIndexedDbFactory();
const base = dependencies(memory);
const newer = createIndexedDbRuntime({
...base,
schemaVersion: 2,
migrations: [
...base.migrations,
{
id: "schema-v2",
fromVersion: 1,
toVersion: 2,
operations: [{ kind: "CREATE_STORE", name: "extra", keyPath: "k" }],
},
],
});
expect(await newer.open()).toMatchObject({ ok: true });
newer.close();
const older = createIndexedDbRuntime(dependencies(memory));
expect(await older.open()).toMatchObject({
ok: false,
error: {
code: "UNSUPPORTED",
operation: "INDEXEDDB_OPEN",
recovery: "ONLINE_ONLY",
},
});
expect(older.getStatus()).toEqual({
kind: "INCOMPATIBLE",
schemaVersion: 1,
});
expect(await older.read("x")).toMatchObject({
ok: false,
error: { code: "UNSUPPORTED", recovery: "ONLINE_ONLY" },
});
});
});
@@ -0,0 +1,120 @@
import { describe, expect, it } from "vitest";
import { MemoryIndexedDbFactory } from "../helpers/memory-indexeddb.ts";
/**
* 05-08. Where the fake used to part from native IndexedDB, each case pins
* the native result, so a test built on the fake cannot pass on a behavior no
* browser has.
*/
function settle<Value>(request: IDBRequest<Value>): Promise<Value> {
return new Promise((resolve, reject) => {
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
}
function outcome(transaction: IDBTransaction): Promise<string> {
return new Promise((resolve) => {
transaction.oncomplete = () => resolve("complete");
transaction.onabort = () =>
resolve(`abort:${transaction.error?.name ?? "none"}`);
});
}
async function openWithStore(
memory: MemoryIndexedDbFactory,
name: string,
): Promise<IDBDatabase> {
const request = memory.factory.open(name, 1);
request.onupgradeneeded = () => {
request.result.createObjectStore("s", { keyPath: "k" });
};
return await settle(request);
}
describe("05-08 memory IndexedDB matches native semantics the adapters rely on", () => {
it("(a) runs overlapping readwrite transactions one after another", async () => {
const memory = new MemoryIndexedDbFactory();
const database = await openWithStore(memory, "a");
memory.seed("s", { k: "counter", n: 0 });
const first = database.transaction(["s"], "readwrite");
const second = database.transaction(["s"], "readwrite");
const settled = [outcome(first), outcome(second)];
for (const transaction of [first, second]) {
const read = transaction.objectStore("s").get("counter");
read.onsuccess = () => {
const { n } = read.result as { n: number };
transaction.objectStore("s").put({ k: "counter", n: n + 1 });
};
}
expect(await Promise.all(settled)).toEqual(["complete", "complete"]);
expect(memory.readRaw("s", "counter")).toEqual({ k: "counter", n: 2 });
});
it("(b) rejects a request placed after the transaction's task ended", async () => {
const memory = new MemoryIndexedDbFactory();
const database = await openWithStore(memory, "b");
const transaction = database.transaction(["s"], "readwrite");
const settled = outcome(transaction);
await new Promise((resolve) => setTimeout(resolve, 5));
expect(() =>
transaction.objectStore("s").put({ k: "late" }),
).toThrow(expect.objectContaining({ name: "TransactionInactiveError" }));
// A transaction with no request left commits on its own.
expect(await settled).toBe("complete");
expect(memory.readRaw("s", "late")).toBeUndefined();
});
it("(b) keeps a transaction active for requests placed in a success callback's microtasks", async () => {
const memory = new MemoryIndexedDbFactory();
const database = await openWithStore(memory, "b2");
const transaction = database.transaction(["s"], "readwrite");
const settled = outcome(transaction);
const read = transaction.objectStore("s").get("absent");
read.onsuccess = () => {
void Promise.resolve().then(() => {
transaction.objectStore("s").put({ k: "chained" });
});
};
expect(await settled).toBe("complete");
expect(memory.readRaw("s", "chained")).toEqual({ k: "chained" });
});
it("(d) keeps databases of different names apart", async () => {
const memory = new MemoryIndexedDbFactory();
const userA = await openWithStore(memory, "user-A");
await settle(
userA.transaction(["s"], "readwrite").objectStore("s").put({
k: "secret-of-A",
}),
);
let upgraded = false;
const request = memory.factory.open("user-B", 1);
request.onupgradeneeded = () => {
upgraded = true;
request.result.createObjectStore("s", { keyPath: "k" });
};
const userB = await settle(request);
expect(upgraded).toBe(true);
expect(
await settle(
userB.transaction(["s"], "readonly").objectStore("s").get(
"secret-of-A",
),
),
).toBeUndefined();
expect(
await settle(
userA.transaction(["s"], "readonly").objectStore("s").get(
"secret-of-A",
),
),
).toEqual({ k: "secret-of-A" });
});
});
+115 -6
View File
@@ -630,7 +630,7 @@ describe("OPFS byte-store coordinator", () => {
},
{
async abortPreparedPut() {
return browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const });
return browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE");
},
},
]) {
@@ -666,11 +666,120 @@ describe("OPFS byte-store coordinator", () => {
}
});
it("does not roll back journal after an unknown worker mutation effect", async () => {
it("05-01 decides a reconciled row from what the journal holds once its lock is taken", async () => {
// Listed while its owner still ran, the row was PREPARING. By the time
// the owner let go it had committed and only finalization failed, so the
// staging receipt names the committed generation. Cleaning it up from the
// stale snapshot would delete a committed object.
const journal = createJournal();
const row: OpfsJournalTransaction = {
transactionId: "transaction_12345678",
fencingToken: "fencing_12345678",
mutation: "PUT",
scope,
phase: "PREPARING",
objectId: "object_12345678",
expectedGeneration: null,
targetGeneration: 1,
targetByteLength: 1,
targetStoragePolicy: storagePolicy,
budgetReservation: {
namespace: scope.namespace,
reservedBytes: 1,
hardBudgetBytes: storagePolicy.hardBudgetBytes,
},
startedAtEpochMs: 100,
};
journal.transactions.set(row.transactionId, {
...row,
phase: "COMMITTED",
preparedObject: committedObject("object_12345678", storagePolicy, 100),
});
vi.spyOn(journal, "listIncomplete").mockResolvedValueOnce(
browserDataSuccess({ transactions: [row], moreAvailable: false }),
);
const worker = createWorker();
const cleanupTransaction = vi.spyOn(worker, "cleanupTransaction");
const finalizePut = vi.spyOn(worker, "finalizePut");
const adapter = createOpfsByteStoreAdapter({
journal,
worker,
scope,
storagePolicy,
now: () => 100,
});
expect(await adapter.maintenance.reconcile()).toMatchObject({
ok: true,
value: { inspectedTransactions: 1, rolledBackTransactions: 0 },
});
expect(cleanupTransaction).not.toHaveBeenCalled();
expect(finalizePut).toHaveBeenCalledTimes(1);
});
it("05-02 skips a row it cannot reconcile, settles the rest and still runs orphan GC", async () => {
const journal = createJournal();
const row = (transactionId: string): OpfsJournalTransaction => ({
transactionId,
fencingToken: "fencing_12345678",
mutation: "PUT",
scope,
phase: "PREPARING",
objectId: `object_${transactionId.slice(-8)}`,
expectedGeneration: null,
targetGeneration: 1,
targetByteLength: 1,
targetStoragePolicy: storagePolicy,
budgetReservation: {
namespace: scope.namespace,
reservedBytes: 1,
hardBudgetBytes: storagePolicy.hardBudgetBytes,
},
startedAtEpochMs: 100,
});
journal.transactions.set("transaction_broken_1", row("transaction_broken_1"));
journal.transactions.set("transaction_healthy_1", row("transaction_healthy_1"));
const worker = createWorker({
async cleanupTransaction(_scope, transactionId) {
return transactionId === "transaction_broken_1"
? browserDataFailure("CORRUPT_DATA", "OBJECT_RECONCILE")
: browserDataSuccess({ kind: "CLEANED" as const });
},
});
const listOrphanCandidates = vi.spyOn(worker, "listOrphanCandidates");
const observations: unknown[] = [];
const adapter = createOpfsByteStoreAdapter({
journal,
worker,
scope,
storagePolicy,
now: () => 100,
observer: (event) => observations.push(event),
});
expect(await adapter.maintenance.reconcile()).toMatchObject({
ok: true,
value: {
inspectedTransactions: 2,
failedTransactions: 1,
rolledBackTransactions: 1,
orphanGcStatus: "COMPLETED",
},
});
expect([...journal.transactions.keys()]).toEqual(["transaction_broken_1"]);
expect(listOrphanCandidates).toHaveBeenCalledTimes(1);
expect(observations).toContainEqual({
operation: "OBJECT_RECONCILE",
outcome: "FAILED",
failureCode: "CORRUPT_DATA",
});
});
it("does not roll back journal after an unconfirmed worker cleanup", async () => {
const journal = createJournal();
const worker = createWorker({
async abortPreparedPut() {
return browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const });
return browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE");
},
});
const rollback = vi.spyOn(journal, "rollback");
@@ -734,8 +843,8 @@ describe("OPFS byte-store coordinator", () => {
async () => browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE"),
],
[
"an unknown cleanup effect",
async () => browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const }),
"an aborted cleanup",
async () => browserDataFailure("ABORTED", "OBJECT_RECONCILE"),
],
])("after %s", async (_label, abortPreparedPut) => {
const journal = createJournal();
@@ -801,7 +910,7 @@ describe("OPFS byte-store coordinator", () => {
await putWith(
{
abortPreparedPut: async () =>
browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const }),
browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE"),
},
journal,
),
+422
View File
@@ -13,6 +13,7 @@ import type {
} from "../../src/application/ports/browser-file-storage/shared.ts";
import { browserDataSuccess } from "../../src/adapters/browser-file-storage/result.ts";
import { createOpfsByteStoreAdapter } from "../../src/adapters/storage/opfs/opfs-byte-store-adapter.ts";
import { createIndexedDbOpfsJournal } from "../../src/adapters/storage/opfs/indexeddb-opfs-journal.ts";
import {
resolveOpfsRuntimePolicy,
} from "../../src/adapters/storage/opfs/opfs-policy.ts";
@@ -33,6 +34,7 @@ import {
type OpfsWorkerMessageHost,
type OpfsWorkerRuntime,
} from "../../src/adapters/storage/opfs/opfs-worker-runtime.ts";
import { MemoryIndexedDbFactory } from "../helpers/memory-indexeddb.ts";
const scopeA: OpfsStorageScope = Object.freeze({
namespace: "durable-objects",
@@ -2083,3 +2085,423 @@ describe("NS-05 a bootstrap failure answers the request it belongs to", () => {
});
}
});
/**
* Two runtimes ("tabs") share one OPFS root, one exclusive worker lock and
* one journal database, the way two tabs of one origin do.
*/
function sharedRootTabs(policy = runtimePolicy) {
const root = new MemoryDirectory();
const leases = exclusiveLeases();
const idb = new MemoryIndexedDbFactory();
const tab = (
tag: string,
extra: Partial<Parameters<typeof createOpfsByteStoreAdapter>[0]> = {},
) => {
const tabScope = extra.scope ?? scopeA;
const runtime = createOpfsWorkerRuntime({
root: root as unknown as FileSystemDirectoryHandle,
crypto: globalThis.crypto,
policy,
leaseManager: leases,
dedicatedWorker: true,
supportsSynchronousAccessHandles: false,
});
let sequence = 0;
return createOpfsByteStoreAdapter({
journal: createIndexedDbOpfsJournal({
authorityToken: tabScope.authorityToken,
factory: idb.factory,
keyRange: idb.keyRange,
}),
worker: loopbackGateway(runtime, policy),
scope: tabScope,
storagePolicy,
policy,
createTransactionId: () => `transaction_${tag}_${++sequence}`,
...extra,
});
};
return { root, leases, idb, tab };
}
async function openedBytes(
adapter: ReturnType<typeof createOpfsByteStoreAdapter>,
objectId: string,
): Promise<Readonly<{ generation: number; bytes: number[] }> | string> {
const opened = await adapter.objects.open({ objectId });
if (!opened.ok) return opened.error.code;
const bytes: number[] = [];
for await (const chunk of opened.value.source.stream(
new AbortController().signal,
)) {
if (!chunk.ok) return chunk.error.code;
bytes.push(...chunk.value);
}
return { generation: opened.value.descriptor.generation, bytes };
}
describe("05-01 reconcile leaves a live put's journal row alone", () => {
it.each([
["a first put", null],
["an overwrite", 1],
] as const)(
"keeps %s that reconcile overlapped in another tab",
async (_label, expectedGeneration) => {
const { tab } = sharedRootTabs();
const tabA = tab("a");
const tabB = tab("b");
if (expectedGeneration !== null) {
expect(
await tabA.objects.put({
objectId: "object_12345678",
expectedGeneration: null,
mediaType: "application/octet-stream",
source: oneByteSource(),
}),
).toMatchObject({ ok: true, value: { generation: 1 } });
}
const stalled = stalledSource();
const putA = tabA.objects.put({
objectId: "object_12345678",
expectedGeneration,
mediaType: "application/octet-stream",
source: stalled.source,
});
// Tab A holds the worker lock and its journal row is PREPARING.
await stalled.streaming;
const reconcileB = tabB.maintenance.reconcile();
await new Promise((resolve) => setTimeout(resolve, 20));
stalled.release();
const [put, reconciled] = await Promise.all([putA, reconcileB]);
const generation = (expectedGeneration ?? 0) + 1;
expect(put).toMatchObject({ ok: true, value: { generation } });
expect(reconciled).toMatchObject({
ok: true,
value: { skippedLiveTransactions: 1, rolledBackTransactions: 0 },
});
expect(await openedBytes(tabA, "object_12345678")).toEqual({
generation,
bytes: [5],
});
},
);
});
describe("05-02 an empty receipt left by a failed first write is no receipt", () => {
it("cleans a crash-left empty receipt and keeps orphan GC sweeping", async () => {
const root = new MemoryDirectory();
const runtime = createOpfsWorkerRuntime({
root: root as unknown as FileSystemDirectoryHandle,
crypto: globalThis.crypto,
policy: runtimePolicy,
leaseManager: immediateLeases({ count: 0 }),
dedicatedWorker: true,
supportsSynchronousAccessHandles: false,
});
// `getFileHandle({create:true})` made the file; the write never landed.
let directory = root as unknown as FileSystemDirectoryHandle;
for (const segment of [
"authorities",
scopeA.authorityToken,
scopeA.namespaceToken,
scopeA.partitionToken,
"staging",
"transaction_empty_0001",
]) {
directory = await directory.getDirectoryHandle(segment, { create: true });
}
await directory.getFileHandle("receipt.json", { create: true });
const orphan = "0".repeat(64);
await seedChunk(root, scopeA, orphan);
expect(
await runtime.handleRequest({
requestId: "request_empty_list",
protocolVersion: OPFS_WORKER_PROTOCOL_VERSION,
kind: "LIST_ORPHAN_CANDIDATES",
scope: scopeA,
olderThanEpochMs: Date.now(),
maxEntries: 10,
}),
).toMatchObject({
ok: true,
value: { safeToSweep: true, digests: [orphan] },
});
expect(
await runtime.handleRequest({
requestId: "request_empty_cleanup",
protocolVersion: OPFS_WORKER_PROTOCOL_VERSION,
kind: "CLEANUP_TRANSACTION",
scope: scopeA,
transactionId: "transaction_empty_0001",
}),
).toMatchObject({ ok: true, value: { kind: "ALREADY_CLEAN" } });
expect(
root.has([
"authorities",
scopeA.authorityToken,
scopeA.namespaceToken,
scopeA.partitionToken,
"staging",
"transaction_empty_0001",
]),
).toBe(false);
});
it("compensates a put whose first receipt write hit the quota, so reconcile runs and the same id retries", async () => {
const { tab } = sharedRootTabs();
const store = tab("q");
const getFileHandle = MemoryDirectory.prototype.getFileHandle;
let failReceipt = true;
vi.spyOn(MemoryDirectory.prototype, "getFileHandle").mockImplementation(
async function (this: MemoryDirectory, name, options) {
const handle = await getFileHandle.call(this, name, options);
if (name !== "receipt.json" || !failReceipt) return handle;
failReceipt = false;
return {
...handle,
async createWritable() {
const writable = await handle.createWritable();
return {
write: (data: FileSystemWriteChunkType) => writable.write(data),
abort: () => writable.abort(),
async close() {
throw new DOMException("Quota exceeded.", "QuotaExceededError");
},
} as unknown as FileSystemWritableFileStream;
},
} as FileSystemFileHandle;
},
);
const put = () =>
store.objects.put({
objectId: "object_12345678",
expectedGeneration: null,
mediaType: "application/octet-stream",
source: oneByteSource(),
});
const failed = await put();
expect(failed).toMatchObject({
ok: false,
error: { code: "QUOTA_EXCEEDED" },
});
expect(failed.ok || failed.error.recovery).not.toBe("RECONCILE");
expect(await store.maintenance.reconcile()).toMatchObject({
ok: true,
value: { inspectedTransactions: 0, orphanGcStatus: "COMPLETED" },
});
expect(await put()).toMatchObject({ ok: true, value: { generation: 1 } });
});
});
describe("05-04 a LOGOUT or ACCOUNT_DELETION purge leaves no partition bytes behind", () => {
const scopeP2: OpfsStorageScope = Object.freeze({
...scopeA,
partitionToken: "partition_87654321",
});
const authority = {
requestMaintenanceAuthority: () => ({
authorized: true as const,
proofToken: "proof_token_1234567890",
expiresAtEpochMs: Date.now() + 60_000,
}),
consumeMaintenanceAuthority: () => true,
};
function filesUnder(root: MemoryDirectory, scope: OpfsStorageScope): string[] {
const files: string[] = [];
const walk = (directory: MemoryDirectory, path: string) => {
for (const [name, node] of directory.children) {
if (node instanceof MemoryDirectory) walk(node, `${path}/${name}`);
else files.push(`${path}/${name}`);
}
};
const partition = [
"authorities",
scope.authorityToken,
scope.namespaceToken,
scope.partitionToken,
].reduce<MemoryNode | undefined>(
(node, segment) =>
node instanceof MemoryDirectory ? node.children.get(segment) : node,
root,
);
if (partition instanceof MemoryDirectory) walk(partition, "");
return files;
}
const putBytes = (
adapter: ReturnType<typeof createOpfsByteStoreAdapter>,
objectId: string,
bytes: number[],
) =>
adapter.objects.put({
objectId,
expectedGeneration: null,
mediaType: "application/octet-stream",
source: bytesSource(new Uint8Array(bytes)),
});
it.each(["LOGOUT", "ACCOUNT_DELETION"] as const)(
"removes the chunks of a %s-purged partition and keeps other partitions",
async (reason) => {
const { root, tab } = sharedRootTabs();
const purged = tab("p", authority);
const kept = tab("k", { ...authority, scope: scopeP2 });
expect(await putBytes(purged, "object_secret_1", [1, 2, 3])).toMatchObject({ ok: true });
expect(await putBytes(purged, "object_secret_2", [4, 5])).toMatchObject({ ok: true });
expect(await putBytes(kept, "object_kept_0001", [7, 7])).toMatchObject({ ok: true });
expect(await purged.maintenance.enforcePolicies({ reason })).toMatchObject({
ok: true,
value: {
removedObjects: 2,
moreObjectsAvailable: false,
deadlineReached: false,
chunkSweep: "COMPLETED",
},
});
expect(filesUnder(root, scopeA)).toEqual([]);
expect(await openedBytes(kept, "object_kept_0001")).toEqual({
generation: 1,
bytes: [7, 7],
});
},
);
it("never sweeps the chunk of a put that runs while the purge does", async () => {
const { tab } = sharedRootTabs();
const purger = tab("p", authority);
const writer = tab("w", authority);
expect(await putBytes(purger, "object_secret_1", [1, 2, 3])).toMatchObject({ ok: true });
const stalled = stalledSource();
const put = writer.objects.put({
objectId: "object_late_0001",
expectedGeneration: null,
mediaType: "application/octet-stream",
source: stalled.source,
});
await stalled.streaming;
const purge = purger.maintenance.enforcePolicies({ reason: "LOGOUT" });
await new Promise((resolve) => setTimeout(resolve, 20));
stalled.release();
expect(await put).toMatchObject({ ok: true });
expect(await purge).toMatchObject({ ok: true, value: { removedObjects: 1 } });
expect(await openedBytes(writer, "object_late_0001")).toEqual({
generation: 1,
bytes: [5],
});
});
});
describe("05-07 a put whose source stops lets go of the origin lock", () => {
afterEach(() => {
vi.useRealTimers();
});
it("fails the stalled put at the idle bound so another tab's put runs", async () => {
const policy = resolveOpfsRuntimePolicy({
...runtimePolicy,
putIdleTimeoutMs: 30_000,
});
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
const { tab, leases } = sharedRootTabs(policy);
const tabA = tab("a");
const tabB = tab("b");
const stalled = stalledSource();
const putA = tabA.objects.put({
objectId: "object_stalled_01",
expectedGeneration: null,
mediaType: "application/octet-stream",
source: stalled.source,
});
await stalled.streaming;
expect(leases.held).toBe(true);
const putB = tabB.objects.put({
objectId: "object_waiting_01",
expectedGeneration: null,
mediaType: "application/octet-stream",
source: oneByteSource(),
});
await vi.advanceTimersByTimeAsync(policy.putIdleTimeoutMs + 1);
vi.useRealTimers();
const settledA = await Promise.race([
putA,
new Promise((resolve) => setTimeout(() => resolve("STILL_RUNNING"), 2_000)),
]);
expect(settledA).toMatchObject({
ok: false,
error: { code: "NOT_READABLE", retryable: true },
});
expect(await putB).toMatchObject({ ok: true, value: { generation: 1 } });
stalled.release();
});
});
describe("05-10 the object id grammar is the prepared-object grammar", () => {
it("rejects a policy that widens the object id grammar when it is resolved", () => {
expect(() =>
resolveOpfsRuntimePolicy({
...runtimePolicy,
isObjectIdAllowed: (objectId) => /^[A-Za-z0-9_.-]{1,200}$/u.test(objectId),
}),
).toThrow(TypeError);
});
});
describe("05-14 a read that outlives its generation", () => {
it("ends with NOT_FOUND / REHYDRATE once GC took the replaced chunks, and a reopen reads the new generation", async () => {
const { root, tab } = sharedRootTabs();
const store = tab("r");
const put = (bytes: number[], expectedGeneration: number | null) =>
store.objects.put({
objectId: "object_12345678",
expectedGeneration,
mediaType: "application/octet-stream",
source: bytesSource(new Uint8Array(bytes)),
});
expect(await put([1], null)).toMatchObject({ ok: true });
const opened = await store.objects.open({ objectId: "object_12345678" });
if (!opened.ok) throw new Error(opened.error.code);
// Generation 2 replaces it, and its old chunk is past the GC grace.
expect(await put([2], 1)).toMatchObject({ ok: true });
const digest = Array.from(
new Uint8Array(
await globalThis.crypto.subtle.digest("SHA-256", new Uint8Array([1])),
),
(byte) => byte.toString(16).padStart(2, "0"),
).join("");
memoryFileAt(root, chunkFilePath(scopeA, digest)).lastModified = 0;
expect(await store.maintenance.reconcile()).toMatchObject({
ok: true,
value: { deletedOrphanChunks: 1 },
});
const chunks = [];
for await (const chunk of opened.value.source.stream(
new AbortController().signal,
)) {
chunks.push(chunk);
}
expect(chunks).toEqual([
expect.objectContaining({
ok: false,
error: expect.objectContaining({
code: "NOT_FOUND",
recovery: "REHYDRATE",
}),
}),
]);
expect(await openedBytes(store, "object_12345678")).toEqual({
generation: 2,
bytes: [2],
});
});
});