Merge lane E: browser storage and adapter kernel fixes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -759,7 +759,7 @@ CLEANED -> journal 제거
|
||||
|
||||
- journal row와 budget reservation은 physical cleanup effect가
|
||||
`CLEANED` 또는 `ALREADY_CLEAN`으로 확인된 뒤에만 해제한다. timeout, crash,
|
||||
malformed response, `EFFECT_UNKNOWN`은 성공이 아니며 `PREPARING`/`FILES_READY`를
|
||||
malformed response, worker cleanup 실패는 성공이 아니며 `PREPARING`/`FILES_READY`를
|
||||
그대로 남긴다. 이때 put은 자기 실패 코드(취소면 `ABORTED`, 쿼터 초과면
|
||||
`QUOTA_EXCEEDED` 등)를 그대로 반환하고 recovery만 `RECONCILE`로 바꾼다
|
||||
(Q-6.5-9). journal rollback이 실패해도 같다.
|
||||
|
||||
@@ -16,15 +16,24 @@ primitive를 coordinator 완료 증거로 사용하지 않는다.
|
||||
|
||||
## OPFS 보상 실패와 reconcile (STO-01)
|
||||
|
||||
`put()`이 실패했는데 보상 cleanup effect가 확인되지 않으면 runtime은 실패를
|
||||
`OBJECT_RECONCILE` / `CONFLICT`(recovery `RETRY`)로 보고하고 journal row를 남긴다.
|
||||
이는 결함이 아니라 설계된 상태다.
|
||||
`put()`이 commit 전에 실패하면 coordinator가 그 transaction의 staging을 지우고
|
||||
journal row를 되돌린다(보상). 보상의 worker cleanup이나 journal rollback이 끝나지
|
||||
않으면 put은 자기 실패 코드(`QUOTA_EXCEEDED`, `ABORTED` 등)와 operation
|
||||
`OBJECT_WRITE`를 그대로 두고 recovery만 `RECONCILE`로 바꿔 반환한다. journal
|
||||
row(`PREPARING`/`FILES_READY`)는 남는다(`opfs-byte-store-adapter.ts`
|
||||
`compensatePreparedPut`, Q-6.5-9). 이는 결함이 아니라 설계된 상태다. worker가
|
||||
cleanup 결과를 확인하지 못하면 성공 값이 아니라 실패로 답한다. 별도의 "결과 불명"
|
||||
값은 없다(05-09).
|
||||
|
||||
1. journal에 `PREPARING` 또는 `FILES_READY` row가 남아 있는지 확인한다. 남아
|
||||
있다면 staging bytes가 아직 존재할 수 있다는 뜻이다.
|
||||
2. `maintenance.reconcile()`을 실행한다. reconcile은 같은 exact physical
|
||||
generation token만 삭제하고, effect가 여전히 `EFFECT_UNKNOWN`이면 journal을
|
||||
유지한 채 다시 `OBJECT_RECONCILE`을 반환한다.
|
||||
2. `maintenance.reconcile()`을 실행한다. reconcile은 exact physical generation
|
||||
token만 삭제한다. report에서 다음을 본다.
|
||||
- `skippedLiveTransactions`: 어느 tab에서든 아직 진행 중인 put/delete의 row다.
|
||||
reconcile은 transaction별 Web Lock을 `ifAvailable`로 잡지 못한 row를 건드리지
|
||||
않는다(05-01, `reconcile`의 `holdTransactionLock`). 결함이 아니다.
|
||||
- `failedTransactions`: 이번에 정리하지 못해 journal에 남긴 row 수다. 나머지
|
||||
row와 orphan GC는 계속 돈다(05-02). 같은 row가 계속 실패하면 §7을 본다.
|
||||
3. journal row를 수동으로 삭제하지 않는다. row가 사라지면 stale staging을 추적할
|
||||
근거가 사라지고 quota만 누수된다.
|
||||
4. OPFS root나 journal database를 통째로 삭제하거나 schema를 downgrade하지
|
||||
@@ -36,6 +45,9 @@ primitive를 coordinator 완료 증거로 사용하지 않는다.
|
||||
incident 중에도 다음 작업은 금지한다.
|
||||
|
||||
- 자동 또는 일괄 `deleteDatabase()`
|
||||
- 예외는 하나다. authority가 승인한 LOGOUT/ACCOUNT_DELETION이 손상된
|
||||
governance 때문에 dataset을 열 수 없을 때, runtime이 그 partition의 DB
|
||||
하나만 지운다(§5 "database 전체 삭제" 참고).
|
||||
- origin의 모든 `caches.keys()` 삭제
|
||||
- user-authored/unsynced IndexedDB 또는 OPFS data 자동 purge
|
||||
- schema version downgrade
|
||||
@@ -154,6 +166,12 @@ read-only나 online-only가 사용자 작성 내용을 잃게 한다면 먼저 e
|
||||
- `versionchange` 뒤 connection이 남음
|
||||
- repeated reload/update loop
|
||||
- open/maintenance의 `POLICY_REJECTED`: immutable dataset binding missing/mismatch
|
||||
- open의 `UNSUPPORTED` / `ONLINE_ONLY`와 status `INCOMPATIBLE`: 더 새 release가
|
||||
schema를 올린 DB를 이전 bundle이 열었다(05-06). migration 실패가 아니므로 §5로
|
||||
가지 않는다. 이 bundle은 그 dataset을 읽지도 쓰지도 않고 online-only로
|
||||
동작하며, 새 release로 reload하면 풀린다
|
||||
(`indexeddb-runtime.ts` open의 `VersionError` 분기). OPFS journal도 같은 답을
|
||||
낸다(`indexeddb-opfs-journal.ts` `translateFor`)
|
||||
|
||||
### 확인 순서
|
||||
|
||||
@@ -170,7 +188,8 @@ read-only나 online-only가 사용자 작성 내용을 잃게 한다면 먼저 e
|
||||
6. 모든 connection에 `versionchange`/forced-close handler가 설치됐는지 확인한다.
|
||||
7. BroadcastChannel prepare hint는 참고만 하고 실제 open/connection registry를
|
||||
확인한다.
|
||||
8. N-1 bundle이 future schema를 read-only/online-only로 처리하는지 확인한다.
|
||||
8. N-1 bundle이 future schema를 `INCOMPATIBLE` / `ONLINE_ONLY`로 처리하는지
|
||||
확인한다. reference runtime은 read-only가 아니라 online-only로 닫는다.
|
||||
|
||||
### 안전한 조치
|
||||
|
||||
@@ -196,6 +215,12 @@ remote tab 강제 종료, database 삭제, 무한 reload는 금지한다.
|
||||
|
||||
- `MIGRATION_FAILED`, unknown codec, schema postcondition failure
|
||||
- `CORRUPT_DATA`, `NOT_READABLE`, forced close
|
||||
- `RETENTION_SWEEP`(`INDEXEDDB_WRITE`)과 `pruneExpiredReceipts`
|
||||
(`INDEXEDDB_MIGRATE`)의 `CORRUPT_DATA` 관측은 실패가 아니다. 손상 행을 지우지
|
||||
않고 건너뛴 수이고, 나머지 정상 행은 정리된다(05-05,
|
||||
`indexeddb-runtime.ts` `purgeEligibleRecords`,
|
||||
`indexeddb-maintenance.ts` `pruneExpiredReceipts`). budget이 삭제 bytes나
|
||||
receipt 수를 감당하지 못하면 그 값은 내리지 않고 둔다
|
||||
- transaction request success 후 commit failure
|
||||
- `LIMIT_EXCEEDED`: dataset logical byte/receipt budget 초과
|
||||
- lifecycle batch가 proof 없이 실행되거나 TTL/UNTIL_SYNCED record가 잘못 노출·삭제
|
||||
@@ -248,7 +273,21 @@ remote tab 강제 종료, database 삭제, 무한 reload는 금지한다.
|
||||
proof를 운영자가 수동 생성·재사용하지 않음
|
||||
|
||||
database 전체 삭제는 data owner가 export/recovery와 영향 scope를 승인한 별도
|
||||
action이다.
|
||||
action이다. runtime이 스스로 DB를 지우는 경우는 하나뿐이다(05-03).
|
||||
|
||||
- 조건: `enforceLifecycleBatch`의 action이 `LOGOUT` 또는 `ACCOUNT_DELETION`이고,
|
||||
`authorizeLifecycle`이 proof를 내준 뒤, open이 `POLICY_REJECTED` 또는
|
||||
`CORRUPT_DATA`로 실패했을 때(budget/binding 행 손상, 바뀐 policy). 행 단위
|
||||
purge는 open이 필요해서 이 상태에서는 영영 돌 수 없다
|
||||
(`src/adapters/storage/indexeddb/indexeddb-runtime.ts` `deleteUnadmittedDataset`).
|
||||
- 범위: 세 opaque token에서 만든 그 partition의 DB 이름 하나
|
||||
(`indexeddb-governance.ts` `indexedDbPhysicalDatabaseName`). 다른 partition과
|
||||
origin의 다른 DB는 건드리지 않는다.
|
||||
- 결과: 지워지면 receipt `COMPLETE`와 `datasetDeleted: true`(행 수는 세지
|
||||
않는다). 다른 context가 연결을 닫지 않아 blocked deadline이 지나면 `BLOCKED`
|
||||
/ `RELOAD_OTHER_CONTEXTS`를 반환한다. 이때 삭제 요청은 브라우저에 살아 있어
|
||||
나중에 적용될 수 있으므로 다른 tab을 닫고 같은 purge를 다시 호출한다.
|
||||
- governance가 정상이면 지금처럼 행 단위 purge만 하고 DB는 지우지 않는다.
|
||||
|
||||
성공 판정:
|
||||
|
||||
@@ -336,6 +375,11 @@ VD-15 목표 runtime을 먼저 구현한다.
|
||||
- committed logical row와 physical manifest/file 불일치
|
||||
- digest/length mismatch
|
||||
- worker crash, handle lock timeout, `NoModificationAllowedError`
|
||||
- put의 retryable `NOT_READABLE`: source가 `putIdleTimeoutMs`(기본 2분) 동안 다음
|
||||
bytes를 주지 않아 put을 끝내고 origin mutation lock을 풀었다(05-07,
|
||||
`opfs-worker-client.ts` `preparePut`, `opfs-policy.ts`). OPFS 쓰기는 origin
|
||||
전체에서 한 번에 하나라서, 이 bound가 없으면 멈춘 source 하나가 모든 tab의 쓰기를
|
||||
`mutationLockWaitTimeoutMs` 뒤 `BLOCKED`로 만든다
|
||||
- sensitive policy maintenance의 `POLICY_REJECTED`, proof replay/expiry 또는
|
||||
authority provider/consumer unavailable
|
||||
|
||||
@@ -371,12 +415,25 @@ VD-15 목표 runtime을 먼저 구현한다.
|
||||
|
||||
### recovery
|
||||
|
||||
- `PREPARING`: partial staging 검증 후 idempotent resume 또는 purge
|
||||
- `FILES_READY`: expected generation/digest 일치 시 logical commit, 아니면
|
||||
quarantine
|
||||
- `PREPARING`: reconcile은 resume하지 않는다. staging receipt가 가리키는 그
|
||||
transaction의 physical generation과 staging을 지우고 journal row를 rollback한다.
|
||||
receipt가 없거나 0바이트면(첫 receipt 쓰기가 quota나 crash로 끝나지 못함) 지울
|
||||
generation이 없으므로 staging만 지운다(05-02). 읽을 수 없는 receipt는 지우지
|
||||
않고 그 row를 `failedTransactions`로 남긴다
|
||||
(`opfs-byte-store-adapter.ts` `reconcileTransaction`,
|
||||
`opfs-worker-runtime.ts` `cleanupTransactionLocked`)
|
||||
- `FILES_READY`: worker가 manifest와 모든 chunk digest를 다시 검증한다. 맞으면
|
||||
logical commit 뒤 finalize한다. 맞지 않으면 quarantine하지 않고 그 generation과
|
||||
staging을 지운 뒤 rollback한다. 지워지는 것은 commit 전 staging뿐이다
|
||||
- `COMMITTED` + file missing/corrupt: reconstructable만 server rehydrate;
|
||||
user-created private는 read-only/export/recovery
|
||||
- stale staging/orphan chunk는 grace period와 bounded mark/sweep 후 제거
|
||||
- `LOGOUT`/`ACCOUNT_DELETION` purge는 객체를 모두 지운 뒤 그 partition의 chunk를
|
||||
grace 없이 sweep한다(05-04, `opfs-byte-store-adapter.ts` `enforcePolicies`).
|
||||
journal이나 staging receipt가 참조하는 chunk는 이때도 지우지 않는다. report의
|
||||
`chunkSweep`이 `COMPLETED`여야 bytes가 지워진 것이다. `DEADLINE_REACHED`면 같은
|
||||
purge를 다시 호출하고, `STAGING_STATE_UNREADABLE`이면 읽을 수 없는 staging부터
|
||||
처리한다
|
||||
- lock/worker 문제면 OPFS write off → approved capped fallback 또는 online-only
|
||||
- scope binding mismatch면 affected scope를 read-only로 격리하고 registry/
|
||||
composition을 forward fix; 다른 token path로 bytes를 이동하거나 추측 복구 금지
|
||||
|
||||
@@ -31,7 +31,7 @@
|
||||
| 파일 | 책임 | 주요 의존성 / 리뷰 결과 |
|
||||
| --- | --- | --- |
|
||||
| `src/adapters/browser-file-storage/index.ts` | browser data 공통 Result와 StorageManager adapter barrel export | 내부 두 모듈만 export. 경계가 작고 유지 대상. |
|
||||
| `src/adapters/browser-file-storage/result.ts` | native 예외를 closed `BrowserDataFailure`로 정규화하고 안전한 observation 제공 | `application/ports/browser-file-storage/shared.ts`; raw path/name/message 비노출, observer 예외 격리가 좋다. |
|
||||
| `src/adapters/browser-file-storage/result.ts` | native 예외를 closed `BrowserDataFailure`로 정규화하고 안전한 observation 제공. IndexedDB 예외 표 `mapIndexedDbException`도 여기 있다(05-11, 옛 `storage/indexeddb/indexeddb-failure.ts`) | `application/ports/browser-file-storage/shared.ts`; raw path/name/message 비노출, observer 예외 격리가 좋다. |
|
||||
| `src/adapters/browser-file-storage/storage-manager-adapter.ts` | `estimate/persisted/persist` snapshot, pressure bucket, user-activation-bound persistence 요청 | storage durability port/result. estimate를 예약량으로 오인하지 않고 irreversible `persist()` truth를 보존한다. origin coordinator는 의도적으로 없음(GAP-03). |
|
||||
|
||||
### 1.2 `browser-files`
|
||||
@@ -66,7 +66,6 @@
|
||||
| `src/adapters/storage/browser-storage-adapter.ts` | registry key별 local/session/memory 저장, TTL, failure overlay/tombstone, quota fallback | `storage-keys`, `StoragePort`, codec, diagnostics. strict registry와 stale persistent suppression을 유지. adjacent physical-key migration/sweep는 문서상 미구현. |
|
||||
| `src/adapters/storage/browser-storage-codec.ts` | bounded exact JSON envelope, exotic/accessor/unsafe-key/cycle/depth/node 거절 | 독립 codec. prototype pollution/JSON silent coercion 방어가 좋다. |
|
||||
| `src/adapters/storage/indexeddb/index.ts` | IndexedDB runtime/maintenance/governance/migration export | native IDB type을 application port 밖으로 내보내지 않는 구조 유지. |
|
||||
| `src/adapters/storage/indexeddb/indexeddb-failure.ts` | IDB/DOM failure를 closed browser failure로 변환 | common Result. raw native detail 비노출 유지. |
|
||||
| `src/adapters/storage/indexeddb/indexeddb-governance.ts` | opaque dataset scope/physical DB identity 및 frozen policy binding | indexeddb/shared ports. account/business ID를 physical name에 쓰지 않는 양방향 binding 유지. |
|
||||
| `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | post-open codec migration 및 idempotency receipt prune, keyset checkpoint, budget/revision fencing | IndexedDB port/types/failure/governance. async transform outside tx, row+sidecar+budget+checkpoint atomic commit은 좋다. STO-06 및 temporal drain lease 개선 후보. |
|
||||
| `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | additive-only contiguous DDL planner/validator | indexeddb types. destructive DDL 거절 유지. |
|
||||
|
||||
@@ -122,38 +122,37 @@
|
||||
| 112 | `src/adapters/storage/browser-storage-codec.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 113 | `src/adapters/storage/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 114 | `src/adapters/storage/indexeddb/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 115 | `src/adapters/storage/indexeddb/indexeddb-failure.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 116 | `src/adapters/storage/indexeddb/indexeddb-governance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 117 | `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 118 | `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 119 | `src/adapters/storage/indexeddb/indexeddb-records.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 120 | `src/adapters/storage/indexeddb/indexeddb-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 121 | `src/adapters/storage/indexeddb/indexeddb-types.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 122 | `src/adapters/storage/opfs/browser-opfs-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 123 | `src/adapters/storage/opfs/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 124 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 125 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 126 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 127 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 128 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 129 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 130 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 131 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 132 | `src/adapters/telemetry/feature-telemetry.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 133 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 134 | `src/adapters/telemetry/telemetry-collector.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 135 | `src/adapters/web-push/codec-primitives.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 136 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 137 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 138 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 139 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 140 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 141 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 142 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 143 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 144 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 145 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 146 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 115 | `src/adapters/storage/indexeddb/indexeddb-governance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 116 | `src/adapters/storage/indexeddb/indexeddb-maintenance.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 117 | `src/adapters/storage/indexeddb/indexeddb-migrations.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 118 | `src/adapters/storage/indexeddb/indexeddb-records.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 119 | `src/adapters/storage/indexeddb/indexeddb-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 120 | `src/adapters/storage/indexeddb/indexeddb-types.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 121 | `src/adapters/storage/opfs/browser-opfs-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 122 | `src/adapters/storage/opfs/index.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 123 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 124 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 125 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 126 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 127 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 128 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 129 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 130 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 131 | `src/adapters/telemetry/feature-telemetry.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 132 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 133 | `src/adapters/telemetry/telemetry-collector.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 134 | `src/adapters/web-push/codec-primitives.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 135 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 136 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 137 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 138 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 139 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 140 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 141 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 142 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 143 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 144 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 145 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
|
||||
합계: **146/146**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
|
||||
합계: **145/145**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
|
||||
|
||||
|
||||
@@ -40,6 +40,12 @@ export function abortedResult(
|
||||
: null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The table for File, Storage Manager, Cache Storage and download DOM errors.
|
||||
* IndexedDB errors use `mapIndexedDbException` below: the same exception name
|
||||
* means something else there (a `NotFoundError` is a missing store, not a
|
||||
* missing file).
|
||||
*/
|
||||
export function mapBrowserDataException(
|
||||
error: unknown,
|
||||
operation: BrowserDataOperation,
|
||||
@@ -89,6 +95,81 @@ export function mapBrowserDataException(
|
||||
}
|
||||
}
|
||||
|
||||
function exceptionName(error: unknown): string {
|
||||
if (
|
||||
error &&
|
||||
typeof error === "object" &&
|
||||
"name" in error &&
|
||||
typeof error.name === "string"
|
||||
) {
|
||||
return error.name;
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
/**
|
||||
* 05-11. The one table for native IndexedDB errors. It lives in this kernel
|
||||
* exception file so the OPFS journal no longer reaches into the IndexedDB
|
||||
* runtime's folder for it. It maps the closed DOMException vocabulary without
|
||||
* exposing an exception object, message, key or stored value across the
|
||||
* adapter boundary.
|
||||
*
|
||||
* Differences from `mapBrowserDataException`, on purpose: a missing store or
|
||||
* index, an inactive or read-only transaction and a `VersionError` are schema
|
||||
* problems (`MIGRATION_FAILED`, READ_ONLY), not a missing file or a transient
|
||||
* failure; quota and constraint failures are not retried blindly.
|
||||
*/
|
||||
export function mapIndexedDbException(
|
||||
error: unknown,
|
||||
operation: BrowserDataOperation,
|
||||
): BrowserDataResult<never> {
|
||||
switch (exceptionName(error)) {
|
||||
case "AbortError":
|
||||
return browserDataFailure("ABORTED", operation);
|
||||
case "ConstraintError":
|
||||
return browserDataFailure("CONFLICT", operation);
|
||||
case "DataCloneError":
|
||||
case "DataError":
|
||||
return browserDataFailure("CORRUPT_DATA", operation, {
|
||||
recovery: "READ_ONLY",
|
||||
});
|
||||
case "InvalidAccessError":
|
||||
case "InvalidStateError":
|
||||
case "NotFoundError":
|
||||
case "ReadOnlyError":
|
||||
case "TransactionInactiveError":
|
||||
case "VersionError":
|
||||
return browserDataFailure("MIGRATION_FAILED", operation, {
|
||||
recovery: "READ_ONLY",
|
||||
});
|
||||
case "NotAllowedError":
|
||||
case "SecurityError":
|
||||
return browserDataFailure("PERMISSION_DENIED", operation, {
|
||||
recovery: "ONLINE_ONLY",
|
||||
});
|
||||
case "NotReadableError":
|
||||
return browserDataFailure("NOT_READABLE", operation, {
|
||||
retryable: true,
|
||||
recovery: "REOPEN",
|
||||
});
|
||||
case "QuotaExceededError":
|
||||
case "NS_ERROR_DOM_QUOTA_REACHED":
|
||||
return browserDataFailure("QUOTA_EXCEEDED", operation, {
|
||||
recovery: "READ_ONLY",
|
||||
});
|
||||
case "UnknownError":
|
||||
return browserDataFailure("UNAVAILABLE", operation, {
|
||||
retryable: true,
|
||||
recovery: "REOPEN",
|
||||
});
|
||||
default:
|
||||
return browserDataFailure("UNAVAILABLE", operation, {
|
||||
retryable: true,
|
||||
recovery: "RETRY",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export function observeBrowserData(
|
||||
observer: BrowserDataObserver | undefined,
|
||||
observation: BrowserDataObservation,
|
||||
|
||||
@@ -427,6 +427,16 @@ export function openIndexedDbDatabase<Failure>(
|
||||
settle(succeeded(database));
|
||||
return;
|
||||
}
|
||||
// 05-13. Admission is asynchronous and nothing else answers another
|
||||
// context's upgrade until the caller adopts the connection, so that
|
||||
// upgrade would wait blocked until its deadline. While admission runs, a
|
||||
// versionchange closes this connection and ends the open as CLOSED. An
|
||||
// `admit` that installs its own handler replaces this one.
|
||||
database.onversionchange = () => {
|
||||
if (settled) return;
|
||||
closeQuietly(database);
|
||||
settleFailure({ kind: "CLOSED" });
|
||||
};
|
||||
let admission: IndexedDbAdmission | Promise<IndexedDbAdmission>;
|
||||
try {
|
||||
admission = input.admit(database);
|
||||
@@ -605,9 +615,9 @@ export function createIndexedDbConnection<Failure>(
|
||||
|
||||
/**
|
||||
* In this module because it is the same `IDBOpenDBRequest` state machine as
|
||||
* `openIndexedDbDatabase`, not because three subsystems need it — only
|
||||
* `indexeddb-checkpoint-store.ts` deletes a database, and nothing here asks the
|
||||
* other three to start. Leaving it out would leave a third hand-written copy of
|
||||
* `openIndexedDbDatabase`. Two callers delete a database:
|
||||
* `indexeddb-checkpoint-store.ts`, and the IndexedDB runtime's authorized
|
||||
* LOGOUT/ACCOUNT_DELETION purge when its dataset fails admission. Leaving it out would leave a third hand-written copy of
|
||||
* the settle-once blocked-deadline latch eight lines away from the kernel's.
|
||||
*
|
||||
* There is deliberately no `signal`: the request cannot be cancelled after
|
||||
|
||||
@@ -2,9 +2,11 @@
|
||||
* 최상위 storage 배럴은 Web Storage 어댑터만 내보낸다.
|
||||
*
|
||||
* `./indexeddb/index.ts`와 `./opfs/index.ts`를 여기서 재수출하지 말 것.
|
||||
* `scripts/test-browser-file-storage-runtime-removal.ts:23-34`가 그 두 폴더만
|
||||
* 삭제한 뒤 잔존 import를 예외로 잡는다 — 재수출하면 제거 드릴이 죽는다.
|
||||
* 두 서브배럴이 각자 제거 가능한 런타임의 경계다.
|
||||
* `scripts/test-browser-file-storage-runtime-removal.ts`의 `runtimePaths`는 이 두
|
||||
* 폴더를 browser-file-storage 번들 전체(ports, browser-files, browser-transfer,
|
||||
* cache-storage 등)와 함께 삭제한 뒤 잔존 import를 예외로 잡는다 — 재수출하면
|
||||
* 제거 드릴이 죽는다. 드릴은 번들을 통째로만 지운다. 두 폴더 중 하나만 지우는
|
||||
* 경우는 드릴이 검증하지 않는다.
|
||||
*/
|
||||
export {
|
||||
createBrowserStorageAdapter,
|
||||
|
||||
@@ -1,72 +0,0 @@
|
||||
import type {
|
||||
BrowserDataOperation,
|
||||
BrowserDataResult,
|
||||
} from "../../../application/ports/browser-file-storage/shared.ts";
|
||||
import { browserDataFailure } from "../../browser-file-storage/result.ts";
|
||||
|
||||
function exceptionName(error: unknown): string {
|
||||
if (
|
||||
error &&
|
||||
typeof error === "object" &&
|
||||
"name" in error &&
|
||||
typeof error.name === "string"
|
||||
) {
|
||||
return error.name;
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Maps the closed DOMException vocabulary without exposing an exception
|
||||
* object, message, key or stored value across the adapter boundary.
|
||||
*/
|
||||
export function mapIndexedDbException(
|
||||
error: unknown,
|
||||
operation: BrowserDataOperation,
|
||||
): BrowserDataResult<never> {
|
||||
switch (exceptionName(error)) {
|
||||
case "AbortError":
|
||||
return browserDataFailure("ABORTED", operation);
|
||||
case "ConstraintError":
|
||||
return browserDataFailure("CONFLICT", operation);
|
||||
case "DataCloneError":
|
||||
case "DataError":
|
||||
return browserDataFailure("CORRUPT_DATA", operation, {
|
||||
recovery: "READ_ONLY",
|
||||
});
|
||||
case "InvalidAccessError":
|
||||
case "InvalidStateError":
|
||||
case "NotFoundError":
|
||||
case "ReadOnlyError":
|
||||
case "TransactionInactiveError":
|
||||
case "VersionError":
|
||||
return browserDataFailure("MIGRATION_FAILED", operation, {
|
||||
recovery: "READ_ONLY",
|
||||
});
|
||||
case "NotAllowedError":
|
||||
case "SecurityError":
|
||||
return browserDataFailure("PERMISSION_DENIED", operation, {
|
||||
recovery: "ONLINE_ONLY",
|
||||
});
|
||||
case "NotReadableError":
|
||||
return browserDataFailure("NOT_READABLE", operation, {
|
||||
retryable: true,
|
||||
recovery: "REOPEN",
|
||||
});
|
||||
case "QuotaExceededError":
|
||||
case "NS_ERROR_DOM_QUOTA_REACHED":
|
||||
return browserDataFailure("QUOTA_EXCEEDED", operation, {
|
||||
recovery: "READ_ONLY",
|
||||
});
|
||||
case "UnknownError":
|
||||
return browserDataFailure("UNAVAILABLE", operation, {
|
||||
retryable: true,
|
||||
recovery: "REOPEN",
|
||||
});
|
||||
default:
|
||||
return browserDataFailure("UNAVAILABLE", operation, {
|
||||
retryable: true,
|
||||
recovery: "RETRY",
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,10 @@ import {
|
||||
assertValidStoragePolicy,
|
||||
type BrowserStoragePolicy,
|
||||
} from "../../../application/ports/browser-file-storage/shared.ts";
|
||||
import {
|
||||
onIndexedDbRequest,
|
||||
runIndexedDbTransaction,
|
||||
} from "../../platform/indexeddb-transaction.ts";
|
||||
import {
|
||||
INDEXEDDB_DATASET_BUDGET_KEY,
|
||||
isStoredDatasetBudget,
|
||||
@@ -219,105 +223,72 @@ export function queueIndexedDbUpgradeBinding(
|
||||
|
||||
/**
|
||||
* Post-open verification protects non-upgrade opens and maintenance callers.
|
||||
* 05-12. It runs on the kernel transaction, so settle-once, caller abort and
|
||||
* the abort-after-commit rule (CP-4) are the kernel's, not a local copy's.
|
||||
*/
|
||||
export function verifyIndexedDbDatasetBinding(
|
||||
export async function verifyIndexedDbDatasetBinding(
|
||||
database: IDBDatabase,
|
||||
governanceStore: string,
|
||||
expected: StoredDatasetBinding,
|
||||
signal?: AbortSignal,
|
||||
): Promise<IndexedDbBindingVerification> {
|
||||
if (signal?.aborted) {
|
||||
return Promise.resolve({ ok: false, reason: "ABORTED" });
|
||||
}
|
||||
let transaction: IDBTransaction;
|
||||
try {
|
||||
transaction = database.transaction(governanceStore, "readonly");
|
||||
} catch (error) {
|
||||
return Promise.resolve({
|
||||
ok: false,
|
||||
reason: "NATIVE_ERROR",
|
||||
error,
|
||||
});
|
||||
}
|
||||
|
||||
return new Promise((resolve) => {
|
||||
let settled = false;
|
||||
let observed: unknown;
|
||||
let observedBudget: unknown;
|
||||
let requestError: unknown;
|
||||
let callerAborted = false;
|
||||
const finish = (result: IndexedDbBindingVerification) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
signal?.removeEventListener("abort", onAbort);
|
||||
resolve(result);
|
||||
};
|
||||
function onAbort(): void {
|
||||
callerAborted = true;
|
||||
try {
|
||||
transaction.abort();
|
||||
} catch {
|
||||
// Completion determines the race.
|
||||
}
|
||||
}
|
||||
signal?.addEventListener("abort", onAbort, { once: true });
|
||||
|
||||
transaction.onerror = () => {
|
||||
requestError ??= transaction.error;
|
||||
};
|
||||
transaction.onabort = () =>
|
||||
finish(
|
||||
callerAborted
|
||||
? { ok: false, reason: "ABORTED" }
|
||||
: {
|
||||
ok: false,
|
||||
reason: "NATIVE_ERROR",
|
||||
error: requestError ?? transaction.error,
|
||||
},
|
||||
const outcome = await runIndexedDbTransaction<
|
||||
IndexedDbBindingVerification,
|
||||
BindingFailure
|
||||
>({
|
||||
database,
|
||||
stores: [governanceStore],
|
||||
mode: "readonly",
|
||||
signal,
|
||||
translate: (cause) =>
|
||||
cause.kind === "CALLER_ABORT"
|
||||
? { ok: false, reason: "ABORTED" }
|
||||
: {
|
||||
ok: false,
|
||||
reason: "NATIVE_ERROR",
|
||||
error: cause.kind === "NATIVE_EXCEPTION" ? cause.error : undefined,
|
||||
},
|
||||
queue: (transaction, context) => {
|
||||
const store = transaction.objectStore(governanceStore);
|
||||
let observed: unknown;
|
||||
onIndexedDbRequest<unknown, BindingFailure>(
|
||||
store.get(INDEXEDDB_DATASET_BINDING_KEY),
|
||||
context,
|
||||
(value) => {
|
||||
observed = value;
|
||||
},
|
||||
);
|
||||
transaction.oncomplete = () => {
|
||||
if (observed === undefined) {
|
||||
finish({ ok: false, reason: "MISSING" });
|
||||
} else if (!isStoredDatasetBinding(observed)) {
|
||||
finish({ ok: false, reason: "CORRUPT" });
|
||||
} else if (!sameIndexedDbDatasetBinding(observed, expected)) {
|
||||
finish({ ok: false, reason: "MISMATCH" });
|
||||
} else if (
|
||||
!isStoredDatasetBudget(observedBudget) ||
|
||||
observedBudget.usedBytes > expected.storagePolicy.hardBudgetBytes
|
||||
) {
|
||||
finish({ ok: false, reason: "CORRUPT" });
|
||||
} else {
|
||||
finish({ ok: true });
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
const request = transaction
|
||||
.objectStore(governanceStore)
|
||||
.get(INDEXEDDB_DATASET_BINDING_KEY);
|
||||
request.onerror = () => {
|
||||
requestError ??= request.error;
|
||||
};
|
||||
request.onsuccess = () => {
|
||||
observed = request.result;
|
||||
};
|
||||
const budgetRequest = transaction
|
||||
.objectStore(governanceStore)
|
||||
.get(INDEXEDDB_DATASET_BUDGET_KEY);
|
||||
budgetRequest.onerror = () => {
|
||||
requestError ??= budgetRequest.error;
|
||||
};
|
||||
budgetRequest.onsuccess = () => {
|
||||
observedBudget = budgetRequest.result;
|
||||
};
|
||||
} catch (error) {
|
||||
requestError = error;
|
||||
try {
|
||||
transaction.abort();
|
||||
} catch {
|
||||
finish({ ok: false, reason: "NATIVE_ERROR", error });
|
||||
}
|
||||
}
|
||||
onIndexedDbRequest<unknown, BindingFailure>(
|
||||
store.get(INDEXEDDB_DATASET_BUDGET_KEY),
|
||||
context,
|
||||
(budget) => {
|
||||
context.succeed(bindingVerdict(observed, budget, expected));
|
||||
},
|
||||
);
|
||||
},
|
||||
});
|
||||
return outcome.ok ? outcome.value : outcome.error;
|
||||
}
|
||||
|
||||
type BindingFailure = Extract<IndexedDbBindingVerification, { ok: false }>;
|
||||
|
||||
function bindingVerdict(
|
||||
observed: unknown,
|
||||
budget: unknown,
|
||||
expected: StoredDatasetBinding,
|
||||
): IndexedDbBindingVerification {
|
||||
if (observed === undefined) return { ok: false, reason: "MISSING" };
|
||||
if (!isStoredDatasetBinding(observed)) {
|
||||
return { ok: false, reason: "CORRUPT" };
|
||||
}
|
||||
if (!sameIndexedDbDatasetBinding(observed, expected)) {
|
||||
return { ok: false, reason: "MISMATCH" };
|
||||
}
|
||||
if (
|
||||
!isStoredDatasetBudget(budget) ||
|
||||
budget.usedBytes > expected.storagePolicy.hardBudgetBytes
|
||||
) {
|
||||
return { ok: false, reason: "CORRUPT" };
|
||||
}
|
||||
return { ok: true };
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
abortedResult,
|
||||
browserDataFailure,
|
||||
browserDataSuccess,
|
||||
mapIndexedDbException,
|
||||
} from "../../browser-file-storage/result.ts";
|
||||
import {
|
||||
openIndexedDbDatabase,
|
||||
@@ -24,7 +25,6 @@ import {
|
||||
type IndexedDbBudget,
|
||||
type IndexedDbTransactionContext,
|
||||
} from "../../platform/indexeddb-transaction.ts";
|
||||
import { mapIndexedDbException } from "./indexeddb-failure.ts";
|
||||
import {
|
||||
createIndexedDbDatasetBinding,
|
||||
indexedDbPhysicalDatabaseName,
|
||||
@@ -146,6 +146,14 @@ function migrationFailure(): BrowserDataFailure {
|
||||
return failureOf(migrationFailed());
|
||||
}
|
||||
|
||||
function corruptFailure(): BrowserDataFailure {
|
||||
return failureOf(
|
||||
browserDataFailure("CORRUPT_DATA", "INDEXEDDB_MIGRATE", {
|
||||
recovery: "READ_ONLY",
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
function unsupportedFailure(): BrowserDataFailure {
|
||||
return failureOf(unsupported());
|
||||
}
|
||||
@@ -314,10 +322,9 @@ export function createIndexedDbMaintenance<WireValue>(
|
||||
}
|
||||
|
||||
/**
|
||||
* STO-06. Maintenance keeps `mapIndexedDbException`, which the runtime and
|
||||
* the OPFS journal share, rather than the checkpoint store's own table: the
|
||||
* same native error is a different answer in the two families, and unifying
|
||||
* them is a separate change from moving the mechanics onto the kernel.
|
||||
* STO-06. Maintenance uses `mapIndexedDbException`, the one IndexedDB table
|
||||
* in `browser-file-storage/result.ts` that the runtime and the OPFS journal
|
||||
* share (05-11).
|
||||
*
|
||||
* One translator covers the whole adapter because the operation label is the
|
||||
* constant `INDEXEDDB_MIGRATE` here — the runtime needs one per operation.
|
||||
@@ -1124,6 +1131,11 @@ export function createIndexedDbMaintenance<WireValue>(
|
||||
const opened = await openExactVersion(input.signal);
|
||||
if (!opened.ok) return observeResult(opened);
|
||||
const database = opened.value;
|
||||
// 05-05. A corrupt receipt is left in place and counted, not a stop: one
|
||||
// bad row used to end every later prune, so receipts piled up until every
|
||||
// write failed with LIMIT_EXCEEDED. A receipt count already at zero stays
|
||||
// there. Both are reported as one CORRUPT_DATA observation.
|
||||
let corruptRows = 0;
|
||||
|
||||
try {
|
||||
const pruned = await runTransaction<
|
||||
@@ -1188,8 +1200,8 @@ export function createIndexedDbMaintenance<WireValue>(
|
||||
String(cursor.primaryKey) ||
|
||||
cursor.value.expiresAtEpochMs > cutoff.value
|
||||
) {
|
||||
context.fail(migrationFailure());
|
||||
return { kind: "SUSPEND" };
|
||||
corruptRows += 1;
|
||||
return { kind: "CONTINUE" };
|
||||
}
|
||||
let deleteRequest: IDBRequest<undefined>;
|
||||
try {
|
||||
@@ -1207,11 +1219,14 @@ export function createIndexedDbMaintenance<WireValue>(
|
||||
budgetRequest.onerror = () =>
|
||||
context.requestFailed(budgetRequest.error);
|
||||
budgetRequest.onsuccess = () => {
|
||||
if (
|
||||
!isStoredDatasetBudget(budgetRequest.result) ||
|
||||
budgetRequest.result.receiptCount < 1
|
||||
) {
|
||||
context.fail(migrationFailure());
|
||||
if (!isStoredDatasetBudget(budgetRequest.result)) {
|
||||
context.fail(corruptFailure());
|
||||
return;
|
||||
}
|
||||
if (budgetRequest.result.receiptCount < 1) {
|
||||
corruptRows += 1;
|
||||
deletedRows += 1;
|
||||
resume({ kind: "CONTINUE" });
|
||||
return;
|
||||
}
|
||||
const budgetWrite = governance.put({
|
||||
@@ -1248,6 +1263,15 @@ export function createIndexedDbMaintenance<WireValue>(
|
||||
});
|
||||
},
|
||||
);
|
||||
if (corruptRows > 0) {
|
||||
observe({
|
||||
operation: "INDEXEDDB_MIGRATE",
|
||||
outcome: "FAILED",
|
||||
schemaVersion: dependencies.schemaVersion,
|
||||
countBucket: countBucket(corruptRows),
|
||||
failureCode: "CORRUPT_DATA",
|
||||
});
|
||||
}
|
||||
return observeResult(
|
||||
pruned,
|
||||
pruned.ok ? pruned.value.deletedRows : 0,
|
||||
|
||||
@@ -20,10 +20,12 @@ import {
|
||||
abortedResult,
|
||||
browserDataFailure,
|
||||
browserDataSuccess,
|
||||
mapIndexedDbException,
|
||||
} from "../../browser-file-storage/result.ts";
|
||||
import { snapshotAbortTimers } from "../../platform/abortable-operation.ts";
|
||||
import {
|
||||
createIndexedDbConnection,
|
||||
deleteIndexedDbDatabase,
|
||||
openIndexedDbDatabase,
|
||||
type IndexedDbTranslate,
|
||||
} from "../../platform/indexeddb-connection.ts";
|
||||
@@ -35,7 +37,6 @@ import {
|
||||
type IndexedDbCursorVisit,
|
||||
type IndexedDbTransactionContext,
|
||||
} from "../../platform/indexeddb-transaction.ts";
|
||||
import { mapIndexedDbException } from "./indexeddb-failure.ts";
|
||||
import {
|
||||
createIndexedDbDatasetBinding,
|
||||
indexedDbPhysicalDatabaseName,
|
||||
@@ -284,6 +285,14 @@ function unavailable(
|
||||
});
|
||||
}
|
||||
|
||||
function incompatible(
|
||||
operation: BrowserDataOperation,
|
||||
): BrowserDataResult<never> {
|
||||
return browserDataFailure("UNSUPPORTED", operation, {
|
||||
recovery: "ONLINE_ONLY",
|
||||
});
|
||||
}
|
||||
|
||||
function mappedFailure(
|
||||
error: unknown,
|
||||
operation: BrowserDataOperation,
|
||||
@@ -550,10 +559,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
}
|
||||
|
||||
/**
|
||||
* STO-06. The runtime keeps `mapIndexedDbException`, which maintenance and
|
||||
* the OPFS journal share, rather than the checkpoint store's own table: the
|
||||
* same native error is a different answer in the two families, and unifying
|
||||
* them is a separate change from moving the mechanics onto the kernel.
|
||||
* STO-06. The runtime uses `mapIndexedDbException`, the one IndexedDB table
|
||||
* in `browser-file-storage/result.ts` that maintenance and the OPFS journal
|
||||
* share (05-11).
|
||||
*
|
||||
* A translator per operation, because a read, a write and an open carry
|
||||
* different labels. `CLOSED` is mapped explicitly: closing the handle while
|
||||
@@ -642,6 +650,7 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
// so the migration count and the binding verdict belong to the attempt.
|
||||
let appliedMigrations = 0;
|
||||
let policyBindingRejected = false;
|
||||
let futureSchema = false;
|
||||
updateStatus({
|
||||
kind: "OPENING",
|
||||
targetVersion: dependencies.schemaVersion,
|
||||
@@ -650,7 +659,20 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
factory,
|
||||
databaseName,
|
||||
version: dependencies.schemaVersion,
|
||||
translate: translateFor(operation),
|
||||
translate: (cause) => {
|
||||
// 05-06. A VersionError on open means a newer release already
|
||||
// upgraded this database. That is not a failed migration: this
|
||||
// bundle can only work online until it is reloaded.
|
||||
if (
|
||||
cause.kind === "NATIVE_EXCEPTION" &&
|
||||
(cause.error as { name?: unknown } | null)?.name ===
|
||||
"VersionError"
|
||||
) {
|
||||
futureSchema = true;
|
||||
return failureOf(incompatible(operation));
|
||||
}
|
||||
return translateFor(operation)(cause);
|
||||
},
|
||||
signal,
|
||||
blockedTimeoutMs,
|
||||
timers,
|
||||
@@ -743,7 +765,14 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
: opened;
|
||||
if (disposed) return result;
|
||||
if (!result.ok) {
|
||||
updateStatus({ kind: "CLOSED", reason: "NOT_OPENED" });
|
||||
updateStatus(
|
||||
futureSchema
|
||||
? {
|
||||
kind: "INCOMPATIBLE",
|
||||
schemaVersion: dependencies.schemaVersion,
|
||||
}
|
||||
: { kind: "CLOSED", reason: "NOT_OPENED" },
|
||||
);
|
||||
// A failed migration is observed against INDEXEDDB_MIGRATE as well
|
||||
// as against the open the caller asked for.
|
||||
if (result.error.operation === "INDEXEDDB_MIGRATE") {
|
||||
@@ -833,6 +862,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
const cancelled = abortedResult(signal, operation);
|
||||
if (cancelled) return Promise.resolve(cancelled);
|
||||
const db = connection.current();
|
||||
if (!db && !disposed && status.kind === "INCOMPATIBLE") {
|
||||
return Promise.resolve(incompatible(operation));
|
||||
}
|
||||
if (!db || disposed) return Promise.resolve(unavailable(operation));
|
||||
|
||||
return runIndexedDbTransaction<ResultValue, BrowserDataFailure>({
|
||||
@@ -2044,12 +2076,21 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
return browserDataSuccess(undefined);
|
||||
}
|
||||
|
||||
function purgeEligibleRecords(
|
||||
/**
|
||||
* 05-05. Like the partition purge (R-5.4), the sweep never stops on a row it
|
||||
* cannot read: failing closed left every later expired row on the device and
|
||||
* failed the next sweep the same way. A corrupt sidecar is left in place,
|
||||
* counted and reported as one CORRUPT_DATA observation per batch. A budget
|
||||
* that cannot account for a deleted row's bytes is left as it is: it can
|
||||
* only read high, which blocks writes and never admits one.
|
||||
*/
|
||||
async function purgeEligibleRecords(
|
||||
input: IndexedDbLifecycleBatchInput,
|
||||
deadline: number,
|
||||
cutoffEpochMs: number,
|
||||
): Promise<BrowserDataResult<IndexedDbLifecycleBatchReceipt>> {
|
||||
return runTransaction(
|
||||
let corruptRows = 0;
|
||||
const result = await runTransaction<IndexedDbLifecycleBatchReceipt>(
|
||||
[
|
||||
dependencies.recordStore,
|
||||
dependencies.governanceStore,
|
||||
@@ -2140,11 +2181,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
"UNTIL_SYNCED" &&
|
||||
cursor.value.synchronization !== "CONFIRMED")
|
||||
) {
|
||||
context.fail(corruptFailure("INDEXEDDB_WRITE"));
|
||||
// `fail` aborts, so the pump has nowhere to go. Suspending
|
||||
// without ever resuming says that without claiming a summary
|
||||
// the sweep never reached.
|
||||
return { kind: "SUSPEND" };
|
||||
scannedRows += 1;
|
||||
corruptRows += 1;
|
||||
return { kind: "CONTINUE" };
|
||||
}
|
||||
scannedRows += 1;
|
||||
const measuredBytes = cursor.value.measuredBytes;
|
||||
@@ -2176,13 +2215,16 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
budgetRequest.onerror = () =>
|
||||
context.requestFailed(budgetRequest.error);
|
||||
budgetRequest.onsuccess = () => {
|
||||
if (
|
||||
!isStoredDatasetBudget(budgetRequest.result) ||
|
||||
budgetRequest.result.usedBytes < measuredBytes
|
||||
) {
|
||||
if (!isStoredDatasetBudget(budgetRequest.result)) {
|
||||
context.fail(corruptFailure("INDEXEDDB_WRITE"));
|
||||
return;
|
||||
}
|
||||
if (budgetRequest.result.usedBytes < measuredBytes) {
|
||||
corruptRows += 1;
|
||||
deletedRows += 1;
|
||||
resume({ kind: "CONTINUE" });
|
||||
return;
|
||||
}
|
||||
const storedBudgetRequest = governance.put({
|
||||
bindingKey: INDEXEDDB_DATASET_BUDGET_KEY,
|
||||
budgetVersion: 1,
|
||||
@@ -2215,6 +2257,16 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
});
|
||||
},
|
||||
);
|
||||
if (corruptRows > 0) {
|
||||
observe({
|
||||
operation: "INDEXEDDB_WRITE",
|
||||
outcome: "FAILED",
|
||||
schemaVersion: dependencies.schemaVersion,
|
||||
countBucket: countBucket(corruptRows),
|
||||
failureCode: "CORRUPT_DATA",
|
||||
});
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -2500,7 +2552,9 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
cutoffEpochMs,
|
||||
);
|
||||
} else {
|
||||
result = await purgePartitionRecords(input, deadline);
|
||||
result =
|
||||
(await deleteUnadmittedDataset(input.signal)) ??
|
||||
(await purgePartitionRecords(input, deadline));
|
||||
}
|
||||
return observeResult(
|
||||
operation,
|
||||
@@ -2509,6 +2563,54 @@ export function createIndexedDbRuntime<Value, WireValue, Query>(
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* 05-03. A LOGOUT or ACCOUNT_DELETION purge must not depend on the dataset
|
||||
* passing admission: a corrupt governance row rejects every open, so no row
|
||||
* could ever be purged one by one. The authority has already approved this
|
||||
* purge, and the database name comes from this partition's own tokens, so
|
||||
* deleting the whole database touches no other partition. Returns `null`
|
||||
* when a connection is ready for the row-by-row purge.
|
||||
*/
|
||||
async function deleteUnadmittedDataset(
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<BrowserDataResult<IndexedDbLifecycleBatchReceipt> | null> {
|
||||
if (connection.current()) return null;
|
||||
const opened = await open(signal);
|
||||
if (opened.ok) return null;
|
||||
if (
|
||||
(opened.error.code !== "POLICY_REJECTED" &&
|
||||
opened.error.code !== "CORRUPT_DATA") ||
|
||||
disposed ||
|
||||
typeof factory?.deleteDatabase !== "function"
|
||||
) {
|
||||
return opened;
|
||||
}
|
||||
const deleted = await deleteIndexedDbDatabase<BrowserDataFailure>({
|
||||
factory,
|
||||
databaseName,
|
||||
translate: translateFor("INDEXEDDB_WRITE"),
|
||||
blockedTimeoutMs,
|
||||
timers,
|
||||
});
|
||||
if (!deleted.ok) return deleted;
|
||||
if (deleted.value.kind === "BLOCKED_DEADLINE") {
|
||||
// The request is still live and may yet land; the next call finds out.
|
||||
return browserDataFailure("BLOCKED", "INDEXEDDB_WRITE", {
|
||||
retryable: true,
|
||||
recovery: "RELOAD_OTHER_CONTEXTS",
|
||||
});
|
||||
}
|
||||
return browserDataSuccess(
|
||||
Object.freeze({
|
||||
state: "COMPLETE" as const,
|
||||
scannedRows: 0,
|
||||
deletedRows: 0,
|
||||
budgetExhausted: false,
|
||||
datasetDeleted: true as const,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
function close(): void {
|
||||
if (disposed) return;
|
||||
disposed = true;
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
import {
|
||||
browserDataFailure,
|
||||
browserDataSuccess,
|
||||
mapIndexedDbException,
|
||||
} from "../../browser-file-storage/result.ts";
|
||||
import { snapshotAbortTimers } from "../../platform/abortable-operation.ts";
|
||||
import {
|
||||
@@ -31,9 +32,8 @@ import {
|
||||
type IndexedDbRequestSink,
|
||||
type IndexedDbTransactionContext,
|
||||
} from "../../platform/indexeddb-transaction.ts";
|
||||
import { mapIndexedDbException } from "../indexeddb/indexeddb-failure.ts";
|
||||
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
|
||||
import { isPreparedObject } from "./opfs-prepared-object.ts";
|
||||
import { isPreparedObject, stableJson } from "./opfs-prepared-object.ts";
|
||||
|
||||
export type IndexedDbOpfsJournalDependencies = Readonly<{
|
||||
authorityToken: string;
|
||||
@@ -1608,28 +1608,6 @@ function isChunkReference(
|
||||
);
|
||||
}
|
||||
|
||||
function stableJson(value: unknown): string {
|
||||
if (
|
||||
value === null ||
|
||||
typeof value === "string" ||
|
||||
typeof value === "boolean" ||
|
||||
typeof value === "number"
|
||||
) {
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
return `[${value.map(stableJson).join(",")}]`;
|
||||
}
|
||||
if (typeof value === "object") {
|
||||
const record = value as Record<string, unknown>;
|
||||
return `{${Object.keys(record)
|
||||
.sort()
|
||||
.map((key) => `${JSON.stringify(key)}:${stableJson(record[key])}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
throw new TypeError("Journal value is not JSON-safe.");
|
||||
}
|
||||
|
||||
/**
|
||||
* `browserDataFailure` and `mapIndexedDbException` build a `Result`, while the
|
||||
* kernel's `translate` and `context.fail` want the failure on its own. Both
|
||||
@@ -1698,10 +1676,9 @@ function limitExceeded(
|
||||
}
|
||||
|
||||
/**
|
||||
* STO-01. The journal keeps `mapIndexedDbException`, which the IndexedDB
|
||||
* runtime and maintenance share, rather than the checkpoint store's own table:
|
||||
* the same native error is a different answer in the two families and unifying
|
||||
* them is a separate change from moving the mechanics onto the kernel.
|
||||
* STO-01. The journal uses `mapIndexedDbException`, the one IndexedDB table
|
||||
* in `browser-file-storage/result.ts` that the IndexedDB runtime and
|
||||
* maintenance share (05-11). It imports nothing from `storage/indexeddb/`.
|
||||
*
|
||||
* A translator per operation, because the journal labels a read and a write
|
||||
* differently. The connection always translates as `INDEXEDDB_OPEN`: an open
|
||||
@@ -1714,6 +1691,17 @@ function translateFor(
|
||||
return (cause) => {
|
||||
switch (cause.kind) {
|
||||
case "NATIVE_EXCEPTION":
|
||||
// 05-06. A journal a newer release upgraded is not a failed
|
||||
// migration; this bundle cannot use it and works online instead.
|
||||
if (
|
||||
(cause.error as { name?: unknown } | null)?.name === "VersionError"
|
||||
) {
|
||||
return failureOf(
|
||||
browserDataFailure("UNSUPPORTED", operation, {
|
||||
recovery: "ONLINE_ONLY",
|
||||
}),
|
||||
);
|
||||
}
|
||||
return failureOf(mapIndexedDbException(cause.error, operation));
|
||||
case "BLOCKED":
|
||||
case "BLOCKED_DEADLINE":
|
||||
|
||||
@@ -96,6 +96,13 @@ export type OpfsByteStoreDependencies = Readonly<{
|
||||
* deliberately separate from any caller signal.
|
||||
*/
|
||||
compensationSignal?: AbortSignal;
|
||||
/**
|
||||
* 05-01 (D-3). Web Locks, `navigator.locks` by default. A put or delete
|
||||
* holds an exclusive per-transaction lock from journal `begin` until it
|
||||
* returns; reconcile only touches rows whose lock it can take at once, so a
|
||||
* live transaction in any tab is never cleaned up underneath its owner.
|
||||
*/
|
||||
lockManager?: Pick<LockManager, "request">;
|
||||
now?: () => number;
|
||||
observer?: OpfsSafeObserver;
|
||||
/**
|
||||
@@ -152,6 +159,12 @@ export function createOpfsByteStoreAdapter(
|
||||
*/
|
||||
const compensationSignal = dependencies.compensationSignal;
|
||||
const now = dependencies.now ?? Date.now;
|
||||
const lockManager =
|
||||
dependencies.lockManager ?? globalThis.navigator?.locks;
|
||||
if (!lockManager || typeof lockManager.request !== "function") {
|
||||
throw new TypeError("OPFS byte store requires Web Locks.");
|
||||
}
|
||||
const requestLock = lockManager.request.bind(lockManager);
|
||||
|
||||
const objects: DurableObjectStorePort = Object.freeze({
|
||||
async capabilities() {
|
||||
@@ -203,146 +216,152 @@ export function createOpfsByteStoreAdapter(
|
||||
}
|
||||
const targetGeneration = (current?.descriptor.generation ?? 0) + 1;
|
||||
const transactionId = createTransactionId();
|
||||
// STO-01. The logical generation is reused across transactions; this
|
||||
// token makes the physical target unique so a late compensation can never
|
||||
// delete a newer transaction's directory.
|
||||
const physicalGenerationId = createPhysicalGenerationId();
|
||||
notifyProgress(request, "PREPARING", 0);
|
||||
const begun = await dependencies.journal.begin({
|
||||
return await withTransactionLock(
|
||||
transactionId,
|
||||
mutation: "PUT",
|
||||
scope: dependencies.scope,
|
||||
objectId: request.objectId,
|
||||
expectedGeneration: request.expectedGeneration,
|
||||
targetGeneration,
|
||||
targetByteLength: request.source.byteLength!,
|
||||
targetStoragePolicy: dependencies.storagePolicy,
|
||||
startedAtEpochMs: now(),
|
||||
});
|
||||
if (!begun.ok) {
|
||||
return observeFailure(
|
||||
rebaseFailure(begun.error, "OBJECT_WRITE"),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
"OBJECT_WRITE",
|
||||
async () => {
|
||||
// STO-01. The logical generation is reused across transactions; this
|
||||
// token makes the physical target unique so a late compensation can never
|
||||
// delete a newer transaction's directory.
|
||||
const physicalGenerationId = createPhysicalGenerationId();
|
||||
notifyProgress(request, "PREPARING", 0);
|
||||
const begun = await dependencies.journal.begin({
|
||||
transactionId,
|
||||
mutation: "PUT",
|
||||
scope: dependencies.scope,
|
||||
objectId: request.objectId,
|
||||
expectedGeneration: request.expectedGeneration,
|
||||
targetGeneration,
|
||||
targetByteLength: request.source.byteLength!,
|
||||
targetStoragePolicy: dependencies.storagePolicy,
|
||||
startedAtEpochMs: now(),
|
||||
});
|
||||
if (!begun.ok) {
|
||||
return observeFailure(
|
||||
rebaseFailure(begun.error, "OBJECT_WRITE"),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
|
||||
const descriptor: Omit<DurableObjectDescriptor, "integrity"> =
|
||||
Object.freeze({
|
||||
objectId: request.objectId,
|
||||
scope: dependencies.scope,
|
||||
generation: targetGeneration,
|
||||
byteLength: request.source.byteLength!,
|
||||
mediaType: request.mediaType,
|
||||
createdAtEpochMs: now(),
|
||||
storagePolicy: dependencies.storagePolicy,
|
||||
});
|
||||
const prepared = await dependencies.worker.preparePut({
|
||||
transactionId,
|
||||
physicalGenerationId,
|
||||
descriptor,
|
||||
source: request.source,
|
||||
signal: request.signal,
|
||||
onProgress: request.onProgress,
|
||||
});
|
||||
if (!prepared.ok) {
|
||||
return observeFailure(
|
||||
await compensatePreparedPut(
|
||||
begun.value,
|
||||
const descriptor: Omit<DurableObjectDescriptor, "integrity"> =
|
||||
Object.freeze({
|
||||
objectId: request.objectId,
|
||||
scope: dependencies.scope,
|
||||
generation: targetGeneration,
|
||||
byteLength: request.source.byteLength!,
|
||||
mediaType: request.mediaType,
|
||||
createdAtEpochMs: now(),
|
||||
storagePolicy: dependencies.storagePolicy,
|
||||
});
|
||||
const prepared = await dependencies.worker.preparePut({
|
||||
transactionId,
|
||||
physicalGenerationId,
|
||||
prepared.error,
|
||||
),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
descriptor,
|
||||
source: request.source,
|
||||
signal: request.signal,
|
||||
onProgress: request.onProgress,
|
||||
});
|
||||
if (!prepared.ok) {
|
||||
return observeFailure(
|
||||
await compensatePreparedPut(
|
||||
begun.value,
|
||||
physicalGenerationId,
|
||||
prepared.error,
|
||||
),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
|
||||
const filesReady = await dependencies.journal.markFilesReady(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
prepared.value,
|
||||
);
|
||||
if (!filesReady.ok) {
|
||||
return observeFailure(
|
||||
await compensatePreparedPut(
|
||||
begun.value,
|
||||
physicalGenerationId,
|
||||
filesReady.error,
|
||||
),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
const filesReady = await dependencies.journal.markFilesReady(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
prepared.value,
|
||||
);
|
||||
if (!filesReady.ok) {
|
||||
return observeFailure(
|
||||
await compensatePreparedPut(
|
||||
begun.value,
|
||||
physicalGenerationId,
|
||||
filesReady.error,
|
||||
),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
|
||||
const committed = await dependencies.journal.commitPut(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
);
|
||||
if (!committed.ok) {
|
||||
// The commit response can be lost after IndexedDB commits. Reconciliation
|
||||
// decides from the durable journal; rolling back here would be unsafe.
|
||||
return observeFailure(
|
||||
rebaseFailure(committed.error, "OBJECT_WRITE"),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
const committed = await dependencies.journal.commitPut(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
);
|
||||
if (!committed.ok) {
|
||||
// The commit response can be lost after IndexedDB commits. Reconciliation
|
||||
// decides from the durable journal; rolling back here would be unsafe.
|
||||
return observeFailure(
|
||||
rebaseFailure(committed.error, "OBJECT_WRITE"),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
|
||||
notifyProgress(
|
||||
request,
|
||||
"FINALIZING",
|
||||
prepared.value.descriptor.byteLength,
|
||||
notifyProgress(
|
||||
request,
|
||||
"FINALIZING",
|
||||
prepared.value.descriptor.byteLength,
|
||||
);
|
||||
const finalized = await dependencies.worker.finalizePut(
|
||||
transactionId,
|
||||
prepared.value,
|
||||
request.signal,
|
||||
);
|
||||
if (!finalized.ok) {
|
||||
// STO-RR-01. The commit fence already passed, so the payload is durable
|
||||
// and the journal keeps its COMMITTED record for reconciliation to
|
||||
// settle. What did not happen is finalization: the previous generation
|
||||
// and the staging directory are still present. Reporting a plain
|
||||
// success here would claim a settled state nobody observed, so the
|
||||
// worker's own failure is surfaced and the record is left recoverable.
|
||||
return observeFailure(
|
||||
rebaseFailure(finalized.error, "OBJECT_WRITE"),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
const completed = await dependencies.journal.complete(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
);
|
||||
if (!completed.ok) {
|
||||
// NS-04. The payload is durable and finalized, so nothing is rolled
|
||||
// back and the `COMMITTED` row stays the reconciler's authority. What
|
||||
// did not happen is settling the transaction, and reporting a plain
|
||||
// success for it claimed a state nobody observed while the reconcile
|
||||
// backlog and its quota pressure grew unseen.
|
||||
observeOpfsSafely(dependencies.observer, {
|
||||
operation: "OBJECT_WRITE",
|
||||
outcome: "DEGRADED",
|
||||
failureCode: completed.error.code,
|
||||
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
|
||||
});
|
||||
return {
|
||||
ok: false as const,
|
||||
error: Object.freeze({
|
||||
...completed.error,
|
||||
operation: "OBJECT_WRITE" as const,
|
||||
retryable: true,
|
||||
recovery: "RECONCILE" as const,
|
||||
}),
|
||||
};
|
||||
}
|
||||
observeOpfsSafely(dependencies.observer, {
|
||||
operation: "OBJECT_WRITE",
|
||||
outcome: "SUCCEEDED",
|
||||
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
|
||||
});
|
||||
return browserDataSuccess(prepared.value.descriptor);
|
||||
},
|
||||
);
|
||||
const finalized = await dependencies.worker.finalizePut(
|
||||
transactionId,
|
||||
prepared.value,
|
||||
request.signal,
|
||||
);
|
||||
if (!finalized.ok) {
|
||||
// STO-RR-01. The commit fence already passed, so the payload is durable
|
||||
// and the journal keeps its COMMITTED record for reconciliation to
|
||||
// settle. What did not happen is finalization: the previous generation
|
||||
// and the staging directory are still present. Reporting a plain
|
||||
// success here would claim a settled state nobody observed, so the
|
||||
// worker's own failure is surfaced and the record is left recoverable.
|
||||
return observeFailure(
|
||||
rebaseFailure(finalized.error, "OBJECT_WRITE"),
|
||||
dependencies.observer,
|
||||
request.source.byteLength!,
|
||||
);
|
||||
}
|
||||
const completed = await dependencies.journal.complete(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
);
|
||||
if (!completed.ok) {
|
||||
// NS-04. The payload is durable and finalized, so nothing is rolled
|
||||
// back and the `COMMITTED` row stays the reconciler's authority. What
|
||||
// did not happen is settling the transaction, and reporting a plain
|
||||
// success for it claimed a state nobody observed while the reconcile
|
||||
// backlog and its quota pressure grew unseen.
|
||||
observeOpfsSafely(dependencies.observer, {
|
||||
operation: "OBJECT_WRITE",
|
||||
outcome: "DEGRADED",
|
||||
failureCode: completed.error.code,
|
||||
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
|
||||
});
|
||||
return {
|
||||
ok: false as const,
|
||||
error: Object.freeze({
|
||||
...completed.error,
|
||||
operation: "OBJECT_WRITE" as const,
|
||||
retryable: true,
|
||||
recovery: "RECONCILE" as const,
|
||||
}),
|
||||
};
|
||||
}
|
||||
observeOpfsSafely(dependencies.observer, {
|
||||
operation: "OBJECT_WRITE",
|
||||
outcome: "SUCCEEDED",
|
||||
byteBucket: byteBucket(prepared.value.descriptor.byteLength),
|
||||
});
|
||||
return browserDataSuccess(prepared.value.descriptor);
|
||||
},
|
||||
|
||||
async open(
|
||||
@@ -469,63 +488,70 @@ export function createOpfsByteStoreAdapter(
|
||||
);
|
||||
}
|
||||
|
||||
const targetStoragePolicy = current.value.descriptor.storagePolicy;
|
||||
const transactionId = createTransactionId();
|
||||
const begun = await dependencies.journal.begin({
|
||||
return await withTransactionLock(
|
||||
transactionId,
|
||||
mutation: "DELETE",
|
||||
scope: dependencies.scope,
|
||||
objectId: request.objectId,
|
||||
expectedGeneration: request.expectedGeneration,
|
||||
targetGeneration: request.expectedGeneration + 1,
|
||||
targetByteLength: 0,
|
||||
targetStoragePolicy: current.value.descriptor.storagePolicy,
|
||||
startedAtEpochMs: now(),
|
||||
});
|
||||
if (!begun.ok) {
|
||||
return observeFailure(
|
||||
rebaseFailure(begun.error, "OBJECT_DELETE"),
|
||||
dependencies.observer,
|
||||
);
|
||||
}
|
||||
const committed = await dependencies.journal.commitDelete(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
);
|
||||
if (!committed.ok) {
|
||||
return observeFailure(
|
||||
rebaseFailure(committed.error, "OBJECT_DELETE"),
|
||||
dependencies.observer,
|
||||
);
|
||||
}
|
||||
const removed = await dependencies.worker.removeObject(
|
||||
dependencies.scope,
|
||||
request.objectId,
|
||||
request.expectedGeneration,
|
||||
request.signal,
|
||||
);
|
||||
const completed = removed.ok
|
||||
? await dependencies.journal.complete(
|
||||
"OBJECT_DELETE",
|
||||
async () => {
|
||||
const begun = await dependencies.journal.begin({
|
||||
transactionId,
|
||||
mutation: "DELETE",
|
||||
scope: dependencies.scope,
|
||||
objectId: request.objectId,
|
||||
expectedGeneration: request.expectedGeneration,
|
||||
targetGeneration: request.expectedGeneration + 1,
|
||||
targetByteLength: 0,
|
||||
targetStoragePolicy,
|
||||
startedAtEpochMs: now(),
|
||||
});
|
||||
if (!begun.ok) {
|
||||
return observeFailure(
|
||||
rebaseFailure(begun.error, "OBJECT_DELETE"),
|
||||
dependencies.observer,
|
||||
);
|
||||
}
|
||||
const committed = await dependencies.journal.commitDelete(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
)
|
||||
: null;
|
||||
// Logical deletion is already committed. Physical cleanup is retryable
|
||||
// maintenance and must not make the caller repeat a non-idempotent delete.
|
||||
// NS-04. It is still not a settled state: an unfinished physical removal
|
||||
// or an unsettled journal row is maintenance debt the reconciler owns, so
|
||||
// it is observed as such instead of as a clean success.
|
||||
const settled = removed.ok && completed !== null && completed.ok;
|
||||
const debtCode = removed.ok
|
||||
? completed !== null && !completed.ok
|
||||
? completed.error.code
|
||||
: undefined
|
||||
: removed.error.code;
|
||||
observeOpfsSafely(dependencies.observer, {
|
||||
operation: "OBJECT_DELETE",
|
||||
outcome: settled ? "SUCCEEDED" : "DEGRADED",
|
||||
...(debtCode ? { failureCode: debtCode } : {}),
|
||||
});
|
||||
return browserDataSuccess(undefined);
|
||||
);
|
||||
if (!committed.ok) {
|
||||
return observeFailure(
|
||||
rebaseFailure(committed.error, "OBJECT_DELETE"),
|
||||
dependencies.observer,
|
||||
);
|
||||
}
|
||||
const removed = await dependencies.worker.removeObject(
|
||||
dependencies.scope,
|
||||
request.objectId,
|
||||
request.expectedGeneration,
|
||||
request.signal,
|
||||
);
|
||||
const completed = removed.ok
|
||||
? await dependencies.journal.complete(
|
||||
transactionId,
|
||||
begun.value.fencingToken,
|
||||
)
|
||||
: null;
|
||||
// Logical deletion is already committed. Physical cleanup is retryable
|
||||
// maintenance and must not make the caller repeat a non-idempotent delete.
|
||||
// NS-04. It is still not a settled state: an unfinished physical removal
|
||||
// or an unsettled journal row is maintenance debt the reconciler owns, so
|
||||
// it is observed as such instead of as a clean success.
|
||||
const settled = removed.ok && completed !== null && completed.ok;
|
||||
const debtCode = removed.ok
|
||||
? completed !== null && !completed.ok
|
||||
? completed.error.code
|
||||
: undefined
|
||||
: removed.error.code;
|
||||
observeOpfsSafely(dependencies.observer, {
|
||||
operation: "OBJECT_DELETE",
|
||||
outcome: settled ? "SUCCEEDED" : "DEGRADED",
|
||||
...(debtCode ? { failureCode: debtCode } : {}),
|
||||
});
|
||||
return browserDataSuccess(undefined);
|
||||
},
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -563,14 +589,14 @@ export function createOpfsByteStoreAdapter(
|
||||
const maxTransactions =
|
||||
request.maxTransactions ?? policy.reconciliationBatchSize;
|
||||
const deadline = now() + budgetMs;
|
||||
const page = await dependencies.journal.listIncomplete(maxTransactions);
|
||||
if (!page.ok) {
|
||||
return rebaseFailure(page.error, "OBJECT_RECONCILE");
|
||||
const listed = await dependencies.journal.listIncomplete(maxTransactions);
|
||||
if (!listed.ok) {
|
||||
return rebaseFailure(listed.error, "OBJECT_RECONCILE");
|
||||
}
|
||||
observeOpfsSafely(dependencies.observer, {
|
||||
operation: "OBJECT_RECONCILE",
|
||||
outcome: "STARTED",
|
||||
transactionBucket: transactionBucket(page.value.transactions.length),
|
||||
transactionBucket: transactionBucket(listed.value.transactions.length),
|
||||
});
|
||||
|
||||
const report: {
|
||||
@@ -586,6 +612,8 @@ export function createOpfsByteStoreAdapter(
|
||||
| "STAGING_STATE_UNREADABLE";
|
||||
moreTransactionsAvailable: boolean;
|
||||
deadlineReached: boolean;
|
||||
skippedLiveTransactions: number;
|
||||
failedTransactions: number;
|
||||
} = {
|
||||
inspectedTransactions: 0,
|
||||
committedTransactions: 0,
|
||||
@@ -594,26 +622,63 @@ export function createOpfsByteStoreAdapter(
|
||||
inspectedOrphanChunks: 0,
|
||||
deletedOrphanChunks: 0,
|
||||
orphanGcStatus: "COMPLETED",
|
||||
moreTransactionsAvailable: page.value.moreAvailable,
|
||||
moreTransactionsAvailable: listed.value.moreAvailable,
|
||||
deadlineReached: false,
|
||||
skippedLiveTransactions: 0,
|
||||
failedTransactions: 0,
|
||||
};
|
||||
for (const transaction of page.value.transactions) {
|
||||
if (request.signal?.aborted) {
|
||||
return browserDataFailure("ABORTED", "OBJECT_RECONCILE");
|
||||
// 05-01 (D-3). A row is reconciled only while this call holds its
|
||||
// per-transaction lock, so a put or delete still running in any tab is
|
||||
// skipped. The page is read again under those locks: a row listed while
|
||||
// its owner still ran may have moved on before the owner let go.
|
||||
const held = new Map<string, () => void>();
|
||||
try {
|
||||
for (const transaction of listed.value.transactions) {
|
||||
let release: (() => void) | null;
|
||||
try {
|
||||
release = await holdTransactionLock(transaction.transactionId);
|
||||
} catch {
|
||||
return browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE", {
|
||||
retryable: true,
|
||||
recovery: "RETRY",
|
||||
});
|
||||
}
|
||||
if (release) held.set(transaction.transactionId, release);
|
||||
else report.skippedLiveTransactions += 1;
|
||||
}
|
||||
if (now() >= deadline) {
|
||||
report.deadlineReached = true;
|
||||
report.moreTransactionsAvailable = true;
|
||||
break;
|
||||
const page =
|
||||
held.size === 0
|
||||
? listed
|
||||
: await dependencies.journal.listIncomplete(maxTransactions);
|
||||
if (!page.ok) {
|
||||
return rebaseFailure(page.error, "OBJECT_RECONCILE");
|
||||
}
|
||||
report.inspectedTransactions += 1;
|
||||
const reconciled = await reconcileTransaction(transaction, request.signal);
|
||||
if (!reconciled.ok) {
|
||||
return observeFailure(reconciled, dependencies.observer);
|
||||
report.moreTransactionsAvailable = page.value.moreAvailable;
|
||||
for (const transaction of page.value.transactions) {
|
||||
if (!held.has(transaction.transactionId)) continue;
|
||||
if (request.signal?.aborted) {
|
||||
return browserDataFailure("ABORTED", "OBJECT_RECONCILE");
|
||||
}
|
||||
if (now() >= deadline) {
|
||||
report.deadlineReached = true;
|
||||
report.moreTransactionsAvailable = true;
|
||||
break;
|
||||
}
|
||||
report.inspectedTransactions += 1;
|
||||
const reconciled = await reconcileTransaction(transaction, request.signal);
|
||||
if (!reconciled.ok) {
|
||||
// 05-02. A row that cannot be settled now stays for a later run;
|
||||
// it must not stop the rest of the page or orphan GC.
|
||||
observeFailure(reconciled, dependencies.observer);
|
||||
report.failedTransactions += 1;
|
||||
continue;
|
||||
}
|
||||
report.committedTransactions += reconciled.value.committed;
|
||||
report.rolledBackTransactions += reconciled.value.rolledBack;
|
||||
report.cleanedTransactions += reconciled.value.cleaned;
|
||||
}
|
||||
report.committedTransactions += reconciled.value.committed;
|
||||
report.rolledBackTransactions += reconciled.value.rolledBack;
|
||||
report.cleanedTransactions += reconciled.value.cleaned;
|
||||
} finally {
|
||||
for (const release of held.values()) release();
|
||||
}
|
||||
if (now() >= deadline) {
|
||||
report.deadlineReached = true;
|
||||
@@ -781,12 +846,38 @@ export function createOpfsByteStoreAdapter(
|
||||
afterObjectId = page.value.nextObjectId;
|
||||
moreObjectsAvailable = page.value.moreAvailable;
|
||||
}
|
||||
// 05-04. Removing an object leaves its chunks, and GC only takes a
|
||||
// chunk 24 h after it was written, so a purged user's bytes stayed on
|
||||
// the device. Once every object is gone the partition's chunks are swept
|
||||
// without that grace. The sweep still skips any chunk the journal or a
|
||||
// staged put references.
|
||||
let chunkSweep: OpfsPolicyMaintenanceReport["chunkSweep"];
|
||||
if (
|
||||
(request.reason === "LOGOUT" ||
|
||||
request.reason === "ACCOUNT_DELETION") &&
|
||||
!moreObjectsAvailable &&
|
||||
!deadlineReached &&
|
||||
!request.signal?.aborted
|
||||
) {
|
||||
const swept = await reconcileOrphanChunks(
|
||||
deadline,
|
||||
request.signal,
|
||||
now(),
|
||||
);
|
||||
if (!swept.ok) return rebaseFailure(swept.error, "OBJECT_RECONCILE");
|
||||
chunkSweep = swept.value.status;
|
||||
if (chunkSweep === "DEADLINE_REACHED") {
|
||||
deadlineReached = true;
|
||||
moreObjectsAvailable = true;
|
||||
}
|
||||
}
|
||||
const report: OpfsPolicyMaintenanceReport = Object.freeze({
|
||||
inspectedObjects,
|
||||
removedObjects,
|
||||
releasedBytes,
|
||||
moreObjectsAvailable,
|
||||
deadlineReached,
|
||||
...(chunkSweep ? { chunkSweep } : {}),
|
||||
});
|
||||
return browserDataSuccess(report);
|
||||
},
|
||||
@@ -905,13 +996,63 @@ export function createOpfsByteStoreAdapter(
|
||||
|
||||
return Object.freeze({ objects, maintenance });
|
||||
|
||||
/**
|
||||
* Takes the exclusive per-transaction Web Lock only if nobody holds it.
|
||||
* Resolves with its release, or `null` while another context holds it.
|
||||
*/
|
||||
async function holdTransactionLock(
|
||||
transactionId: string,
|
||||
): Promise<(() => void) | null> {
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
return await new Promise((resolve, reject) => {
|
||||
requestLock(
|
||||
`${policy.mutationLockName}:transaction:${scope.authorityToken}:${transactionId}`,
|
||||
{ mode: "exclusive", ifAvailable: true },
|
||||
async (lock) => {
|
||||
if (!lock) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
resolve(release);
|
||||
await held;
|
||||
},
|
||||
).catch(reject);
|
||||
});
|
||||
}
|
||||
|
||||
async function withTransactionLock<Value>(
|
||||
transactionId: string,
|
||||
operation: "OBJECT_WRITE" | "OBJECT_DELETE",
|
||||
run: () => Promise<BrowserDataResult<Value>>,
|
||||
): Promise<BrowserDataResult<Value>> {
|
||||
let release: (() => void) | null;
|
||||
try {
|
||||
release = await holdTransactionLock(transactionId);
|
||||
} catch {
|
||||
return browserDataFailure("UNAVAILABLE", operation, {
|
||||
retryable: true,
|
||||
recovery: "RETRY",
|
||||
});
|
||||
}
|
||||
// A fresh transaction id is never held; a holder means a duplicate id.
|
||||
if (!release) return browserDataFailure("CONFLICT", operation);
|
||||
try {
|
||||
return await run();
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* STO-01. The compensating half of the put saga.
|
||||
*
|
||||
* The journal row and its budget reservation are the only durable evidence
|
||||
* that a physical staging generation may still exist, so they are released
|
||||
* exactly when the physical effect is confirmed `CLEANED` or
|
||||
* `ALREADY_CLEAN`. A timeout, crash, malformed response or `EFFECT_UNKNOWN`
|
||||
* `ALREADY_CLEAN`. A timeout, crash, malformed response or worker failure
|
||||
* keeps `PREPARING`/`FILES_READY` in place and asks for reconciliation.
|
||||
*/
|
||||
/**
|
||||
@@ -933,7 +1074,6 @@ export function createOpfsByteStoreAdapter(
|
||||
});
|
||||
const compensated =
|
||||
cleanup.ok &&
|
||||
cleanup.value.kind !== "EFFECT_UNKNOWN" &&
|
||||
(
|
||||
await dependencies.journal.rollback(
|
||||
transaction.transactionId,
|
||||
@@ -961,11 +1101,6 @@ export function createOpfsByteStoreAdapter(
|
||||
signal,
|
||||
);
|
||||
if (!cleaned.ok) return cleaned;
|
||||
if (cleaned.value.kind === "EFFECT_UNKNOWN") {
|
||||
return browserDataFailure("CONFLICT", "OBJECT_RECONCILE", {
|
||||
recovery: "RETRY",
|
||||
});
|
||||
}
|
||||
const rolledBack = await dependencies.journal.rollback(
|
||||
transaction.transactionId,
|
||||
transaction.fencingToken,
|
||||
@@ -1073,6 +1208,7 @@ export function createOpfsByteStoreAdapter(
|
||||
async function reconcileOrphanChunks(
|
||||
deadline: number,
|
||||
signal: AbortSignal | undefined,
|
||||
olderThanEpochMs = Math.max(0, now() - policy.orphanGracePeriodMs),
|
||||
): Promise<
|
||||
BrowserDataResult<
|
||||
Readonly<{
|
||||
@@ -1085,10 +1221,6 @@ export function createOpfsByteStoreAdapter(
|
||||
}>
|
||||
>
|
||||
> {
|
||||
const olderThanEpochMs = Math.max(
|
||||
0,
|
||||
now() - policy.orphanGracePeriodMs,
|
||||
);
|
||||
let inspected = 0;
|
||||
let deleted = 0;
|
||||
// R-5.1. Candidates include committed chunks, so a page can be all
|
||||
|
||||
@@ -20,6 +20,13 @@ export type OpfsRuntimePolicy = Readonly<{
|
||||
* is bounded on its own and never counted against `rpcTimeoutMs`.
|
||||
*/
|
||||
mutationLockWaitTimeoutMs: number;
|
||||
/**
|
||||
* 05-07. How long a put may wait for its source's next bytes. The put holds
|
||||
* the origin mutation lock from BEGIN to FINISH, so a stalled source would
|
||||
* otherwise stop every tab's writes until their lock wait ran out. Keep it
|
||||
* below `mutationLockWaitTimeoutMs` so a queued writer outlasts it.
|
||||
*/
|
||||
putIdleTimeoutMs: number;
|
||||
reconciliationBudgetMs: number;
|
||||
reconciliationBatchSize: number;
|
||||
orphanGracePeriodMs: number;
|
||||
@@ -56,6 +63,7 @@ export const DEFAULT_OPFS_RUNTIME_POLICY: OpfsRuntimePolicy = Object.freeze({
|
||||
maxChunkCount: 512,
|
||||
rpcTimeoutMs: 60_000,
|
||||
mutationLockWaitTimeoutMs: 10 * 60_000,
|
||||
putIdleTimeoutMs: 2 * 60_000,
|
||||
reconciliationBudgetMs: 5_000,
|
||||
reconciliationBatchSize: 100,
|
||||
orphanGracePeriodMs: 24 * 60 * 60 * 1_000,
|
||||
@@ -94,6 +102,9 @@ export function assertOpfsRuntimePolicy(policy: OpfsRuntimePolicy): void {
|
||||
!Number.isSafeInteger(policy.mutationLockWaitTimeoutMs) ||
|
||||
policy.mutationLockWaitTimeoutMs < 1_000 ||
|
||||
policy.mutationLockWaitTimeoutMs > 60 * 60_000 ||
|
||||
!Number.isSafeInteger(policy.putIdleTimeoutMs) ||
|
||||
policy.putIdleTimeoutMs < 1_000 ||
|
||||
policy.putIdleTimeoutMs > 60 * 60_000 ||
|
||||
!Number.isSafeInteger(policy.reconciliationBudgetMs) ||
|
||||
policy.reconciliationBudgetMs < 1 ||
|
||||
policy.reconciliationBudgetMs > 60_000 ||
|
||||
@@ -109,12 +120,45 @@ export function assertOpfsRuntimePolicy(policy: OpfsRuntimePolicy): void {
|
||||
policy.maxCancellationTombstones < 16 ||
|
||||
policy.maxCancellationTombstones > 10_000 ||
|
||||
typeof policy.isObjectIdAllowed !== "function" ||
|
||||
typeof policy.isMediaTypeAllowed !== "function"
|
||||
typeof policy.isMediaTypeAllowed !== "function" ||
|
||||
widensObjectIdGrammar(policy.isObjectIdAllowed)
|
||||
) {
|
||||
throw new TypeError("OPFS runtime policy is invalid.");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 05-10. Stored prepared objects, journal rows and physical paths all use the
|
||||
* opaque id grammar, so a policy may narrow it but never widen it: a wider
|
||||
* policy let BEGIN through and failed every put at its last step. Ids just
|
||||
* outside the grammar catch the usual widenings when the policy is resolved.
|
||||
*/
|
||||
const OBJECT_IDS_OUTSIDE_GRAMMAR = Object.freeze([
|
||||
"",
|
||||
"short_7",
|
||||
"a".repeat(129),
|
||||
"object 12345",
|
||||
"object.12345",
|
||||
"object/12345",
|
||||
"object~12345",
|
||||
]);
|
||||
|
||||
function widensObjectIdGrammar(
|
||||
isObjectIdAllowed: (objectId: string) => boolean,
|
||||
): boolean {
|
||||
try {
|
||||
return OBJECT_IDS_OUTSIDE_GRAMMAR.some((objectId) =>
|
||||
isObjectIdAllowed(objectId),
|
||||
);
|
||||
} catch {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
export function isOpaqueObjectId(value: string): boolean {
|
||||
return OPAQUE_OBJECT_ID.test(value);
|
||||
}
|
||||
|
||||
export function validateObjectWriteInput(
|
||||
input: Readonly<{
|
||||
scope: OpfsStorageScope;
|
||||
|
||||
@@ -3,9 +3,8 @@ import type {
|
||||
OpfsStorageScope,
|
||||
} from "../../../application/ports/browser-file-storage/opfs-ports.ts";
|
||||
import { assertValidStoragePolicy } from "../../../application/ports/browser-file-storage/shared.ts";
|
||||
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
|
||||
import { isOpaqueObjectId, isValidOpfsStorageScope } from "./opfs-policy.ts";
|
||||
|
||||
const OPAQUE_OBJECT_ID = /^[A-Za-z0-9_-]{8,128}$/u;
|
||||
const SHA256_HEX = /^[a-f0-9]{64}$/u;
|
||||
|
||||
/**
|
||||
@@ -53,7 +52,7 @@ export function isPreparedObject(
|
||||
: null;
|
||||
if (
|
||||
typeof descriptor.objectId !== "string" ||
|
||||
!OPAQUE_OBJECT_ID.test(descriptor.objectId) ||
|
||||
!isOpaqueObjectId(descriptor.objectId) ||
|
||||
!descriptor.scope ||
|
||||
typeof descriptor.scope !== "object" ||
|
||||
!isValidOpfsStorageScope(descriptor.scope as OpfsStorageScope) ||
|
||||
@@ -129,3 +128,36 @@ export function isPreparedObject(
|
||||
Math.ceil(descriptor.byteLength / integrity.chunkSizeBytes)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* 05-10. The one canonical JSON for comparing OPFS records, used by the
|
||||
* worker and the journal alike. A value JSON cannot carry exactly (a
|
||||
* non-finite number, `undefined`, a function) throws instead of being
|
||||
* written as something else, so two different values never compare equal.
|
||||
*/
|
||||
export function stableJson(value: unknown): string {
|
||||
if (
|
||||
value === null ||
|
||||
typeof value === "string" ||
|
||||
typeof value === "boolean"
|
||||
) {
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
if (typeof value === "number") {
|
||||
if (!Number.isFinite(value)) {
|
||||
throw new TypeError("OPFS value is not JSON-safe.");
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
return `[${value.map(stableJson).join(",")}]`;
|
||||
}
|
||||
if (typeof value === "object") {
|
||||
const record = value as Record<string, unknown>;
|
||||
return `{${Object.keys(record)
|
||||
.sort()
|
||||
.map((key) => `${JSON.stringify(key)}:${stableJson(record[key])}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
throw new TypeError("OPFS value is not JSON-safe.");
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
browserDataFailure,
|
||||
browserDataSuccess,
|
||||
} from "../../browser-file-storage/result.ts";
|
||||
import { createAbortableOperation } from "../../platform/abortable-operation.ts";
|
||||
import type { OpfsRuntimePolicy } from "./opfs-policy.ts";
|
||||
import { isPreparedObject } from "./opfs-prepared-object.ts";
|
||||
import {
|
||||
@@ -272,13 +273,36 @@ export function createOpfsWorkerGateway(
|
||||
let sequence = 0;
|
||||
let transferredBytes = 0;
|
||||
notifyProgress(request, "TRANSFERRING", 0);
|
||||
// 05-07. BEGIN took the origin mutation lock, so every wait for the
|
||||
// source is bounded by `putIdleTimeoutMs`. A source that stops fails the
|
||||
// put, and the coordinator's compensation releases the lock.
|
||||
const sourceStop = new AbortController();
|
||||
const stopSource = () => sourceStop.abort();
|
||||
request.signal?.addEventListener("abort", stopSource, { once: true });
|
||||
if (request.signal?.aborted) stopSource();
|
||||
const chunks = rechunk(
|
||||
request.source.stream(sourceStop.signal),
|
||||
dependencies.policy.chunkSizeBytes,
|
||||
dependencies.policy.maxObjectBytes,
|
||||
request.signal,
|
||||
);
|
||||
try {
|
||||
for await (const chunk of rechunk(
|
||||
request.source.stream(requiredSignal(request.signal)),
|
||||
dependencies.policy.chunkSizeBytes,
|
||||
dependencies.policy.maxObjectBytes,
|
||||
request.signal,
|
||||
)) {
|
||||
for (;;) {
|
||||
const idle = createAbortableOperation({
|
||||
signal: request.signal,
|
||||
timeoutMs: dependencies.policy.putIdleTimeoutMs,
|
||||
});
|
||||
const next = await idle.race(chunks.next());
|
||||
idle.close();
|
||||
if (next.kind === "REJECTED") throw next.reason;
|
||||
if (next.kind === "TERMINAL") {
|
||||
stopSource();
|
||||
throw new OpfsRpcError(
|
||||
next.terminal === "DEADLINE" ? "NOT_READABLE" : "ABORTED",
|
||||
);
|
||||
}
|
||||
if (next.value.done) break;
|
||||
const chunk = next.value.value;
|
||||
const chunkByteLength = chunk.byteLength;
|
||||
const append = await invoke<void>(
|
||||
"OBJECT_WRITE",
|
||||
@@ -320,6 +344,8 @@ export function createOpfsWorkerGateway(
|
||||
error instanceof OpfsRpcError ? error.code : "NOT_READABLE",
|
||||
"OBJECT_WRITE",
|
||||
);
|
||||
} finally {
|
||||
request.signal?.removeEventListener("abort", stopSource);
|
||||
}
|
||||
},
|
||||
|
||||
@@ -585,10 +611,6 @@ async function* rechunk(
|
||||
}
|
||||
}
|
||||
|
||||
function requiredSignal(signal: AbortSignal | undefined): AbortSignal {
|
||||
return signal ?? new AbortController().signal;
|
||||
}
|
||||
|
||||
function notifyProgress(
|
||||
request: PreparePhysicalObjectRequest,
|
||||
phase: "TRANSFERRING" | "VERIFYING",
|
||||
|
||||
@@ -16,7 +16,7 @@ import {
|
||||
isValidOpfsStorageScope,
|
||||
type OpfsRuntimePolicy,
|
||||
} from "./opfs-policy.ts";
|
||||
import { isPreparedObject } from "./opfs-prepared-object.ts";
|
||||
import { isPreparedObject, stableJson } from "./opfs-prepared-object.ts";
|
||||
import {
|
||||
OPFS_WORKER_PROTOCOL_VERSION,
|
||||
} from "./opfs-worker-protocol.ts";
|
||||
@@ -890,16 +890,11 @@ export function createOpfsWorkerRuntime(
|
||||
}
|
||||
{
|
||||
if (removePreparedGeneration) {
|
||||
let receipt: unknown;
|
||||
try {
|
||||
receipt = await readJson(receiptPath(scope, transactionId));
|
||||
} catch (error) {
|
||||
if (isNotFound(error)) {
|
||||
await removeEntryIfPresent(staging, transactionId, true);
|
||||
return CLEANUP_ALREADY_CLEAN;
|
||||
}
|
||||
// Keep unreadable staging in place so orphan GC fails closed.
|
||||
throw error;
|
||||
// Keep unreadable staging in place so orphan GC fails closed.
|
||||
const receipt = await readReceipt(scope, transactionId);
|
||||
if (receipt === undefined) {
|
||||
await removeEntryIfPresent(staging, transactionId, true);
|
||||
return CLEANUP_ALREADY_CLEAN;
|
||||
}
|
||||
const target = extractReceiptPhysicalTarget(receipt, scope);
|
||||
if (!target) {
|
||||
@@ -1133,10 +1128,11 @@ export function createOpfsWorkerRuntime(
|
||||
}
|
||||
let receipt: unknown;
|
||||
try {
|
||||
receipt = await readJson(receiptPath(scope, transactionId));
|
||||
receipt = await readReceipt(scope, transactionId);
|
||||
} catch {
|
||||
return { safeToSweep: false, digests };
|
||||
}
|
||||
if (receipt === undefined) continue;
|
||||
if (!receiptBelongsToScope(receipt, scope)) {
|
||||
return { safeToSweep: false, digests };
|
||||
}
|
||||
@@ -1303,8 +1299,31 @@ export function createOpfsWorkerRuntime(
|
||||
await writeWithAtomicWritable(handle, bytes);
|
||||
}
|
||||
|
||||
/**
|
||||
* 05-02. `writeJsonAtomic` creates the file before its first write lands,
|
||||
* so a first receipt write that hit the quota or a crash leaves a zero-byte
|
||||
* file. BEGIN writes that receipt before any chunk or generation directory,
|
||||
* so an empty receipt, like a missing one, has nothing to clean up.
|
||||
*/
|
||||
async function readReceipt(
|
||||
scope: OpfsStorageScope,
|
||||
transactionId: string,
|
||||
): Promise<unknown> {
|
||||
let bytes: Uint8Array;
|
||||
try {
|
||||
bytes = await readFile(receiptPath(scope, transactionId));
|
||||
} catch (error) {
|
||||
if (isNotFound(error)) return undefined;
|
||||
throw error;
|
||||
}
|
||||
return bytes.byteLength === 0 ? undefined : parseJson(bytes);
|
||||
}
|
||||
|
||||
async function readJson(path: readonly string[]): Promise<unknown> {
|
||||
const bytes = await readFile(path);
|
||||
return parseJson(await readFile(path));
|
||||
}
|
||||
|
||||
function parseJson(bytes: Uint8Array): unknown {
|
||||
if (bytes.byteLength > MAX_MANIFEST_BYTES) {
|
||||
throw new OpfsRuntimeFailure("CORRUPT_DATA");
|
||||
}
|
||||
@@ -1720,31 +1739,6 @@ function bytesToHex(bytes: Uint8Array): string {
|
||||
return hex;
|
||||
}
|
||||
|
||||
function stableJson(value: unknown): string {
|
||||
if (
|
||||
value === null ||
|
||||
typeof value === "string" ||
|
||||
typeof value === "boolean"
|
||||
) {
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
if (typeof value === "number") {
|
||||
if (!Number.isFinite(value)) throw new OpfsRuntimeFailure("CORRUPT_DATA");
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
return `[${value.map(stableJson).join(",")}]`;
|
||||
}
|
||||
if (typeof value === "object") {
|
||||
const record = value as Record<string, unknown>;
|
||||
return `{${Object.keys(record)
|
||||
.sort()
|
||||
.map((key) => `${JSON.stringify(key)}:${stableJson(record[key])}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
throw new OpfsRuntimeFailure("CORRUPT_DATA");
|
||||
}
|
||||
|
||||
function success(
|
||||
requestId: string,
|
||||
kind: OpfsWorkerRequest["kind"],
|
||||
|
||||
@@ -43,6 +43,13 @@ export type IndexedDbConnectionStatus =
|
||||
targetVersion: number;
|
||||
}>
|
||||
| Readonly<{ kind: "READY"; schemaVersion: number }>
|
||||
/**
|
||||
* 05-06. The database was upgraded past `schemaVersion` by a newer release
|
||||
* (another tab, or the bundle rolled back). This runtime cannot read or
|
||||
* write it: open and every operation answer `UNSUPPORTED` with recovery
|
||||
* `ONLINE_ONLY`, and a reload onto the newer release is the way back.
|
||||
*/
|
||||
| Readonly<{ kind: "INCOMPATIBLE"; schemaVersion: number }>
|
||||
| Readonly<{ kind: "DISPOSED" }>;
|
||||
|
||||
export type IndexedDbCursorKey =
|
||||
@@ -144,6 +151,13 @@ export type IndexedDbLifecycleBatchReceipt = Readonly<{
|
||||
scannedRows: number;
|
||||
deletedRows: number;
|
||||
budgetExhausted: boolean;
|
||||
/**
|
||||
* 05-03. Present when an authorized LOGOUT or ACCOUNT_DELETION found the
|
||||
* dataset's governance rows unreadable, so no row could be purged one by
|
||||
* one, and deleted the partition's whole database instead. Rows are then
|
||||
* not counted.
|
||||
*/
|
||||
datasetDeleted?: true;
|
||||
}>;
|
||||
|
||||
export type IndexedDbLifecycleAuthorityRequest = Readonly<{
|
||||
|
||||
@@ -76,9 +76,23 @@ export type OpfsCapabilities = Readonly<{
|
||||
|
||||
export interface DurableObjectStorePort {
|
||||
capabilities(): Promise<BrowserDataResult<OpfsCapabilities>>;
|
||||
/**
|
||||
* 05-07. OPFS writes are one at a time across the whole origin: a put holds
|
||||
* the origin mutation lock while it streams, and every put, remove and GC in
|
||||
* every tab waits for it. A source that stops for the runtime's put idle
|
||||
* bound fails the put with a retryable `NOT_READABLE` and frees the lock.
|
||||
*/
|
||||
put(
|
||||
request: PutDurableObjectRequest,
|
||||
): Promise<BrowserDataResult<DurableObjectDescriptor>>;
|
||||
/**
|
||||
* 05-14. `open` checks the manifest; the bytes are read and verified while
|
||||
* the source streams. A put that replaces the object meanwhile leaves the
|
||||
* older generation's chunks to orphan GC, whose grace counts from when a
|
||||
* chunk was written, not from when it lost its last reference. A stream
|
||||
* that reaches a chunk GC already took ends with `NOT_FOUND` / `REHYDRATE`.
|
||||
* Nothing is lost: open again to read the generation that replaced it.
|
||||
*/
|
||||
open(
|
||||
request: OpenDurableObjectRequest,
|
||||
): Promise<BrowserDataResult<OpenedDurableObject>>;
|
||||
@@ -100,6 +114,17 @@ export type OpfsReconciliationReport = Readonly<{
|
||||
| "STAGING_STATE_UNREADABLE";
|
||||
moreTransactionsAvailable: boolean;
|
||||
deadlineReached: boolean;
|
||||
/**
|
||||
* 05-01. Rows whose owner still holds the per-transaction lock. They are
|
||||
* a live put or delete, not debris, and are left for a later reconcile.
|
||||
*/
|
||||
skippedLiveTransactions: number;
|
||||
/**
|
||||
* 05-02. Rows that failed to reconcile this time. Each stays in the journal
|
||||
* for a later run and is observed as a `FAILED` reconcile; it no longer
|
||||
* stops the rest of the page or orphan GC.
|
||||
*/
|
||||
failedTransactions: number;
|
||||
}>;
|
||||
|
||||
export type OpfsPolicyMaintenanceReport = Readonly<{
|
||||
@@ -108,6 +133,12 @@ export type OpfsPolicyMaintenanceReport = Readonly<{
|
||||
releasedBytes: number;
|
||||
moreObjectsAvailable: boolean;
|
||||
deadlineReached: boolean;
|
||||
/**
|
||||
* 05-04. LOGOUT and ACCOUNT_DELETION only, once every object is removed:
|
||||
* how the sweep of the partition's now unreferenced chunks ended. Chunk
|
||||
* bytes are gone only when it is `COMPLETED`.
|
||||
*/
|
||||
chunkSweep?: "COMPLETED" | "DEADLINE_REACHED" | "STAGING_STATE_UNREADABLE";
|
||||
}>;
|
||||
|
||||
export type OpfsSensitiveMaintenanceReason =
|
||||
@@ -210,11 +241,12 @@ export type OpfsPreparedObject = OpfsPreparedObjectV1 | OpfsPreparedObjectV2;
|
||||
|
||||
/**
|
||||
* STO-01. Compensation is only allowed to release journal and budget state
|
||||
* after the physical effect is confirmed. `EFFECT_UNKNOWN` is never a success.
|
||||
* after the physical effect is confirmed. An effect the worker cannot confirm
|
||||
* comes back as a failure, never as a success value (05-09).
|
||||
*/
|
||||
export type OpfsCleanupEffect =
|
||||
| Readonly<{ kind: "CLEANED" | "ALREADY_CLEAN" }>
|
||||
| Readonly<{ kind: "EFFECT_UNKNOWN" }>;
|
||||
export type OpfsCleanupEffect = Readonly<{
|
||||
kind: "CLEANED" | "ALREADY_CLEAN";
|
||||
}>;
|
||||
|
||||
export type OpfsJournalMutation = "PUT" | "DELETE";
|
||||
export type OpfsJournalPhase =
|
||||
|
||||
@@ -23,12 +23,14 @@ async function startLifecycleRuntime(
|
||||
name: string,
|
||||
targetVersion: 1 | 2,
|
||||
blockedTimeoutMs = 1_000,
|
||||
partitioned = false,
|
||||
): Promise<void> {
|
||||
await page.evaluate(
|
||||
async ({
|
||||
currentDatabaseName,
|
||||
currentTargetVersion,
|
||||
currentBlockedTimeoutMs,
|
||||
currentPartitioned,
|
||||
}) => {
|
||||
const modulePath =
|
||||
"/src/adapters/storage/indexeddb/indexeddb-runtime.ts";
|
||||
@@ -38,24 +40,31 @@ async function startLifecycleRuntime(
|
||||
type LifecycleValue = Readonly<{ value: string }>;
|
||||
const [authorityToken, namespaceToken, partitionToken] =
|
||||
currentDatabaseName.slice("ca-idb-v1:".length).split(".");
|
||||
const accountScope = currentPartitioned
|
||||
? ("OPAQUE_PARTITION" as const)
|
||||
: ("ORIGIN_SHARED" as const);
|
||||
const scope = {
|
||||
authorityToken,
|
||||
namespaceToken,
|
||||
partitionToken,
|
||||
accountScope: "ORIGIN_SHARED" as const,
|
||||
accountScope,
|
||||
};
|
||||
const storagePolicy = {
|
||||
owner: "platform-storage",
|
||||
namespace: "indexeddb-browser-lifecycle",
|
||||
classification: "INTERNAL" as const,
|
||||
authority: "SERVER" as const,
|
||||
accountScope: "ORIGIN_SHARED" as const,
|
||||
accountScope,
|
||||
retention: { kind: "EXPLICIT_DELETE" as const },
|
||||
softBudgetBytes: 1_000_000,
|
||||
hardBudgetBytes: 2_000_000,
|
||||
evictionPriority: "SYNCED_COPY" as const,
|
||||
logoutAction: "KEEP_ORIGIN_SHARED" as const,
|
||||
accountDeletionAction: "KEEP_ORIGIN_SHARED" as const,
|
||||
logoutAction: currentPartitioned
|
||||
? ("PURGE_PARTITION" as const)
|
||||
: ("KEEP_ORIGIN_SHARED" as const),
|
||||
accountDeletionAction: currentPartitioned
|
||||
? ("PURGE_PARTITION" as const)
|
||||
: ("KEEP_ORIGIN_SHARED" as const),
|
||||
pressureAction: "RETAIN" as const,
|
||||
unavailableFallback: "ONLINE_ONLY" as const,
|
||||
};
|
||||
@@ -184,6 +193,7 @@ async function startLifecycleRuntime(
|
||||
currentDatabaseName: name,
|
||||
currentTargetVersion: targetVersion,
|
||||
currentBlockedTimeoutMs: blockedTimeoutMs,
|
||||
currentPartitioned: partitioned,
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -1067,3 +1077,155 @@ test("fails a blocked v2 upgrade closed, then closes its late connection after t
|
||||
await blockerPage.close();
|
||||
}
|
||||
});
|
||||
|
||||
test("05-03 deletes a partition's dataset for an authorized ACCOUNT_DELETION when its budget row is corrupt", async ({
|
||||
page,
|
||||
}) => {
|
||||
const name = databaseName("corrupt-governance");
|
||||
await page.goto("/");
|
||||
await startLifecycleRuntime(page, name, 1, 1_000, true);
|
||||
expect(await settleLifecycleRuntime(page)).toMatchObject({
|
||||
opened: { ok: true },
|
||||
});
|
||||
expect(
|
||||
await page.evaluate(async () => {
|
||||
const runtime = (
|
||||
globalThis as unknown as {
|
||||
__indexedDbLifecycleRuntime: {
|
||||
compareAndSwap(input: unknown): Promise<unknown>;
|
||||
close(): void;
|
||||
};
|
||||
}
|
||||
).__indexedDbLifecycleRuntime;
|
||||
const written = await runtime.compareAndSwap({
|
||||
key: "personal-1",
|
||||
value: { value: "mine" },
|
||||
expectedRevision: null,
|
||||
idempotencyKey: "personal-create",
|
||||
});
|
||||
runtime.close();
|
||||
return written;
|
||||
}),
|
||||
).toMatchObject({ ok: true });
|
||||
expect(
|
||||
await page.evaluate(
|
||||
async (currentDatabaseName) =>
|
||||
await new Promise<string>((resolve) => {
|
||||
const request = indexedDB.open(currentDatabaseName, 1);
|
||||
request.onerror = () =>
|
||||
resolve(request.error?.name ?? "UNKNOWN_ERROR");
|
||||
request.onsuccess = () => {
|
||||
const database = request.result;
|
||||
const transaction = database.transaction(
|
||||
"governance",
|
||||
"readwrite",
|
||||
);
|
||||
transaction.objectStore("governance").put({
|
||||
bindingKey: "dataset-budget",
|
||||
budgetVersion: 1,
|
||||
usedBytes: -5,
|
||||
receiptCount: 1,
|
||||
});
|
||||
transaction.oncomplete = () => {
|
||||
database.close();
|
||||
resolve("CORRUPTED");
|
||||
};
|
||||
transaction.onabort = () => {
|
||||
database.close();
|
||||
resolve("ABORTED");
|
||||
};
|
||||
};
|
||||
}),
|
||||
name,
|
||||
),
|
||||
).toBe("CORRUPTED");
|
||||
|
||||
await startLifecycleRuntime(page, name, 1, 1_000, true);
|
||||
expect(await settleLifecycleRuntime(page)).toMatchObject({
|
||||
opened: { ok: false, error: { code: "POLICY_REJECTED" } },
|
||||
});
|
||||
const purged = await page.evaluate(async () => {
|
||||
const runtime = (
|
||||
globalThis as unknown as {
|
||||
__indexedDbLifecycleRuntime: {
|
||||
enforceLifecycleBatch(input: unknown): Promise<unknown>;
|
||||
};
|
||||
}
|
||||
).__indexedDbLifecycleRuntime;
|
||||
return await runtime.enforceLifecycleBatch({
|
||||
action: "ACCOUNT_DELETION",
|
||||
maxRows: 100,
|
||||
maxDurationMs: 5_000,
|
||||
});
|
||||
});
|
||||
expect(purged).toMatchObject({
|
||||
ok: true,
|
||||
value: { state: "COMPLETE", datasetDeleted: true },
|
||||
});
|
||||
const remaining = await page.evaluate(async (currentDatabaseName) => {
|
||||
(
|
||||
globalThis as unknown as {
|
||||
__indexedDbLifecycleRuntime?: { close(): void };
|
||||
}
|
||||
).__indexedDbLifecycleRuntime?.close();
|
||||
// A fresh open of a deleted database runs its upgrade from version 0.
|
||||
return await new Promise<number>((resolve, reject) => {
|
||||
const request = indexedDB.open(currentDatabaseName);
|
||||
let oldVersion = -1;
|
||||
request.onupgradeneeded = (event) => {
|
||||
oldVersion = event.oldVersion;
|
||||
};
|
||||
request.onsuccess = () => {
|
||||
request.result.close();
|
||||
const deletion = indexedDB.deleteDatabase(currentDatabaseName);
|
||||
deletion.onsuccess = () => resolve(oldVersion);
|
||||
deletion.onerror = () => reject(deletion.error);
|
||||
};
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
}, name);
|
||||
expect(remaining).toBe(0);
|
||||
});
|
||||
|
||||
test("05-06 an older tab meets a schema a newer tab upgraded and goes online-only", async ({
|
||||
page,
|
||||
}) => {
|
||||
const olderPage = await page.context().newPage();
|
||||
const name = databaseName("future-schema");
|
||||
try {
|
||||
await Promise.all([page.goto("/"), olderPage.goto("/")]);
|
||||
expect(await installLifecycleRuntime(page, name, 2)).toMatchObject({
|
||||
opened: { ok: true },
|
||||
status: { kind: "READY", schemaVersion: 2 },
|
||||
});
|
||||
await closeLifecycleRuntime(page);
|
||||
|
||||
expect(await installLifecycleRuntime(olderPage, name, 1)).toEqual({
|
||||
opened: {
|
||||
ok: false,
|
||||
error: {
|
||||
code: "UNSUPPORTED",
|
||||
operation: "INDEXEDDB_OPEN",
|
||||
retryable: false,
|
||||
recovery: "ONLINE_ONLY",
|
||||
},
|
||||
},
|
||||
status: { kind: "INCOMPATIBLE", schemaVersion: 1 },
|
||||
});
|
||||
await closeLifecycleRuntime(olderPage);
|
||||
const deletion = await page.evaluate(
|
||||
async (currentDatabaseName) =>
|
||||
await new Promise<string>((resolve) => {
|
||||
const request = indexedDB.deleteDatabase(currentDatabaseName);
|
||||
request.onsuccess = () => resolve("DELETED");
|
||||
request.onerror = () =>
|
||||
resolve(request.error?.name ?? "UNKNOWN_ERROR");
|
||||
request.onblocked = () => resolve("BLOCKED");
|
||||
}),
|
||||
name,
|
||||
);
|
||||
expect(deletion).toBe("DELETED");
|
||||
} finally {
|
||||
await olderPage.close();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -219,3 +219,193 @@ test("runs put, open, remove and reconcile through a native DedicatedWorker, OPF
|
||||
expect(result.deletion).toBe("DELETED");
|
||||
expect(["DELETED", "ABSENT"]).toContain(result.opfsCleanup);
|
||||
});
|
||||
|
||||
type OpfsTabScope = Readonly<{
|
||||
namespace: string;
|
||||
authorityToken: string;
|
||||
namespaceToken: string;
|
||||
partitionToken: string;
|
||||
}>;
|
||||
|
||||
/**
|
||||
* Installs an OPFS runtime for `scope` in this tab and keeps it on
|
||||
* `globalThis.__opfsTab` for later evaluate calls.
|
||||
*/
|
||||
async function installOpfsTab(
|
||||
page: import("@playwright/test").Page,
|
||||
scope: OpfsTabScope,
|
||||
): Promise<boolean> {
|
||||
return await page.evaluate(async (tabScope) => {
|
||||
const runtimeModulePath =
|
||||
"/src/adapters/storage/opfs/browser-opfs-runtime.ts";
|
||||
const policyModulePath = "/src/adapters/storage/opfs/opfs-policy.ts";
|
||||
const { createBrowserOpfsRuntime, inspectBrowserOpfsSupport } =
|
||||
(await import(
|
||||
/* @vite-ignore */ runtimeModulePath
|
||||
)) as typeof import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts");
|
||||
const { resolveOpfsRuntimePolicy } = (await import(
|
||||
/* @vite-ignore */ policyModulePath
|
||||
)) as typeof import("../../src/adapters/storage/opfs/opfs-policy.ts");
|
||||
const policy = resolveOpfsRuntimePolicy();
|
||||
if (!inspectBrowserOpfsSupport(policy).ok) return false;
|
||||
(globalThis as unknown as { __opfsTab: unknown }).__opfsTab =
|
||||
createBrowserOpfsRuntime({
|
||||
workerUrl: new URL(
|
||||
"/tests/browser-capabilities/opfs-test.worker.ts",
|
||||
location.href,
|
||||
),
|
||||
workerName: `opfs-tab-${crypto.randomUUID()}`,
|
||||
scope: tabScope,
|
||||
storagePolicy: {
|
||||
owner: "browser-capability",
|
||||
namespace: tabScope.namespace,
|
||||
classification: "PERSONAL",
|
||||
authority: "LOCAL_FIRST",
|
||||
accountScope: "OPAQUE_PARTITION",
|
||||
retention: { kind: "EXPLICIT_DELETE" },
|
||||
softBudgetBytes: 1024 * 1024,
|
||||
hardBudgetBytes: 2 * 1024 * 1024,
|
||||
evictionPriority: "USER_AUTHORED",
|
||||
logoutAction: "EXPORT_THEN_PURGE",
|
||||
accountDeletionAction: "PURGE_PARTITION",
|
||||
pressureAction: "RETAIN",
|
||||
unavailableFallback: "EXPORT_REQUIRED",
|
||||
},
|
||||
policy,
|
||||
});
|
||||
return true;
|
||||
}, scope);
|
||||
}
|
||||
|
||||
test("05-01 a reconcile in another tab skips a live put and the put stays readable", async ({
|
||||
page,
|
||||
}) => {
|
||||
const secondPage = await page.context().newPage();
|
||||
const suffix = crypto.randomUUID().replaceAll("-", "");
|
||||
const scope: OpfsTabScope = {
|
||||
namespace: "durable-objects",
|
||||
authorityToken: `authority_${suffix}`,
|
||||
namespaceToken: `namespace_${suffix}`,
|
||||
partitionToken: `partition_${suffix}`,
|
||||
};
|
||||
type Tab = import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts").BrowserOpfsRuntime;
|
||||
try {
|
||||
await Promise.all([page.goto("/"), secondPage.goto("/")]);
|
||||
if (!(await installOpfsTab(page, scope))) return;
|
||||
expect(await installOpfsTab(secondPage, scope)).toBe(true);
|
||||
|
||||
// Tab A starts a put whose source stalls, so its journal row stays
|
||||
// PREPARING and its worker holds the OPFS mutation lock.
|
||||
await page.evaluate(async () => {
|
||||
const state = globalThis as unknown as {
|
||||
__opfsTab: Tab;
|
||||
__release?: () => void;
|
||||
__put?: Promise<unknown>;
|
||||
};
|
||||
let streaming!: () => void;
|
||||
const started = new Promise<void>((resolve) => {
|
||||
streaming = resolve;
|
||||
});
|
||||
const gate = new Promise<void>((resolve) => {
|
||||
state.__release = resolve;
|
||||
});
|
||||
state.__put = state.__opfsTab.objects.put({
|
||||
objectId: "object_tab_12345678",
|
||||
expectedGeneration: null,
|
||||
mediaType: "application/octet-stream",
|
||||
source: {
|
||||
byteLength: 4,
|
||||
async *stream() {
|
||||
streaming();
|
||||
await gate;
|
||||
yield { ok: true as const, value: new Uint8Array([2, 7, 1, 8]) };
|
||||
},
|
||||
},
|
||||
});
|
||||
await started;
|
||||
});
|
||||
|
||||
// Tab B reconciles while tab A's put is in flight.
|
||||
await secondPage.evaluate(() => {
|
||||
const state = globalThis as unknown as {
|
||||
__opfsTab: Tab;
|
||||
__reconcile?: Promise<unknown>;
|
||||
};
|
||||
state.__reconcile = state.__opfsTab.maintenance.reconcile();
|
||||
});
|
||||
await page.waitForTimeout(250);
|
||||
const put = await page.evaluate(async () => {
|
||||
const state = globalThis as unknown as {
|
||||
__release: () => void;
|
||||
__put: Promise<unknown>;
|
||||
};
|
||||
state.__release();
|
||||
return await state.__put;
|
||||
});
|
||||
const reconciled = await secondPage.evaluate(
|
||||
async () =>
|
||||
await (globalThis as unknown as { __reconcile: Promise<unknown> })
|
||||
.__reconcile,
|
||||
);
|
||||
const opened = await page.evaluate(async () => {
|
||||
const runtime = (globalThis as unknown as { __opfsTab: Tab }).__opfsTab;
|
||||
const result = await runtime.objects.open({
|
||||
objectId: "object_tab_12345678",
|
||||
});
|
||||
if (!result.ok) return result.error.code;
|
||||
const bytes: number[] = [];
|
||||
for await (const chunk of result.value.source.stream(
|
||||
new AbortController().signal,
|
||||
)) {
|
||||
if (!chunk.ok) return chunk.error.code;
|
||||
bytes.push(...chunk.value);
|
||||
}
|
||||
return bytes;
|
||||
});
|
||||
|
||||
expect(put).toMatchObject({ ok: true, value: { generation: 1 } });
|
||||
expect(reconciled).toMatchObject({
|
||||
ok: true,
|
||||
value: { skippedLiveTransactions: 1, rolledBackTransactions: 0 },
|
||||
});
|
||||
expect(opened).toEqual([2, 7, 1, 8]);
|
||||
} finally {
|
||||
for (const tab of [page, secondPage]) {
|
||||
await tab.evaluate(() =>
|
||||
(globalThis as unknown as { __opfsTab?: Tab }).__opfsTab?.close(),
|
||||
);
|
||||
}
|
||||
const cleanup = await page.evaluate(async (tabScope) => {
|
||||
const journalModulePath =
|
||||
"/src/adapters/storage/opfs/indexeddb-opfs-journal.ts";
|
||||
const policyModulePath = "/src/adapters/storage/opfs/opfs-policy.ts";
|
||||
const { opfsJournalDatabaseName } = (await import(
|
||||
/* @vite-ignore */ journalModulePath
|
||||
)) as typeof import("../../src/adapters/storage/opfs/indexeddb-opfs-journal.ts");
|
||||
const { DEFAULT_OPFS_RUNTIME_POLICY } = (await import(
|
||||
/* @vite-ignore */ policyModulePath
|
||||
)) as typeof import("../../src/adapters/storage/opfs/opfs-policy.ts");
|
||||
const deletion = await new Promise<string>((resolve) => {
|
||||
const request = indexedDB.deleteDatabase(
|
||||
opfsJournalDatabaseName(tabScope.authorityToken),
|
||||
);
|
||||
request.onsuccess = () => resolve("DELETED");
|
||||
request.onerror = () => resolve(request.error?.name ?? "UNKNOWN");
|
||||
request.onblocked = () => resolve("BLOCKED");
|
||||
});
|
||||
try {
|
||||
const root = await (
|
||||
await navigator.storage.getDirectory()
|
||||
).getDirectoryHandle(DEFAULT_OPFS_RUNTIME_POLICY.rootDirectoryName);
|
||||
await (
|
||||
await root.getDirectoryHandle("authorities")
|
||||
).removeEntry(tabScope.authorityToken, { recursive: true });
|
||||
} catch {
|
||||
// Nothing was written for this authority.
|
||||
}
|
||||
return deletion;
|
||||
}, scope);
|
||||
expect(cleanup).toBe("DELETED");
|
||||
await secondPage.close();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -24,6 +24,31 @@ type StoreState = {
|
||||
indexes: Map<string, IndexState>;
|
||||
};
|
||||
|
||||
/** One named database: its schema, rows and version. */
|
||||
type DatabaseState = {
|
||||
stores: Map<string, StoreState>;
|
||||
version: number;
|
||||
latest: FakeDatabase | null;
|
||||
/** Connections handed out by a successful open, closed or not. */
|
||||
connections: FakeDatabase[];
|
||||
/** Transactions that have not finished, in creation order. */
|
||||
transactions: FakeTransaction[];
|
||||
/** A `deleteDatabase` waiting for the open connections to close. */
|
||||
pendingDelete: (() => void) | null;
|
||||
};
|
||||
|
||||
function storeOf(state: DatabaseState, name: string): StoreState {
|
||||
const store = state.stores.get(name);
|
||||
if (!store) {
|
||||
throw new DOMException("Store does not exist.", "NotFoundError");
|
||||
}
|
||||
return store;
|
||||
}
|
||||
|
||||
// The fake decides when a transaction's task has ended with a real macrotask,
|
||||
// so a test that fakes timers cannot keep a transaction alive by accident.
|
||||
const realSetTimeout = globalThis.setTimeout.bind(globalThis);
|
||||
|
||||
type FakeKeyRange =
|
||||
| Readonly<{
|
||||
fakeKind: "ONLY";
|
||||
@@ -221,23 +246,26 @@ class FakeDatabase {
|
||||
onversionchange: ((event: IDBVersionChangeEvent) => unknown) | null = null;
|
||||
onclose: ((event: Event) => unknown) | null = null;
|
||||
|
||||
constructor(private readonly owner: MemoryIndexedDbFactory) {}
|
||||
constructor(
|
||||
private readonly owner: MemoryIndexedDbFactory,
|
||||
readonly state: DatabaseState,
|
||||
) {}
|
||||
|
||||
get objectStoreNames(): DOMStringList {
|
||||
return stringList(this.owner.storeNames());
|
||||
return stringList(this.state.stores.keys());
|
||||
}
|
||||
|
||||
createObjectStore(
|
||||
name: string,
|
||||
options?: IDBObjectStoreParameters,
|
||||
): IDBObjectStore {
|
||||
if (this.owner.hasStore(name)) {
|
||||
if (this.state.stores.has(name)) {
|
||||
throw new DOMException("Store already exists.", "ConstraintError");
|
||||
}
|
||||
if (typeof options?.keyPath !== "string") {
|
||||
throw new DOMException("The fake requires a string key path.", "DataError");
|
||||
}
|
||||
this.owner.createStore(name, {
|
||||
this.state.stores.set(name, {
|
||||
keyPath: options.keyPath,
|
||||
autoIncrement: options.autoIncrement ?? false,
|
||||
data: new Map(),
|
||||
@@ -247,13 +275,13 @@ class FakeDatabase {
|
||||
}
|
||||
|
||||
deleteObjectStore(name: string): void {
|
||||
if (!this.owner.deleteStore(name)) {
|
||||
if (!this.state.stores.delete(name)) {
|
||||
throw new DOMException("Store does not exist.", "NotFoundError");
|
||||
}
|
||||
}
|
||||
|
||||
schemaObjectStore(name: string): IDBObjectStore {
|
||||
const state = this.owner.store(name);
|
||||
const state = storeOf(this.state, name);
|
||||
return new FakeObjectStore(null, state) as unknown as IDBObjectStore;
|
||||
}
|
||||
|
||||
@@ -261,7 +289,7 @@ class FakeDatabase {
|
||||
name: string,
|
||||
transaction: FakeUpgradeTransaction,
|
||||
): IDBObjectStore {
|
||||
const state = this.owner.store(name);
|
||||
const state = storeOf(this.state, name);
|
||||
return new FakeObjectStore(
|
||||
null,
|
||||
state,
|
||||
@@ -278,16 +306,22 @@ class FakeDatabase {
|
||||
}
|
||||
const names =
|
||||
typeof storeNames === "string" ? [storeNames] : [...storeNames];
|
||||
const transaction = new FakeTransaction(this.owner, names, mode);
|
||||
const transaction = new FakeTransaction(
|
||||
this.owner,
|
||||
this.state,
|
||||
names,
|
||||
mode,
|
||||
);
|
||||
this.owner.noteTransaction(transaction);
|
||||
return transaction as unknown as IDBTransaction;
|
||||
}
|
||||
|
||||
close(): void {
|
||||
this.closed = true;
|
||||
this.state.pendingDelete?.();
|
||||
}
|
||||
|
||||
fireVersionChange(newVersion: number): void {
|
||||
fireVersionChange(newVersion: number | null): void {
|
||||
this.onversionchange?.({
|
||||
oldVersion: this.version,
|
||||
newVersion,
|
||||
@@ -307,19 +341,108 @@ class FakeTransaction {
|
||||
error: DOMException | null = null;
|
||||
readonly mode: IDBTransactionMode;
|
||||
private active = true;
|
||||
/**
|
||||
* Native "active" state: a request may be placed only in the task that
|
||||
* created the transaction or while one of its request events (and the
|
||||
* microtasks that event queued) is being handled.
|
||||
*/
|
||||
private accepting = true;
|
||||
/** Held back until every earlier overlapping transaction finishes. */
|
||||
private started = false;
|
||||
private readonly deferred: Array<() => void> = [];
|
||||
private completionQueued = false;
|
||||
private pending = 0;
|
||||
private readonly workingStores: Map<string, StoreState>;
|
||||
|
||||
constructor(
|
||||
private readonly owner: MemoryIndexedDbFactory,
|
||||
private readonly database: DatabaseState,
|
||||
private readonly names: readonly string[],
|
||||
mode: IDBTransactionMode,
|
||||
) {
|
||||
this.mode = mode;
|
||||
this.workingStores = new Map(
|
||||
names.map((name) => [name, cloneStore(owner.store(name))]),
|
||||
names.map((name) => [name, cloneStore(storeOf(database, name))]),
|
||||
);
|
||||
database.transactions.push(this);
|
||||
this.startIfUnblocked();
|
||||
this.deactivateAfterTask();
|
||||
}
|
||||
|
||||
/**
|
||||
* Native IndexedDB runs transactions with overlapping scopes in creation
|
||||
* order unless both are readonly. A later one starts from the rows the
|
||||
* earlier one committed.
|
||||
*/
|
||||
startIfUnblocked(): void {
|
||||
if (this.started || !this.active) return;
|
||||
const earlier = this.database.transactions.slice(
|
||||
0,
|
||||
this.database.transactions.indexOf(this),
|
||||
);
|
||||
if (earlier.some((transaction) => transaction.blocks(this))) return;
|
||||
this.started = true;
|
||||
for (const name of this.names) {
|
||||
const fresh = cloneStore(storeOf(this.database, name));
|
||||
const working = this.workingStores.get(name)!;
|
||||
working.data = fresh.data;
|
||||
working.indexes = fresh.indexes;
|
||||
}
|
||||
for (const callback of this.deferred.splice(0)) {
|
||||
this.owner.scheduleTransaction(callback, this.mode);
|
||||
}
|
||||
if (!this.accepting) this.commitIfIdle();
|
||||
}
|
||||
|
||||
private blocks(later: FakeTransaction): boolean {
|
||||
return (
|
||||
this.active &&
|
||||
(this.mode === "readwrite" || later.mode === "readwrite") &&
|
||||
this.names.some((name) => later.names.includes(name))
|
||||
);
|
||||
}
|
||||
|
||||
private schedule(callback: () => void): void {
|
||||
if (this.started) {
|
||||
this.owner.scheduleTransaction(callback, this.mode);
|
||||
} else {
|
||||
this.deferred.push(callback);
|
||||
}
|
||||
}
|
||||
|
||||
private deactivateAfterTask(): void {
|
||||
realSetTimeout(() => {
|
||||
this.accepting = false;
|
||||
this.commitIfIdle();
|
||||
}, 0);
|
||||
}
|
||||
|
||||
/** A native transaction with no request left commits once inactive. */
|
||||
private commitIfIdle(): void {
|
||||
if (this.started && this.pending === 0) this.queueCompletion();
|
||||
}
|
||||
|
||||
private assertAccepting(): void {
|
||||
if (!this.active || !this.accepting) {
|
||||
throw new DOMException(
|
||||
"The transaction is not active.",
|
||||
"TransactionInactiveError",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private dispatch(handler: () => void): void {
|
||||
this.accepting = true;
|
||||
handler();
|
||||
this.deactivateAfterTask();
|
||||
}
|
||||
|
||||
private finish(): void {
|
||||
const index = this.database.transactions.indexOf(this);
|
||||
if (index >= 0) this.database.transactions.splice(index, 1);
|
||||
for (const transaction of [...this.database.transactions]) {
|
||||
transaction.startIfUnblocked();
|
||||
}
|
||||
}
|
||||
|
||||
objectStore(name: string): IDBObjectStore {
|
||||
@@ -341,22 +464,23 @@ class FakeTransaction {
|
||||
}
|
||||
|
||||
request<Value>(operation: () => Value): IDBRequest<Value> {
|
||||
this.assertAccepting();
|
||||
const request = createRequest<Value>();
|
||||
this.pending += 1;
|
||||
this.owner.scheduleTransaction(() => {
|
||||
this.schedule(() => {
|
||||
if (!this.active) return;
|
||||
try {
|
||||
request.result = structuredClone(operation());
|
||||
request.onsuccess?.(new Event("success"));
|
||||
this.dispatch(() => request.onsuccess?.(new Event("success")));
|
||||
} catch (error) {
|
||||
request.error = asException(error);
|
||||
request.onerror?.(new Event("error"));
|
||||
this.dispatch(() => request.onerror?.(new Event("error")));
|
||||
this.abortWith(request.error);
|
||||
}
|
||||
if (!this.active) return;
|
||||
this.pending -= 1;
|
||||
this.queueCompletion();
|
||||
}, this.mode);
|
||||
});
|
||||
return request as unknown as IDBRequest<Value>;
|
||||
}
|
||||
|
||||
@@ -368,12 +492,13 @@ class FakeTransaction {
|
||||
}>[],
|
||||
direction: IDBCursorDirection,
|
||||
): IDBRequest<IDBCursorWithValue | null> {
|
||||
this.assertAccepting();
|
||||
const request = createRequest<IDBCursorWithValue | null>();
|
||||
let position = 0;
|
||||
this.pending += 1;
|
||||
|
||||
const emit = () => {
|
||||
this.owner.scheduleTransaction(() => {
|
||||
this.schedule(() => {
|
||||
if (!this.active) return;
|
||||
let continued = false;
|
||||
let eventReturned = false;
|
||||
@@ -445,12 +570,12 @@ class FakeTransaction {
|
||||
},
|
||||
} as unknown as IDBCursorWithValue;
|
||||
}
|
||||
request.onsuccess?.(new Event("success"));
|
||||
this.dispatch(() => request.onsuccess?.(new Event("success")));
|
||||
eventReturned = true;
|
||||
if (continued || !this.active) return;
|
||||
this.pending -= 1;
|
||||
this.queueCompletion();
|
||||
}, this.mode);
|
||||
});
|
||||
};
|
||||
|
||||
emit();
|
||||
@@ -472,7 +597,7 @@ class FakeTransaction {
|
||||
return;
|
||||
}
|
||||
this.completionQueued = true;
|
||||
this.owner.scheduleTransaction(() => {
|
||||
this.schedule(() => {
|
||||
this.completionQueued = false;
|
||||
if (!this.active || this.pending !== 0) return;
|
||||
const commitFailure =
|
||||
@@ -485,18 +610,20 @@ class FakeTransaction {
|
||||
}
|
||||
if (this.mode === "readwrite") {
|
||||
for (const [name, state] of this.workingStores) {
|
||||
this.owner.replaceStore(name, cloneStore(state));
|
||||
this.database.stores.set(name, cloneStore(state));
|
||||
}
|
||||
}
|
||||
this.active = false;
|
||||
this.finish();
|
||||
this.oncomplete?.(new Event("complete"));
|
||||
}, this.mode);
|
||||
});
|
||||
}
|
||||
|
||||
private abortWith(error: DOMException): void {
|
||||
if (!this.active) return;
|
||||
this.active = false;
|
||||
this.error = error;
|
||||
this.finish();
|
||||
this.owner.scheduleLifecycle(() => {
|
||||
this.onerror?.(new Event("error"));
|
||||
this.onabort?.(new Event("abort"));
|
||||
@@ -721,14 +848,36 @@ class FakeIndex {
|
||||
}
|
||||
|
||||
/**
|
||||
* Minimal deterministic IndexedDB model for adapter contract tests. It models
|
||||
* request ordering, transaction-level commit/rollback and upgrade blocking;
|
||||
* it is not intended to polyfill IndexedDB for application code.
|
||||
* Minimal deterministic IndexedDB model for adapter contract tests. It is not
|
||||
* intended to polyfill IndexedDB for application code.
|
||||
*
|
||||
* Modeled like a browser:
|
||||
* - request ordering and transaction-level commit/rollback;
|
||||
* - one database per name (`open(name)`), so two names never share rows;
|
||||
* - transactions with overlapping scopes run one after another unless both
|
||||
* are readonly, and a later one sees what the earlier one committed;
|
||||
* - a request placed after the transaction's task ended, outside a request
|
||||
* event and its microtasks, throws `TransactionInactiveError`, and a
|
||||
* transaction with nothing left to do commits on its own;
|
||||
* - upgrade blocking, through `blockNextOpen`/`releaseBlockedOpen`.
|
||||
*
|
||||
* Not modeled (05-08) — use a browser capability spec for these:
|
||||
* - key comparison is by `String`/JSON, and a compound (array) index key path
|
||||
* is not read, so such an index orders by primary key;
|
||||
* - opening a higher version does not fire `versionchange` on open
|
||||
* connections, and `onblocked` fires only for `blockNextOpen`; tests drive
|
||||
* `triggerVersionChange` by hand;
|
||||
* - the upgrade transaction ends synchronously, so an abort requested from an
|
||||
* asynchronous step inside `onupgradeneeded` is not rolled back;
|
||||
* - quota: writes never fail for lack of space unless a test injects it with
|
||||
* `failNextWriteCommit`.
|
||||
*
|
||||
* `seed`, `readRaw` and the lifecycle triggers act on the database opened
|
||||
* most recently.
|
||||
*/
|
||||
export class MemoryIndexedDbFactory {
|
||||
private stores = new Map<string, StoreState>();
|
||||
private databaseVersion = 0;
|
||||
private latestDatabase = new FakeDatabase(this);
|
||||
private readonly databases = new Map<string, DatabaseState>();
|
||||
private current: DatabaseState = this.databaseNamed("");
|
||||
private blockedRequest: Readonly<{
|
||||
request: MutableOpenRequest;
|
||||
targetVersion: number;
|
||||
@@ -741,8 +890,10 @@ export class MemoryIndexedDbFactory {
|
||||
lastTransaction: FakeTransaction | null = null;
|
||||
|
||||
readonly factory: IDBFactory = {
|
||||
open: (_name: string, version?: number) =>
|
||||
this.open(version ?? 1) as unknown as IDBOpenDBRequest,
|
||||
open: (name: string, version?: number) =>
|
||||
this.open(name, version ?? 1) as unknown as IDBOpenDBRequest,
|
||||
deleteDatabase: (name: string) =>
|
||||
this.deleteDatabase(name) as unknown as IDBOpenDBRequest,
|
||||
cmp: compareKeys,
|
||||
} as IDBFactory;
|
||||
|
||||
@@ -782,10 +933,12 @@ export class MemoryIndexedDbFactory {
|
||||
}) as unknown as IDBKeyRange,
|
||||
};
|
||||
|
||||
open(version: number): MutableOpenRequest {
|
||||
const database = new FakeDatabase(this);
|
||||
database.version = this.databaseVersion;
|
||||
this.latestDatabase = database;
|
||||
open(name: string, version: number): MutableOpenRequest {
|
||||
const state = this.databaseNamed(name);
|
||||
const database = new FakeDatabase(this, state);
|
||||
database.version = state.version;
|
||||
state.latest = database;
|
||||
this.current = state;
|
||||
const request: MutableOpenRequest = {
|
||||
...createRequest<IDBDatabase>(),
|
||||
result: database as unknown as IDBDatabase,
|
||||
@@ -802,7 +955,7 @@ export class MemoryIndexedDbFactory {
|
||||
database,
|
||||
};
|
||||
request.onblocked?.({
|
||||
oldVersion: this.databaseVersion,
|
||||
oldVersion: state.version,
|
||||
newVersion: version,
|
||||
} as IDBVersionChangeEvent);
|
||||
return;
|
||||
@@ -812,6 +965,51 @@ export class MemoryIndexedDbFactory {
|
||||
return request;
|
||||
}
|
||||
|
||||
/**
|
||||
* Like a browser: open connections get `versionchange` (new version
|
||||
* `null`); if any stays open the request reports `blocked` and completes
|
||||
* once the last one closes. The name then holds an empty version-0
|
||||
* database, so `hasStore` answers false.
|
||||
*/
|
||||
deleteDatabase(name: string): MutableOpenRequest {
|
||||
const request: MutableOpenRequest = {
|
||||
...createRequest<IDBDatabase>(),
|
||||
transaction: null,
|
||||
onblocked: null,
|
||||
onupgradeneeded: null,
|
||||
};
|
||||
queueMicrotask(() => {
|
||||
const state = this.databaseNamed(name);
|
||||
const oldVersion = state.version;
|
||||
const finish = () => {
|
||||
if (state.connections.some((connection) => !connection.closed)) {
|
||||
return;
|
||||
}
|
||||
state.pendingDelete = null;
|
||||
state.stores = new Map();
|
||||
state.version = 0;
|
||||
state.connections = [];
|
||||
request.onsuccess?.({
|
||||
oldVersion,
|
||||
newVersion: null,
|
||||
} as unknown as Event);
|
||||
};
|
||||
for (const connection of state.connections) {
|
||||
if (!connection.closed) connection.fireVersionChange(null);
|
||||
}
|
||||
if (state.connections.some((connection) => !connection.closed)) {
|
||||
state.pendingDelete = finish;
|
||||
request.onblocked?.({
|
||||
oldVersion,
|
||||
newVersion: null,
|
||||
} as IDBVersionChangeEvent);
|
||||
return;
|
||||
}
|
||||
finish();
|
||||
});
|
||||
return request;
|
||||
}
|
||||
|
||||
blockNextOpen(): void {
|
||||
this.shouldBlockNextOpen = true;
|
||||
}
|
||||
@@ -848,25 +1046,25 @@ export class MemoryIndexedDbFactory {
|
||||
}
|
||||
|
||||
triggerVersionChange(newVersion: number): void {
|
||||
this.latestDatabase.fireVersionChange(newVersion);
|
||||
this.current.latest?.fireVersionChange(newVersion);
|
||||
}
|
||||
|
||||
triggerForcedClose(): void {
|
||||
this.latestDatabase.fireForcedClose();
|
||||
this.current.latest?.fireForcedClose();
|
||||
}
|
||||
|
||||
isConnectionClosed(): boolean {
|
||||
return this.latestDatabase.closed;
|
||||
return this.current.latest?.closed ?? true;
|
||||
}
|
||||
|
||||
seed(storeName: string, value: unknown): void {
|
||||
const state = this.store(storeName);
|
||||
const state = storeOf(this.current, storeName);
|
||||
const key = primaryKey(state, value);
|
||||
state.data.set(key, structuredClone(value));
|
||||
}
|
||||
|
||||
readRaw(storeName: string, key: string): unknown {
|
||||
const value = this.store(storeName).data.get(key);
|
||||
const value = storeOf(this.current, storeName).data.get(key);
|
||||
return value === undefined ? undefined : structuredClone(value);
|
||||
}
|
||||
|
||||
@@ -896,39 +1094,36 @@ export class MemoryIndexedDbFactory {
|
||||
}
|
||||
|
||||
storeNames(): Iterable<string> {
|
||||
return this.stores.keys();
|
||||
return this.current.stores.keys();
|
||||
}
|
||||
|
||||
hasStore(name: string): boolean {
|
||||
return this.stores.has(name);
|
||||
return this.current.stores.has(name);
|
||||
}
|
||||
|
||||
store(name: string): StoreState {
|
||||
const state = this.stores.get(name);
|
||||
private databaseNamed(name: string): DatabaseState {
|
||||
let state = this.databases.get(name);
|
||||
if (!state) {
|
||||
throw new DOMException("Store does not exist.", "NotFoundError");
|
||||
state = {
|
||||
stores: new Map(),
|
||||
version: 0,
|
||||
latest: null,
|
||||
connections: [],
|
||||
transactions: [],
|
||||
pendingDelete: null,
|
||||
};
|
||||
this.databases.set(name, state);
|
||||
}
|
||||
return state;
|
||||
}
|
||||
|
||||
createStore(name: string, state: StoreState): void {
|
||||
this.stores.set(name, state);
|
||||
}
|
||||
|
||||
replaceStore(name: string, state: StoreState): void {
|
||||
this.stores.set(name, state);
|
||||
}
|
||||
|
||||
deleteStore(name: string): boolean {
|
||||
return this.stores.delete(name);
|
||||
}
|
||||
|
||||
private finishOpen(
|
||||
request: MutableOpenRequest,
|
||||
targetVersion: number,
|
||||
database: FakeDatabase,
|
||||
): void {
|
||||
if (targetVersion < this.databaseVersion) {
|
||||
const state = database.state;
|
||||
if (targetVersion < state.version) {
|
||||
request.error = new DOMException(
|
||||
"Requested version is older.",
|
||||
"VersionError",
|
||||
@@ -936,9 +1131,9 @@ export class MemoryIndexedDbFactory {
|
||||
request.onerror?.(new Event("error"));
|
||||
return;
|
||||
}
|
||||
if (targetVersion > this.databaseVersion) {
|
||||
const before = cloneStores(this.stores);
|
||||
const oldVersion = this.databaseVersion;
|
||||
if (targetVersion > state.version) {
|
||||
const before = cloneStores(state.stores);
|
||||
const oldVersion = state.version;
|
||||
const upgrade = new FakeUpgradeTransaction(database);
|
||||
request.transaction = upgrade as unknown as IDBTransaction;
|
||||
database.version = targetVersion;
|
||||
@@ -948,7 +1143,7 @@ export class MemoryIndexedDbFactory {
|
||||
} as IDBVersionChangeEvent);
|
||||
request.transaction = null;
|
||||
if (upgrade.aborted) {
|
||||
this.stores = before;
|
||||
state.stores = before;
|
||||
database.version = oldVersion;
|
||||
request.error = new DOMException(
|
||||
"Upgrade transaction aborted.",
|
||||
@@ -957,9 +1152,10 @@ export class MemoryIndexedDbFactory {
|
||||
request.onerror?.(new Event("error"));
|
||||
return;
|
||||
}
|
||||
this.databaseVersion = targetVersion;
|
||||
state.version = targetVersion;
|
||||
}
|
||||
database.version = this.databaseVersion;
|
||||
database.version = state.version;
|
||||
state.connections.push(database);
|
||||
request.result = database as unknown as IDBDatabase;
|
||||
request.onsuccess?.(new Event("success"));
|
||||
}
|
||||
|
||||
@@ -943,3 +943,35 @@ describe("IndexedDB database deletion", () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-13 a versionchange that arrives while admission runs", () => {
|
||||
it("closes the connection and ends the open instead of blocking the upgrade", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
let admitting!: () => void;
|
||||
const admissionStarted = new Promise<void>((resolve) => {
|
||||
admitting = resolve;
|
||||
});
|
||||
let finishAdmission!: () => void;
|
||||
const pending = openIndexedDbDatabase<TestFailure>({
|
||||
factory: memory.factory,
|
||||
databaseName: "kernel",
|
||||
version: 1,
|
||||
translate: recordingTranslate([]),
|
||||
upgrade: () => ({ kind: "APPLIED" }),
|
||||
admit: async () => {
|
||||
admitting();
|
||||
await new Promise<void>((resolve) => {
|
||||
finishAdmission = resolve;
|
||||
});
|
||||
return { kind: "ADMIT" };
|
||||
},
|
||||
});
|
||||
await admissionStarted;
|
||||
|
||||
// Another context asks to upgrade while this one is still admitting.
|
||||
memory.triggerVersionChange(2);
|
||||
expect(memory.isConnectionClosed()).toBe(true);
|
||||
finishAdmission();
|
||||
expect(await pending).toEqual({ ok: false, error: { cause: "CLOSED" } });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1123,3 +1123,40 @@ describe("Q-6.5-7 stored-row rules stay the same at every call site", () => {
|
||||
).toMatchObject({ ok: false, error: { code: "POLICY_REJECTED" } });
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-05 a receipt prune steps over a corrupt receipt", () => {
|
||||
it("deletes every expired healthy receipt, leaves the corrupt one and reports CORRUPT_DATA", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
await prepareSchema(memory);
|
||||
seedReceipt(memory, "expired-a", 100);
|
||||
seedReceipt(memory, "expired-b", 200);
|
||||
memory.seed("receipts", { idempotencyKey: "broken", expiresAtEpochMs: 0 });
|
||||
const observations: IndexedDbObservation[] = [];
|
||||
const maintenance = createMaintenance(memory, defaultPolicy(), {
|
||||
nowEpochMilliseconds: () => 300,
|
||||
observe: (event) => observations.push(event),
|
||||
});
|
||||
|
||||
expect(
|
||||
await maintenance.pruneExpiredReceipts({
|
||||
maxRows: 10,
|
||||
maxDurationMs: 10_000,
|
||||
}),
|
||||
).toMatchObject({
|
||||
ok: true,
|
||||
value: { state: "COMPLETE", deletedRows: 2 },
|
||||
});
|
||||
expect(memory.readRaw("receipts", "expired-a")).toBeUndefined();
|
||||
expect(memory.readRaw("receipts", "expired-b")).toBeUndefined();
|
||||
expect(memory.readRaw("receipts", "broken")).toBeDefined();
|
||||
expect(
|
||||
memory.readRaw("governance", "dataset-budget"),
|
||||
).toMatchObject({ receiptCount: 0 });
|
||||
expect(observations).toContainEqual(
|
||||
expect.objectContaining({
|
||||
outcome: "FAILED",
|
||||
failureCode: "CORRUPT_DATA",
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -795,3 +795,28 @@ describe("IndexedDB OPFS journal", () => {
|
||||
).toMatchObject({ ok: false, error: { code: "UNSUPPORTED" } });
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-06 a journal from a newer release", () => {
|
||||
it("answers online-only instead of a migration failure", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const name = opfsJournalDatabaseName("authority_12345678");
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const request = memory.factory.open(name, 2);
|
||||
request.onsuccess = () => {
|
||||
request.result.close();
|
||||
resolve();
|
||||
};
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
const journal = createIndexedDbOpfsJournal({
|
||||
authorityToken: "authority_12345678",
|
||||
factory: memory.factory,
|
||||
keyRange: memory.keyRange,
|
||||
});
|
||||
|
||||
expect(await journal.listIncomplete(10)).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: "UNSUPPORTED", recovery: "ONLINE_ONLY" },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,7 +6,7 @@ import type {
|
||||
IndexedDbObservation,
|
||||
IndexedDbRuntimeDependencies,
|
||||
} from "../../src/adapters/storage/indexeddb/indexeddb-types.ts";
|
||||
import { mapIndexedDbException } from "../../src/adapters/storage/indexeddb/indexeddb-failure.ts";
|
||||
import { mapIndexedDbException } from "../../src/adapters/browser-file-storage/result.ts";
|
||||
import { MemoryIndexedDbFactory } from "../helpers/memory-indexeddb.ts";
|
||||
|
||||
type Item = Readonly<{
|
||||
@@ -1035,9 +1035,10 @@ describe("IndexedDB runtime", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("aborts a sweep whose budget row cannot account for the deleted bytes", async () => {
|
||||
it("05-05 sweeps a row the budget cannot account for and leaves the budget high", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
let epochMs = 1_000;
|
||||
const observations: IndexedDbObservation[] = [];
|
||||
const runtime = createIndexedDbRuntime(
|
||||
dependencies(memory, {
|
||||
storagePolicy: {
|
||||
@@ -1054,6 +1055,7 @@ describe("IndexedDB runtime", () => {
|
||||
authorized: true,
|
||||
proofToken: "sweep_authority_proof_001",
|
||||
}),
|
||||
observe: (event) => observations.push(event),
|
||||
}),
|
||||
);
|
||||
expect(await runtime.open()).toMatchObject({ ok: true });
|
||||
@@ -1066,9 +1068,8 @@ describe("IndexedDB runtime", () => {
|
||||
}),
|
||||
).toMatchObject({ ok: true });
|
||||
|
||||
// A budget that cannot account for the record's bytes is corrupt, and the
|
||||
// sweep only discovers it after both deletions have already succeeded
|
||||
// inside the transaction.
|
||||
// A budget that cannot account for the record's bytes is corrupt. The
|
||||
// sweep finds it only after both deletions succeeded in the transaction.
|
||||
memory.seed("governance", {
|
||||
bindingKey: "dataset-budget",
|
||||
budgetVersion: 1,
|
||||
@@ -1085,15 +1086,20 @@ describe("IndexedDB runtime", () => {
|
||||
maxDurationMs: 1_000,
|
||||
}),
|
||||
).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: "CORRUPT_DATA" },
|
||||
ok: true,
|
||||
value: { state: "COMPLETE", deletedRows: 1 },
|
||||
});
|
||||
// `fail` aborts. Recording the failure without aborting would report the
|
||||
// same code while letting the two deletions commit against a budget that
|
||||
// never moved.
|
||||
expect(aborts.count).toBe(1);
|
||||
expect(memory.readRaw("records", "sweep-record")).toBeDefined();
|
||||
expect(memory.readRaw("retention", "sweep-record")).toBeDefined();
|
||||
expect(aborts.count).toBe(0);
|
||||
expect(memory.readRaw("records", "sweep-record")).toBeUndefined();
|
||||
expect(memory.readRaw("retention", "sweep-record")).toBeUndefined();
|
||||
// Not lowered below what it cannot account for: a budget that reads high
|
||||
// blocks writes, one that reads low would admit them.
|
||||
expect(memory.readRaw("governance", "dataset-budget")).toMatchObject({
|
||||
usedBytes: 0,
|
||||
});
|
||||
expect(observations).toContainEqual(
|
||||
expect.objectContaining({ failureCode: "CORRUPT_DATA" }),
|
||||
);
|
||||
});
|
||||
|
||||
it("requires lifecycle authority and only deletes sync-confirmed rows", async () => {
|
||||
@@ -1453,3 +1459,260 @@ describe("IndexedDB runtime", () => {
|
||||
expect(hardMemory.readRaw("records", "x")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-03 a LOGOUT or ACCOUNT_DELETION purge survives corrupt governance", () => {
|
||||
const partitionScope = {
|
||||
...TEST_SCOPE,
|
||||
accountScope: "OPAQUE_PARTITION" as const,
|
||||
};
|
||||
const otherScope = {
|
||||
...partitionScope,
|
||||
partitionToken: "partitiontoken_002",
|
||||
};
|
||||
const partitionPolicy = {
|
||||
...TEST_STORAGE_POLICY,
|
||||
accountScope: "OPAQUE_PARTITION" as const,
|
||||
logoutAction: "PURGE_PARTITION" as const,
|
||||
accountDeletionAction: "PURGE_PARTITION" as const,
|
||||
};
|
||||
const batch = { maxRows: 100, maxDurationMs: 1_000 } as const;
|
||||
|
||||
/**
|
||||
* Another partition and this one each hold a record in one factory; this
|
||||
* partition's budget row is then corrupted, so a fresh runtime cannot pass
|
||||
* admission (E1).
|
||||
*/
|
||||
async function corruptedPartition(
|
||||
authorizeLifecycle: IndexedDbRuntimeDependencies<
|
||||
Item,
|
||||
Item,
|
||||
Query
|
||||
>["authorizeLifecycle"] = () => ({
|
||||
authorized: true,
|
||||
proofToken: "authorityproof_001",
|
||||
}),
|
||||
) {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const other = createIndexedDbRuntime(
|
||||
dependencies(memory, {
|
||||
scope: otherScope,
|
||||
storagePolicy: partitionPolicy,
|
||||
}),
|
||||
);
|
||||
expect(await other.open()).toMatchObject({ ok: true });
|
||||
await other.compareAndSwap({
|
||||
key: "kept",
|
||||
value: { label: "other", rank: 1 },
|
||||
expectedRevision: null,
|
||||
idempotencyKey: "kept-create",
|
||||
});
|
||||
const writer = createIndexedDbRuntime(
|
||||
dependencies(memory, {
|
||||
scope: partitionScope,
|
||||
storagePolicy: partitionPolicy,
|
||||
}),
|
||||
);
|
||||
expect(await writer.open()).toMatchObject({ ok: true });
|
||||
await writer.compareAndSwap({
|
||||
key: "personal-1",
|
||||
value: { label: "mine", rank: 1 },
|
||||
expectedRevision: null,
|
||||
idempotencyKey: "personal-create",
|
||||
});
|
||||
writer.close();
|
||||
memory.seed("governance", {
|
||||
bindingKey: "dataset-budget",
|
||||
budgetVersion: 1,
|
||||
usedBytes: -5,
|
||||
receiptCount: 1,
|
||||
});
|
||||
const deleteDatabase = vi.spyOn(memory.factory, "deleteDatabase");
|
||||
const fresh = createIndexedDbRuntime(
|
||||
dependencies(memory, {
|
||||
scope: partitionScope,
|
||||
storagePolicy: partitionPolicy,
|
||||
authorizeLifecycle,
|
||||
}),
|
||||
);
|
||||
expect(await fresh.open()).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: "POLICY_REJECTED" },
|
||||
});
|
||||
return { memory, other, fresh, deleteDatabase };
|
||||
}
|
||||
|
||||
it.each(["LOGOUT", "ACCOUNT_DELETION"] as const)(
|
||||
"deletes the partition's whole dataset for an authorized %s and leaves other partitions alone",
|
||||
async (action) => {
|
||||
const { memory, other, fresh, deleteDatabase } =
|
||||
await corruptedPartition();
|
||||
|
||||
expect(
|
||||
await fresh.enforceLifecycleBatch({ action, ...batch }),
|
||||
).toMatchObject({
|
||||
ok: true,
|
||||
value: { state: "COMPLETE", datasetDeleted: true },
|
||||
});
|
||||
expect(deleteDatabase).toHaveBeenCalledTimes(1);
|
||||
expect(deleteDatabase).toHaveBeenCalledWith(
|
||||
indexedDbPhysicalDatabaseName(partitionScope),
|
||||
);
|
||||
expect(memory.hasStore("records")).toBe(false);
|
||||
expect(await other.read("kept")).toMatchObject({
|
||||
ok: true,
|
||||
value: { value: { label: "other", rank: 1 } },
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps the dataset when the authority does not approve", async () => {
|
||||
const { memory, fresh, deleteDatabase } = await corruptedPartition(
|
||||
() => ({ authorized: false }),
|
||||
);
|
||||
|
||||
expect(
|
||||
await fresh.enforceLifecycleBatch({ action: "ACCOUNT_DELETION", ...batch }),
|
||||
).toMatchObject({ ok: false, error: { code: "POLICY_REJECTED" } });
|
||||
expect(deleteDatabase).not.toHaveBeenCalled();
|
||||
expect(memory.readRaw("records", "personal-1")).toBeDefined();
|
||||
});
|
||||
|
||||
it("purges row by row, never the database, when governance is healthy", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const deleteDatabase = vi.spyOn(memory.factory, "deleteDatabase");
|
||||
const runtime = createIndexedDbRuntime(
|
||||
dependencies(memory, {
|
||||
scope: partitionScope,
|
||||
storagePolicy: partitionPolicy,
|
||||
}),
|
||||
);
|
||||
expect(await runtime.open()).toMatchObject({ ok: true });
|
||||
await runtime.compareAndSwap({
|
||||
key: "personal-1",
|
||||
value: { label: "mine", rank: 1 },
|
||||
expectedRevision: null,
|
||||
idempotencyKey: "personal-create",
|
||||
});
|
||||
runtime.close();
|
||||
const fresh = createIndexedDbRuntime(
|
||||
dependencies(memory, {
|
||||
scope: partitionScope,
|
||||
storagePolicy: partitionPolicy,
|
||||
}),
|
||||
);
|
||||
|
||||
const purged = await fresh.enforceLifecycleBatch({
|
||||
action: "LOGOUT",
|
||||
...batch,
|
||||
});
|
||||
expect(purged).toMatchObject({ ok: true, value: { state: "COMPLETE" } });
|
||||
expect(purged.ok && purged.value).not.toHaveProperty("datasetDeleted");
|
||||
expect(deleteDatabase).not.toHaveBeenCalled();
|
||||
expect(memory.hasStore("records")).toBe(true);
|
||||
expect(memory.readRaw("records", "personal-1")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-05 a retention sweep steps over a corrupt row", () => {
|
||||
it("deletes every expired healthy row, leaves the corrupt sidecar and reports it", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
let epochMs = 1_000;
|
||||
const observations: IndexedDbObservation[] = [];
|
||||
const runtime = createIndexedDbRuntime(
|
||||
dependencies(memory, {
|
||||
storagePolicy: {
|
||||
...TEST_STORAGE_POLICY,
|
||||
retention: { kind: "TTL", maxAgeMs: 10 },
|
||||
},
|
||||
keyRange:
|
||||
memory.keyRange as NonNullable<
|
||||
IndexedDbRuntimeDependencies<Item, Item, Query>["keyRange"]
|
||||
>,
|
||||
nowEpochMilliseconds: () => epochMs,
|
||||
nowMonotonicMilliseconds: () => 0,
|
||||
observe: (event) => observations.push(event),
|
||||
}),
|
||||
);
|
||||
expect(await runtime.open()).toMatchObject({ ok: true });
|
||||
for (const key of ["a", "b", "c"]) {
|
||||
await runtime.compareAndSwap({
|
||||
key,
|
||||
value: { label: key, rank: 1 },
|
||||
expectedRevision: null,
|
||||
idempotencyKey: `create-${key}`,
|
||||
});
|
||||
}
|
||||
// Sorts first on the eligibility index and names a record it is not keyed by.
|
||||
memory.seed("retention", {
|
||||
recordKey: "zzz",
|
||||
writtenAtEpochMs: 0,
|
||||
synchronization: "NONE",
|
||||
measuredBytes: 0,
|
||||
eligibleAtEpochMs: 0,
|
||||
});
|
||||
epochMs = 5_000;
|
||||
|
||||
for (let attempt = 0; attempt < 2; attempt += 1) {
|
||||
expect(
|
||||
await runtime.enforceLifecycleBatch({
|
||||
action: "RETENTION_SWEEP",
|
||||
maxRows: 100,
|
||||
maxDurationMs: 1_000,
|
||||
}),
|
||||
).toMatchObject({ ok: true, value: { state: "COMPLETE" } });
|
||||
}
|
||||
expect(
|
||||
["a", "b", "c"].filter(
|
||||
(key) => memory.readRaw("records", key) !== undefined,
|
||||
),
|
||||
).toEqual([]);
|
||||
expect(memory.readRaw("retention", "zzz")).toBeDefined();
|
||||
expect(observations).toContainEqual(
|
||||
expect.objectContaining({
|
||||
operation: "INDEXEDDB_WRITE",
|
||||
outcome: "FAILED",
|
||||
failureCode: "CORRUPT_DATA",
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-06 an older tab that meets a newer schema", () => {
|
||||
it("reports INCOMPATIBLE and online-only, not a migration failure", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const base = dependencies(memory);
|
||||
const newer = createIndexedDbRuntime({
|
||||
...base,
|
||||
schemaVersion: 2,
|
||||
migrations: [
|
||||
...base.migrations,
|
||||
{
|
||||
id: "schema-v2",
|
||||
fromVersion: 1,
|
||||
toVersion: 2,
|
||||
operations: [{ kind: "CREATE_STORE", name: "extra", keyPath: "k" }],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(await newer.open()).toMatchObject({ ok: true });
|
||||
newer.close();
|
||||
const older = createIndexedDbRuntime(dependencies(memory));
|
||||
|
||||
expect(await older.open()).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
code: "UNSUPPORTED",
|
||||
operation: "INDEXEDDB_OPEN",
|
||||
recovery: "ONLINE_ONLY",
|
||||
},
|
||||
});
|
||||
expect(older.getStatus()).toEqual({
|
||||
kind: "INCOMPATIBLE",
|
||||
schemaVersion: 1,
|
||||
});
|
||||
expect(await older.read("x")).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: "UNSUPPORTED", recovery: "ONLINE_ONLY" },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { MemoryIndexedDbFactory } from "../helpers/memory-indexeddb.ts";
|
||||
|
||||
/**
|
||||
* 05-08. Where the fake used to part from native IndexedDB, each case pins
|
||||
* the native result, so a test built on the fake cannot pass on a behavior no
|
||||
* browser has.
|
||||
*/
|
||||
|
||||
function settle<Value>(request: IDBRequest<Value>): Promise<Value> {
|
||||
return new Promise((resolve, reject) => {
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
}
|
||||
|
||||
function outcome(transaction: IDBTransaction): Promise<string> {
|
||||
return new Promise((resolve) => {
|
||||
transaction.oncomplete = () => resolve("complete");
|
||||
transaction.onabort = () =>
|
||||
resolve(`abort:${transaction.error?.name ?? "none"}`);
|
||||
});
|
||||
}
|
||||
|
||||
async function openWithStore(
|
||||
memory: MemoryIndexedDbFactory,
|
||||
name: string,
|
||||
): Promise<IDBDatabase> {
|
||||
const request = memory.factory.open(name, 1);
|
||||
request.onupgradeneeded = () => {
|
||||
request.result.createObjectStore("s", { keyPath: "k" });
|
||||
};
|
||||
return await settle(request);
|
||||
}
|
||||
|
||||
describe("05-08 memory IndexedDB matches native semantics the adapters rely on", () => {
|
||||
it("(a) runs overlapping readwrite transactions one after another", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const database = await openWithStore(memory, "a");
|
||||
memory.seed("s", { k: "counter", n: 0 });
|
||||
const first = database.transaction(["s"], "readwrite");
|
||||
const second = database.transaction(["s"], "readwrite");
|
||||
const settled = [outcome(first), outcome(second)];
|
||||
for (const transaction of [first, second]) {
|
||||
const read = transaction.objectStore("s").get("counter");
|
||||
read.onsuccess = () => {
|
||||
const { n } = read.result as { n: number };
|
||||
transaction.objectStore("s").put({ k: "counter", n: n + 1 });
|
||||
};
|
||||
}
|
||||
|
||||
expect(await Promise.all(settled)).toEqual(["complete", "complete"]);
|
||||
expect(memory.readRaw("s", "counter")).toEqual({ k: "counter", n: 2 });
|
||||
});
|
||||
|
||||
it("(b) rejects a request placed after the transaction's task ended", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const database = await openWithStore(memory, "b");
|
||||
const transaction = database.transaction(["s"], "readwrite");
|
||||
const settled = outcome(transaction);
|
||||
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||
|
||||
expect(() =>
|
||||
transaction.objectStore("s").put({ k: "late" }),
|
||||
).toThrow(expect.objectContaining({ name: "TransactionInactiveError" }));
|
||||
// A transaction with no request left commits on its own.
|
||||
expect(await settled).toBe("complete");
|
||||
expect(memory.readRaw("s", "late")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("(b) keeps a transaction active for requests placed in a success callback's microtasks", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const database = await openWithStore(memory, "b2");
|
||||
const transaction = database.transaction(["s"], "readwrite");
|
||||
const settled = outcome(transaction);
|
||||
const read = transaction.objectStore("s").get("absent");
|
||||
read.onsuccess = () => {
|
||||
void Promise.resolve().then(() => {
|
||||
transaction.objectStore("s").put({ k: "chained" });
|
||||
});
|
||||
};
|
||||
|
||||
expect(await settled).toBe("complete");
|
||||
expect(memory.readRaw("s", "chained")).toEqual({ k: "chained" });
|
||||
});
|
||||
|
||||
it("(d) keeps databases of different names apart", async () => {
|
||||
const memory = new MemoryIndexedDbFactory();
|
||||
const userA = await openWithStore(memory, "user-A");
|
||||
await settle(
|
||||
userA.transaction(["s"], "readwrite").objectStore("s").put({
|
||||
k: "secret-of-A",
|
||||
}),
|
||||
);
|
||||
let upgraded = false;
|
||||
const request = memory.factory.open("user-B", 1);
|
||||
request.onupgradeneeded = () => {
|
||||
upgraded = true;
|
||||
request.result.createObjectStore("s", { keyPath: "k" });
|
||||
};
|
||||
const userB = await settle(request);
|
||||
|
||||
expect(upgraded).toBe(true);
|
||||
expect(
|
||||
await settle(
|
||||
userB.transaction(["s"], "readonly").objectStore("s").get(
|
||||
"secret-of-A",
|
||||
),
|
||||
),
|
||||
).toBeUndefined();
|
||||
expect(
|
||||
await settle(
|
||||
userA.transaction(["s"], "readonly").objectStore("s").get(
|
||||
"secret-of-A",
|
||||
),
|
||||
),
|
||||
).toEqual({ k: "secret-of-A" });
|
||||
});
|
||||
});
|
||||
@@ -630,7 +630,7 @@ describe("OPFS byte-store coordinator", () => {
|
||||
},
|
||||
{
|
||||
async abortPreparedPut() {
|
||||
return browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const });
|
||||
return browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE");
|
||||
},
|
||||
},
|
||||
]) {
|
||||
@@ -666,11 +666,120 @@ describe("OPFS byte-store coordinator", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("does not roll back journal after an unknown worker mutation effect", async () => {
|
||||
it("05-01 decides a reconciled row from what the journal holds once its lock is taken", async () => {
|
||||
// Listed while its owner still ran, the row was PREPARING. By the time
|
||||
// the owner let go it had committed and only finalization failed, so the
|
||||
// staging receipt names the committed generation. Cleaning it up from the
|
||||
// stale snapshot would delete a committed object.
|
||||
const journal = createJournal();
|
||||
const row: OpfsJournalTransaction = {
|
||||
transactionId: "transaction_12345678",
|
||||
fencingToken: "fencing_12345678",
|
||||
mutation: "PUT",
|
||||
scope,
|
||||
phase: "PREPARING",
|
||||
objectId: "object_12345678",
|
||||
expectedGeneration: null,
|
||||
targetGeneration: 1,
|
||||
targetByteLength: 1,
|
||||
targetStoragePolicy: storagePolicy,
|
||||
budgetReservation: {
|
||||
namespace: scope.namespace,
|
||||
reservedBytes: 1,
|
||||
hardBudgetBytes: storagePolicy.hardBudgetBytes,
|
||||
},
|
||||
startedAtEpochMs: 100,
|
||||
};
|
||||
journal.transactions.set(row.transactionId, {
|
||||
...row,
|
||||
phase: "COMMITTED",
|
||||
preparedObject: committedObject("object_12345678", storagePolicy, 100),
|
||||
});
|
||||
vi.spyOn(journal, "listIncomplete").mockResolvedValueOnce(
|
||||
browserDataSuccess({ transactions: [row], moreAvailable: false }),
|
||||
);
|
||||
const worker = createWorker();
|
||||
const cleanupTransaction = vi.spyOn(worker, "cleanupTransaction");
|
||||
const finalizePut = vi.spyOn(worker, "finalizePut");
|
||||
const adapter = createOpfsByteStoreAdapter({
|
||||
journal,
|
||||
worker,
|
||||
scope,
|
||||
storagePolicy,
|
||||
now: () => 100,
|
||||
});
|
||||
|
||||
expect(await adapter.maintenance.reconcile()).toMatchObject({
|
||||
ok: true,
|
||||
value: { inspectedTransactions: 1, rolledBackTransactions: 0 },
|
||||
});
|
||||
expect(cleanupTransaction).not.toHaveBeenCalled();
|
||||
expect(finalizePut).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("05-02 skips a row it cannot reconcile, settles the rest and still runs orphan GC", async () => {
|
||||
const journal = createJournal();
|
||||
const row = (transactionId: string): OpfsJournalTransaction => ({
|
||||
transactionId,
|
||||
fencingToken: "fencing_12345678",
|
||||
mutation: "PUT",
|
||||
scope,
|
||||
phase: "PREPARING",
|
||||
objectId: `object_${transactionId.slice(-8)}`,
|
||||
expectedGeneration: null,
|
||||
targetGeneration: 1,
|
||||
targetByteLength: 1,
|
||||
targetStoragePolicy: storagePolicy,
|
||||
budgetReservation: {
|
||||
namespace: scope.namespace,
|
||||
reservedBytes: 1,
|
||||
hardBudgetBytes: storagePolicy.hardBudgetBytes,
|
||||
},
|
||||
startedAtEpochMs: 100,
|
||||
});
|
||||
journal.transactions.set("transaction_broken_1", row("transaction_broken_1"));
|
||||
journal.transactions.set("transaction_healthy_1", row("transaction_healthy_1"));
|
||||
const worker = createWorker({
|
||||
async cleanupTransaction(_scope, transactionId) {
|
||||
return transactionId === "transaction_broken_1"
|
||||
? browserDataFailure("CORRUPT_DATA", "OBJECT_RECONCILE")
|
||||
: browserDataSuccess({ kind: "CLEANED" as const });
|
||||
},
|
||||
});
|
||||
const listOrphanCandidates = vi.spyOn(worker, "listOrphanCandidates");
|
||||
const observations: unknown[] = [];
|
||||
const adapter = createOpfsByteStoreAdapter({
|
||||
journal,
|
||||
worker,
|
||||
scope,
|
||||
storagePolicy,
|
||||
now: () => 100,
|
||||
observer: (event) => observations.push(event),
|
||||
});
|
||||
|
||||
expect(await adapter.maintenance.reconcile()).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
inspectedTransactions: 2,
|
||||
failedTransactions: 1,
|
||||
rolledBackTransactions: 1,
|
||||
orphanGcStatus: "COMPLETED",
|
||||
},
|
||||
});
|
||||
expect([...journal.transactions.keys()]).toEqual(["transaction_broken_1"]);
|
||||
expect(listOrphanCandidates).toHaveBeenCalledTimes(1);
|
||||
expect(observations).toContainEqual({
|
||||
operation: "OBJECT_RECONCILE",
|
||||
outcome: "FAILED",
|
||||
failureCode: "CORRUPT_DATA",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not roll back journal after an unconfirmed worker cleanup", async () => {
|
||||
const journal = createJournal();
|
||||
const worker = createWorker({
|
||||
async abortPreparedPut() {
|
||||
return browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const });
|
||||
return browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE");
|
||||
},
|
||||
});
|
||||
const rollback = vi.spyOn(journal, "rollback");
|
||||
@@ -734,8 +843,8 @@ describe("OPFS byte-store coordinator", () => {
|
||||
async () => browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE"),
|
||||
],
|
||||
[
|
||||
"an unknown cleanup effect",
|
||||
async () => browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const }),
|
||||
"an aborted cleanup",
|
||||
async () => browserDataFailure("ABORTED", "OBJECT_RECONCILE"),
|
||||
],
|
||||
])("after %s", async (_label, abortPreparedPut) => {
|
||||
const journal = createJournal();
|
||||
@@ -801,7 +910,7 @@ describe("OPFS byte-store coordinator", () => {
|
||||
await putWith(
|
||||
{
|
||||
abortPreparedPut: async () =>
|
||||
browserDataSuccess({ kind: "EFFECT_UNKNOWN" as const }),
|
||||
browserDataFailure("UNAVAILABLE", "OBJECT_RECONCILE"),
|
||||
},
|
||||
journal,
|
||||
),
|
||||
|
||||
@@ -13,6 +13,7 @@ import type {
|
||||
} from "../../src/application/ports/browser-file-storage/shared.ts";
|
||||
import { browserDataSuccess } from "../../src/adapters/browser-file-storage/result.ts";
|
||||
import { createOpfsByteStoreAdapter } from "../../src/adapters/storage/opfs/opfs-byte-store-adapter.ts";
|
||||
import { createIndexedDbOpfsJournal } from "../../src/adapters/storage/opfs/indexeddb-opfs-journal.ts";
|
||||
import {
|
||||
resolveOpfsRuntimePolicy,
|
||||
} from "../../src/adapters/storage/opfs/opfs-policy.ts";
|
||||
@@ -33,6 +34,7 @@ import {
|
||||
type OpfsWorkerMessageHost,
|
||||
type OpfsWorkerRuntime,
|
||||
} from "../../src/adapters/storage/opfs/opfs-worker-runtime.ts";
|
||||
import { MemoryIndexedDbFactory } from "../helpers/memory-indexeddb.ts";
|
||||
|
||||
const scopeA: OpfsStorageScope = Object.freeze({
|
||||
namespace: "durable-objects",
|
||||
@@ -2083,3 +2085,423 @@ describe("NS-05 a bootstrap failure answers the request it belongs to", () => {
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Two runtimes ("tabs") share one OPFS root, one exclusive worker lock and
|
||||
* one journal database, the way two tabs of one origin do.
|
||||
*/
|
||||
function sharedRootTabs(policy = runtimePolicy) {
|
||||
const root = new MemoryDirectory();
|
||||
const leases = exclusiveLeases();
|
||||
const idb = new MemoryIndexedDbFactory();
|
||||
const tab = (
|
||||
tag: string,
|
||||
extra: Partial<Parameters<typeof createOpfsByteStoreAdapter>[0]> = {},
|
||||
) => {
|
||||
const tabScope = extra.scope ?? scopeA;
|
||||
const runtime = createOpfsWorkerRuntime({
|
||||
root: root as unknown as FileSystemDirectoryHandle,
|
||||
crypto: globalThis.crypto,
|
||||
policy,
|
||||
leaseManager: leases,
|
||||
dedicatedWorker: true,
|
||||
supportsSynchronousAccessHandles: false,
|
||||
});
|
||||
let sequence = 0;
|
||||
return createOpfsByteStoreAdapter({
|
||||
journal: createIndexedDbOpfsJournal({
|
||||
authorityToken: tabScope.authorityToken,
|
||||
factory: idb.factory,
|
||||
keyRange: idb.keyRange,
|
||||
}),
|
||||
worker: loopbackGateway(runtime, policy),
|
||||
scope: tabScope,
|
||||
storagePolicy,
|
||||
policy,
|
||||
createTransactionId: () => `transaction_${tag}_${++sequence}`,
|
||||
...extra,
|
||||
});
|
||||
};
|
||||
return { root, leases, idb, tab };
|
||||
}
|
||||
|
||||
async function openedBytes(
|
||||
adapter: ReturnType<typeof createOpfsByteStoreAdapter>,
|
||||
objectId: string,
|
||||
): Promise<Readonly<{ generation: number; bytes: number[] }> | string> {
|
||||
const opened = await adapter.objects.open({ objectId });
|
||||
if (!opened.ok) return opened.error.code;
|
||||
const bytes: number[] = [];
|
||||
for await (const chunk of opened.value.source.stream(
|
||||
new AbortController().signal,
|
||||
)) {
|
||||
if (!chunk.ok) return chunk.error.code;
|
||||
bytes.push(...chunk.value);
|
||||
}
|
||||
return { generation: opened.value.descriptor.generation, bytes };
|
||||
}
|
||||
|
||||
describe("05-01 reconcile leaves a live put's journal row alone", () => {
|
||||
it.each([
|
||||
["a first put", null],
|
||||
["an overwrite", 1],
|
||||
] as const)(
|
||||
"keeps %s that reconcile overlapped in another tab",
|
||||
async (_label, expectedGeneration) => {
|
||||
const { tab } = sharedRootTabs();
|
||||
const tabA = tab("a");
|
||||
const tabB = tab("b");
|
||||
if (expectedGeneration !== null) {
|
||||
expect(
|
||||
await tabA.objects.put({
|
||||
objectId: "object_12345678",
|
||||
expectedGeneration: null,
|
||||
mediaType: "application/octet-stream",
|
||||
source: oneByteSource(),
|
||||
}),
|
||||
).toMatchObject({ ok: true, value: { generation: 1 } });
|
||||
}
|
||||
|
||||
const stalled = stalledSource();
|
||||
const putA = tabA.objects.put({
|
||||
objectId: "object_12345678",
|
||||
expectedGeneration,
|
||||
mediaType: "application/octet-stream",
|
||||
source: stalled.source,
|
||||
});
|
||||
// Tab A holds the worker lock and its journal row is PREPARING.
|
||||
await stalled.streaming;
|
||||
const reconcileB = tabB.maintenance.reconcile();
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
stalled.release();
|
||||
|
||||
const [put, reconciled] = await Promise.all([putA, reconcileB]);
|
||||
const generation = (expectedGeneration ?? 0) + 1;
|
||||
expect(put).toMatchObject({ ok: true, value: { generation } });
|
||||
expect(reconciled).toMatchObject({
|
||||
ok: true,
|
||||
value: { skippedLiveTransactions: 1, rolledBackTransactions: 0 },
|
||||
});
|
||||
expect(await openedBytes(tabA, "object_12345678")).toEqual({
|
||||
generation,
|
||||
bytes: [5],
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("05-02 an empty receipt left by a failed first write is no receipt", () => {
|
||||
it("cleans a crash-left empty receipt and keeps orphan GC sweeping", async () => {
|
||||
const root = new MemoryDirectory();
|
||||
const runtime = createOpfsWorkerRuntime({
|
||||
root: root as unknown as FileSystemDirectoryHandle,
|
||||
crypto: globalThis.crypto,
|
||||
policy: runtimePolicy,
|
||||
leaseManager: immediateLeases({ count: 0 }),
|
||||
dedicatedWorker: true,
|
||||
supportsSynchronousAccessHandles: false,
|
||||
});
|
||||
// `getFileHandle({create:true})` made the file; the write never landed.
|
||||
let directory = root as unknown as FileSystemDirectoryHandle;
|
||||
for (const segment of [
|
||||
"authorities",
|
||||
scopeA.authorityToken,
|
||||
scopeA.namespaceToken,
|
||||
scopeA.partitionToken,
|
||||
"staging",
|
||||
"transaction_empty_0001",
|
||||
]) {
|
||||
directory = await directory.getDirectoryHandle(segment, { create: true });
|
||||
}
|
||||
await directory.getFileHandle("receipt.json", { create: true });
|
||||
const orphan = "0".repeat(64);
|
||||
await seedChunk(root, scopeA, orphan);
|
||||
|
||||
expect(
|
||||
await runtime.handleRequest({
|
||||
requestId: "request_empty_list",
|
||||
protocolVersion: OPFS_WORKER_PROTOCOL_VERSION,
|
||||
kind: "LIST_ORPHAN_CANDIDATES",
|
||||
scope: scopeA,
|
||||
olderThanEpochMs: Date.now(),
|
||||
maxEntries: 10,
|
||||
}),
|
||||
).toMatchObject({
|
||||
ok: true,
|
||||
value: { safeToSweep: true, digests: [orphan] },
|
||||
});
|
||||
expect(
|
||||
await runtime.handleRequest({
|
||||
requestId: "request_empty_cleanup",
|
||||
protocolVersion: OPFS_WORKER_PROTOCOL_VERSION,
|
||||
kind: "CLEANUP_TRANSACTION",
|
||||
scope: scopeA,
|
||||
transactionId: "transaction_empty_0001",
|
||||
}),
|
||||
).toMatchObject({ ok: true, value: { kind: "ALREADY_CLEAN" } });
|
||||
expect(
|
||||
root.has([
|
||||
"authorities",
|
||||
scopeA.authorityToken,
|
||||
scopeA.namespaceToken,
|
||||
scopeA.partitionToken,
|
||||
"staging",
|
||||
"transaction_empty_0001",
|
||||
]),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("compensates a put whose first receipt write hit the quota, so reconcile runs and the same id retries", async () => {
|
||||
const { tab } = sharedRootTabs();
|
||||
const store = tab("q");
|
||||
const getFileHandle = MemoryDirectory.prototype.getFileHandle;
|
||||
let failReceipt = true;
|
||||
vi.spyOn(MemoryDirectory.prototype, "getFileHandle").mockImplementation(
|
||||
async function (this: MemoryDirectory, name, options) {
|
||||
const handle = await getFileHandle.call(this, name, options);
|
||||
if (name !== "receipt.json" || !failReceipt) return handle;
|
||||
failReceipt = false;
|
||||
return {
|
||||
...handle,
|
||||
async createWritable() {
|
||||
const writable = await handle.createWritable();
|
||||
return {
|
||||
write: (data: FileSystemWriteChunkType) => writable.write(data),
|
||||
abort: () => writable.abort(),
|
||||
async close() {
|
||||
throw new DOMException("Quota exceeded.", "QuotaExceededError");
|
||||
},
|
||||
} as unknown as FileSystemWritableFileStream;
|
||||
},
|
||||
} as FileSystemFileHandle;
|
||||
},
|
||||
);
|
||||
const put = () =>
|
||||
store.objects.put({
|
||||
objectId: "object_12345678",
|
||||
expectedGeneration: null,
|
||||
mediaType: "application/octet-stream",
|
||||
source: oneByteSource(),
|
||||
});
|
||||
|
||||
const failed = await put();
|
||||
expect(failed).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: "QUOTA_EXCEEDED" },
|
||||
});
|
||||
expect(failed.ok || failed.error.recovery).not.toBe("RECONCILE");
|
||||
expect(await store.maintenance.reconcile()).toMatchObject({
|
||||
ok: true,
|
||||
value: { inspectedTransactions: 0, orphanGcStatus: "COMPLETED" },
|
||||
});
|
||||
expect(await put()).toMatchObject({ ok: true, value: { generation: 1 } });
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-04 a LOGOUT or ACCOUNT_DELETION purge leaves no partition bytes behind", () => {
|
||||
const scopeP2: OpfsStorageScope = Object.freeze({
|
||||
...scopeA,
|
||||
partitionToken: "partition_87654321",
|
||||
});
|
||||
const authority = {
|
||||
requestMaintenanceAuthority: () => ({
|
||||
authorized: true as const,
|
||||
proofToken: "proof_token_1234567890",
|
||||
expiresAtEpochMs: Date.now() + 60_000,
|
||||
}),
|
||||
consumeMaintenanceAuthority: () => true,
|
||||
};
|
||||
|
||||
function filesUnder(root: MemoryDirectory, scope: OpfsStorageScope): string[] {
|
||||
const files: string[] = [];
|
||||
const walk = (directory: MemoryDirectory, path: string) => {
|
||||
for (const [name, node] of directory.children) {
|
||||
if (node instanceof MemoryDirectory) walk(node, `${path}/${name}`);
|
||||
else files.push(`${path}/${name}`);
|
||||
}
|
||||
};
|
||||
const partition = [
|
||||
"authorities",
|
||||
scope.authorityToken,
|
||||
scope.namespaceToken,
|
||||
scope.partitionToken,
|
||||
].reduce<MemoryNode | undefined>(
|
||||
(node, segment) =>
|
||||
node instanceof MemoryDirectory ? node.children.get(segment) : node,
|
||||
root,
|
||||
);
|
||||
if (partition instanceof MemoryDirectory) walk(partition, "");
|
||||
return files;
|
||||
}
|
||||
|
||||
const putBytes = (
|
||||
adapter: ReturnType<typeof createOpfsByteStoreAdapter>,
|
||||
objectId: string,
|
||||
bytes: number[],
|
||||
) =>
|
||||
adapter.objects.put({
|
||||
objectId,
|
||||
expectedGeneration: null,
|
||||
mediaType: "application/octet-stream",
|
||||
source: bytesSource(new Uint8Array(bytes)),
|
||||
});
|
||||
|
||||
it.each(["LOGOUT", "ACCOUNT_DELETION"] as const)(
|
||||
"removes the chunks of a %s-purged partition and keeps other partitions",
|
||||
async (reason) => {
|
||||
const { root, tab } = sharedRootTabs();
|
||||
const purged = tab("p", authority);
|
||||
const kept = tab("k", { ...authority, scope: scopeP2 });
|
||||
expect(await putBytes(purged, "object_secret_1", [1, 2, 3])).toMatchObject({ ok: true });
|
||||
expect(await putBytes(purged, "object_secret_2", [4, 5])).toMatchObject({ ok: true });
|
||||
expect(await putBytes(kept, "object_kept_0001", [7, 7])).toMatchObject({ ok: true });
|
||||
|
||||
expect(await purged.maintenance.enforcePolicies({ reason })).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
removedObjects: 2,
|
||||
moreObjectsAvailable: false,
|
||||
deadlineReached: false,
|
||||
chunkSweep: "COMPLETED",
|
||||
},
|
||||
});
|
||||
expect(filesUnder(root, scopeA)).toEqual([]);
|
||||
expect(await openedBytes(kept, "object_kept_0001")).toEqual({
|
||||
generation: 1,
|
||||
bytes: [7, 7],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("never sweeps the chunk of a put that runs while the purge does", async () => {
|
||||
const { tab } = sharedRootTabs();
|
||||
const purger = tab("p", authority);
|
||||
const writer = tab("w", authority);
|
||||
expect(await putBytes(purger, "object_secret_1", [1, 2, 3])).toMatchObject({ ok: true });
|
||||
const stalled = stalledSource();
|
||||
const put = writer.objects.put({
|
||||
objectId: "object_late_0001",
|
||||
expectedGeneration: null,
|
||||
mediaType: "application/octet-stream",
|
||||
source: stalled.source,
|
||||
});
|
||||
await stalled.streaming;
|
||||
const purge = purger.maintenance.enforcePolicies({ reason: "LOGOUT" });
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
stalled.release();
|
||||
|
||||
expect(await put).toMatchObject({ ok: true });
|
||||
expect(await purge).toMatchObject({ ok: true, value: { removedObjects: 1 } });
|
||||
expect(await openedBytes(writer, "object_late_0001")).toEqual({
|
||||
generation: 1,
|
||||
bytes: [5],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-07 a put whose source stops lets go of the origin lock", () => {
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it("fails the stalled put at the idle bound so another tab's put runs", async () => {
|
||||
const policy = resolveOpfsRuntimePolicy({
|
||||
...runtimePolicy,
|
||||
putIdleTimeoutMs: 30_000,
|
||||
});
|
||||
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
|
||||
const { tab, leases } = sharedRootTabs(policy);
|
||||
const tabA = tab("a");
|
||||
const tabB = tab("b");
|
||||
const stalled = stalledSource();
|
||||
const putA = tabA.objects.put({
|
||||
objectId: "object_stalled_01",
|
||||
expectedGeneration: null,
|
||||
mediaType: "application/octet-stream",
|
||||
source: stalled.source,
|
||||
});
|
||||
await stalled.streaming;
|
||||
expect(leases.held).toBe(true);
|
||||
const putB = tabB.objects.put({
|
||||
objectId: "object_waiting_01",
|
||||
expectedGeneration: null,
|
||||
mediaType: "application/octet-stream",
|
||||
source: oneByteSource(),
|
||||
});
|
||||
|
||||
await vi.advanceTimersByTimeAsync(policy.putIdleTimeoutMs + 1);
|
||||
vi.useRealTimers();
|
||||
const settledA = await Promise.race([
|
||||
putA,
|
||||
new Promise((resolve) => setTimeout(() => resolve("STILL_RUNNING"), 2_000)),
|
||||
]);
|
||||
expect(settledA).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: "NOT_READABLE", retryable: true },
|
||||
});
|
||||
expect(await putB).toMatchObject({ ok: true, value: { generation: 1 } });
|
||||
stalled.release();
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-10 the object id grammar is the prepared-object grammar", () => {
|
||||
it("rejects a policy that widens the object id grammar when it is resolved", () => {
|
||||
expect(() =>
|
||||
resolveOpfsRuntimePolicy({
|
||||
...runtimePolicy,
|
||||
isObjectIdAllowed: (objectId) => /^[A-Za-z0-9_.-]{1,200}$/u.test(objectId),
|
||||
}),
|
||||
).toThrow(TypeError);
|
||||
});
|
||||
});
|
||||
|
||||
describe("05-14 a read that outlives its generation", () => {
|
||||
it("ends with NOT_FOUND / REHYDRATE once GC took the replaced chunks, and a reopen reads the new generation", async () => {
|
||||
const { root, tab } = sharedRootTabs();
|
||||
const store = tab("r");
|
||||
const put = (bytes: number[], expectedGeneration: number | null) =>
|
||||
store.objects.put({
|
||||
objectId: "object_12345678",
|
||||
expectedGeneration,
|
||||
mediaType: "application/octet-stream",
|
||||
source: bytesSource(new Uint8Array(bytes)),
|
||||
});
|
||||
expect(await put([1], null)).toMatchObject({ ok: true });
|
||||
const opened = await store.objects.open({ objectId: "object_12345678" });
|
||||
if (!opened.ok) throw new Error(opened.error.code);
|
||||
|
||||
// Generation 2 replaces it, and its old chunk is past the GC grace.
|
||||
expect(await put([2], 1)).toMatchObject({ ok: true });
|
||||
const digest = Array.from(
|
||||
new Uint8Array(
|
||||
await globalThis.crypto.subtle.digest("SHA-256", new Uint8Array([1])),
|
||||
),
|
||||
(byte) => byte.toString(16).padStart(2, "0"),
|
||||
).join("");
|
||||
memoryFileAt(root, chunkFilePath(scopeA, digest)).lastModified = 0;
|
||||
expect(await store.maintenance.reconcile()).toMatchObject({
|
||||
ok: true,
|
||||
value: { deletedOrphanChunks: 1 },
|
||||
});
|
||||
|
||||
const chunks = [];
|
||||
for await (const chunk of opened.value.source.stream(
|
||||
new AbortController().signal,
|
||||
)) {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
expect(chunks).toEqual([
|
||||
expect.objectContaining({
|
||||
ok: false,
|
||||
error: expect.objectContaining({
|
||||
code: "NOT_FOUND",
|
||||
recovery: "REHYDRATE",
|
||||
}),
|
||||
}),
|
||||
]);
|
||||
expect(await openedBytes(store, "object_12345678")).toEqual({
|
||||
generation: 2,
|
||||
bytes: [2],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user