fix(storage): verify stored OPFS objects by their own chunk size (R-5.5)
Prepared objects were validated in three places with different rules. The
worker required integrity.chunkSizeBytes, chunk sizes, chunk count and
object size to match the current runtime policy; the journal checked the
object against itself; the page only checked the shape. Changing
chunkSizeBytes (allowed by opfs-policy) made every existing object fail:
open returned INTEGRITY_FAILED, FILES_READY reconcile failed CORRUPT_DATA
(and, had cleanup succeeded, would have rolled back a valid object), and
COMMITTED reconcile failed INVALID_INPUT at FINALIZE_PUT.
The journal's policy-free rules move to opfs-prepared-object.ts, the one
definition of isPreparedObject (and of the physical schema check). The
worker's VERIFY_OBJECT, READ_CHUNK, FINALIZE_PUT and FILES_READY receipt
paths and the page's FINISH_PUT parse use it, so a stored object is
checked against its own manifest. The current policy still governs new
writes only: validateObjectWriteInput, BEGIN_PUT's chunk size check and
APPEND_CHUNK's size and count limits are unchanged. stableJson keeps its
two copies; they differ in how a non-finite number fails and unifying
them would change error mapping.
Verified: new test "R-5.5 stored objects are checked against their own
chunk size" writes an object with 64 KiB chunks, then reconciles and opens
it through a 128 KiB runtime. Before the fix reconcile failed CORRUPT_DATA
and open failed INTEGRITY_FAILED; now reconcile commits without rollback,
the bytes read back, and a new put uses 128 KiB chunks. A new journal test
pins that a 4 MiB-chunk object round-trips and an inconsistent one is
still rejected (passes before and after). Inventory and the storage
detail review list the new file.
Gates: check:architecture, check:types:app/test/web-worker, lint,
test:component pass; test:unit passes except removal-fixture.test.ts
(needs gitignored release artifacts absent from a fresh worktree);
check:adapter-inventory accepts the new row but still reports two rows
already missing at f0d3953 (http/platform-error-mapping.ts,
telemetry/feature-telemetry.ts).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
46af07b4fe
commit
e74d835d6a
@@ -79,6 +79,7 @@
|
||||
| `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | logical object/journal/budget/chunk refcount의 IDB authority; begin/files-ready/commit/rollback/reconcile pages | OPFS ports, IDB failure, policy. journal+object+budget CAS atomicity가 좋다. STO-01 수정에서 incomplete row를 cleanup 확인 전 삭제하지 않아야 한다. |
|
||||
| `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | logical journal과 physical worker를 saga로 조정, put/open/remove, reconcile/policy maintenance | OPFS/shared ports, journal, worker gateway, policy. STO-01의 journal/physical compensation ordering 결함 위치. |
|
||||
| `src/adapters/storage/opfs/opfs-policy.ts` | root/lock/chunk/object/RPC/reconcile/GC hard limits 및 scope validation | shared/opfs ports. opaque physical path와 absolute caps 유지. |
|
||||
| `src/adapters/storage/opfs/opfs-prepared-object.ts` | page·journal·worker가 함께 쓰는 prepared-object 검증기 하나 | OPFS/shared ports, opfs-policy scope 검증. R-5.5: 저장된 객체는 자기 manifest의 `chunkSizeBytes`로 검증하고 현재 정책값은 새 쓰기에만 적용. |
|
||||
| `src/adapters/storage/opfs/opfs-worker-client.ts` | request correlation/timeout/abort/transferable chunking, worker gateway, streamed reads | protocol/policy/shared Result. STO-01의 untracked abort cleanup 및 STO-07의 shallow response parse. |
|
||||
| `src/adapters/storage/opfs/opfs-worker-protocol.ts` | page↔DedicatedWorker request/response union 및 gateway contract | OPFS/shared ports. STO-07; protocol version/kind/effect certainty 추가 필요. |
|
||||
| `src/adapters/storage/opfs/opfs-worker-runtime.ts` | DedicatedWorker OPFS physical layout, lock lease, immutable chunk, manifest/staging receipt, abort/finalize/remove/GC | protocol/policy/OPFS ports/Web Lock/Crypto. STO-01의 generation-only cleanup과 lease release 순서. sync handle `finally close` 등은 유지. |
|
||||
|
||||
@@ -125,22 +125,23 @@
|
||||
| 115 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 116 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 117 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 118 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 119 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 120 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 121 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 122 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 123 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 124 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 125 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 126 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 127 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 128 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 129 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 130 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 131 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 132 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 133 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 118 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 119 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 120 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 121 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
|
||||
| 122 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 123 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
|
||||
| 124 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 125 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 126 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 127 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 128 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 129 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 130 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 131 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 132 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 133 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
| 134 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
|
||||
|
||||
합계: **133/133**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
|
||||
합계: **134/134**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
|
||||
|
||||
|
||||
@@ -33,6 +33,7 @@ import {
|
||||
} from "../../platform/indexeddb-transaction.ts";
|
||||
import { mapIndexedDbException } from "../indexeddb/indexeddb-failure.ts";
|
||||
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
|
||||
import { isPreparedObject } from "./opfs-prepared-object.ts";
|
||||
|
||||
export type IndexedDbOpfsJournalDependencies = Readonly<{
|
||||
authorityToken: string;
|
||||
@@ -1607,118 +1608,6 @@ function isChunkReference(
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
|
||||
* window; v2 additionally carries a transaction-unique physical fencing token.
|
||||
*/
|
||||
function isSupportedPhysicalSchema(value: object): boolean {
|
||||
const record = value as Record<string, unknown>;
|
||||
if (record.physicalSchemaVersion === 1) return true;
|
||||
return (
|
||||
record.physicalSchemaVersion === 2 &&
|
||||
typeof record.physicalGenerationId === "string" &&
|
||||
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
|
||||
);
|
||||
}
|
||||
|
||||
function isPreparedObject(value: unknown): value is OpfsPreparedObject {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== "object" ||
|
||||
!("physicalSchemaVersion" in value) ||
|
||||
!isSupportedPhysicalSchema(value) ||
|
||||
!("descriptor" in value) ||
|
||||
!value.descriptor ||
|
||||
typeof value.descriptor !== "object" ||
|
||||
!("chunks" in value) ||
|
||||
!Array.isArray(value.chunks)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
const descriptor = value.descriptor as Record<string, unknown>;
|
||||
const integrity =
|
||||
descriptor.integrity && typeof descriptor.integrity === "object"
|
||||
? (descriptor.integrity as Record<string, unknown>)
|
||||
: null;
|
||||
if (
|
||||
typeof descriptor.objectId !== "string" ||
|
||||
!SAFE_BOUNDARY_ID.test(descriptor.objectId) ||
|
||||
!descriptor.scope ||
|
||||
typeof descriptor.scope !== "object" ||
|
||||
!isValidOpfsStorageScope(descriptor.scope as OpfsStorageScope) ||
|
||||
typeof descriptor.generation !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.generation) ||
|
||||
descriptor.generation < 1 ||
|
||||
typeof descriptor.byteLength !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.byteLength) ||
|
||||
descriptor.byteLength < 0 ||
|
||||
typeof descriptor.mediaType !== "string" ||
|
||||
descriptor.mediaType.length < 1 ||
|
||||
typeof descriptor.createdAtEpochMs !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.createdAtEpochMs) ||
|
||||
descriptor.createdAtEpochMs < 0 ||
|
||||
!integrity ||
|
||||
integrity.algorithm !== "SHA-256-TREE-V1" ||
|
||||
typeof integrity.rootDigestHex !== "string" ||
|
||||
!SHA256_HEX.test(integrity.rootDigestHex) ||
|
||||
typeof integrity.chunkSizeBytes !== "number" ||
|
||||
!Number.isSafeInteger(integrity.chunkSizeBytes) ||
|
||||
integrity.chunkSizeBytes < 1 ||
|
||||
!descriptor.storagePolicy ||
|
||||
typeof descriptor.storagePolicy !== "object"
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
assertValidStoragePolicy(
|
||||
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"],
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
(descriptor.scope as OpfsStorageScope).namespace !==
|
||||
(
|
||||
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"]
|
||||
).namespace
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
|
||||
let totalBytes = 0;
|
||||
for (let index = 0; index < value.chunks.length; index += 1) {
|
||||
const chunk = value.chunks[index] as unknown;
|
||||
if (
|
||||
!chunk ||
|
||||
typeof chunk !== "object" ||
|
||||
!("sequence" in chunk) ||
|
||||
chunk.sequence !== index ||
|
||||
!("byteLength" in chunk) ||
|
||||
typeof chunk.byteLength !== "number" ||
|
||||
!Number.isSafeInteger(chunk.byteLength) ||
|
||||
chunk.byteLength < 1 ||
|
||||
chunk.byteLength > integrity.chunkSizeBytes ||
|
||||
!("digestHex" in chunk) ||
|
||||
typeof chunk.digestHex !== "string" ||
|
||||
!SHA256_HEX.test(chunk.digestHex)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
index < value.chunks.length - 1 &&
|
||||
chunk.byteLength !== integrity.chunkSizeBytes
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
totalBytes += chunk.byteLength;
|
||||
}
|
||||
return (
|
||||
totalBytes === descriptor.byteLength &&
|
||||
value.chunks.length ===
|
||||
Math.ceil(descriptor.byteLength / integrity.chunkSizeBytes)
|
||||
);
|
||||
}
|
||||
|
||||
function stableJson(value: unknown): string {
|
||||
if (
|
||||
value === null ||
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
import type {
|
||||
OpfsPreparedObject,
|
||||
OpfsStorageScope,
|
||||
} from "../../../application/ports/browser-file-storage/opfs-ports.ts";
|
||||
import { assertValidStoragePolicy } from "../../../application/ports/browser-file-storage/shared.ts";
|
||||
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
|
||||
|
||||
const OPAQUE_OBJECT_ID = /^[A-Za-z0-9_-]{8,128}$/u;
|
||||
const SHA256_HEX = /^[a-f0-9]{64}$/u;
|
||||
|
||||
/**
|
||||
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
|
||||
* window; v2 additionally carries a transaction-unique physical fencing token.
|
||||
*/
|
||||
function isSupportedPhysicalSchema(value: object): boolean {
|
||||
const record = value as Record<string, unknown>;
|
||||
if (record.physicalSchemaVersion === 1) return true;
|
||||
return (
|
||||
record.physicalSchemaVersion === 2 &&
|
||||
typeof record.physicalGenerationId === "string" &&
|
||||
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* R-5.5. The one prepared-object validator for the page, the journal and the
|
||||
* worker. An object is checked against itself, including its own
|
||||
* `chunkSizeBytes`, never against the current runtime policy: the policy
|
||||
* governs new writes only (validateObjectWriteInput, BEGIN_PUT, APPEND_CHUNK),
|
||||
* so changing the chunk size or another limit cannot make stored objects
|
||||
* unreadable or unreconcilable.
|
||||
*/
|
||||
export function isPreparedObject(
|
||||
value: unknown,
|
||||
): value is OpfsPreparedObject {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== "object" ||
|
||||
!("physicalSchemaVersion" in value) ||
|
||||
!isSupportedPhysicalSchema(value) ||
|
||||
!("descriptor" in value) ||
|
||||
!value.descriptor ||
|
||||
typeof value.descriptor !== "object" ||
|
||||
!("chunks" in value) ||
|
||||
!Array.isArray(value.chunks)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
const descriptor = value.descriptor as Record<string, unknown>;
|
||||
const integrity =
|
||||
descriptor.integrity && typeof descriptor.integrity === "object"
|
||||
? (descriptor.integrity as Record<string, unknown>)
|
||||
: null;
|
||||
if (
|
||||
typeof descriptor.objectId !== "string" ||
|
||||
!OPAQUE_OBJECT_ID.test(descriptor.objectId) ||
|
||||
!descriptor.scope ||
|
||||
typeof descriptor.scope !== "object" ||
|
||||
!isValidOpfsStorageScope(descriptor.scope as OpfsStorageScope) ||
|
||||
typeof descriptor.generation !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.generation) ||
|
||||
descriptor.generation < 1 ||
|
||||
typeof descriptor.byteLength !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.byteLength) ||
|
||||
descriptor.byteLength < 0 ||
|
||||
typeof descriptor.mediaType !== "string" ||
|
||||
descriptor.mediaType.length < 1 ||
|
||||
typeof descriptor.createdAtEpochMs !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.createdAtEpochMs) ||
|
||||
descriptor.createdAtEpochMs < 0 ||
|
||||
!integrity ||
|
||||
integrity.algorithm !== "SHA-256-TREE-V1" ||
|
||||
typeof integrity.rootDigestHex !== "string" ||
|
||||
!SHA256_HEX.test(integrity.rootDigestHex) ||
|
||||
typeof integrity.chunkSizeBytes !== "number" ||
|
||||
!Number.isSafeInteger(integrity.chunkSizeBytes) ||
|
||||
integrity.chunkSizeBytes < 1 ||
|
||||
!descriptor.storagePolicy ||
|
||||
typeof descriptor.storagePolicy !== "object"
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
assertValidStoragePolicy(
|
||||
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"],
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
(descriptor.scope as OpfsStorageScope).namespace !==
|
||||
(
|
||||
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"]
|
||||
).namespace
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
|
||||
let totalBytes = 0;
|
||||
for (let index = 0; index < value.chunks.length; index += 1) {
|
||||
const chunk = value.chunks[index] as unknown;
|
||||
if (
|
||||
!chunk ||
|
||||
typeof chunk !== "object" ||
|
||||
!("sequence" in chunk) ||
|
||||
chunk.sequence !== index ||
|
||||
!("byteLength" in chunk) ||
|
||||
typeof chunk.byteLength !== "number" ||
|
||||
!Number.isSafeInteger(chunk.byteLength) ||
|
||||
chunk.byteLength < 1 ||
|
||||
chunk.byteLength > integrity.chunkSizeBytes ||
|
||||
!("digestHex" in chunk) ||
|
||||
typeof chunk.digestHex !== "string" ||
|
||||
!SHA256_HEX.test(chunk.digestHex)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
index < value.chunks.length - 1 &&
|
||||
chunk.byteLength !== integrity.chunkSizeBytes
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
totalBytes += chunk.byteLength;
|
||||
}
|
||||
return (
|
||||
totalBytes === descriptor.byteLength &&
|
||||
value.chunks.length ===
|
||||
Math.ceil(descriptor.byteLength / integrity.chunkSizeBytes)
|
||||
);
|
||||
}
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
browserDataSuccess,
|
||||
} from "../../browser-file-storage/result.ts";
|
||||
import type { OpfsRuntimePolicy } from "./opfs-policy.ts";
|
||||
import { isPreparedObject } from "./opfs-prepared-object.ts";
|
||||
import {
|
||||
OPFS_LOCK_WAITING_REQUEST_KINDS,
|
||||
OPFS_WORKER_PROTOCOL_VERSION,
|
||||
@@ -648,20 +649,6 @@ function parseCapabilities(value: unknown): OpfsCapabilities | null {
|
||||
return value as OpfsCapabilities;
|
||||
}
|
||||
|
||||
/**
|
||||
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
|
||||
* window; v2 additionally carries a transaction-unique physical fencing token.
|
||||
*/
|
||||
function isSupportedPhysicalSchema(value: object): boolean {
|
||||
const record = value as Record<string, unknown>;
|
||||
if (record.physicalSchemaVersion === 1) return true;
|
||||
return (
|
||||
record.physicalSchemaVersion === 2 &&
|
||||
typeof record.physicalGenerationId === "string" &&
|
||||
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
|
||||
);
|
||||
}
|
||||
|
||||
function parseCleanupEffect(value: unknown): OpfsCleanupEffect | null {
|
||||
if (!value || typeof value !== "object") return null;
|
||||
const kind = (value as Record<string, unknown>).kind;
|
||||
@@ -671,18 +658,7 @@ function parseCleanupEffect(value: unknown): OpfsCleanupEffect | null {
|
||||
}
|
||||
|
||||
function parsePreparedObject(value: unknown): OpfsPreparedObject | null {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== "object" ||
|
||||
!("physicalSchemaVersion" in value) ||
|
||||
!isSupportedPhysicalSchema(value) ||
|
||||
!("descriptor" in value) ||
|
||||
!("chunks" in value) ||
|
||||
!Array.isArray(value.chunks)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return value as OpfsPreparedObject;
|
||||
return isPreparedObject(value) ? value : null;
|
||||
}
|
||||
|
||||
function parseOrphanCandidateBatch(
|
||||
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
isValidOpfsStorageScope,
|
||||
type OpfsRuntimePolicy,
|
||||
} from "./opfs-policy.ts";
|
||||
import { isPreparedObject } from "./opfs-prepared-object.ts";
|
||||
import {
|
||||
OPFS_WORKER_PROTOCOL_VERSION,
|
||||
} from "./opfs-worker-protocol.ts";
|
||||
@@ -693,7 +694,7 @@ export function createOpfsWorkerRuntime(
|
||||
async function verifyObject(
|
||||
expected: OpfsPreparedObject,
|
||||
): Promise<boolean> {
|
||||
if (!isPreparedObjectSafe(expected, dependencies.policy)) return false;
|
||||
if (!isPreparedObject(expected)) return false;
|
||||
let stored: unknown;
|
||||
try {
|
||||
stored = await readJson(manifestPath(expected));
|
||||
@@ -702,7 +703,7 @@ export function createOpfsWorkerRuntime(
|
||||
throw error;
|
||||
}
|
||||
if (
|
||||
!isPreparedObjectSafe(stored, dependencies.policy) ||
|
||||
!isPreparedObject(stored) ||
|
||||
stableJson(stored) !== stableJson(expected)
|
||||
) {
|
||||
return false;
|
||||
@@ -741,7 +742,7 @@ export function createOpfsWorkerRuntime(
|
||||
sequence: number,
|
||||
): Promise<ArrayBuffer> {
|
||||
if (
|
||||
!isPreparedObjectSafe(expected, dependencies.policy) ||
|
||||
!isPreparedObject(expected) ||
|
||||
!Number.isSafeInteger(sequence) ||
|
||||
sequence < 0
|
||||
) {
|
||||
@@ -908,7 +909,7 @@ export function createOpfsWorkerRuntime(
|
||||
assertWorkerAvailable();
|
||||
if (
|
||||
!SAFE_TRANSACTION_ID.test(transactionId) ||
|
||||
!isPreparedObjectSafe(preparedObject, dependencies.policy)
|
||||
!isPreparedObject(preparedObject)
|
||||
) {
|
||||
throw new OpfsRuntimeFailure("INVALID_INPUT");
|
||||
}
|
||||
@@ -1160,7 +1161,7 @@ export function createOpfsWorkerRuntime(
|
||||
}
|
||||
if (
|
||||
record.phase === "FILES_READY" &&
|
||||
isPreparedObjectSafe(record.preparedObject, dependencies.policy) &&
|
||||
isPreparedObject(record.preparedObject) &&
|
||||
sameScope(record.preparedObject.descriptor.scope, scope)
|
||||
) {
|
||||
const prepared = record.preparedObject;
|
||||
@@ -1727,118 +1728,6 @@ function stableJson(value: unknown): string {
|
||||
throw new OpfsRuntimeFailure("CORRUPT_DATA");
|
||||
}
|
||||
|
||||
/**
|
||||
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
|
||||
* window; v2 additionally carries a transaction-unique physical fencing token.
|
||||
*/
|
||||
function isSupportedPhysicalSchema(value: object): boolean {
|
||||
const record = value as Record<string, unknown>;
|
||||
if (record.physicalSchemaVersion === 1) return true;
|
||||
return (
|
||||
record.physicalSchemaVersion === 2 &&
|
||||
typeof record.physicalGenerationId === "string" &&
|
||||
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
|
||||
);
|
||||
}
|
||||
|
||||
function isPreparedObjectSafe(
|
||||
value: unknown,
|
||||
policy: OpfsRuntimePolicy,
|
||||
): value is OpfsPreparedObject {
|
||||
if (
|
||||
!value ||
|
||||
typeof value !== "object" ||
|
||||
!("physicalSchemaVersion" in value) ||
|
||||
!isSupportedPhysicalSchema(value) ||
|
||||
!("descriptor" in value) ||
|
||||
!value.descriptor ||
|
||||
typeof value.descriptor !== "object" ||
|
||||
!("chunks" in value) ||
|
||||
!Array.isArray(value.chunks)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
const descriptor = value.descriptor as Record<string, unknown>;
|
||||
const integrity =
|
||||
descriptor.integrity && typeof descriptor.integrity === "object"
|
||||
? (descriptor.integrity as Record<string, unknown>)
|
||||
: null;
|
||||
if (
|
||||
typeof descriptor.objectId !== "string" ||
|
||||
!policy.isObjectIdAllowed(descriptor.objectId) ||
|
||||
!("scope" in descriptor) ||
|
||||
!descriptor.scope ||
|
||||
typeof descriptor.scope !== "object" ||
|
||||
!isValidOpfsStorageScope(
|
||||
descriptor.scope as OpfsPreparedObject["descriptor"]["scope"],
|
||||
) ||
|
||||
(descriptor.scope as OpfsPreparedObject["descriptor"]["scope"])
|
||||
.namespace !==
|
||||
(descriptor.storagePolicy as
|
||||
| OpfsPreparedObject["descriptor"]["storagePolicy"]
|
||||
| undefined)?.namespace ||
|
||||
typeof descriptor.generation !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.generation) ||
|
||||
descriptor.generation < 1 ||
|
||||
typeof descriptor.byteLength !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.byteLength) ||
|
||||
descriptor.byteLength < 0 ||
|
||||
descriptor.byteLength > policy.maxObjectBytes ||
|
||||
typeof descriptor.mediaType !== "string" ||
|
||||
!policy.isMediaTypeAllowed(descriptor.mediaType) ||
|
||||
typeof descriptor.createdAtEpochMs !== "number" ||
|
||||
!Number.isSafeInteger(descriptor.createdAtEpochMs) ||
|
||||
descriptor.createdAtEpochMs < 0 ||
|
||||
!integrity ||
|
||||
integrity.algorithm !== "SHA-256-TREE-V1" ||
|
||||
typeof integrity.rootDigestHex !== "string" ||
|
||||
!SHA256_HEX.test(integrity.rootDigestHex) ||
|
||||
integrity.chunkSizeBytes !== policy.chunkSizeBytes ||
|
||||
!("storagePolicy" in descriptor) ||
|
||||
!descriptor.storagePolicy ||
|
||||
typeof descriptor.storagePolicy !== "object" ||
|
||||
value.chunks.length > policy.maxChunkCount
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
assertValidStoragePolicy(
|
||||
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"],
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
let total = 0;
|
||||
for (let index = 0; index < value.chunks.length; index += 1) {
|
||||
const chunk = value.chunks[index] as unknown;
|
||||
if (
|
||||
!chunk ||
|
||||
typeof chunk !== "object" ||
|
||||
!("sequence" in chunk) ||
|
||||
chunk.sequence !== index ||
|
||||
!("byteLength" in chunk) ||
|
||||
typeof chunk.byteLength !== "number" ||
|
||||
!Number.isSafeInteger(chunk.byteLength) ||
|
||||
chunk.byteLength < 1 ||
|
||||
chunk.byteLength > policy.chunkSizeBytes ||
|
||||
!("digestHex" in chunk) ||
|
||||
typeof chunk.digestHex !== "string" ||
|
||||
!SHA256_HEX.test(chunk.digestHex)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (index < value.chunks.length - 1 && chunk.byteLength !== policy.chunkSizeBytes) {
|
||||
return false;
|
||||
}
|
||||
total += chunk.byteLength;
|
||||
}
|
||||
return (
|
||||
total === descriptor.byteLength &&
|
||||
value.chunks.length ===
|
||||
Math.ceil(descriptor.byteLength / policy.chunkSizeBytes)
|
||||
);
|
||||
}
|
||||
|
||||
function success(
|
||||
requestId: string,
|
||||
kind: OpfsWorkerRequest["kind"],
|
||||
|
||||
@@ -710,6 +710,61 @@ describe("IndexedDB OPFS journal", () => {
|
||||
expect(second.value.nextObjectId).toBeNull();
|
||||
});
|
||||
|
||||
it("R-5.5 keeps an object readable by its own chunk size and still rejects an inconsistent one", async () => {
|
||||
const { journal } = createHarness();
|
||||
const chunkSizeBytes = 4 * 1024 * 1024;
|
||||
const byteLength = chunkSizeBytes + 1;
|
||||
const input: BeginOpfsJournalTransaction = {
|
||||
...beginInput("transaction_chunk001", "object_chunk_001", byteLength),
|
||||
targetStoragePolicy: {
|
||||
...policy,
|
||||
softBudgetBytes: 2 * byteLength,
|
||||
hardBudgetBytes: 2 * byteLength,
|
||||
},
|
||||
};
|
||||
const base = prepared(input);
|
||||
const stored: OpfsPreparedObject = {
|
||||
...base,
|
||||
descriptor: {
|
||||
...base.descriptor,
|
||||
integrity: { ...base.descriptor.integrity, chunkSizeBytes },
|
||||
},
|
||||
chunks: [
|
||||
{ sequence: 0, byteLength: chunkSizeBytes, digestHex: "c".repeat(64) },
|
||||
{ sequence: 1, byteLength: 1, digestHex: "d".repeat(64) },
|
||||
],
|
||||
};
|
||||
const begun = await journal.begin(input);
|
||||
if (!begun.ok) throw new Error("begin failed");
|
||||
const token = begun.value.fencingToken;
|
||||
|
||||
// A first chunk shorter than the object's own chunk size is not a
|
||||
// well-formed object under any policy.
|
||||
expect(
|
||||
await journal.markFilesReady(input.transactionId, token, {
|
||||
...stored,
|
||||
chunks: [
|
||||
{ sequence: 0, byteLength: 1, digestHex: "c".repeat(64) },
|
||||
{ sequence: 1, byteLength: chunkSizeBytes, digestHex: "d".repeat(64) },
|
||||
],
|
||||
}),
|
||||
).toMatchObject({ ok: false, error: { code: "INVALID_INPUT" } });
|
||||
|
||||
expect(
|
||||
await journal.markFilesReady(input.transactionId, token, stored),
|
||||
).toMatchObject({ ok: true, value: { phase: "FILES_READY" } });
|
||||
expect(await journal.listIncomplete(10)).toMatchObject({
|
||||
ok: true,
|
||||
value: { transactions: [{ preparedObject: stored }] },
|
||||
});
|
||||
expect(
|
||||
await journal.commitPut(input.transactionId, token),
|
||||
).toMatchObject({ ok: true });
|
||||
expect(
|
||||
await journal.getCommittedObject(scope, input.objectId),
|
||||
).toEqual({ ok: true, value: stored });
|
||||
});
|
||||
|
||||
it("answers UNSUPPORTED once the journal is closed", async () => {
|
||||
const { factory, journal } = createHarness();
|
||||
expect(
|
||||
|
||||
@@ -2,6 +2,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import type {
|
||||
OpfsJournalPort,
|
||||
OpfsJournalTransaction,
|
||||
OpfsPreparedObject,
|
||||
OpfsPhysicalGenerationId,
|
||||
OpfsStorageScope,
|
||||
@@ -904,24 +905,29 @@ function stalledSource(): Readonly<{
|
||||
};
|
||||
}
|
||||
|
||||
function oneByteSource(): ByteSource {
|
||||
function bytesSource(bytes: Uint8Array): ByteSource {
|
||||
return {
|
||||
byteLength: 1,
|
||||
byteLength: bytes.byteLength,
|
||||
async *stream() {
|
||||
yield browserDataSuccess(new Uint8Array([6]));
|
||||
yield browserDataSuccess(Uint8Array.from(bytes));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function oneByteSource(): ByteSource {
|
||||
return bytesSource(new Uint8Array([6]));
|
||||
}
|
||||
|
||||
/** `suffix` is a hex digit so it also forms a physical generation id. */
|
||||
function putRequest(
|
||||
suffix: string,
|
||||
source: ByteSource,
|
||||
): Parameters<OpfsWorkerGateway["preparePut"]>[0] {
|
||||
return {
|
||||
transactionId: `transaction_lock_${suffix}`,
|
||||
transactionId: `transaction_put_${suffix}`,
|
||||
physicalGenerationId: suffix.padStart(32, "0") as OpfsPhysicalGenerationId,
|
||||
descriptor: {
|
||||
objectId: `object_lock_${suffix}`,
|
||||
objectId: `object_put_${suffix}`,
|
||||
scope: scopeA,
|
||||
generation: 1,
|
||||
byteLength: source.byteLength!,
|
||||
@@ -988,7 +994,7 @@ describe("R-5.3 lock waiting is not counted against the RPC timeout", () => {
|
||||
expect(await putA).toMatchObject({ ok: true });
|
||||
expect(await putB).toMatchObject({
|
||||
ok: true,
|
||||
value: { descriptor: { objectId: "object_lock_b" } },
|
||||
value: { descriptor: { objectId: "object_put_b" } },
|
||||
});
|
||||
expect(await removal).toMatchObject({ ok: true });
|
||||
expect(leases.held).toBe(false);
|
||||
@@ -1014,6 +1020,122 @@ describe("R-5.3 lock waiting is not counted against the RPC timeout", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("R-5.5 stored objects are checked against their own chunk size", () => {
|
||||
const oldPolicy = resolveOpfsRuntimePolicy({
|
||||
chunkSizeBytes: 64 * 1024,
|
||||
maxObjectBytes: 128 * 1024,
|
||||
maxChunkCount: 2,
|
||||
orphanGracePeriodMs: 60_000,
|
||||
});
|
||||
const newPolicy = resolveOpfsRuntimePolicy({
|
||||
chunkSizeBytes: 128 * 1024,
|
||||
maxObjectBytes: 128 * 1024,
|
||||
maxChunkCount: 1,
|
||||
orphanGracePeriodMs: 60_000,
|
||||
});
|
||||
|
||||
async function readAll(source: ByteSource): Promise<number[]> {
|
||||
const bytes: number[] = [];
|
||||
for await (const chunk of source.stream(new AbortController().signal)) {
|
||||
if (!chunk.ok) throw new Error(chunk.error.code);
|
||||
bytes.push(...chunk.value);
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
|
||||
it("reconciles and opens an object written before a chunk-size change and writes new objects at the new size", async () => {
|
||||
const root = new MemoryDirectory();
|
||||
const runtimeFor = (policy: typeof oldPolicy) =>
|
||||
createOpfsWorkerRuntime({
|
||||
root: root as unknown as FileSystemDirectoryHandle,
|
||||
crypto: globalThis.crypto,
|
||||
policy,
|
||||
leaseManager: immediateLeases({ count: 0 }),
|
||||
dedicatedWorker: true,
|
||||
supportsSynchronousAccessHandles: false,
|
||||
});
|
||||
const payload = Uint8Array.from(
|
||||
{ length: 64 * 1024 + 1 },
|
||||
(_, index) => index % 251,
|
||||
);
|
||||
|
||||
// Written with 64 KiB chunks and left FILES_READY, as a crash right
|
||||
// after FINISH_PUT would leave it.
|
||||
const before = loopbackGateway(runtimeFor(oldPolicy), oldPolicy);
|
||||
const written = await before.preparePut(
|
||||
putRequest("c", bytesSource(payload)),
|
||||
);
|
||||
if (!written.ok) throw new Error(written.error.code);
|
||||
const stored = written.value;
|
||||
expect(stored.descriptor.integrity.chunkSizeBytes).toBe(64 * 1024);
|
||||
expect(stored.chunks).toHaveLength(2);
|
||||
|
||||
// The chunk size is now 128 KiB.
|
||||
const after = loopbackGateway(runtimeFor(newPolicy), newPolicy);
|
||||
const transaction: OpfsJournalTransaction = {
|
||||
transactionId: "transaction_put_c",
|
||||
fencingToken: "fencing_12345678",
|
||||
mutation: "PUT",
|
||||
scope: scopeA,
|
||||
phase: "FILES_READY",
|
||||
objectId: stored.descriptor.objectId,
|
||||
expectedGeneration: null,
|
||||
targetGeneration: 1,
|
||||
targetByteLength: payload.byteLength,
|
||||
targetStoragePolicy: storagePolicy,
|
||||
budgetReservation: {
|
||||
namespace: scopeA.namespace,
|
||||
reservedBytes: payload.byteLength,
|
||||
hardBudgetBytes: storagePolicy.hardBudgetBytes,
|
||||
},
|
||||
startedAtEpochMs: 100,
|
||||
preparedObject: stored,
|
||||
};
|
||||
const rollback = vi.fn(async () => browserDataSuccess(undefined));
|
||||
const adapter = createOpfsByteStoreAdapter({
|
||||
journal: {
|
||||
...referenceJournal(
|
||||
new Set(stored.chunks.map((chunk) => chunk.digestHex)),
|
||||
),
|
||||
async listIncomplete() {
|
||||
return browserDataSuccess({
|
||||
transactions: [transaction],
|
||||
moreAvailable: false,
|
||||
});
|
||||
},
|
||||
async commitPut() {
|
||||
return browserDataSuccess({ ...transaction, phase: "COMMITTED" });
|
||||
},
|
||||
async complete() {
|
||||
return browserDataSuccess(undefined);
|
||||
},
|
||||
rollback,
|
||||
},
|
||||
worker: after,
|
||||
scope: scopeA,
|
||||
storagePolicy,
|
||||
policy: newPolicy,
|
||||
});
|
||||
expect(await adapter.maintenance.reconcile()).toMatchObject({
|
||||
ok: true,
|
||||
value: { committedTransactions: 1, rolledBackTransactions: 0 },
|
||||
});
|
||||
expect(rollback).not.toHaveBeenCalled();
|
||||
|
||||
const opened = await after.openObject(stored);
|
||||
if (!opened.ok) throw new Error(opened.error.code);
|
||||
expect(await readAll(opened.value)).toEqual([...payload]);
|
||||
|
||||
// A new write still follows the current policy.
|
||||
const fresh = await after.preparePut(putRequest("d", bytesSource(payload)));
|
||||
expect(fresh).toMatchObject({
|
||||
ok: true,
|
||||
value: { descriptor: { integrity: { chunkSizeBytes: 128 * 1024 } } },
|
||||
});
|
||||
expect(fresh.ok && fresh.value.chunks).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("OPFS worker client lifecycle", () => {
|
||||
it("rejects a colliding concurrent request id without replacing the first RPC", async () => {
|
||||
let listener: ((event: MessageEvent<unknown>) => void) | undefined;
|
||||
|
||||
Reference in New Issue
Block a user