fix(storage): verify stored OPFS objects by their own chunk size (R-5.5)

Prepared objects were validated in three places with different rules. The
worker required integrity.chunkSizeBytes, chunk sizes, chunk count and
object size to match the current runtime policy; the journal checked the
object against itself; the page only checked the shape. Changing
chunkSizeBytes (allowed by opfs-policy) made every existing object fail:
open returned INTEGRITY_FAILED, FILES_READY reconcile failed CORRUPT_DATA
(and, had cleanup succeeded, would have rolled back a valid object), and
COMMITTED reconcile failed INVALID_INPUT at FINALIZE_PUT.

The journal's policy-free rules move to opfs-prepared-object.ts, the one
definition of isPreparedObject (and of the physical schema check). The
worker's VERIFY_OBJECT, READ_CHUNK, FINALIZE_PUT and FILES_READY receipt
paths and the page's FINISH_PUT parse use it, so a stored object is
checked against its own manifest. The current policy still governs new
writes only: validateObjectWriteInput, BEGIN_PUT's chunk size check and
APPEND_CHUNK's size and count limits are unchanged. stableJson keeps its
two copies; they differ in how a non-finite number fails and unifying
them would change error mapping.

Verified: new test "R-5.5 stored objects are checked against their own
chunk size" writes an object with 64 KiB chunks, then reconciles and opens
it through a 128 KiB runtime. Before the fix reconcile failed CORRUPT_DATA
and open failed INTEGRITY_FAILED; now reconcile commits without rollback,
the bytes read back, and a new put uses 128 KiB chunks. A new journal test
pins that a 4 MiB-chunk object round-trips and an inconsistent one is
still rejected (passes before and after). Inventory and the storage
detail review list the new file.
Gates: check:architecture, check:types:app/test/web-worker, lint,
test:component pass; test:unit passes except removal-fixture.test.ts
(needs gitignored release artifacts absent from a fresh worktree);
check:adapter-inventory accepts the new row but still reports two rows
already missing at f0d3953 (http/platform-error-mapping.ts,
telemetry/feature-telemetry.ts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
2026-09-24 12:27:48 +09:00
co-authored by Claude Opus 5.5
parent 46af07b4fe
commit e74d835d6a
8 changed files with 342 additions and 278 deletions
@@ -79,6 +79,7 @@
| `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | logical object/journal/budget/chunk refcount의 IDB authority; begin/files-ready/commit/rollback/reconcile pages | OPFS ports, IDB failure, policy. journal+object+budget CAS atomicity가 좋다. STO-01 수정에서 incomplete row를 cleanup 확인 전 삭제하지 않아야 한다. |
| `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | logical journal과 physical worker를 saga로 조정, put/open/remove, reconcile/policy maintenance | OPFS/shared ports, journal, worker gateway, policy. STO-01의 journal/physical compensation ordering 결함 위치. |
| `src/adapters/storage/opfs/opfs-policy.ts` | root/lock/chunk/object/RPC/reconcile/GC hard limits 및 scope validation | shared/opfs ports. opaque physical path와 absolute caps 유지. |
| `src/adapters/storage/opfs/opfs-prepared-object.ts` | page·journal·worker가 함께 쓰는 prepared-object 검증기 하나 | OPFS/shared ports, opfs-policy scope 검증. R-5.5: 저장된 객체는 자기 manifest의 `chunkSizeBytes`로 검증하고 현재 정책값은 새 쓰기에만 적용. |
| `src/adapters/storage/opfs/opfs-worker-client.ts` | request correlation/timeout/abort/transferable chunking, worker gateway, streamed reads | protocol/policy/shared Result. STO-01의 untracked abort cleanup 및 STO-07의 shallow response parse. |
| `src/adapters/storage/opfs/opfs-worker-protocol.ts` | page↔DedicatedWorker request/response union 및 gateway contract | OPFS/shared ports. STO-07; protocol version/kind/effect certainty 추가 필요. |
| `src/adapters/storage/opfs/opfs-worker-runtime.ts` | DedicatedWorker OPFS physical layout, lock lease, immutable chunk, manifest/staging receipt, abort/finalize/remove/GC | protocol/policy/OPFS ports/Web Lock/Crypto. STO-01의 generation-only cleanup과 lease release 순서. sync handle `finally close` 등은 유지. |
+18 -17
View File
@@ -125,22 +125,23 @@
| 115 | `src/adapters/storage/opfs/indexeddb-opfs-journal.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 116 | `src/adapters/storage/opfs/opfs-byte-store-adapter.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 117 | `src/adapters/storage/opfs/opfs-policy.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 118 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 119 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 120 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 121 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
| 122 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
| 123 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 124 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 125 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 126 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 127 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 128 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 129 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 130 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 131 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 132 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 133 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 118 | `src/adapters/storage/opfs/opfs-prepared-object.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 119 | `src/adapters/storage/opfs/opfs-worker-client.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 120 | `src/adapters/storage/opfs/opfs-worker-protocol.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 121 | `src/adapters/storage/opfs/opfs-worker-runtime.ts` | [Storage/files](./03-storage-and-browser-files.md) |
| 122 | `src/adapters/telemetry/best-effort-telemetry.ts` | [Network/state](./01-network-and-state.md) |
| 123 | `src/adapters/telemetry/index.ts` | [Network/state](./01-network-and-state.md) |
| 124 | `src/adapters/web-push/inbound/notification-click-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 125 | `src/adapters/web-push/inbound/push-event-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 126 | `src/adapters/web-push/index.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 127 | `src/adapters/web-push/notification-registry.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 128 | `src/adapters/web-push/push-association-fence-store.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 129 | `src/adapters/web-push/push-codec.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 130 | `src/adapters/web-push/push-registration-gateway.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 131 | `src/adapters/web-push/push-subscription-adapter.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 132 | `src/adapters/web-push/runtime-support.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 133 | `src/adapters/web-push/service-worker-runtime.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
| 134 | `src/adapters/web-push/service-worker-scope-host.ts` | [Worker/push](./05-service-worker-and-web-push.md) |
합계: **133/133**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
합계: **134/134**. 새 adapter 파일이 추가되면 이 ledger와 해당 상세 리뷰 inventory를 같은 변경에서 갱신한다.
@@ -33,6 +33,7 @@ import {
} from "../../platform/indexeddb-transaction.ts";
import { mapIndexedDbException } from "../indexeddb/indexeddb-failure.ts";
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
import { isPreparedObject } from "./opfs-prepared-object.ts";
export type IndexedDbOpfsJournalDependencies = Readonly<{
authorityToken: string;
@@ -1607,118 +1608,6 @@ function isChunkReference(
);
}
/**
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
* window; v2 additionally carries a transaction-unique physical fencing token.
*/
function isSupportedPhysicalSchema(value: object): boolean {
const record = value as Record<string, unknown>;
if (record.physicalSchemaVersion === 1) return true;
return (
record.physicalSchemaVersion === 2 &&
typeof record.physicalGenerationId === "string" &&
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
);
}
function isPreparedObject(value: unknown): value is OpfsPreparedObject {
if (
!value ||
typeof value !== "object" ||
!("physicalSchemaVersion" in value) ||
!isSupportedPhysicalSchema(value) ||
!("descriptor" in value) ||
!value.descriptor ||
typeof value.descriptor !== "object" ||
!("chunks" in value) ||
!Array.isArray(value.chunks)
) {
return false;
}
const descriptor = value.descriptor as Record<string, unknown>;
const integrity =
descriptor.integrity && typeof descriptor.integrity === "object"
? (descriptor.integrity as Record<string, unknown>)
: null;
if (
typeof descriptor.objectId !== "string" ||
!SAFE_BOUNDARY_ID.test(descriptor.objectId) ||
!descriptor.scope ||
typeof descriptor.scope !== "object" ||
!isValidOpfsStorageScope(descriptor.scope as OpfsStorageScope) ||
typeof descriptor.generation !== "number" ||
!Number.isSafeInteger(descriptor.generation) ||
descriptor.generation < 1 ||
typeof descriptor.byteLength !== "number" ||
!Number.isSafeInteger(descriptor.byteLength) ||
descriptor.byteLength < 0 ||
typeof descriptor.mediaType !== "string" ||
descriptor.mediaType.length < 1 ||
typeof descriptor.createdAtEpochMs !== "number" ||
!Number.isSafeInteger(descriptor.createdAtEpochMs) ||
descriptor.createdAtEpochMs < 0 ||
!integrity ||
integrity.algorithm !== "SHA-256-TREE-V1" ||
typeof integrity.rootDigestHex !== "string" ||
!SHA256_HEX.test(integrity.rootDigestHex) ||
typeof integrity.chunkSizeBytes !== "number" ||
!Number.isSafeInteger(integrity.chunkSizeBytes) ||
integrity.chunkSizeBytes < 1 ||
!descriptor.storagePolicy ||
typeof descriptor.storagePolicy !== "object"
) {
return false;
}
try {
assertValidStoragePolicy(
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"],
);
} catch {
return false;
}
if (
(descriptor.scope as OpfsStorageScope).namespace !==
(
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"]
).namespace
) {
return false;
}
let totalBytes = 0;
for (let index = 0; index < value.chunks.length; index += 1) {
const chunk = value.chunks[index] as unknown;
if (
!chunk ||
typeof chunk !== "object" ||
!("sequence" in chunk) ||
chunk.sequence !== index ||
!("byteLength" in chunk) ||
typeof chunk.byteLength !== "number" ||
!Number.isSafeInteger(chunk.byteLength) ||
chunk.byteLength < 1 ||
chunk.byteLength > integrity.chunkSizeBytes ||
!("digestHex" in chunk) ||
typeof chunk.digestHex !== "string" ||
!SHA256_HEX.test(chunk.digestHex)
) {
return false;
}
if (
index < value.chunks.length - 1 &&
chunk.byteLength !== integrity.chunkSizeBytes
) {
return false;
}
totalBytes += chunk.byteLength;
}
return (
totalBytes === descriptor.byteLength &&
value.chunks.length ===
Math.ceil(descriptor.byteLength / integrity.chunkSizeBytes)
);
}
function stableJson(value: unknown): string {
if (
value === null ||
@@ -0,0 +1,131 @@
import type {
OpfsPreparedObject,
OpfsStorageScope,
} from "../../../application/ports/browser-file-storage/opfs-ports.ts";
import { assertValidStoragePolicy } from "../../../application/ports/browser-file-storage/shared.ts";
import { isValidOpfsStorageScope } from "./opfs-policy.ts";
const OPAQUE_OBJECT_ID = /^[A-Za-z0-9_-]{8,128}$/u;
const SHA256_HEX = /^[a-f0-9]{64}$/u;
/**
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
* window; v2 additionally carries a transaction-unique physical fencing token.
*/
function isSupportedPhysicalSchema(value: object): boolean {
const record = value as Record<string, unknown>;
if (record.physicalSchemaVersion === 1) return true;
return (
record.physicalSchemaVersion === 2 &&
typeof record.physicalGenerationId === "string" &&
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
);
}
/**
* R-5.5. The one prepared-object validator for the page, the journal and the
* worker. An object is checked against itself, including its own
* `chunkSizeBytes`, never against the current runtime policy: the policy
* governs new writes only (validateObjectWriteInput, BEGIN_PUT, APPEND_CHUNK),
* so changing the chunk size or another limit cannot make stored objects
* unreadable or unreconcilable.
*/
export function isPreparedObject(
value: unknown,
): value is OpfsPreparedObject {
if (
!value ||
typeof value !== "object" ||
!("physicalSchemaVersion" in value) ||
!isSupportedPhysicalSchema(value) ||
!("descriptor" in value) ||
!value.descriptor ||
typeof value.descriptor !== "object" ||
!("chunks" in value) ||
!Array.isArray(value.chunks)
) {
return false;
}
const descriptor = value.descriptor as Record<string, unknown>;
const integrity =
descriptor.integrity && typeof descriptor.integrity === "object"
? (descriptor.integrity as Record<string, unknown>)
: null;
if (
typeof descriptor.objectId !== "string" ||
!OPAQUE_OBJECT_ID.test(descriptor.objectId) ||
!descriptor.scope ||
typeof descriptor.scope !== "object" ||
!isValidOpfsStorageScope(descriptor.scope as OpfsStorageScope) ||
typeof descriptor.generation !== "number" ||
!Number.isSafeInteger(descriptor.generation) ||
descriptor.generation < 1 ||
typeof descriptor.byteLength !== "number" ||
!Number.isSafeInteger(descriptor.byteLength) ||
descriptor.byteLength < 0 ||
typeof descriptor.mediaType !== "string" ||
descriptor.mediaType.length < 1 ||
typeof descriptor.createdAtEpochMs !== "number" ||
!Number.isSafeInteger(descriptor.createdAtEpochMs) ||
descriptor.createdAtEpochMs < 0 ||
!integrity ||
integrity.algorithm !== "SHA-256-TREE-V1" ||
typeof integrity.rootDigestHex !== "string" ||
!SHA256_HEX.test(integrity.rootDigestHex) ||
typeof integrity.chunkSizeBytes !== "number" ||
!Number.isSafeInteger(integrity.chunkSizeBytes) ||
integrity.chunkSizeBytes < 1 ||
!descriptor.storagePolicy ||
typeof descriptor.storagePolicy !== "object"
) {
return false;
}
try {
assertValidStoragePolicy(
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"],
);
} catch {
return false;
}
if (
(descriptor.scope as OpfsStorageScope).namespace !==
(
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"]
).namespace
) {
return false;
}
let totalBytes = 0;
for (let index = 0; index < value.chunks.length; index += 1) {
const chunk = value.chunks[index] as unknown;
if (
!chunk ||
typeof chunk !== "object" ||
!("sequence" in chunk) ||
chunk.sequence !== index ||
!("byteLength" in chunk) ||
typeof chunk.byteLength !== "number" ||
!Number.isSafeInteger(chunk.byteLength) ||
chunk.byteLength < 1 ||
chunk.byteLength > integrity.chunkSizeBytes ||
!("digestHex" in chunk) ||
typeof chunk.digestHex !== "string" ||
!SHA256_HEX.test(chunk.digestHex)
) {
return false;
}
if (
index < value.chunks.length - 1 &&
chunk.byteLength !== integrity.chunkSizeBytes
) {
return false;
}
totalBytes += chunk.byteLength;
}
return (
totalBytes === descriptor.byteLength &&
value.chunks.length ===
Math.ceil(descriptor.byteLength / integrity.chunkSizeBytes)
);
}
@@ -15,6 +15,7 @@ import {
browserDataSuccess,
} from "../../browser-file-storage/result.ts";
import type { OpfsRuntimePolicy } from "./opfs-policy.ts";
import { isPreparedObject } from "./opfs-prepared-object.ts";
import {
OPFS_LOCK_WAITING_REQUEST_KINDS,
OPFS_WORKER_PROTOCOL_VERSION,
@@ -648,20 +649,6 @@ function parseCapabilities(value: unknown): OpfsCapabilities | null {
return value as OpfsCapabilities;
}
/**
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
* window; v2 additionally carries a transaction-unique physical fencing token.
*/
function isSupportedPhysicalSchema(value: object): boolean {
const record = value as Record<string, unknown>;
if (record.physicalSchemaVersion === 1) return true;
return (
record.physicalSchemaVersion === 2 &&
typeof record.physicalGenerationId === "string" &&
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
);
}
function parseCleanupEffect(value: unknown): OpfsCleanupEffect | null {
if (!value || typeof value !== "object") return null;
const kind = (value as Record<string, unknown>).kind;
@@ -671,18 +658,7 @@ function parseCleanupEffect(value: unknown): OpfsCleanupEffect | null {
}
function parsePreparedObject(value: unknown): OpfsPreparedObject | null {
if (
!value ||
typeof value !== "object" ||
!("physicalSchemaVersion" in value) ||
!isSupportedPhysicalSchema(value) ||
!("descriptor" in value) ||
!("chunks" in value) ||
!Array.isArray(value.chunks)
) {
return null;
}
return value as OpfsPreparedObject;
return isPreparedObject(value) ? value : null;
}
function parseOrphanCandidateBatch(
@@ -16,6 +16,7 @@ import {
isValidOpfsStorageScope,
type OpfsRuntimePolicy,
} from "./opfs-policy.ts";
import { isPreparedObject } from "./opfs-prepared-object.ts";
import {
OPFS_WORKER_PROTOCOL_VERSION,
} from "./opfs-worker-protocol.ts";
@@ -693,7 +694,7 @@ export function createOpfsWorkerRuntime(
async function verifyObject(
expected: OpfsPreparedObject,
): Promise<boolean> {
if (!isPreparedObjectSafe(expected, dependencies.policy)) return false;
if (!isPreparedObject(expected)) return false;
let stored: unknown;
try {
stored = await readJson(manifestPath(expected));
@@ -702,7 +703,7 @@ export function createOpfsWorkerRuntime(
throw error;
}
if (
!isPreparedObjectSafe(stored, dependencies.policy) ||
!isPreparedObject(stored) ||
stableJson(stored) !== stableJson(expected)
) {
return false;
@@ -741,7 +742,7 @@ export function createOpfsWorkerRuntime(
sequence: number,
): Promise<ArrayBuffer> {
if (
!isPreparedObjectSafe(expected, dependencies.policy) ||
!isPreparedObject(expected) ||
!Number.isSafeInteger(sequence) ||
sequence < 0
) {
@@ -908,7 +909,7 @@ export function createOpfsWorkerRuntime(
assertWorkerAvailable();
if (
!SAFE_TRANSACTION_ID.test(transactionId) ||
!isPreparedObjectSafe(preparedObject, dependencies.policy)
!isPreparedObject(preparedObject)
) {
throw new OpfsRuntimeFailure("INVALID_INPUT");
}
@@ -1160,7 +1161,7 @@ export function createOpfsWorkerRuntime(
}
if (
record.phase === "FILES_READY" &&
isPreparedObjectSafe(record.preparedObject, dependencies.policy) &&
isPreparedObject(record.preparedObject) &&
sameScope(record.preparedObject.descriptor.scope, scope)
) {
const prepared = record.preparedObject;
@@ -1727,118 +1728,6 @@ function stableJson(value: unknown): string {
throw new OpfsRuntimeFailure("CORRUPT_DATA");
}
/**
* STO-01 expand phase. v1 prepared objects stay readable through the rollback
* window; v2 additionally carries a transaction-unique physical fencing token.
*/
function isSupportedPhysicalSchema(value: object): boolean {
const record = value as Record<string, unknown>;
if (record.physicalSchemaVersion === 1) return true;
return (
record.physicalSchemaVersion === 2 &&
typeof record.physicalGenerationId === "string" &&
/^[0-9a-f]{32}$/u.test(record.physicalGenerationId)
);
}
function isPreparedObjectSafe(
value: unknown,
policy: OpfsRuntimePolicy,
): value is OpfsPreparedObject {
if (
!value ||
typeof value !== "object" ||
!("physicalSchemaVersion" in value) ||
!isSupportedPhysicalSchema(value) ||
!("descriptor" in value) ||
!value.descriptor ||
typeof value.descriptor !== "object" ||
!("chunks" in value) ||
!Array.isArray(value.chunks)
) {
return false;
}
const descriptor = value.descriptor as Record<string, unknown>;
const integrity =
descriptor.integrity && typeof descriptor.integrity === "object"
? (descriptor.integrity as Record<string, unknown>)
: null;
if (
typeof descriptor.objectId !== "string" ||
!policy.isObjectIdAllowed(descriptor.objectId) ||
!("scope" in descriptor) ||
!descriptor.scope ||
typeof descriptor.scope !== "object" ||
!isValidOpfsStorageScope(
descriptor.scope as OpfsPreparedObject["descriptor"]["scope"],
) ||
(descriptor.scope as OpfsPreparedObject["descriptor"]["scope"])
.namespace !==
(descriptor.storagePolicy as
| OpfsPreparedObject["descriptor"]["storagePolicy"]
| undefined)?.namespace ||
typeof descriptor.generation !== "number" ||
!Number.isSafeInteger(descriptor.generation) ||
descriptor.generation < 1 ||
typeof descriptor.byteLength !== "number" ||
!Number.isSafeInteger(descriptor.byteLength) ||
descriptor.byteLength < 0 ||
descriptor.byteLength > policy.maxObjectBytes ||
typeof descriptor.mediaType !== "string" ||
!policy.isMediaTypeAllowed(descriptor.mediaType) ||
typeof descriptor.createdAtEpochMs !== "number" ||
!Number.isSafeInteger(descriptor.createdAtEpochMs) ||
descriptor.createdAtEpochMs < 0 ||
!integrity ||
integrity.algorithm !== "SHA-256-TREE-V1" ||
typeof integrity.rootDigestHex !== "string" ||
!SHA256_HEX.test(integrity.rootDigestHex) ||
integrity.chunkSizeBytes !== policy.chunkSizeBytes ||
!("storagePolicy" in descriptor) ||
!descriptor.storagePolicy ||
typeof descriptor.storagePolicy !== "object" ||
value.chunks.length > policy.maxChunkCount
) {
return false;
}
try {
assertValidStoragePolicy(
descriptor.storagePolicy as OpfsPreparedObject["descriptor"]["storagePolicy"],
);
} catch {
return false;
}
let total = 0;
for (let index = 0; index < value.chunks.length; index += 1) {
const chunk = value.chunks[index] as unknown;
if (
!chunk ||
typeof chunk !== "object" ||
!("sequence" in chunk) ||
chunk.sequence !== index ||
!("byteLength" in chunk) ||
typeof chunk.byteLength !== "number" ||
!Number.isSafeInteger(chunk.byteLength) ||
chunk.byteLength < 1 ||
chunk.byteLength > policy.chunkSizeBytes ||
!("digestHex" in chunk) ||
typeof chunk.digestHex !== "string" ||
!SHA256_HEX.test(chunk.digestHex)
) {
return false;
}
if (index < value.chunks.length - 1 && chunk.byteLength !== policy.chunkSizeBytes) {
return false;
}
total += chunk.byteLength;
}
return (
total === descriptor.byteLength &&
value.chunks.length ===
Math.ceil(descriptor.byteLength / policy.chunkSizeBytes)
);
}
function success(
requestId: string,
kind: OpfsWorkerRequest["kind"],
+55
View File
@@ -710,6 +710,61 @@ describe("IndexedDB OPFS journal", () => {
expect(second.value.nextObjectId).toBeNull();
});
it("R-5.5 keeps an object readable by its own chunk size and still rejects an inconsistent one", async () => {
const { journal } = createHarness();
const chunkSizeBytes = 4 * 1024 * 1024;
const byteLength = chunkSizeBytes + 1;
const input: BeginOpfsJournalTransaction = {
...beginInput("transaction_chunk001", "object_chunk_001", byteLength),
targetStoragePolicy: {
...policy,
softBudgetBytes: 2 * byteLength,
hardBudgetBytes: 2 * byteLength,
},
};
const base = prepared(input);
const stored: OpfsPreparedObject = {
...base,
descriptor: {
...base.descriptor,
integrity: { ...base.descriptor.integrity, chunkSizeBytes },
},
chunks: [
{ sequence: 0, byteLength: chunkSizeBytes, digestHex: "c".repeat(64) },
{ sequence: 1, byteLength: 1, digestHex: "d".repeat(64) },
],
};
const begun = await journal.begin(input);
if (!begun.ok) throw new Error("begin failed");
const token = begun.value.fencingToken;
// A first chunk shorter than the object's own chunk size is not a
// well-formed object under any policy.
expect(
await journal.markFilesReady(input.transactionId, token, {
...stored,
chunks: [
{ sequence: 0, byteLength: 1, digestHex: "c".repeat(64) },
{ sequence: 1, byteLength: chunkSizeBytes, digestHex: "d".repeat(64) },
],
}),
).toMatchObject({ ok: false, error: { code: "INVALID_INPUT" } });
expect(
await journal.markFilesReady(input.transactionId, token, stored),
).toMatchObject({ ok: true, value: { phase: "FILES_READY" } });
expect(await journal.listIncomplete(10)).toMatchObject({
ok: true,
value: { transactions: [{ preparedObject: stored }] },
});
expect(
await journal.commitPut(input.transactionId, token),
).toMatchObject({ ok: true });
expect(
await journal.getCommittedObject(scope, input.objectId),
).toEqual({ ok: true, value: stored });
});
it("answers UNSUPPORTED once the journal is closed", async () => {
const { factory, journal } = createHarness();
expect(
+128 -6
View File
@@ -2,6 +2,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";
import type {
OpfsJournalPort,
OpfsJournalTransaction,
OpfsPreparedObject,
OpfsPhysicalGenerationId,
OpfsStorageScope,
@@ -904,24 +905,29 @@ function stalledSource(): Readonly<{
};
}
function oneByteSource(): ByteSource {
function bytesSource(bytes: Uint8Array): ByteSource {
return {
byteLength: 1,
byteLength: bytes.byteLength,
async *stream() {
yield browserDataSuccess(new Uint8Array([6]));
yield browserDataSuccess(Uint8Array.from(bytes));
},
};
}
function oneByteSource(): ByteSource {
return bytesSource(new Uint8Array([6]));
}
/** `suffix` is a hex digit so it also forms a physical generation id. */
function putRequest(
suffix: string,
source: ByteSource,
): Parameters<OpfsWorkerGateway["preparePut"]>[0] {
return {
transactionId: `transaction_lock_${suffix}`,
transactionId: `transaction_put_${suffix}`,
physicalGenerationId: suffix.padStart(32, "0") as OpfsPhysicalGenerationId,
descriptor: {
objectId: `object_lock_${suffix}`,
objectId: `object_put_${suffix}`,
scope: scopeA,
generation: 1,
byteLength: source.byteLength!,
@@ -988,7 +994,7 @@ describe("R-5.3 lock waiting is not counted against the RPC timeout", () => {
expect(await putA).toMatchObject({ ok: true });
expect(await putB).toMatchObject({
ok: true,
value: { descriptor: { objectId: "object_lock_b" } },
value: { descriptor: { objectId: "object_put_b" } },
});
expect(await removal).toMatchObject({ ok: true });
expect(leases.held).toBe(false);
@@ -1014,6 +1020,122 @@ describe("R-5.3 lock waiting is not counted against the RPC timeout", () => {
});
});
describe("R-5.5 stored objects are checked against their own chunk size", () => {
const oldPolicy = resolveOpfsRuntimePolicy({
chunkSizeBytes: 64 * 1024,
maxObjectBytes: 128 * 1024,
maxChunkCount: 2,
orphanGracePeriodMs: 60_000,
});
const newPolicy = resolveOpfsRuntimePolicy({
chunkSizeBytes: 128 * 1024,
maxObjectBytes: 128 * 1024,
maxChunkCount: 1,
orphanGracePeriodMs: 60_000,
});
async function readAll(source: ByteSource): Promise<number[]> {
const bytes: number[] = [];
for await (const chunk of source.stream(new AbortController().signal)) {
if (!chunk.ok) throw new Error(chunk.error.code);
bytes.push(...chunk.value);
}
return bytes;
}
it("reconciles and opens an object written before a chunk-size change and writes new objects at the new size", async () => {
const root = new MemoryDirectory();
const runtimeFor = (policy: typeof oldPolicy) =>
createOpfsWorkerRuntime({
root: root as unknown as FileSystemDirectoryHandle,
crypto: globalThis.crypto,
policy,
leaseManager: immediateLeases({ count: 0 }),
dedicatedWorker: true,
supportsSynchronousAccessHandles: false,
});
const payload = Uint8Array.from(
{ length: 64 * 1024 + 1 },
(_, index) => index % 251,
);
// Written with 64 KiB chunks and left FILES_READY, as a crash right
// after FINISH_PUT would leave it.
const before = loopbackGateway(runtimeFor(oldPolicy), oldPolicy);
const written = await before.preparePut(
putRequest("c", bytesSource(payload)),
);
if (!written.ok) throw new Error(written.error.code);
const stored = written.value;
expect(stored.descriptor.integrity.chunkSizeBytes).toBe(64 * 1024);
expect(stored.chunks).toHaveLength(2);
// The chunk size is now 128 KiB.
const after = loopbackGateway(runtimeFor(newPolicy), newPolicy);
const transaction: OpfsJournalTransaction = {
transactionId: "transaction_put_c",
fencingToken: "fencing_12345678",
mutation: "PUT",
scope: scopeA,
phase: "FILES_READY",
objectId: stored.descriptor.objectId,
expectedGeneration: null,
targetGeneration: 1,
targetByteLength: payload.byteLength,
targetStoragePolicy: storagePolicy,
budgetReservation: {
namespace: scopeA.namespace,
reservedBytes: payload.byteLength,
hardBudgetBytes: storagePolicy.hardBudgetBytes,
},
startedAtEpochMs: 100,
preparedObject: stored,
};
const rollback = vi.fn(async () => browserDataSuccess(undefined));
const adapter = createOpfsByteStoreAdapter({
journal: {
...referenceJournal(
new Set(stored.chunks.map((chunk) => chunk.digestHex)),
),
async listIncomplete() {
return browserDataSuccess({
transactions: [transaction],
moreAvailable: false,
});
},
async commitPut() {
return browserDataSuccess({ ...transaction, phase: "COMMITTED" });
},
async complete() {
return browserDataSuccess(undefined);
},
rollback,
},
worker: after,
scope: scopeA,
storagePolicy,
policy: newPolicy,
});
expect(await adapter.maintenance.reconcile()).toMatchObject({
ok: true,
value: { committedTransactions: 1, rolledBackTransactions: 0 },
});
expect(rollback).not.toHaveBeenCalled();
const opened = await after.openObject(stored);
if (!opened.ok) throw new Error(opened.error.code);
expect(await readAll(opened.value)).toEqual([...payload]);
// A new write still follows the current policy.
const fresh = await after.preparePut(putRequest("d", bytesSource(payload)));
expect(fresh).toMatchObject({
ok: true,
value: { descriptor: { integrity: { chunkSizeBytes: 128 * 1024 } } },
});
expect(fresh.ok && fresh.value.chunks).toHaveLength(1);
});
});
describe("OPFS worker client lifecycle", () => {
it("rejects a colliding concurrent request id without replacing the first RPC", async () => {
let listener: ((event: MessageEvent<unknown>) => void) | undefined;