diff --git a/README.md b/README.md index a5e0fbd8..787e9ca2 100644 --- a/README.md +++ b/README.md @@ -82,16 +82,14 @@ list is in [`docs/operations/ci-quality-gates.md`](docs/operations/ci-quality-ga ## Included starter experience The default build mounts a domain-neutral application shell with a header, -responsive sidebar, route focus management, session integration status, and a -persistent `system` / `light` / `dark` theme selector. +responsive sidebar, route focus management, session integration status, a +persistent `system` / `light` / `dark` theme selector, and a language selector. -| Route | Purpose | -| --- | --- | -| `/` | implementation readiness and starter links | -| `/examples/ui` | buttons, fields, cards, alerts, badges, modal, and tokens | -| `/examples/states` | loading, refresh, empty, error, auth, forbidden, and not-found states | -| `/examples/auth` | reactive external-auth integration seam | -| `/examples/reference-resources` | removable, session-required reference feature | +The route list is `ROUTE_REGISTRY` (`src/features/installed-feature-contracts.ts`): +the platform routes of `src/contracts/routes.ts` plus the routes of every +installed feature, the removable reference feature included. The platform +overview page (`/examples/platform`) shows the composed list at runtime, so +this README keeps no copy of it and adding a feature does not edit it. `AUTH_MODE=demo` is credential-free and accepted only in local/development environments. Deployments use `AUTH_MODE=external` and provide the opaque auth @@ -295,12 +293,10 @@ corepack pnpm exec playwright install --with-deps chromium firefox webkit Two gates intentionally need external evidence: - `review:a11y-manual` (release gate FE-GATE-028) needs a signed human - keyboard/focus/screen-reader review for all ten registered routes: `APP_HOME`, `EXAMPLES_PLATFORM`, - `EXAMPLES_UI`, `EXAMPLES_STATES`, `EXAMPLES_AUTH`, `NOT_FOUND`, - `REFERENCE_RESOURCE_LIST`, `REFERENCE_RESOURCE_DETAIL`, - `REFERENCE_RESOURCE_FORM` and `REFERENCE_RESOURCE_STATUS`. - `verify:documentation` derives that list from the route registry and fails if - this paragraph falls behind it. The merge gate FE-GATE-009 runs only + keyboard/focus/screen-reader review of every registered route, one record + per route at `artifacts/tests/a11y-manual/.md`. The routes are + `ROUTE_REGISTRY`; `verify:documentation` fails, naming the route ID, when a + registered route has no record. The merge gate FE-GATE-009 runs only `review:a11y-manual -- --format-only`, which checks that every route record keeps the template's fields, reviewed or still pending. - `collect:web-vitals-evidence` stays `FAIL_UNVERIFIED` until a reviewed minimum diff --git a/docs/README.md b/docs/README.md index e0ac866f..d0086516 100644 --- a/docs/README.md +++ b/docs/README.md @@ -37,11 +37,18 @@ README와 입구 문서는 영어, recipe·ledger·스타일 문서는 한국어 - 디자인 토큰과 수동 접근성 검토: [`styling/design-tokens.md`](styling/design-tokens.md), [`accessibility/manual-checklist.md`](accessibility/manual-checklist.md) - 버전별 변경: [`../CHANGELOG.md`](../CHANGELOG.md) +`verify:documentation`은 이 파일과 이 절에서 링크한 문서(폴더 링크는 그 안의 `.md` 전부)에 +적힌 repo 경로·링크·package script·FE-GATE ID가 실제로 있는지 검사한다. `CHANGELOG.md`는 +지난 상태를 적는 문서라 빼고, git이 무시하는 생성 산출물과 복사한 프로젝트가 지운 작업 이력은 +없어도 된다. 설명·수치가 코드와 맞는지는 검사하지 않는다. 규칙과 예외의 정본은 +`scripts/verify-documentation-readiness.ts`다. + 게이트가 직접 읽는 파일이라 지우거나 옮기면 게이트가 실패한다. | 파일 | 읽는 명령 | |---|---| -| `architecture/review-ledger.json`, `architecture/review-evidence.md`, `accessibility/manual-checklist.md` | `verify:documentation` (FE-GATE-017) | +| 이 파일, `architecture/review-ledger.json`, `architecture/review-evidence.md` | `verify:documentation` (FE-GATE-017) | +| repo 루트의 `artifacts/tests/a11y-manual/` 라우트별 기록 | `verify:documentation` (FE-GATE-017), `review:a11y-manual` (FE-GATE-009, FE-GATE-028) | | `reviews/adapters/INVENTORY.md` | `check:adapter-inventory` (FE-GATE-010) | | `operations/adapter-remediation-ledger.md`, `operations/adapter-remediation-dispositions.json` | `check:remediation-ledger` (게이트 밖. 이유는 `config/ci/ungated-scripts.json`) | diff --git a/docs/accessibility/manual-checklist.md b/docs/accessibility/manual-checklist.md index 77b804f7..8dad35c9 100644 --- a/docs/accessibility/manual-checklist.md +++ b/docs/accessibility/manual-checklist.md @@ -1,14 +1,12 @@ # Manual accessibility review checklist Automated axe checks do not establish WCAG conformance. A human reviewer must -review all ten route records in `artifacts/tests/a11y-manual/` against one -release candidate and sign them. The required scope is derived from the route -registry: `APP_HOME`, `EXAMPLES_PLATFORM`, `EXAMPLES_UI`, `EXAMPLES_STATES`, -`EXAMPLES_AUTH`, `NOT_FOUND`, `REFERENCE_RESOURCE_LIST`, -`REFERENCE_RESOURCE_DETAIL`, `REFERENCE_RESOURCE_FORM` and -`REFERENCE_RESOURCE_STATUS`. Copy the template fields exactly; the gate rejects -blank identity/timestamp/signature fields, pending verdicts, mismatched release -IDs, or missing routes. +review every route record in `artifacts/tests/a11y-manual/` against one +release candidate and sign them. The required scope is the route registry +(`ROUTE_REGISTRY` in `src/features/installed-feature-contracts.ts`): one +record per route, named `.md`. Copy the template fields exactly; the +gate rejects blank identity/timestamp/signature fields, pending verdicts, +mismatched release IDs, or missing routes. The signed review is the release gate FE-GATE-028 (`review:a11y-manual`), so it is required once per release candidate, not on every merge. The merge gate @@ -16,11 +14,11 @@ FE-GATE-009 runs `review:a11y-manual -- --format-only`: each record must keep the fields below with `pending`, `pass` or `not-applicable ()` verdicts, and a record marked `reviewed` must be complete. -This list is not maintained by hand: `verify:documentation` compares it against -the installed route registry and fails when a registered route is absent. It -said six routes while ten were registered, which put the platform overview and -the three reference-resource screens outside the declared manual review scope -without anyone deciding they should be. +This document keeps no route list. A list here once said six routes while ten +were registered, and keeping it current made every new feature edit this file. +Instead `verify:documentation` fails, naming the route ID, when a registered +route has no record file. A new route starts from a copy of a pending record +with its title and `Route ID` changed. Allowed item verdicts: diff --git a/docs/contracts/compatibility.md b/docs/contracts/compatibility.md index 8191cb93..49c43497 100644 --- a/docs/contracts/compatibility.md +++ b/docs/contracts/compatibility.md @@ -20,15 +20,15 @@ reader that accepts it must ship before any writer sends it. | Boundary | Reader policy | Adding a field or value | Code | |---|---|---|---| -| Runtime config V2 (`/config.json`) | strict object; `CONFIG_SCHEMA_VERSION` is the literal `"2.0"` | breaking | `src/contracts/release-artifacts.ts:163-171` | -| `CAPABILITY_OVERRIDES` | strict; unknown capability rejected | breaking | `src/contracts/release-artifacts.ts:50-63` | -| `FEATURE_OVERRIDES` | record; unknown feature id ignored | additive | `src/contracts/release-artifacts.ts:65-80` | -| Release manifest V2 | strict object | breaking | `src/contracts/release-artifacts.ts:200-208` | -| HTTP V2 success/failure envelope | strict object (`meta` strips) | breaking | `src/adapters/http/schema-registry.ts:12-34` | -| HTTP V3 response DTO (reference feature) | `.strip()` drops unknown fields | additive | `src/features/reference-feature/contracts/reference-schemas.ts:29-35` | -| Realtime envelope | exact key set; unknown `eventType` is `MALFORMED_EVENT` | breaking | `src/adapters/realtime/event-codec.ts:126,149` | -| Browser RPC transport error code | closed vocabulary; unknown code is a protocol violation | breaking | `src/adapters/browser-rpc/browser-rpc-runtime.ts:1473` | -| Persisted client storage | versioned key; incompatible value discarded | use a new key version | `src/contracts/storage-keys.ts:148-150` | +| Runtime config V2 (`/config.json`) | strict object; `CONFIG_SCHEMA_VERSION` is the literal `"2.0"` | breaking | `runtimeConfigV2ArtifactSchema` in `src/contracts/release-artifacts.ts` | +| `CAPABILITY_OVERRIDES` | strict; unknown capability rejected | breaking | `capabilityOverrideArtifactSchema` in `src/contracts/release-artifacts.ts` | +| `FEATURE_OVERRIDES` | record; unknown feature id ignored | additive | `featureOverrideArtifactSchema` in `src/contracts/release-artifacts.ts` | +| Release manifest V2 | strict object | breaking | `releaseManifestV2ArtifactSchema` in `src/contracts/release-artifacts.ts` | +| HTTP V2 success/failure envelope | strict object (`meta` strips) | breaking | `successEnvelopeSchema`, `failureEnvelopeSchema` in `src/adapters/http/schema-registry.ts` | +| HTTP V3 response DTO (reference feature) | `.strip()` drops unknown fields | additive | `referenceResourceDtoSchema` in `src/features/reference-feature/contracts/reference-schemas.ts` | +| Realtime envelope | exact key set; unknown `eventType` is `MALFORMED_EVENT` | breaking | `decodeUnsafe` in `src/adapters/realtime/event-codec.ts` | +| Browser RPC transport error code | closed vocabulary; unknown code is a protocol violation | breaking | `decodeTransportFailure` in `src/adapters/browser-rpc/browser-rpc-runtime.ts` | +| Persisted client storage | versioned key; incompatible value discarded | use a new key version | `defineStorageKey` in `src/contracts/storage-keys.ts` | `CAPABILITY_OVERRIDES` and `FEATURE_OVERRIDES` differ on purpose. Feature ids vary per build because each downstream project installs its own features, and @@ -39,7 +39,7 @@ addition. ## Schema change classifier and FE-GATE-014 -`classifyObjectSchemaChange` (`src/contracts/compatibility.ts:110`) answers +`classifyObjectSchemaChange` (`src/contracts/compatibility.ts`) answers for a reader built against `before`. It is fail-closed: only a new property under a reader without `additionalProperties: false` is `additive`. A removed or changed property (type, enum, nullable, format, ...), a different required diff --git a/docs/operations/adapter-remediation-ledger.md b/docs/operations/adapter-remediation-ledger.md index 94e784cd..1bc49b0f 100644 --- a/docs/operations/adapter-remediation-ledger.md +++ b/docs/operations/adapter-remediation-ledger.md @@ -5,6 +5,11 @@ > > Plan: [2026-08-13 adapter remediation](../superpowers/plans/2026-08-13-adapter-remediation.md). > Baseline revision: `develop` / `4dc033cf33a5b6173bbf960d5eb464a406dc4c92`. +> +> Test paths are given at their current location. Two suites moved after +> these runs: `ci-artifact-contract.test.ts` from `tests/unit/` to +> `tests/system/ci-runner/`, and `provider-guardian-transaction.test.ts` from +> `tests/unit/` to `tests/integration/`. ## Containment state (plan Task 1, Step 1) @@ -44,7 +49,7 @@ Captured on the baseline revision before any source change. ### `test:unit` failure attribution -The single failing file is `tests/unit/ci-artifact-contract.test.ts`. All 19 failures come from +The single failing file is `tests/system/ci-runner/ci-artifact-contract.test.ts`. All 19 failures come from child-process, cgroup, and filesystem-permission behavior of the sandboxed execution environment, not from adapter code. Verbatim causes recorded from the run log: @@ -208,7 +213,7 @@ No finding remains `NOT_STARTED`. | Gate | Status | Attribution | | --- | --- | --- | -| `tests/unit/ci-artifact-contract.test.ts` | 19 failed | Identical to the baseline capture. Sandbox child-process, cgroup and `/tmp` permission behavior; unrelated to adapters. Count did not change across any task. | +| `tests/system/ci-runner/ci-artifact-contract.test.ts` | 19 failed | Identical to the baseline capture. Sandbox child-process, cgroup and `/tmp` permission behavior; unrelated to adapters. Count did not change across any task. | | `corepack pnpm test:browser-capabilities` (webkit) | 6 failed / 24 passed | WebKit cannot launch: missing `libevent-2.1-7t64` and `libavif16`. Chromium passes. **UNVERIFIED**, not PASS. | | `corepack pnpm test:browser-file-storage-removal` | exit 1 | The reduced-removal-fixture prunes the CI contract to 77/89/102 while `scripts/contracts/ci-gates.ts:481` demands exactly 81/93/105. That file, the CI contract and `scripts/lib/removal-fixture.ts` are **unchanged since the baseline revision** (`git diff --name-only 4dc033c..HEAD` outside `src/`, `tests/` and `docs/` lists only the two Service Worker build scripts), so this is pre-existing, not a regression from this work. | | `corepack pnpm test:realtime-removal` | exit 1 | Same pre-existing reduced-fixture arithmetic. | @@ -336,8 +341,8 @@ Run on the landed tree. Only what actually passed is claimed as passing. | Gate | Status | Attribution | | --- | --- | --- | | `tests/unit/ci-workflow-generation.test.ts` | 82 failed / 325 passed | Identical on the pre-change baseline (`git stash` comparison). The subprocess gates it spawns cannot run in this sandbox. | -| `tests/unit/ci-artifact-contract.test.ts` | fails | Unchanged pre-existing sandbox, cgroup and `/tmp` permission behaviour. | -| `tests/unit/security-followup.test.ts`, `tests/unit/provider-guardian-transaction.test.ts`, `tests/unit/risk-coverage.test.ts` | flaky under full-suite load | Historical baseline result. All three passed in a fresh process (78 passed together). | +| `tests/system/ci-runner/ci-artifact-contract.test.ts` | fails | Unchanged pre-existing sandbox, cgroup and `/tmp` permission behaviour. | +| `tests/unit/security-followup.test.ts`, `tests/integration/provider-guardian-transaction.test.ts`, `tests/unit/risk-coverage.test.ts` | flaky under full-suite load | Historical baseline result. All three passed in a fresh process (78 passed together). | > Current classification update (2026-09-18): the provider guardian transaction > suite moved to `tests/integration/provider-guardian-transaction.test.ts`. @@ -356,7 +361,7 @@ fixture with a single directory symlink at `/node_modules`: - `scripts/lib/removal-fixture.ts` - `scripts/check-supply-chain-provider-fixtures.ts` - `tests/integration/security-followup-archive.test.ts` -- `tests/unit/ci-artifact-contract.test.ts` +- `tests/system/ci-runner/ci-artifact-contract.test.ts` Each fixture then runs `pnpm` inside itself. pnpm does not recognise the modules directory it finds there and purges it; with `CI=true` it does so without a @@ -380,9 +385,9 @@ files above plus the two flaky-under-load ones: | File | Failed | Attribution | | --- | ---: | --- | | `tests/unit/ci-workflow-generation.test.ts` | 82 | Identical on the pre-change baseline (`git stash` comparison). Its subprocess gates cannot run in this sandbox. | -| `tests/unit/ci-artifact-contract.test.ts` | 19 | Unchanged pre-existing sandbox, cgroup and `/tmp` permission behaviour. | +| `tests/system/ci-runner/ci-artifact-contract.test.ts` | 19 | Unchanged pre-existing sandbox, cgroup and `/tmp` permission behaviour. | | `tests/unit/security-followup.test.ts` | 2 | Passes in isolation. | -| `tests/unit/provider-guardian-transaction.test.ts` (historical path; now `tests/integration/provider-guardian-transaction.test.ts`) | 1 | Historical baseline: passed in isolation; current suite is classified as process integration. | +| `tests/integration/provider-guardian-transaction.test.ts` | 1 | Historical baseline: passed in isolation; current suite is classified as process integration. | 1619 passed / 1723 total, and `tests/unit/removal-fixture.test.ts`, `tests/unit/supply-chain.test.ts` and @@ -465,7 +470,7 @@ forward as if it were green. The full `tests/unit` + `tests/integration` run is **1,845 passed / 1,864**, 128 of 129 files green. Every one of the 19 failures is in -`tests/unit/ci-artifact-contract.test.ts` and is the pre-existing sandbox, +`tests/system/ci-runner/ci-artifact-contract.test.ts` and is the pre-existing sandbox, cgroup, RLIMIT and `/tmp` permission behaviour already recorded above — the same file failed identically before this work. No adapter test fails. diff --git a/docs/operations/ci-quality-gates.md b/docs/operations/ci-quality-gates.md index 20a4fbf9..6e24b5de 100644 --- a/docs/operations/ci-quality-gates.md +++ b/docs/operations/ci-quality-gates.md @@ -126,11 +126,16 @@ DOCUMENTATION_READY (off-chain) `DOCUMENTATION_READY`(`verify:documentation`)가 검사하는 범위는 좁다 (`scripts/verify-documentation-readiness.ts`). -- diagram 리뷰 ledger(`docs/architecture/review-ledger.json`)와 - `review-evidence.md`가 같은 경로·해시·점수를 담는지 -- README와 `docs/accessibility/manual-checklist.md`가 설치된 route ID를 모두 - 적는지 +- 설치된 라우트마다 수동 접근성 기록 `artifacts/tests/a11y-manual/.md`가 + 있는지. 없으면 route ID를 적고 실패한다 - README가 빌드와 승인 단계가 읽는 환경 변수를 모두 적는지 +- 입구 문서가 실제 `src` 최상위 폴더와 `.dependency-cruiser.json` 규칙 이름을 + 적는지 +- `docs/README.md` 1절의 현재 계약 문서에 적힌 repo 경로·링크·package script· + FE-GATE ID가 실제로 있는지 +- 2026-07-18 diagram 리뷰 기록의 두 사본(`docs/architecture/review-ledger.json`, + `review-evidence.md`)이 같은 해시·점수를 담는지. 원본 diagram은 repo 밖에 + 있으므로 이 항목은 구조가 바뀌어도 통과한다 문서의 다른 내용(수치, 목록, 설명)이 코드와 맞는지는 검사하지 않는다(리뷰 14-07). diff --git a/docs/styling/design-tokens.md b/docs/styling/design-tokens.md index b2940c30..1a2d94c5 100644 --- a/docs/styling/design-tokens.md +++ b/docs/styling/design-tokens.md @@ -1,8 +1,32 @@ # Design-token styling contract -`src/presentation/styles/theme.css` is the styling SSOT. Components consume -semantic color, spacing, typography, status, and radius tokens through static -classes. `/examples/ui` is the executable token and primitive gallery. +Tokens are defined in three layers under `src/presentation/design-system/tokens/`: +`primitive.css` holds the raw palette and scales, `semantic.css` maps them to +roles (surface, content, border, action, status) and overrides those roles for +the dark theme, and `component.css` maps roles to per-component values. +`src/presentation/styles/theme.css` only imports the three files and the style +files under `src/presentation/styles/theme/`. Components consume semantic +color, spacing, typography, status, and radius tokens through static classes. +`/examples/ui` is the executable token and primitive gallery. + +## Rules a rebrand must keep + +`check:design-system` (`scripts/check-design-system.ts`) fails when a token +edit breaks one of these. The reasons are next to each rule in +`scripts/lib/design-system-rules.ts`. + +- UI-01: `--color-focus` has at least 3:1 contrast against every surface a + focus ring can sit on, in both themes. Those tokens must resolve to opaque + `oklch()` values so the contrast can be computed. +- UI-02: `color-mix()` mixes in `oklab`, never in a hue-based space. +- UI-03: every component token is read by some style. +- UI-05: `--font-sans` starts with a generic family or one the repository + ships with `@font-face`. + +The same check requires the token names in `token-contract.ts`, the dark +overrides of the core semantic roles, a forced-colors fallback in +`component.css`, no raw palette values outside the token files, and no direct +headless-library imports. ## Theme contract diff --git a/scripts/contracts/release-artifacts.ts b/scripts/contracts/release-artifacts.ts index 6d2f9c54..d70ed79a 100644 --- a/scripts/contracts/release-artifacts.ts +++ b/scripts/contracts/release-artifacts.ts @@ -1393,9 +1393,9 @@ export const documentationReviewArtifactSchema = z .strict(), reportDigestValid: z.boolean(), /** - * The declared review scope, derived from the installed route registry - * rather than read off a sentence. Both scope documents claimed six routes - * while ten were registered. + * The manual accessibility review scope: the installed routes that have no + * record file under artifacts/tests/a11y-manual/. It is derived from the + * route registry, not read off a document (1101/14-05). */ routeScope: z.array( z @@ -1421,6 +1421,18 @@ export const documentationReviewArtifactSchema = z }) .strict(), ).min(1), + // 1101/14-05: repo paths, package scripts and gate IDs that the current + // contract documents (docs/README.md §1) name but that do not exist. + documentReferences: z.array( + z + .object({ + path: nonEmptyString, + missingPaths: z.array(nonEmptyString), + missingScripts: z.array(nonEmptyString), + unknownGateIds: z.array(nonEmptyString), + }) + .strict(), + ).min(1), results: z.array( z .object({ @@ -1466,9 +1478,15 @@ export const documentationReviewArtifactSchema = z entry.unknownFolders.length === 0 && entry.missingRules.length === 0, ) && + artifact.documentReferences.every( + (entry) => + entry.missingPaths.length === 0 && + entry.missingScripts.length === 0 && + entry.unknownGateIds.length === 0, + ) && artifact.results.every(({ passed }) => passed)) ) { - context.addIssue({ code: "custom", path: ["passed"], message: "must agree with report digest, documented scope, source structure and review results" }); + context.addIssue({ code: "custom", path: ["passed"], message: "must agree with report digest, documented scope, source structure, document references and review results" }); } }); diff --git a/scripts/verify-documentation-readiness.ts b/scripts/verify-documentation-readiness.ts index 8aec9db6..0165ec7e 100644 --- a/scripts/verify-documentation-readiness.ts +++ b/scripts/verify-documentation-readiness.ts @@ -1,24 +1,22 @@ -import { access, mkdir, readdir, readFile } from "node:fs/promises"; +import { existsSync } from "node:fs"; +import { mkdir, readdir, readFile } from "node:fs/promises"; +import { dirname, join, matchesGlob, normalize } from "node:path/posix"; +import { fileURLToPath } from "node:url"; -import { ROUTE_REGISTRY } from "../src/features/installed-feature-contracts.ts"; +import { RETIRED_GATE_IDS } from "./contracts/ci-gates.ts"; import { documentationReviewArtifactSchema } from "./contracts/release-artifacts.ts"; import { RUNTIME_OVERRIDES } from "./generate-runtime-config.ts"; +import { MANUAL_A11Y_ROUTE_IDS } from "./lib/manual-a11y-evidence.ts"; import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts"; /** - * Documents that state the review scope. The route registry is the source of - * truth for what that scope is, so these have to enumerate exactly the - * installed routes. - * - * Both said "six routes" while ten were registered: the four newest — the - * platform overview and three reference-resource screens — were outside the - * declared manual accessibility scope without anybody deciding they should be. - * A hand-typed count drifts silently, so it is derived here instead. + * The manual accessibility review scope is one record per installed route. + * README and the checklist used to list the route IDs and this script checked + * the lists (they had said six routes while ten were registered). That made + * every new feature edit two template-owned documents (1101/14-05), so the + * documents now name this folder and the record files are what is checked. */ -const ROUTE_SCOPE_DOCUMENTS = Object.freeze([ - "README.md", - "docs/accessibility/manual-checklist.md", -]); +const MANUAL_A11Y_RECORDS = "artifacts/tests/a11y-manual/"; type DocumentationReview = Readonly<{ sourcePath: string; @@ -77,25 +75,13 @@ const reportDigestValid = /^[0-9a-f]{64}$/.test(ledger.evidenceReport.canonicalSha256) && evidence.includes(ledger.evidenceReport.canonicalSha256); -const installedRouteIds = Object.values(ROUTE_REGISTRY) - .map((route) => route.routeId) - .sort(); -const routeScope = []; -for (const path of ROUTE_SCOPE_DOCUMENTS) { - let text: string; - try { - await access(path); - text = await readFile(path, "utf8"); - } catch { - routeScope.push({ path, missingRouteIds: [...installedRouteIds], documented: false }); - continue; - } - const missingRouteIds = installedRouteIds.filter( - (routeId) => !text.includes(routeId), - ); - routeScope.push({ path, missingRouteIds, documented: missingRouteIds.length === 0 }); -} -const routeScopeDocumented = routeScope.every((entry) => entry.documented); +const missingRouteIds = MANUAL_A11Y_ROUTE_IDS.filter( + (routeId) => !existsSync(`${MANUAL_A11Y_RECORDS}${routeId}.md`), +).sort(); +const routeScope = [ + { path: MANUAL_A11Y_RECORDS, missingRouteIds, documented: missingRouteIds.length === 0 }, +]; +const routeScopeDocumented = missingRouteIds.length === 0; // 13-15. README is where a project learns how to build for production. It // named none of the variables the build and admission read, so each one is @@ -164,16 +150,150 @@ const sourceStructureDocumented = sourceStructure.every( entry.missingRules.length === 0, ); +// 1101/14-05. docs/README.md §1 says the gates check the current contract +// documents, while only the checks above did. Every repo path, package script +// and FE-GATE ID that the index and the documents its §1 links name must +// exist. Documents are read from the working directory; what they name is +// looked up in this repository, as for the src structure above. +const DOCUMENT_INDEX = "docs/README.md"; +// Past states are its subject: it names removed scripts and paths on purpose. +const UNCHECKED_DOCUMENTS = new Set(["CHANGELOG.md"]); +const NOT_CREATED = new Set([ + // Reserved by generic-worker-has-no-network-or-credentials. The generic CPU + // worker is NOT_SELECTED (tsconfig.web-worker.json). + "src/adapters/web-worker", +]); +// pnpm's own commands, which are not package scripts. +const PNPM_COMMANDS = new Set(["add", "dlx", "exec", "install", "list", "why"]); + +const rootDirectory = fileURLToPath(repositoryRoot); +const exists = (file: string) => existsSync(join(rootDirectory, file)); +const topLevelEntries = new Set(await readdir(rootDirectory)); +const scripts = new Set( + Object.keys( + (JSON.parse(await readFile(join(rootDirectory, "package.json"), "utf8")) as { scripts: Record }) + .scripts, + ), +); +const scriptPrefixes = new Set([...scripts].map((name) => name.split(":")[0])); +const gateIds = new Set([ + ...( + JSON.parse(await readFile(join(rootDirectory, "config/ci/gates.json"), "utf8")) as { gates: { id: string }[] } + ).gates.map((gate) => gate.id), + ...RETIRED_GATE_IDS, +]); +// Generated output is what git ignores, so a document may name it before a +// run has written it. +const ignorePatterns = (await readFile(join(rootDirectory, ".gitignore"), "utf8").catch(() => "")) + .split("\n") + .map((line) => line.trim()) + .filter((line) => line !== "" && !line.startsWith("#")); +function generated(file: string): boolean { + let ignored = false; + for (const line of ignorePatterns) { + const pattern = line.replace(/^!/u, "").replace(/^\/|\/$/gu, ""); + const glob = pattern.includes("/") ? pattern : `**/${pattern}`; + if (matchesGlob(file, glob) || matchesGlob(file, `${glob}/**`)) ignored = !line.startsWith("!"); + } + return ignored; +} +// A copy may delete the history README lists under "Files you can delete +// after copying". A path under an entry that is gone is not drift. +const deletedHistory = [ + ...(readme.match(/^### Files you can delete after copying\n[\s\S]*?(?=^#)/mu)?.[0] ?? "").matchAll( + /^\| (.+?) \|/gmu, + ), +] + .flatMap(([, cell = ""]) => [...cell.matchAll(/`([^`]+)`/gu)].map(([, entry = ""]) => entry.replace(/\/$/u, ""))) + .filter((entry) => !exists(entry)); + +// A backtick span is a path when it starts at a top-level entry or at an +// entry beside the document; line numbers and anchors are dropped. +function namedPath(token: string, document: string): string | null { + const candidate = token.replace(/#.*$/u, "").replace(/:\d[\d,:-]*$/u, ""); + if (!candidate.includes("/") || candidate.startsWith("/") || /[\s*<>{}$()|]|\.\.\./u.test(candidate)) return null; + const first = candidate.split("/")[0] ?? ""; + if (topLevelEntries.has(first)) return candidate; + return first !== "." && first !== ".." && exists(join(dirname(document), first)) + ? join(dirname(document), candidate) + : null; +} +function linkedPath(target: string, document: string): string | null { + if (/^(?:[a-z][a-z+.-]*:|#)/iu.test(target)) return null; + const resolved = normalize(join(dirname(document), target.replace(/#.*$/u, ""))); + return resolved.startsWith("../") ? null : resolved; +} + +const index = await readFile(DOCUMENT_INDEX, "utf8").catch(() => ""); +const contractDocuments = new Set([DOCUMENT_INDEX]); +for (const [, target = ""] of (index.match(/^## 1\.[\s\S]*?(?=^## )/mu)?.[0] ?? "").matchAll(/\]\(([^)#\s]+)\)/gu)) { + const linked = normalize(join(dirname(DOCUMENT_INDEX), target)); + const entries = await readdir(linked).catch(() => null); + const documents = + entries === null ? [linked] : entries.filter((name) => name.endsWith(".md")).map((name) => join(linked, name)); + for (const document of documents) if (!UNCHECKED_DOCUMENTS.has(document)) contractDocuments.add(document); +} +const documentReferences = []; +for (const document of contractDocuments) { + const text = await readFile(document, "utf8").catch(() => null); + if (text === null) { + // A linked document that is gone is reported as the index's broken link. + if (document === DOCUMENT_INDEX) { + documentReferences.push({ path: document, missingPaths: [document], missingScripts: [], unknownGateIds: [] }); + } + continue; + } + const named = [ + ...[...text.matchAll(/`([^`\n]+)`/gu)].map(([, token = ""]) => namedPath(token, document)), + ...[...text.matchAll(/\]\(([^)\s]+)\)/gu)].map(([, target = ""]) => linkedPath(target, document)), + ]; + const missingPaths = new Set(); + for (const file of named) { + if (file === null) continue; + const trimmed = file.replace(/\/$/u, ""); + if ( + !exists(trimmed) && + !NOT_CREATED.has(trimmed) && + !generated(trimmed) && + !deletedHistory.some((entry) => trimmed === entry || trimmed.startsWith(`${entry}/`)) + ) { + missingPaths.add(trimmed); + } + } + const missingScripts = new Set(); + for (const [, name = ""] of text.matchAll(/(?:corepack pnpm|`pnpm) (?:run )?([a-z](?:[\w:-]*\w)?)/gu)) { + if (!PNPM_COMMANDS.has(name) && !scripts.has(name)) missingScripts.add(name); + } + for (const [, name = ""] of text.matchAll(/`([a-z][\w-]*:[\w:-]*\w)(?=[` ])/gu)) { + if (scriptPrefixes.has(name.split(":")[0]) && !scripts.has(name)) missingScripts.add(name); + } + documentReferences.push({ + path: document, + missingPaths: [...missingPaths].sort(), + missingScripts: [...missingScripts].sort(), + unknownGateIds: [...new Set([...text.matchAll(/\bFE-GATE-\d+\b/gu)].map(([id]) => id))] + .filter((id) => !gateIds.has(id)) + .sort(), + }); +} +const documentReferencesValid = documentReferences.every( + (entry) => entry.missingPaths.length === 0 && entry.missingScripts.length === 0 && entry.unknownGateIds.length === 0, +); + // 1101/14-06. Each failed check is named, so the stderr points at the file to -// fix rather than at the diagram review whatever failed. +// fix rather than at the diagram review whatever failed. 1101/14-05: the last +// two only compare the two hand-kept copies of the 2026-07-18 diagram review; +// the diagrams are outside this repository, so they pass whatever the +// structure becomes, and their names say so. const failedChecks = [ ...(routeScopeDocumented ? [] : ["route scope"]), ...(missingRuntimeVariables.length === 0 ? [] : ["runtime variables"]), ...(sourceStructureDocumented ? [] : ["source structure"]), - ...(reportDigestValid ? [] : ["review report digest"]), + ...(documentReferencesValid ? [] : ["document references"]), + ...(reportDigestValid ? [] : ["2026-07-18 diagram review copy digest"]), ...(results.length === 2 && results.every((result) => result.passed) ? [] - : ["diagram reviews"]), + : ["2026-07-18 diagram review copies"]), ]; const passed = failedChecks.length === 0; await mkdir("artifacts/quality", { recursive: true }); @@ -192,14 +312,14 @@ await writeValidatedJsonArtifact({ routeScopeDocumented, missingRuntimeVariables, sourceStructure, + documentReferences, results, passed, }, }); if (!passed) { - for (const entry of routeScope) { - if (entry.documented) continue; - process.stderr.write(`${entry.path} does not name route IDs: ${entry.missingRouteIds.join(", ")}\n`); + if (!routeScopeDocumented) { + process.stderr.write(`${MANUAL_A11Y_RECORDS} has no record for route IDs: ${missingRouteIds.join(", ")}\n`); } if (missingRuntimeVariables.length > 0) { process.stderr.write(`README.md does not name: ${missingRuntimeVariables.join(", ")}\n`); @@ -217,6 +337,19 @@ if (!passed) { ); } } + for (const entry of documentReferences) { + if (entry.missingPaths.length > 0) { + process.stderr.write(`${entry.path} names paths that do not exist: ${entry.missingPaths.join(", ")}\n`); + } + if (entry.missingScripts.length > 0) { + process.stderr.write(`${entry.path} names package scripts that do not exist: ${entry.missingScripts.join(", ")}\n`); + } + if (entry.unknownGateIds.length > 0) { + process.stderr.write( + `${entry.path} names gate IDs that config/ci/gates.json does not define: ${entry.unknownGateIds.join(", ")}\n`, + ); + } + } if (!reportDigestValid) { process.stderr.write( `${ledger.evidenceReport.repoPath} does not contain the ledger's canonicalSha256\n`, diff --git a/tests/unit/documentation-readiness.test.ts b/tests/unit/documentation-readiness.test.ts index 5eeb3a7d..f639b9aa 100644 --- a/tests/unit/documentation-readiness.test.ts +++ b/tests/unit/documentation-readiness.test.ts @@ -5,23 +5,25 @@ import path from "node:path"; import { describe, expect, it } from "vitest"; -const INPUTS = [ - "README.md", - "docs/accessibility/manual-checklist.md", - "docs/architecture/review-ledger.json", - "docs/architecture/review-evidence.md", - "docs/architecture/layers.md", - "docs/architecture/overview.md", - "docs/architecture/frontend-application-foundation.md", -]; +const INPUTS = ["README.md", "docs", "artifacts/tests/a11y-manual"]; -/** Runs verify:documentation on a copy of its inputs, `mutate` applied to `document`. */ -async function verifyWithReadme(mutate: (readme: string) => string, document = "README.md") { +/** + * Runs verify:documentation on a copy of its inputs, each `changes` entry + * applied to its document and every `removed` file deleted. Code facts (src, + * package.json, gates.json) are read from this repository, not the copy. + */ +async function verify( + changes: Readonly string>>, + removed: readonly string[] = [], +) { const root = await mkdtemp(path.join(tmpdir(), "documentation-readiness-")); try { for (const input of INPUTS) await cp(input, path.join(root, input), { recursive: true }); - const target = path.join(root, document); - await writeFile(target, mutate(await readFile(target, "utf8"))); + for (const [document, mutate] of Object.entries(changes)) { + const target = path.join(root, document); + await writeFile(target, mutate(await readFile(target, "utf8"))); + } + for (const file of removed) await rm(path.join(root, file)); return spawnSync(process.execPath, [path.resolve("scripts/verify-documentation-readiness.ts")], { cwd: root, encoding: "utf8", @@ -31,32 +33,101 @@ async function verifyWithReadme(mutate: (readme: string) => string, document = " } } +function verifyWithReadme(mutate: (readme: string) => string, document = "README.md") { + return verify({ [document]: mutate }); +} + // 1101/14-06. A route missing from a scope document used to fail with only // "canonical scoped-review evidence is incomplete", which points at the -// diagram review instead of the document to fix. +// diagram review instead of the file to fix. describe("verify:documentation failure output (1101/14-06)", () => { it("passes the documents as they are", async () => { const result = await verifyWithReadme((readme) => readme); expect(result.status, result.stderr).toBe(0); }); - it("names the document and the route ID it is missing", async () => { - const result = await verifyWithReadme((readme) => readme.replaceAll("`EXAMPLES_PLATFORM`", "")); + it("names the route ID that has no manual review record", async () => { + const result = await verify({}, ["artifacts/tests/a11y-manual/EXAMPLES_PLATFORM.md"]); expect(result.status).toBe(1); - expect(result.stderr).toContain("README.md does not name route IDs: EXAMPLES_PLATFORM"); + expect(result.stderr).toContain("artifacts/tests/a11y-manual/ has no record for route IDs: EXAMPLES_PLATFORM"); expect(result.stderr).toMatch(/FAIL_UNVERIFIED \(failed: route scope\)/u); expect(result.stderr).not.toContain("scoped-review evidence"); }); it("names every failed check", async () => { - const result = await verifyWithReadme((readme) => - readme.replaceAll("`EXAMPLES_PLATFORM`", "").replaceAll("`RELEASE_TARGET`", ""), + const result = await verify( + { "README.md": (readme) => readme.replaceAll("`RELEASE_TARGET`", "") }, + ["artifacts/tests/a11y-manual/EXAMPLES_PLATFORM.md"], ); expect(result.stderr).toContain("README.md does not name: RELEASE_TARGET"); expect(result.stderr).toMatch(/\(failed: route scope, runtime variables\)/u); }); }); +// 1101/14-05. README and the manual checklist listed every route ID, so each +// new feature edited two template-owned documents. The scope is the record +// files now, and the current contract documents (docs/README.md §1) may only +// name paths, package scripts and gate IDs that exist. +describe("verify:documentation current contract documents (1101/14-05)", () => { + const ROUTE_IDS = /`(?:APP_HOME|EXAMPLES_[A-Z]+|NOT_FOUND|REFERENCE_RESOURCE_[A-Z]+)`/gu; + + it("does not need README or the checklist to list route IDs", async () => { + const result = await verify({ + "README.md": (readme) => readme.replaceAll(ROUTE_IDS, ""), + "docs/accessibility/manual-checklist.md": (checklist) => checklist.replaceAll(ROUTE_IDS, ""), + }); + expect(result.status, result.stderr).toBe(0); + }); + + it.each([ + ["README.md", "scripts/verify-documentation-readiness.ts", "scripts/verify-documentation-readyness.ts"], + ["docs/contracts/compatibility.md", "scripts/check-compatibility.ts", "scripts/check-compatibilty.ts"], + // reached through the `operations/` directory link + ["docs/operations/release-cache-rollback.md", "scripts/lib/hosting-probe.ts", "scripts/lib/hosting-prob.ts"], + ])("fails when %s names a path that does not exist", async (document, before, after) => { + const result = await verifyWithReadme((text) => text.replace(`\`${before}\``, `\`${after}\``), document); + expect(result.status).toBe(1); + expect(result.stderr).toContain(`${document} names paths that do not exist: ${after}`); + expect(result.stderr).toMatch(/\(failed: document references\)/u); + }); + + it("fails when docs/README.md links a document that does not exist", async () => { + const result = await verifyWithReadme( + (index) => index.replace("(architecture/layers.md)", "(architecture/layer.md)"), + "docs/README.md", + ); + expect(result.status).toBe(1); + expect(result.stderr).toContain("docs/README.md names paths that do not exist: docs/architecture/layer.md"); + }); + + it("fails when a document names a package script or a gate ID that does not exist", async () => { + const result = await verify({ + "README.md": (readme) => readme.replace("`verify:documentation` fails", "`verify:documentaton` fails"), + "docs/README.md": (index) => index.replace("(FE-GATE-017)", "(FE-GATE-099)"), + }); + expect(result.status).toBe(1); + expect(result.stderr).toContain("README.md names package scripts that do not exist: verify:documentaton"); + expect(result.stderr).toContain("docs/README.md names gate IDs that config/ci/gates.json does not define: FE-GATE-099"); + }); + + it("does not ask generated output to exist", async () => { + const result = await verifyWithReadme((readme) => `${readme}\nSee \`artifacts/quality/not-generated-yet.json\`.\n`); + expect(result.status, result.stderr).toBe(0); + }); + + it("accepts a path under history a copy deleted, and only that", async () => { + const naming = (readme: string) => `${readme}\nSee \`docs/retired-notes/plan.md\`.\n`; + const deleted = await verifyWithReadme((readme) => + naming(readme).replace("| Path | Contents |\n| --- | --- |\n", "$&| `docs/retired-notes/` | deleted |\n"), + ); + expect(deleted.status, deleted.stderr).toBe(0); + + const missing = await verifyWithReadme(naming); + expect(missing.status).toBe(1); + expect(missing.stderr).toContain("README.md names paths that do not exist: docs/retired-notes/plan.md"); + }); +}); + // 1101/14-03. The entry documents drew a top-level domain layer that does not // exist, left out three of the nine src folders, and layers.md named a // minority of the architecture rules while claiming it could not drift.