diff --git a/docs/operations/ci-quality-gates.md b/docs/operations/ci-quality-gates.md index ae54a31d..42988613 100644 --- a/docs/operations/ci-quality-gates.md +++ b/docs/operations/ci-quality-gates.md @@ -221,6 +221,15 @@ namespace 점검을 돌아, 이 조건이 없는 runner에서 이름과 host 설 stderr, 위 sysctl 값, 고치는 방법을 적고 exit 1로 멈춘다. 전에는 이 검사가 PASS를 낸 뒤 provider 테스트가 bwrap 원문 오류로 실패했다(2026-11-05 desktop에서 114건 중 18건). +- 같은 검사가 provider의 cgroup 준비도 provider와 같은 호출로 해 본다. + `/usr/bin/prlimit`·`/usr/bin/systemd-run`이 실행 가능한지 보고, + `systemctl --user show --property=Version --value`로 user manager와 버전(254 + 이상)을 확인한 뒤, provider scope와 같은 속성·같은 cgroup 강제 확인으로 + `systemd-run --user --scope`에서 `/bin/true`를 한 번 띄운다 + (`scripts/lib/provider-cgroup.ts`). user bus가 없거나, 버전이 낮거나, memory·pids·cpu + controller가 위임되지 않은 호스트는 그 원인을 이름으로 대며 exit 1이다. 전에는 + system manager bus만 보고 PASS를 낸 뒤 provider 테스트가 하나씩 + `provider cgroup unavailable: ...`로 실패했다. - 고치는 방법은 `scripts/check-system-test-prerequisites.ts`가 실패할 때 출력한다. 호스트의 보안 설정을 바꾸는 일이므로 호스트 소유자가 고른다. 호스트를 바꾸는 설정 파일과 명령은 runner를 만드는 쪽의 인프라 repo가 갖는다. @@ -443,7 +452,18 @@ provider 확인이 필요하다고 표시한 것은 모두 `UNVERIFIED`다. `FAI drill은 `FAIL`(exit 1)이다. 그렇지 않고 `UNVERIFIED`가 하나라도 있으면 `UNVERIFIED`(exit 3)이고, 게이트는 `PASS`가 아니다. 그래서 provider 증거가 없으면 production 단계는 닫히지 않는다. 의도한 결과다. 로컬 drill은 승격한 release를 rollback할 -수 있다는 것을 보여 줄 수 없다. FE-GATE-016(`rollback-drill`)은 FE-GATE-025와 같은 +수 있다는 것을 보여 줄 수 없다. + +provider 증거는 `--provider-evidence <파일>`로 넘긴다(0929/11-02). 파일은 +`providerDrillEvidenceSchema`(`scripts/contracts/release-artifacts.ts`) 꼴이다: runbook ID, +release ID, 관찰 시각, provider 쪽 run ID, 그리고 항목마다 `PASS`/`FAIL`과 근거다. 항목 +이름은 `UNVERIFIED` 줄이 적는 이름(`trigger`, recovery assertion 이름, `provider +verification` 등)이다. drill은 runbook과 release가 같고, 관찰이 7일 안이고, 로컬 drill이 +보지 못한 항목만 답한 증거만 받는다. 받은 증거는 그 항목의 `UNVERIFIED`만 바꾸고, 로컬에서 +본 `PASS`·`FAIL`은 그대로 둔다. 모든 항목이 답을 얻고 `FAIL`이 없을 때만 drill이 `PASS`다. +받지 않은 증거는 이유와 함께 record의 `providerEvidence`에 남고 drill은 `UNVERIFIED`다. +템플릿의 production 게이트 명령은 아직 이 인자를 넘기지 않는다. provider를 연결한 제품이 +증거 파일을 만들고 drill 명령에 넘겨야 그 단계가 닫힌다. FE-GATE-016(`rollback-drill`)은 FE-GATE-025와 같은 FE-RB-005 drill을 돌렸으므로 폐기했다(`scripts/contracts/ci-gates.ts`의 `RETIRED_GATE_IDS`). @@ -482,6 +502,14 @@ gate의 realtime source/fixture 검사는 native SSE/WebSocket/Web Push API가 bootstrap/installed-feature 조합을 차단한다. optional recipe source gate는 uncomposed runtime 전체를 tree-shaking 없이 합성해 catalog의 gzip 예산도 blocking으로 검증한다. +같은 gate의 `check:request-fanout`은 `src/features`·`src/application`에서 루프나 배열 +순회 callback 안의 요청(`fetch()`, 이름이 transport인 receiver의 method)을 거절한다. +실패 줄은 두 탈출 방법을 함께 적는다: 기능 `adapters/`에서 +`createCursorPaginationRuntime`(`src/adapters/query-cache`)으로 페이지를 받거나, 상한이 +있고 검토한 반복이면 그 호출 줄에 `// fanout-reviewed: <이유>`를 적는다. 결과 줄은 +주석으로 넘긴 호출의 수와 위치를 적는다. 이름이 다른 receiver(`repository.load()`), +루프가 부르는 이름 있는 함수 안의 요청, 재귀, 순회 전에 만든 promise는 보지 못한다 +(`scripts/check-request-fanout.ts` 머리 주석). `FE-GATE-030`(main 단계)은 reference runtime source·전용 test·catalog metadata를 제거한 임시 repository에서도 base typecheck, architecture, test, build가 통과하는지 검증해 skeleton의 선택성을 유지한다. browser file/storage와 realtime runtime은 diff --git a/docs/operations/release-cache-rollback.md b/docs/operations/release-cache-rollback.md index f9377e17..79906c02 100644 --- a/docs/operations/release-cache-rollback.md +++ b/docs/operations/release-cache-rollback.md @@ -130,9 +130,12 @@ probe하지 않고 `UNVERIFIED`로 끝난다. 슬롯을 채우는 방법은 **주소와 연결** -1. 앱은 HTTPS origin의 루트(`/`)에서 내보낸다. 인증서가 유효하고, 어느 surface도 그 - origin 밖으로 redirect하지 않는다. — FE-GATE-019가 잰다. `HOSTING_BASE_URL`은 경로 없는 - HTTPS 루트 URL이어야 하고, surface마다 최종 origin을 본다. +1. 앱은 HTTPS origin의 루트(`/`)나 빌드한 하위 경로(`VITE_ROUTER_BASE_PATH`, 예 `/app/`)에서 + 내보낸다. 인증서가 유효하고, 어느 surface도 그 origin 밖으로 redirect하지 않는다. — + FE-GATE-019가 잰다. `HOSTING_BASE_URL`은 release의 기준 URL(루트 또는 `/`로 끝나는 그 + 하위 경로, query·fragment 없음)이다. probe는 surface 경로(`config/hosting/cache-policy.json`)를 + 그 아래에서 요청하고, 후보의 진입 script가 그 경로 아래에 있는지와 surface마다 최종 + origin을 본다. 2. TLS에서 HTTP/2를 맺는다(ALPN `h2`). — FE-GATE-019가 잰다(`transport.alpnProtocols`). **surface별 응답** (surface: probe가 요청하는 경로 하나) diff --git a/eslint.config.ts b/eslint.config.ts index fbdcd82a..f0079032 100644 --- a/eslint.config.ts +++ b/eslint.config.ts @@ -937,6 +937,48 @@ export default [ "react-hooks/rules-of-hooks": "off", }, }, + { + // 1112/03-04. A browser spec takes `test` from the strict fixture, which + // fails it on console errors, page errors, failed requests and unused + // preloads. FE-GATE-008 used to look for the fixture's path as text in + // tests/e2e only, so a path in a comment passed and the other browser + // folders were not looked at. tests/unit/browser-spec-imports.test.ts + // lints a fixture spec in each folder against this entry. + files: [`tests/{e2e,browser-capabilities,visual,storybook}/**/*.spec.ts`], + rules: { + "no-restricted-imports": [ + "error", + { + paths: [ + { + name: "@playwright/test", + importNames: ["test", "default"], + message: + "import test from tests/support/browser/strict-browser-test.ts; the strict fixture fails a spec on browser errors (.claude/skills/fe-qa).", + }, + ], + patterns: [ + ...deprecatedAliasImports, + { + group: ["**/support/browser/browser-test.ts", "**/support/browser/browser-test"], + importNames: ["test"], + message: + "import test from tests/support/browser/strict-browser-test.ts; the base test does not check browser errors.", + }, + ], + }, + ], + }, + }, + { + // 1112/03-04. The two specs that need the plain test on purpose: + // preload-use proves the strict fixture's own record, and chunk-load-error + // provokes the 404 a missing chunk gives to classify the engine's wording. + files: ["tests/e2e/preload-use.spec.ts", "tests/browser-capabilities/chunk-load-error.spec.ts"], + rules: { + "no-restricted-imports": ["error", { patterns: deprecatedAliasImports }], + }, + }, { files: [ `tests/fixtures/browser-file-storage-boundaries/forbidden/**/*.${sourceExtensions}`, diff --git a/scripts/check-app-build-input.ts b/scripts/check-app-build-input.ts index 9dc7d023..9193106a 100644 --- a/scripts/check-app-build-input.ts +++ b/scripts/check-app-build-input.ts @@ -5,6 +5,7 @@ import { createFileSystemDeclarationSource, type TopologyDeclarationSource, } from "./generate-product-feature-index.ts"; +import { checkoutBuildReadFailure } from "./lib/run-isolation.ts"; /** * scale-architecture.md §3 / §37. Build-input regression gate for deployable-app isolation. @@ -123,6 +124,13 @@ async function main(): Promise { // without `--require-bundle` the gate checks the generated indexes only, so a // stale inventory from another build can never pass or fail it. const requireBundle = process.argv.includes("--require-bundle"); + // 1106/12-02. The inventory is read where the checkout keeps it; under + // RUN_OUTPUT_ROOT the build wrote elsewhere and this would judge an earlier one. + const outputRootFailure = requireBundle ? checkoutBuildReadFailure("scripts/check-app-build-input.ts") : undefined; + if (outputRootFailure !== undefined) { + process.stderr.write(`${outputRootFailure}\n`); + process.exit(1); + } const report = await inspectAppBuildInput(process.cwd(), requireBundle ? {} : { bundleInventory: null }); if (requireBundle && !report.bundle.checked) { process.stderr.write("app build input: no production module inventory; run the build first\n"); diff --git a/scripts/check-optional-recipes.ts b/scripts/check-optional-recipes.ts index 8a7206f4..8c3f556d 100644 --- a/scripts/check-optional-recipes.ts +++ b/scripts/check-optional-recipes.ts @@ -11,6 +11,7 @@ import { validateRecipeCatalog, } from "./lib/optional-recipes.ts"; import { assertMatchesJsonSchema } from "./lib/json-schema.ts"; +import { checkoutBuildReadFailure } from "./lib/run-isolation.ts"; import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts"; type GateViolation = Readonly<{ @@ -36,6 +37,15 @@ const catalogPath = argument( ); const sourceRoot = argument("--source-root", "src"); const distRoot = argument("--dist-root", "dist"); +// 1106/12-02. Without --dist-root this scans the checkout's dist; under +// RUN_OUTPUT_ROOT the build wrote elsewhere, so it would judge an earlier build. +const outputRootFailure = process.argv.includes("--dist-root") + ? undefined + : checkoutBuildReadFailure("scripts/check-optional-recipes.ts"); +if (outputRootFailure !== undefined) { + process.stderr.write(`${outputRootFailure}\n`); + process.exit(1); +} const artifactPath = argument( "--artifact", "artifacts/quality/optional-recipes.json", diff --git a/scripts/check-request-fanout.ts b/scripts/check-request-fanout.ts index 3fcbe287..1a994599 100644 --- a/scripts/check-request-fanout.ts +++ b/scripts/check-request-fanout.ts @@ -17,6 +17,16 @@ import { parseAsync } from "@babel/core"; * The HTTP runtime itself (src/adapters/http) owns its bounded retry loop and is * out of scope. A reviewed exception is written on the call's line as * `// fanout-reviewed: `. + * + * 0929/12-01. A failure names both ways out: fetch page after page through + * `createCursorPaginationRuntime` (src/adapters/query-cache) assembled in the + * feature's `adapters/`, or mark a bounded, reviewed repetition on the call's + * line. Every reviewed exception is counted and listed in the result line. + * + * What it does not see: a dispatch through a receiver named otherwise + * (`repository.load()`, a destructured `get`), a dispatch in a named function + * that a loop calls, recursion, and promises built before an iteration and + * awaited inside it. Reviews still look for those. */ export const REQUEST_FANOUT_ROOTS = Object.freeze(["src/features", "src/application"]); @@ -44,6 +54,14 @@ const FUNCTION_TYPES = new Set(["ArrowFunctionExpression", "FunctionExpression"] type Node = { type: string; loc?: { start: { line: number } } } & Record; export type FanoutViolation = Readonly<{ file: string; line: number; call: string; context: string }>; +export type FanoutReviewedException = Readonly<{ file: string; line: number; call: string; reason: string }>; + +const REVIEWED = /fanout-reviewed:\s*(\S.*?)\s*$/u; + +/** How to fix a failure; both ways the source gate accepts. */ +export const FANOUT_REMEDY = + "load one page per request with createCursorPaginationRuntime (src/adapters/query-cache) assembled in the " + + "feature's adapters/, or, for a bounded repetition someone reviewed, write `// fanout-reviewed: ` on the call's line"; function receiverName(node: Node | undefined): string | null { if (!node) return null; @@ -81,6 +99,14 @@ export async function findRequestFanout( file: string, sourceText: string, ): Promise { + return (await inspectRequestFanout(file, sourceText)).violations; +} + +/** Per-row dispatches, and the ones a `fanout-reviewed:` comment let through. */ +export async function inspectRequestFanout( + file: string, + sourceText: string, +): Promise> { const plugins: ("jsx" | ["typescript", { dts: boolean }])[] = [["typescript", { dts: false }]]; if (file.endsWith(".tsx")) plugins.push("jsx"); const tree = (await parseAsync(sourceText, { @@ -93,6 +119,7 @@ export async function findRequestFanout( if (!tree) throw new Error(`cannot parse ${file}`); const lines = sourceText.split("\n"); const violations: FanoutViolation[] = []; + const reviewed: FanoutReviewedException[] = []; const visit = (node: Node, parent: Node | undefined, repeating: string | null): void => { let context = repeating; @@ -106,9 +133,9 @@ export async function findRequestFanout( if (node.type === "CallExpression" && context) { const call = dispatchDescription(node); const line = node.loc?.start.line ?? 0; - if (call && !/fanout-reviewed:\s*\S/u.test(lines[line - 1] ?? "")) { - violations.push({ file, line, call, context }); - } + const reason = REVIEWED.exec(lines[line - 1] ?? "")?.[1]; + if (call && reason !== undefined) reviewed.push({ file, line, call, reason }); + else if (call) violations.push({ file, line, call, context }); } for (const [key, value] of Object.entries(node)) { if (key === "loc" || key === "start" || key === "end" || key === "extra") continue; @@ -121,7 +148,7 @@ export async function findRequestFanout( } }; visit(tree, undefined, null); - return violations; + return { violations, reviewed }; } async function sourceFiles(directory: string): Promise { @@ -143,39 +170,62 @@ async function sourceFiles(directory: string): Promise { export async function scanRequestFanout( root: string, roots: readonly string[] = REQUEST_FANOUT_ROOTS, -): Promise> { +): Promise> { const files = (await Promise.all(roots.map((entry) => sourceFiles(path.join(root, entry))))).flat(); const violations: FanoutViolation[] = []; + const reviewed: FanoutReviewedException[] = []; for (const file of files) { - violations.push( - ...(await findRequestFanout(path.relative(root, file), await readFile(file, "utf8"))), - ); + const found = await inspectRequestFanout(path.relative(root, file), await readFile(file, "utf8")); + violations.push(...found.violations); + reviewed.push(...found.reviewed); } - return { files: files.length, violations }; + return { files: files.length, violations, reviewed }; +} + +export function formatFanoutViolation(violation: FanoutViolation): string { + return `FAIL ${violation.file}:${violation.line} ${violation.call} inside ${violation.context}: ${FANOUT_REMEDY}`; +} + +function formatReviewed(reviewed: readonly FanoutReviewedException[]): string { + if (reviewed.length === 0) return "0 reviewed exceptions"; + return `${reviewed.length} reviewed exception${reviewed.length === 1 ? "" : "s"} (` + + `${reviewed.map((entry) => `${entry.file}:${entry.line} ${entry.reason}`).join("; ")})`; +} + +/** + * The forbidden fixture must be rejected; the allowed fixture must pass, with + * its reviewed cursor loop counted as a reviewed exception. + */ +export async function checkRequestFanoutFixtures(root: string): Promise> { + const forbidden = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/forbidden"]); + const allowed = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/allowed"]); + const ok = forbidden.violations.length >= forbidden.files && forbidden.files > 0 && + allowed.violations.length === 0 && allowed.reviewed.length > 0; + return { + ok, + line: `request fan-out fixtures: forbidden ${forbidden.violations.length} rejected in ${forbidden.files} files, ` + + `allowed ${allowed.violations.length} violations and ${formatReviewed(allowed.reviewed)} ${ok ? "PASS" : "FAIL"}`, + }; } async function main(): Promise { const fixture = process.argv.includes("--fixture"); const root = process.cwd(); if (fixture) { - // The forbidden fixture must be rejected; the allowed fixture must pass. - const forbidden = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/forbidden"]); - const allowed = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/allowed"]); - const ok = forbidden.violations.length >= forbidden.files && forbidden.files > 0 && allowed.violations.length === 0; - process.stdout.write( - `request fan-out fixtures: forbidden ${forbidden.violations.length} rejected in ${forbidden.files} files, allowed ${allowed.violations.length} violations ${ok ? "PASS" : "FAIL"}\n`, - ); - if (!ok) process.exitCode = 1; + const result = await checkRequestFanoutFixtures(root); + process.stdout.write(`${result.line}\n`); + if (!result.ok) process.exitCode = 1; return; } const result = await scanRequestFanout(root); - for (const violation of result.violations) { - process.stderr.write( - `FAIL ${violation.file}:${violation.line} ${violation.call} inside ${violation.context}\n`, - ); - } + for (const violation of result.violations) process.stderr.write(`${formatFanoutViolation(violation)}\n`); process.stdout.write( - `request fan-out: ${result.files} files scanned, ${result.violations.length} per-row dispatches ${result.violations.length === 0 ? "PASS" : "FAIL"}\n`, + `request fan-out: ${result.files} files scanned, ${result.violations.length} per-row dispatches, ` + + `${formatReviewed(result.reviewed)} ${result.violations.length === 0 ? "PASS" : "FAIL"}\n`, ); if (result.violations.length > 0) process.exitCode = 1; } diff --git a/scripts/check-system-test-prerequisites.ts b/scripts/check-system-test-prerequisites.ts index f3aa5ec9..1619ea71 100644 --- a/scripts/check-system-test-prerequisites.ts +++ b/scripts/check-system-test-prerequisites.ts @@ -1,9 +1,16 @@ +import { spawnSync } from "node:child_process"; +import { randomBytes } from "node:crypto"; import { constants } from "node:fs"; import { access, readFile } from "node:fs/promises"; -import { spawnSync } from "node:child_process"; import path from "node:path"; import { fileURLToPath } from "node:url"; +import { + formatProviderPrerequisiteUnitName, + MINIMUM_PROVIDER_SYSTEMD_VERSION, + systemdRunPrerequisiteProbeArguments, + USER_MANAGER_VERSION_ARGUMENTS, +} from "./lib/provider-cgroup.ts"; import { PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS } from "./lib/provider-sandbox-namespaces.ts"; /** What the check reads from the host; a test passes a fake. */ @@ -15,7 +22,9 @@ export type SystemTestHost = Readonly<{ command: string, args: readonly string[], timeoutMs: number, - ): Readonly<{ status: number | null; stderr: string; error?: Error | undefined }>; + ): Readonly<{ status: number | null; stdout?: string; stderr: string; error?: Error | undefined }>; + /** Names the provider-shaped scope the probe starts; a test passes a fixed one. */ + probeUnitName?: () => string; }>; export const nodeSystemTestHost: SystemTestHost = Object.freeze({ @@ -23,12 +32,18 @@ export const nodeSystemTestHost: SystemTestHost = Object.freeze({ readFile: async (target: string) => readFile(target, "utf8"), spawn: (command: string, args: readonly string[], timeoutMs: number) => { const result = spawnSync(command, [...args], { encoding: "utf8", timeout: timeoutMs }); - return { status: result.status, stderr: result.stderr ?? "", error: result.error }; + return { status: result.status, stdout: result.stdout ?? "", stderr: result.stderr ?? "", error: result.error }; }, + probeUnitName: defaultProbeUnitName, }); +function defaultProbeUnitName(): string { + return formatProviderPrerequisiteUnitName(process.pid, randomBytes(12).toString("hex")); +} + const BWRAP = "/usr/bin/bwrap"; const SYSTEMCTL = "/usr/bin/systemctl"; +const SYSTEMD_RUN = "/usr/bin/systemd-run"; /** * 1105/12-01. The provider sandbox's own namespace options on /bin/true. The @@ -50,6 +65,7 @@ const USER_NAMESPACE_SETTINGS = Object.freeze([ ] as const); const PROBE_TIMEOUT_MS = 5_000; +const SCOPE_PROBE_TIMEOUT_MS = 15_000; const MAX_REPORTED_STDERR_LINES = 4; export async function findSystemTestPrerequisiteFailures( @@ -59,6 +75,8 @@ export async function findSystemTestPrerequisiteFailures( const executable = new Map(); for (const [target, label] of [ [BWRAP, "bubblewrap"], + ["/usr/bin/prlimit", "prlimit"], + [SYSTEMD_RUN, "systemd-run"], [SYSTEMCTL, "systemctl"], ["/usr/bin/tar", "tar"], ] as const) { @@ -80,10 +98,18 @@ export async function findSystemTestPrerequisiteFailures( failures.push("cgroup v2 is required at /sys/fs/cgroup/cgroup.controllers"); } + // 1112/08-03. The provider asks the user manager (not the system manager) + // for its version and then starts its sandbox in a `systemd-run --user + // --scope` (scripts/run-and-validate-provider.ts). The check makes the same + // calls, so a host without a user bus, with an old manager, or without + // delegated controllers fails here by name instead of in each provider test. if (executable.get(SYSTEMCTL)) { - const probe = host.spawn(SYSTEMCTL, ["show-environment"], 3_000); - if (probe.error || probe.status !== 0) { - failures.push("a reachable systemd manager bus is required"); + const userManager = findUserManagerFailure(host); + if (userManager !== undefined) { + failures.push(userManager); + } else if (executable.get(SYSTEMD_RUN)) { + const scope = findProviderScopeFailure(host); + if (scope !== undefined) failures.push(scope); } } @@ -94,6 +120,52 @@ export async function findSystemTestPrerequisiteFailures( return failures; } +function findUserManagerFailure(host: SystemTestHost): string | undefined { + const probe = host.spawn(SYSTEMCTL, USER_MANAGER_VERSION_ARGUMENTS, 3_000); + const command = `${SYSTEMCTL} ${USER_MANAGER_VERSION_ARGUMENTS.join(" ")}`; + if (probe.error || probe.status !== 0) { + return [ + `a reachable systemd user manager is required (${command} ${outcomeOf(probe)}).`, + ...stderrLines(probe.stderr, "systemctl"), + " The provider runs in a `systemd-run --user --scope`; give the runner user a user", + " manager (for example `loginctl enable-linger `) and a user bus in XDG_RUNTIME_DIR.", + ].join("\n"); + } + const reported = (probe.stdout ?? "").trim(); + const match = /^([0-9]+)/u.exec(reported); + const version = match ? Number(match[1]) : Number.NaN; + if (!Number.isSafeInteger(version) || version < MINIMUM_PROVIDER_SYSTEMD_VERSION) { + return `systemd user manager ${MINIMUM_PROVIDER_SYSTEMD_VERSION} or newer is required (${command} reported ${JSON.stringify(reported)})`; + } + return undefined; +} + +function findProviderScopeFailure(host: SystemTestHost): string | undefined { + const args = systemdRunPrerequisiteProbeArguments((host.probeUnitName ?? defaultProbeUnitName)()); + const probe = host.spawn(SYSTEMD_RUN, args, SCOPE_PROBE_TIMEOUT_MS); + if (!probe.error && probe.status === 0) return undefined; + return [ + "systemd-run could not start the provider's cgroup scope around /bin/true " + + `(${SYSTEMD_RUN} --user --scope ... ${outcomeOf(probe)}).`, + ...stderrLines(probe.stderr, "systemd-run"), + " The scope needs the memory, pids and cpu controllers delegated to the user manager", + " (Delegate= in user@.service) and cgroup v2. Runner requirements: docs/operations/ci-quality-gates.md.", + ].join("\n"); +} + +function outcomeOf(probe: ReturnType): string { + return probe.error ? `did not finish: ${probe.error.message}` : `exited ${String(probe.status)}`; +} + +function stderrLines(stderr: string, label: string): string[] { + const lines = stderr + .split("\n") + .map((line) => line.trim()) + .filter((line) => line.length > 0) + .slice(0, MAX_REPORTED_STDERR_LINES); + return lines.length > 0 ? lines.map((line) => ` ${label} stderr: ${line}`) : [` ${label} stderr: (empty)`]; +} + /** * 1101/13-03. Whether this host lets the provider sandbox create its * namespaces, and if not, why and what to do. The release provider jobs run @@ -118,19 +190,11 @@ async function namespaceFailure( // Absent on this kernel; the others still say what the host allows. } } - const stderr = probe.stderr - .split("\n") - .map((line) => line.trim()) - .filter((line) => line.length > 0) - .slice(0, MAX_REPORTED_STDERR_LINES); - const outcome = probe.error - ? `did not finish: ${probe.error.message}` - : `exited ${String(probe.status)}`; return [ "this host does not allow unprivileged user and network namespaces, which the provider " + `sandbox needs (bwrap ${PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS.join(" ")}).`, - ` probe: ${BWRAP} ${NAMESPACE_PROBE_ARGUMENTS.join(" ")} ${outcome}`, - ...(stderr.length > 0 ? stderr.map((line) => ` bwrap stderr: ${line}`) : [" bwrap stderr: (empty)"]), + ` probe: ${BWRAP} ${NAMESPACE_PROBE_ARGUMENTS.join(" ")} ${outcomeOf(probe)}`, + ...stderrLines(probe.stderr, "bwrap"), ...(settings.length > 0 ? [` host settings: ${settings.join(", ")}`] : []), " To fix, do one of these:", " - give /usr/bin/bwrap an AppArmor profile that allows `userns` (Ubuntu 23.10 and later),", diff --git a/scripts/check-test-evidence.ts b/scripts/check-test-evidence.ts index 6b16b483..4c9002b2 100644 --- a/scripts/check-test-evidence.ts +++ b/scripts/check-test-evidence.ts @@ -64,6 +64,12 @@ const policyPath = argumentValue( "config/testing/test-evidence.json", ); const explicitCatalogPath = optionalArgumentValue("--catalog"); +// 1112/03-03. The forbidOnly rule read only the repository's configs and ran +// only in the full check, so no fixture could show it still fires. +// `--playwright-config ` (repeatable) checks those files in any mode. +const explicitPlaywrightConfigs = process.argv.flatMap((argument, index) => + argument === "--playwright-config" && process.argv[index + 1] ? [process.argv[index + 1]!] : [], +); const explicitReceiptPath = optionalArgumentValue("--receipt"); const fixtureMode = sourceRoot !== "tests"; const sourceOnly = process.argv.includes("--source-only"); @@ -78,6 +84,7 @@ const facts = { sharedScenarios: 0, declaredScenarioExecutions: 0, executedScenarioExecutions: 0, + playwrightConfigs: [] as string[], }; async function filesBelow(target: string): Promise { @@ -365,6 +372,26 @@ if (parsedPolicy) { } } +async function missingForbidOnly(configs: readonly string[]): Promise { + const missing: string[] = []; + for (const config of configs) { + if (!(await readFile(config, "utf8")).includes("forbidOnly: true")) { + missing.push(`${config}: missing forbidOnly: true, so a committed test.only passes the gate`); + } + } + return missing; +} + +// The repository's configs unless the run names others; every mode but the +// scenario check reads them, and the report lists what it read. +if (!scenarioOnly) { + // playwright.dev.config.ts spreads playwright.config.ts. + facts.playwrightConfigs = explicitPlaywrightConfigs.length > 0 + ? explicitPlaywrightConfigs + : [...selectRepositoryPaths("playwright-configs")]; + failures.push(...(await missingForbidOnly(facts.playwrightConfigs))); +} + if (!fixtureMode && !scenarioOnly) { const e2eConfig = await readFile("playwright.config.ts", "utf8"); for (const token of [ @@ -381,22 +408,9 @@ if (!fixtureMode && !scenarioOnly) { failures.push(`playwright.config.ts missing release evidence token ${token}`); } } - // playwright.dev.config.ts spreads playwright.config.ts. - for (const config of selectRepositoryPaths("playwright-configs")) { - if (!(await readFile(config, "utf8")).includes("forbidOnly: true")) { - failures.push(`${config}: missing forbidOnly: true, so a committed test.only passes the gate`); - } - } - - const e2eFiles = (await filesBelow("tests/e2e")).filter((file) => - /\.spec\.ts$/.test(file), - ); - for (const file of e2eFiles) { - const source = await readFile(file, "utf8"); - if (!source.includes("support/browser/strict-browser-test")) { - failures.push(`${file}: bypasses strict browser fixture`); - } - } + // 1112/03-04. That a browser spec takes the strict fixture's `test` is an + // ESLint rule now (eslint.config.ts, tests/unit/browser-spec-imports.test.ts), + // which reads the import instead of looking for the path as text. const baselineFiles = (await filesBelow("tests/visual/__snapshots__")).filter( (file) => file.endsWith(".png"), diff --git a/scripts/contracts/release-artifacts.ts b/scripts/contracts/release-artifacts.ts index 13d30bdd..2b98f9df 100644 --- a/scripts/contracts/release-artifacts.ts +++ b/scripts/contracts/release-artifacts.ts @@ -746,16 +746,52 @@ const drillVerdict = z.enum(["PASS", "FAIL", "UNVERIFIED"]); export type DrillVerdict = z.infer; -/** FAIL beats UNVERIFIED beats PASS; provider scope alone keeps a drill UNVERIFIED. */ +/** + * FAIL beats UNVERIFIED beats PASS. A drill with provider scope stays + * UNVERIFIED until accepted provider evidence says the provider side passed + * (0929/11-02). + */ export function combineDrillVerdicts( verdicts: readonly DrillVerdict[], providerVerificationRequired: boolean, + providerVerification: DrillVerdict = "UNVERIFIED", ): DrillVerdict { - if (verdicts.includes("FAIL")) return "FAIL"; - if (verdicts.includes("UNVERIFIED") || providerVerificationRequired) return "UNVERIFIED"; + const all = providerVerificationRequired ? [...verdicts, providerVerification] : verdicts; + if (all.includes("FAIL")) return "FAIL"; + if (all.includes("UNVERIFIED")) return "UNVERIFIED"; return "PASS"; } +/** + * 0929/11-02. What the hosting provider observed for one drill of one + * release: the provider-side items a local drill cannot see (a smoke after the + * switch, live traffic, the trigger), named as the drill's UNVERIFIED line + * names them, plus `provider verification` itself. + */ +export const providerDrillEvidenceSchema = z + .object({ + schemaVersion: z.literal(1), + artifactType: z.literal("provider-drill-evidence"), + runbookId: z.string().regex(/^FE-RB-00[1-5]$/u), + releaseId: nonEmptyString, + observedAt: timestamp, + runId: nonEmptyString, + observations: z + .array( + z + .object({ item: nonEmptyString, result: z.enum(["PASS", "FAIL"]), evidence: nonEmptyString }) + .strict(), + ) + .min(1) + .max(32) + .refine((items) => new Set(items.map(({ item }) => item)).size === items.length, { + message: "must name each item once", + }), + }) + .strict(); + +export type ProviderDrillEvidence = z.infer; + export const runbookRecordArtifactSchema = z .object({ schemaVersion: z.literal(2), @@ -778,6 +814,19 @@ export const runbookRecordArtifactSchema = z negativeFixtureFailedAsExpected: drillVerdict, windowObservedBucket: nonEmptyString, providerVerificationRequired: z.boolean(), + providerEvidence: z + .object({ + accepted: z.boolean(), + reason: nonEmptyString.optional(), + runId: nonEmptyString.optional(), + observedAt: timestamp.optional(), + providerVerification: drillVerdict, + }) + .strict() + .refine((evidence) => evidence.accepted || (evidence.reason !== undefined && evidence.providerVerification === "UNVERIFIED"), { + message: "rejected provider evidence must say why and verify nothing", + }) + .optional(), result: drillVerdict, }) .strict() @@ -791,6 +840,7 @@ export const runbookRecordArtifactSchema = z record.negativeFixtureFailedAsExpected, ], record.providerVerificationRequired, + record.providerEvidence?.providerVerification, ); if (record.result !== expected) { context.addIssue({ diff --git a/scripts/create-release-candidate.ts b/scripts/create-release-candidate.ts index 7ac2e792..cd618d1b 100644 --- a/scripts/create-release-candidate.ts +++ b/scripts/create-release-candidate.ts @@ -1,5 +1,6 @@ import { mkdir } from "node:fs/promises"; +import { checkoutBuildReadFailure } from "./lib/run-isolation.ts"; import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts"; import { localEvidenceAssessmentArtifactSchema } from "./contracts/release-artifacts.ts"; import { createLocalEvidenceAssessment } from "./lib/local-release-evidence.ts"; @@ -10,6 +11,15 @@ import { releaseCandidateManifestSchema, } from "./lib/release-candidate.ts"; +// 1106/12-02. This digests dist and the evidence under artifacts/ where the +// checkout keeps them (scripts/lib/release-candidate.ts). Under RUN_OUTPUT_ROOT +// the build wrote elsewhere, so the candidate would package an earlier build. +const outputRootFailure = checkoutBuildReadFailure("scripts/create-release-candidate.ts"); +if (outputRootFailure !== undefined) { + process.stderr.write(`${outputRootFailure}\n`); + process.exit(1); +} + const assessment = await createLocalEvidenceAssessment(); await mkdir("artifacts/security", { recursive: true }); await writeValidatedJsonArtifact({ diff --git a/scripts/drill-runbook.ts b/scripts/drill-runbook.ts index c1b03c2d..c610a8a8 100644 --- a/scripts/drill-runbook.ts +++ b/scripts/drill-runbook.ts @@ -22,8 +22,10 @@ import { projectTelemetryEvent } from "../src/contracts/telemetry.ts"; import { EXPECTED_CONTRACT_SET_PACKAGES } from "../src/features/installed-contract-contributions.ts"; import { combineDrillVerdicts, + providerDrillEvidenceSchema, runbookRecordArtifactSchema, type DrillVerdict, + type ProviderDrillEvidence, } from "./contracts/release-artifacts.ts"; import { verifyReleaseRuntimeCoherence } from "./lib/release-runtime-coherence.ts"; import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts"; @@ -87,8 +89,25 @@ export type DrillOptions = Readonly<{ */ releaseRoot?: string; contractPackages?: readonly InstalledContractPackageIdentity[]; + /** + * 0929/11-02. The provider's record for this drill (`--provider-evidence`), + * parsed JSON. Without it a drill with provider scope cannot pass. + */ + providerEvidence?: unknown; + /** The clock evidence age is judged against; a test passes a fixed one. */ + now?: Date; }>; +/** Provider evidence older than this no longer describes the provider as it is. */ +export const PROVIDER_EVIDENCE_MAX_AGE_MS = 7 * 24 * 60 * 60 * 1_000; +/** How far ahead of this clock a provider's timestamp may be. */ +const PROVIDER_EVIDENCE_CLOCK_SKEW_MS = 5 * 60 * 1_000; +/** The item names a drill's UNVERIFIED line uses, and provider evidence answers. */ +const ITEM_TRIGGER = "trigger"; +const ITEM_CONTAINMENT = "containment"; +const ITEM_NEGATIVE_FIXTURE = "negative fixture"; +export const ITEM_PROVIDER_VERIFICATION = "provider verification"; + type DrillContext = Readonly<{ root: string; contractPackages: readonly InstalledContractPackageIdentity[]; @@ -490,14 +509,29 @@ export async function runRunbookDrill(runbookId: string, options: DrillOptions = const release = parseReleaseArtifact( await rootedReader(context.root)(DEFAULT_ROLLBACK_PATHS.release), ); - const result = await drill(context); + const local = await drill(context); + const now = options.now ?? new Date(); + const judged = options.providerEvidence === undefined + ? undefined + : judgeProviderEvidence(options.providerEvidence, runbookId, release.releaseId, openItems(local), now); + const result = judged?.accepted ? applyProviderEvidence(local, judged.evidence) : local; + const providerEvidence = judged === undefined + ? undefined + : judged.accepted + ? { + accepted: true as const, + runId: judged.evidence.runId, + observedAt: judged.evidence.observedAt, + providerVerification: observed(judged.evidence, ITEM_PROVIDER_VERIFICATION) ?? "UNVERIFIED", + } + : { accepted: false as const, reason: judged.reason, providerVerification: "UNVERIFIED" as const }; const escalationPathAsserted = verdict(specification.escalation.length >= 2); return { schemaVersion: 2 as const, runbookId, gateId: specification.gateId, releaseId: release.releaseId, - drillTimestamp: new Date().toISOString(), + drillTimestamp: now.toISOString(), triggerInjected: specification.triggerKinds[0] ?? "", triggerAsserted: result.triggerAsserted, containmentAsserted: result.containmentAsserted, @@ -506,6 +540,7 @@ export async function runRunbookDrill(runbookId: string, options: DrillOptions = negativeFixtureFailedAsExpected: result.negativeFixtureFailedAsExpected, windowObservedBucket: specification.window, providerVerificationRequired: result.providerVerificationRequired, + ...(providerEvidence === undefined ? {} : { providerEvidence }), result: combineDrillVerdicts( [ result.triggerAsserted, @@ -515,24 +550,117 @@ export async function runRunbookDrill(runbookId: string, options: DrillOptions = result.negativeFixtureFailedAsExpected, ], result.providerVerificationRequired, + providerEvidence?.providerVerification, ), }; } +/** The items the local drill left UNVERIFIED, by the names provider evidence uses. */ +function openItems(result: DrillResult): string[] { + return [ + ...(result.triggerAsserted === "UNVERIFIED" ? [ITEM_TRIGGER] : []), + ...(result.containmentAsserted === "UNVERIFIED" ? [ITEM_CONTAINMENT] : []), + ...result.recoveryAssertions.filter((item) => item.result === "UNVERIFIED").map((item) => item.assertion), + ...(result.negativeFixtureFailedAsExpected === "UNVERIFIED" ? [ITEM_NEGATIVE_FIXTURE] : []), + ...(result.providerVerificationRequired ? [ITEM_PROVIDER_VERIFICATION] : []), + ]; +} + +type JudgedProviderEvidence = + | Readonly<{ accepted: true; evidence: ProviderDrillEvidence }> + | Readonly<{ accepted: false; reason: string }>; + +/** + * 0929/11-02. Provider evidence counts only for this runbook, this release, + * a recent observation, and the items the local drill could not observe. + */ +export function judgeProviderEvidence( + value: unknown, + runbookId: string, + releaseId: string, + open: readonly string[], + now: Date, +): JudgedProviderEvidence { + const parsed = providerDrillEvidenceSchema.safeParse(value); + if (!parsed.success) { + const issue = parsed.error.issues[0]; + return { accepted: false, reason: `not provider drill evidence (${issue?.path.join(".") || "root"}: ${issue?.message ?? "invalid"})` }; + } + const evidence = parsed.data; + if (evidence.runbookId !== runbookId) { + return { accepted: false, reason: `evidence is for ${evidence.runbookId}, not ${runbookId}` }; + } + if (evidence.releaseId !== releaseId) { + return { accepted: false, reason: `evidence is for release ${evidence.releaseId}, not ${releaseId}` }; + } + const age = now.getTime() - Date.parse(evidence.observedAt); + if (age > PROVIDER_EVIDENCE_MAX_AGE_MS) { + return { accepted: false, reason: `evidence observed at ${evidence.observedAt} is older than ${PROVIDER_EVIDENCE_MAX_AGE_MS / 86_400_000} days` }; + } + if (age < -PROVIDER_EVIDENCE_CLOCK_SKEW_MS) { + return { accepted: false, reason: `evidence observed at ${evidence.observedAt} is in the future` }; + } + const extra = evidence.observations.map(({ item }) => item).filter((item) => !open.includes(item)); + if (extra.length > 0) { + return { accepted: false, reason: `evidence names items the local drill observed or does not have: ${extra.join(", ")}` }; + } + return { accepted: true, evidence }; +} + +function observed(evidence: ProviderDrillEvidence, item: string): DrillVerdict | undefined { + return evidence.observations.find((observation) => observation.item === item)?.result; +} + +/** Replaces each UNVERIFIED item the evidence answers; local PASS and FAIL stay. */ +function applyProviderEvidence(result: DrillResult, evidence: ProviderDrillEvidence): DrillResult { + const answer = (item: string, local: DrillVerdict): DrillVerdict => + local === "UNVERIFIED" ? observed(evidence, item) ?? local : local; + const source = (item: string) => + `provider evidence ${evidence.runId} at ${evidence.observedAt}: ${ + evidence.observations.find((observation) => observation.item === item)?.evidence ?? "" + }`; + return { + ...result, + triggerAsserted: answer(ITEM_TRIGGER, result.triggerAsserted), + containmentAsserted: answer(ITEM_CONTAINMENT, result.containmentAsserted), + recoveryAssertions: result.recoveryAssertions.map((item) => + item.result === "UNVERIFIED" && observed(evidence, item.assertion) !== undefined + ? { ...item, evidence: source(item.assertion), result: observed(evidence, item.assertion)! } + : item, + ), + negativeFixtureFailedAsExpected: answer(ITEM_NEGATIVE_FIXTURE, result.negativeFixtureFailedAsExpected), + }; +} + async function main(): Promise { const runbookId = process.argv .slice(2) .find((argument) => /^FE-RB-00[1-5]$/.test(argument)); const releaseRootFlag = process.argv.indexOf("--release-root"); const releaseRoot = releaseRootFlag >= 0 ? process.argv[releaseRootFlag + 1] : undefined; - if (!runbookId || (releaseRootFlag >= 0 && !releaseRoot)) { + const evidenceFlag = process.argv.indexOf("--provider-evidence"); + const evidencePath = evidenceFlag >= 0 ? process.argv[evidenceFlag + 1] : undefined; + if (!runbookId || (releaseRootFlag >= 0 && !releaseRoot) || (evidenceFlag >= 0 && !evidencePath)) { process.stderr.write( - "Usage: drill:runbook -- FE-RB-001..FE-RB-005 [--release-root ]\n", + "Usage: drill:runbook -- FE-RB-001..FE-RB-005 [--release-root ]" + + " [--provider-evidence ]\n", ); process.exit(2); } + let providerEvidence: unknown; + if (evidencePath !== undefined) { + try { + providerEvidence = JSON.parse(await readFile(evidencePath, "utf8")); + } catch (error) { + process.stderr.write(`${runbookId} drill: cannot read provider evidence ${evidencePath}: ${String(error)}\n`); + process.exit(2); + } + } - const { gateId, ...record } = await runRunbookDrill(runbookId, releaseRoot ? { releaseRoot } : {}); + const { gateId, ...record } = await runRunbookDrill(runbookId, { + ...(releaseRoot ? { releaseRoot } : {}), + ...(evidencePath !== undefined ? { providerEvidence } : {}), + }); const artifactDirectory = `artifacts/runbooks/${runbookId}`; await mkdir(artifactDirectory, { recursive: true }); await writeValidatedJsonArtifact({ @@ -547,14 +675,18 @@ async function main(): Promise { if (record.result === "UNVERIFIED") { // Not a pass: the gate stays red until provider evidence exists (D-14). const open = [ - ["trigger", record.triggerAsserted], - ["containment", record.containmentAsserted], + [ITEM_TRIGGER, record.triggerAsserted], + [ITEM_CONTAINMENT, record.containmentAsserted], ...record.recoveryAssertions.map((item) => [item.assertion, item.result] as const), - ["negative fixture", record.negativeFixtureFailedAsExpected], - ["provider verification", record.providerVerificationRequired ? "UNVERIFIED" : "PASS"], + [ITEM_NEGATIVE_FIXTURE, record.negativeFixtureFailedAsExpected], + [ + ITEM_PROVIDER_VERIFICATION, + record.providerVerificationRequired ? record.providerEvidence?.providerVerification ?? "UNVERIFIED" : "PASS", + ], ].filter(([, result]) => result === "UNVERIFIED").map(([name]) => name); + const rejected = record.providerEvidence?.accepted === false ? `; provider evidence rejected: ${record.providerEvidence.reason}` : ""; process.stderr.write( - `${runbookId} drill: UNVERIFIED (${gateId}; no local check failed; not observed locally: ${open.join(", ")})\n`, + `${runbookId} drill: UNVERIFIED (${gateId}; no local check failed; not observed locally: ${open.join(", ")}${rejected})\n`, ); process.exit(3); } diff --git a/scripts/generate-build-manifest.ts b/scripts/generate-build-manifest.ts index 2d47f7d7..58c1de8a 100644 --- a/scripts/generate-build-manifest.ts +++ b/scripts/generate-build-manifest.ts @@ -20,7 +20,7 @@ import { buildDate, } from "./lib/build-environment.ts"; import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts"; -import { CANONICAL_VITE_MANIFEST_PATH } from "./lib/build-manifest-outputs.ts"; +import { CANONICAL_VITE_MANIFEST_PATH, findRouteChunk } from "./lib/build-manifest-outputs.ts"; import { runOutputPath } from "./lib/run-isolation.ts"; assertCiBuildEnvironment(process.env); @@ -68,14 +68,13 @@ const routeChunks: Record = {}; const runtimeContracts: Readonly> = ROUTE_RUNTIME_CONTRACT; for (const definition of Object.values(ROUTE_REGISTRY)) { - const runtime = runtimeContracts[definition.routeId]; - const asset = Object.values(viteManifestObject).find( - (entry) => entry.name === runtime?.moduleId && entry.isDynamicEntry, + const chunk = findRouteChunk( + viteManifestObject, + definition.routeId, + runtimeContracts[definition.routeId]?.moduleId, ); - if (!runtime || !asset?.file) { - throw new Error(`Missing built route chunk: ${definition.routeId}`); - } - routeChunks[definition.chunkId] = asset.file; + if ("error" in chunk) throw new Error(chunk.error); + routeChunks[definition.chunkId] = chunk.file; } const runtimeConfigJsonSchema = z.toJSONSchema(runtimeConfigV2ArtifactSchema); diff --git a/scripts/lib/build-manifest-outputs.ts b/scripts/lib/build-manifest-outputs.ts index 0f7dea44..bdd8f268 100644 --- a/scripts/lib/build-manifest-outputs.ts +++ b/scripts/lib/build-manifest-outputs.ts @@ -186,3 +186,52 @@ export async function verifyBuildManifestOutputs( } return mismatches; } + +/** + * 0929/13-03. The built chunk of a route's lazy page, for the build manifest + * and the two release checks that compare its `routeChunks`. A route runtime's + * `moduleId` is the file name of the page it imports + * (`src/presentation/routes/route-runtime.tsx`, a generated feature's + * `-route-runtime.ts`), so the chunk is the dynamic entry whose source + * file — the Vite manifest key — has that name. The chunk's own `name` used to + * be matched instead, so a chunk-naming change in vite.config.ts stopped the + * build with a reason-less `Missing built route chunk`. + */ +export type RouteChunkLookup = Readonly<{ file: string }> | Readonly<{ error: string }>; + +const SOURCE_EXTENSION = /\.(?:tsx|ts|jsx|js)$/u; + +export function findRouteChunk( + manifest: Readonly>, + routeId: string, + moduleId: string | undefined, +): RouteChunkLookup { + if (moduleId === undefined) { + return { error: `Missing built route chunk: ${routeId} has no route runtime contract (ROUTE_RUNTIME_CONTRACT)` }; + } + const matches: { source: string; file: string }[] = []; + for (const [key, entry] of Object.entries(manifest)) { + if (!isManifestRecord(entry) || entry["isDynamicEntry"] !== true || typeof entry["file"] !== "string") continue; + const source = typeof entry["src"] === "string" ? entry["src"] : key; + if (path.posix.basename(source).replace(SOURCE_EXTENSION, "") === moduleId) { + matches.push({ source, file: entry["file"] }); + } + } + if (matches.length === 1) return { file: matches[0]!.file }; + if (matches.length === 0) { + return { + error: + `Missing built route chunk: ${routeId}: no lazily imported module in the Vite manifest has the source ` + + `file ${moduleId}.tsx (or .ts). A route runtime's moduleId must be the file name of the page it imports.`, + }; + } + return { + error: + `Ambiguous built route chunk: ${routeId}: the lazily imported modules ` + + `${matches.map(({ source }) => source).join(", ")} are all named ${moduleId}.`, + }; +} + +function isManifestRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} diff --git a/scripts/lib/hosting-probe.ts b/scripts/lib/hosting-probe.ts index 02a997b5..640fc236 100644 --- a/scripts/lib/hosting-probe.ts +++ b/scripts/lib/hosting-probe.ts @@ -142,9 +142,9 @@ export async function probeLiveRelease( const negotiateAlpn = transport.negotiateAlpn ?? negotiatedAlpnProtocol; const responses: ResponseHeaders = {}; const results: ProbeResult[] = []; - const probe = async (surface: string, pathname: string): Promise => { + const probe = async (surface: string, target: URL): Promise => { try { - const response = await fetcher(new URL(pathname, base)); + const response = await fetcher(target); const finalOrigin = new URL(response.url).origin; results.push( { @@ -191,11 +191,12 @@ export async function probeLiveRelease( const bodies: Record = {}; for (const [surface, pathname] of Object.entries(paths)) { - bodies[surface] = await probe(surface, pathname); + bodies[surface] = await probe(surface, surfaceUrl(pathname, base)); } const entryScript = servedEntryScript(bodies["index"] ?? ""); if (entryScript) { - await probe("hashedAsset", entryScript); + // The served index.html names it as the build wrote it (under the base path). + await probe("hashedAsset", new URL(entryScript, base)); } else { results.push({ surface: "hashedAsset", @@ -226,6 +227,21 @@ export async function probeLiveRelease( }); } const candidateEntryScript = servedEntryScript(candidate.indexHtml); + // 0929/13-06. The candidate was built for one base path; a HOSTING_BASE_URL + // with another path probes surfaces the release does not serve. + // The build writes its scripts to `assets/`, so the entry script's + // folder says which base the candidate was built for, in both directions. + const builtFor = candidateEntryScript?.startsWith(`${base.pathname}${BUILD_ASSET_DIRECTORY}`) === true; + results.push({ + surface: "deployment", + header: "base-path", + expected: `${base.pathname}${BUILD_ASSET_DIRECTORY}`, + observed: candidateEntryScript, + ...(builtFor + ? {} + : { reason: "the candidate was built for another base path than the HOSTING_BASE_URL path (VITE_ROUTER_BASE_PATH of the build)" }), + passed: builtFor, + }); results.push({ surface: "index", header: "candidate-entry-script", @@ -405,10 +421,19 @@ export function checkPublicServiceWorkers( }; } +/** + * 0929/13-06. The surfaces in config/hosting/cache-policy.json are written + * from the release's root (`/config.json`); a release served under a base + * path (`VITE_ROUTER_BASE_PATH=/app/`) serves them under that path. + */ +export function surfaceUrl(pathname: string, base: URL): URL { + return new URL(`.${pathname.startsWith("/") ? pathname : `/${pathname}`}`, base); +} + /** * A release gate must not promote a local preview server as live hosting - * evidence. - * + * evidence. 0929/13-06: the URL is the release's base URL, the root or the + * sub-path the build was made for (`/app/`). */ export function classifyLiveHostingBaseUrl(value: string): | { @@ -467,10 +492,10 @@ export function classifyLiveHostingBaseUrl(value: string): observedOrigin, }; } - if (url.pathname !== "/" || url.search || url.hash) { + if (!url.pathname.endsWith("/") || url.search || url.hash) { return { passed: false, - reason: "HOSTING_BASE_URL must be the canonical root URL", + reason: "HOSTING_BASE_URL must be the release's base URL: a path ending in \"/\", without query or fragment", url, observedOrigin, }; diff --git a/scripts/lib/local-release-evidence.ts b/scripts/lib/local-release-evidence.ts index 9149872e..3e1423c9 100644 --- a/scripts/lib/local-release-evidence.ts +++ b/scripts/lib/local-release-evidence.ts @@ -26,6 +26,7 @@ import { } from "../contracts/release-artifacts.ts"; import { CANONICAL_VITE_MANIFEST_PATH, + findRouteChunk, verifyBuildManifestOutputs, } from "./build-manifest-outputs.ts"; import { assertMatchesJsonSchema } from "./json-schema.ts"; @@ -1255,14 +1256,12 @@ async function verifyReleaseOutputs( const runtimeContracts: Readonly> = ROUTE_RUNTIME_CONTRACT; for (const definition of Object.values(ROUTE_REGISTRY)) { - const runtime = runtimeContracts[definition.routeId]; - const viteEntry = Object.values(viteManifest).find( - (entry) => - isRecord(entry) && - entry.name === runtime?.moduleId && - entry.isDynamicEntry === true, + const chunk = findRouteChunk( + viteManifest, + definition.routeId, + runtimeContracts[definition.routeId]?.moduleId, ); - const file = isRecord(viteEntry) ? viteEntry.file : null; + const file = "error" in chunk ? null : chunk.file; if ( typeof file !== "string" || release.routeChunks[definition.chunkId] !== file || diff --git a/scripts/lib/provider-cgroup.ts b/scripts/lib/provider-cgroup.ts index 74ae58d4..770f26b3 100644 --- a/scripts/lib/provider-cgroup.ts +++ b/scripts/lib/provider-cgroup.ts @@ -1,10 +1,24 @@ export type ProviderKind = "vulnerability" | "provenance"; +/** The oldest systemd user manager whose `--scope` properties the provider relies on. */ +export const MINIMUM_PROVIDER_SYSTEMD_VERSION = 254; + +/** `systemctl` arguments that ask the user manager its version; the provider and the prerequisite check both use them. */ +export const USER_MANAGER_VERSION_ARGUMENTS: readonly string[] = Object.freeze([ + "--user", + "show", + "--property=Version", + "--value", +]); + const MEMORY_MAX = 1_073_741_824; const TASKS_MAX = 64; const STOP_TIMEOUT_MS = 5_000; const RUNTIME_GRACE_MS = 10_000; const UNIT_NAME = /^ca-provider-(?:vulnerability|provenance)-[1-9][0-9]*-[0-9a-f]{24}\.scope$/u; +const PROBE_UNIT_NAME = /^ca-provider-prerequisite-[1-9][0-9]*-[0-9a-f]{24}\.scope$/u; +/** How long the prerequisite probe's scope may live; it runs /bin/true. */ +const PROBE_RUNTIME_MS = 5_000; const UNIT_NONCE = /^[0-9a-f]{24}$/u; const ENVIRONMENT_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/u; const ENFORCEMENT_GATE = [ @@ -79,6 +93,37 @@ export function systemdRunProviderArguments( ) { throw new TypeError("provider scope wrapper path is invalid"); } + return [ + ...scopeArguments(unit, timeoutMs + RUNTIME_GRACE_MS), + nodeExecutable, + wrapperScript, + String(cpuSeconds), + reportPath, + String(reportDev), + String(reportIno), + ]; +} + +export function formatProviderPrerequisiteUnitName(supervisorPid: number, nonce: string): string { + if (!Number.isSafeInteger(supervisorPid) || supervisorPid <= 0 || !UNIT_NONCE.test(nonce)) { + throw new TypeError("provider cgroup unit identity is invalid"); + } + return `ca-provider-prerequisite-${supervisorPid}-${nonce}.scope`; +} + +/** + * 1112/08-03. The provider's own scope — same properties, same check that the + * unit's cgroup enforces them — around /bin/true. A host whose user manager + * cannot start the scope, or does not delegate the memory, pids and cpu + * controllers, fails here instead of in each provider test. + */ +export function systemdRunPrerequisiteProbeArguments(unit: string): string[] { + if (!PROBE_UNIT_NAME.test(unit)) throw new TypeError("provider cgroup unit is invalid"); + return [...scopeArguments(unit, PROBE_RUNTIME_MS), "/bin/true"]; +} + +/** `systemd-run` options and the enforcement gate up to its `$0`; the caller appends what the gate execs. */ +function scopeArguments(unit: string, runtimeMaxMs: number): string[] { return [ "--user", "--scope", @@ -94,19 +139,13 @@ export function systemdRunProviderArguments( "--property=KillMode=control-group", "--property=SendSIGKILL=yes", `--property=TimeoutStopSec=${STOP_TIMEOUT_MS}ms`, - `--property=RuntimeMaxSec=${timeoutMs + RUNTIME_GRACE_MS}ms`, + `--property=RuntimeMaxSec=${runtimeMaxMs}ms`, "--", "/bin/sh", "-eu", "-c", ENFORCEMENT_GATE, unit, - nodeExecutable, - wrapperScript, - String(cpuSeconds), - reportPath, - String(reportDev), - String(reportIno), ]; } diff --git a/scripts/lib/test-evidence-artifact.ts b/scripts/lib/test-evidence-artifact.ts index 1f86f3f7..dcc1de8c 100644 --- a/scripts/lib/test-evidence-artifact.ts +++ b/scripts/lib/test-evidence-artifact.ts @@ -12,6 +12,8 @@ export const testEvidenceReportSchema = z sharedScenarios: z.number().int().nonnegative(), declaredScenarioExecutions: z.number().int().nonnegative(), executedScenarioExecutions: z.number().int().nonnegative(), + // 1112/03-03. The Playwright configs the forbidOnly rule read. + playwrightConfigs: z.array(z.string().min(1)).optional(), }) .strict(), failures: z.array(z.string()), diff --git a/scripts/run-and-validate-provider.ts b/scripts/run-and-validate-provider.ts index a662ca3f..0be35287 100644 --- a/scripts/run-and-validate-provider.ts +++ b/scripts/run-and-validate-provider.ts @@ -20,8 +20,10 @@ import { encodeProviderBwrapInput, encodeProviderScopeFrame, formatProviderCgroupUnitName, + MINIMUM_PROVIDER_SYSTEMD_VERSION, systemctlKillProviderArguments, systemdRunProviderArguments, + USER_MANAGER_VERSION_ARGUMENTS, } from "./lib/provider-cgroup.ts"; import { assertProviderGuardianLeasePaths, @@ -49,7 +51,6 @@ const PROVIDER_REAP_TIMEOUT_MS = 5_000; const PROVIDER_CONTROL_TIMEOUT_MS = 5_000; const PROVIDER_CONTROL_MAX_OUTPUT_BYTES = 65_536; const PROVIDER_CONTROL_POLL_MS = 25; -const MINIMUM_PROVIDER_SYSTEMD_VERSION = 254; if (kind !== "vulnerability" && kind !== "provenance") { process.stderr.write("Usage: run-and-validate-provider --kind vulnerability|provenance\n"); process.exit(2); @@ -501,7 +502,7 @@ async function waitForProviderClose( async function assertProviderCgroupManagerAvailable(): Promise { const output = await runBoundedSystemctl( - ["--user", "show", "--property=Version", "--value"], + USER_MANAGER_VERSION_ARGUMENTS, "provider cgroup user manager probe", ); const match = /^([0-9]+)/u.exec(output.trim()); diff --git a/scripts/verify-documentation-readiness.ts b/scripts/verify-documentation-readiness.ts index 1032181d..3abdd9fe 100644 --- a/scripts/verify-documentation-readiness.ts +++ b/scripts/verify-documentation-readiness.ts @@ -304,16 +304,44 @@ function linkedPath(target: string, document: string): string | null { // never looked up. Leading `NAME=value` assignments are skipped, a // `--flag=value` argument is read as its value, and placeholders (``, // globs, `$VAR`) are left alone by namedPath. -const COMMAND_HEADS = new Set(["corepack", "pnpm", "npx", "node", "vitest", "playwright", "tsc", "eslint", "bash", "sh"]); -function commandPaths(command: string, document: string): (string | null)[] { +const COMMAND_HEADS = new Set([ + "corepack", "pnpm", "npx", "node", "vitest", "playwright", "tsc", "eslint", "bash", "sh", "python3", +]); +// 1112/08-06. `scripts/verify.sh` is this repository's main entry and most +// documented commands go through it: the entry is itself a path, `exec --` +// runs a command whose paths are looked up like any other, and `script` +// names package scripts (verifyScripts). +const VERIFY_ENTRY = /^(?:\.\/)?scripts\/verify\.sh$/u; +function commandWords(command: string): Readonly<{ words: string[]; head: number }> { const words = command.trim().replace(/^\$\s+/u, "").split(/\s+/u); let head = 0; while (/^[A-Z_][A-Z0-9_]*=/u.test(words[head] ?? "")) head += 1; - if (!COMMAND_HEADS.has(words[head] ?? "")) return []; + return { words, head }; +} +function commandPaths(command: string, document: string): (string | null)[] { + const { words, head } = commandWords(command); + const first = words[head] ?? ""; + if (VERIFY_ENTRY.test(first)) { + const separator = words.indexOf("--", head + 1); + const executed = words[head + 1] === "exec" && separator > head ? words.slice(separator + 1).join(" ") : ""; + return [namedPath(first, document), ...(executed === "" ? [] : commandPaths(executed, document))]; + } + if (!COMMAND_HEADS.has(first)) return []; return words .slice(head + 1) .map((word) => namedPath(word.replace(/^--?[\w.-]+=/u, "").replace(/^["']|["']$/gu, ""), document)); } +/** The package scripts `scripts/verify.sh script … [-- args]` runs; placeholders are left alone. */ +function verifyScripts(command: string): string[] { + const { words, head } = commandWords(command); + if (!VERIFY_ENTRY.test(words[head] ?? "") || words[head + 1] !== "script") return []; + const names: string[] = []; + for (const word of words.slice(head + 2)) { + if (word.startsWith("-")) break; + names.push(word); + } + return names.filter((name) => /^[a-z](?:[\w:-]*\w)?$/u.test(name)); +} // Lines of fenced shell blocks (```bash, ```sh, ```shell, ```console or no // language), which is where most documented commands are. function fencedCommands(text: string): string[] { @@ -398,6 +426,10 @@ for (const document of contractDocuments) { for (const [, name = ""] of text.matchAll(/`([a-z][\w-]*:[\w:-]*\w)(?=[` ])/gu)) { if (scriptPrefixes.has(name.split(":")[0]) && missingScript(name)) missingScripts.add(name); } + const commands = [...[...text.matchAll(/`([^`\n]+)`/gu)].map(([, token = ""]) => token), ...fencedCommands(text)]; + for (const name of commands.flatMap(verifyScripts)) { + if (missingScript(name)) missingScripts.add(name); + } documentReferences.push({ path: document, missingPaths: [...missingPaths].sort(), diff --git a/scripts/verify-hosting-headers.ts b/scripts/verify-hosting-headers.ts index 7ec12a74..e3ccb6af 100644 --- a/scripts/verify-hosting-headers.ts +++ b/scripts/verify-hosting-headers.ts @@ -121,7 +121,7 @@ if (liveTarget?.passed && !slotEmpty) { probeResults.push({ surface: "deployment", header: "base-url", - expected: "canonical non-loopback HTTPS root URL", + expected: "non-loopback HTTPS base URL of the release", observed: liveTarget.observedOrigin, reason: liveTarget.reason, passed: false, diff --git a/scripts/verify-release.ts b/scripts/verify-release.ts index c4eb5a9f..c332cd69 100644 --- a/scripts/verify-release.ts +++ b/scripts/verify-release.ts @@ -24,6 +24,7 @@ import { import { assertMatchesJsonSchema } from "./lib/json-schema.ts"; import { CANONICAL_VITE_MANIFEST_PATH, + findRouteChunk, verifyBuildManifestOutputs, } from "./lib/build-manifest-outputs.ts"; import { verifyReleaseRuntimeCoherence } from "./lib/release-runtime-coherence.ts"; @@ -168,12 +169,13 @@ for (const chunkId of actualChunkIds) { const runtimeContracts: Readonly> = ROUTE_RUNTIME_CONTRACT; for (const definition of Object.values(ROUTE_REGISTRY)) { - const runtime = runtimeContracts[definition.routeId]; - const viteEntry = Object.values(viteManifestObject).find( - (entry) => entry.name === runtime?.moduleId && entry.isDynamicEntry, + const viteEntry = findRouteChunk( + viteManifestObject, + definition.routeId, + runtimeContracts[definition.routeId]?.moduleId, ); const routeAsset = release.routeChunks[definition.chunkId]; - if (!runtime || !viteEntry || routeAsset !== viteEntry.file) { + if ("error" in viteEntry || routeAsset !== viteEntry.file) { artifactMismatches.push(`routeChunk:mismatch:${definition.chunkId}`); continue; } diff --git a/scripts/verify-reproducible-build.ts b/scripts/verify-reproducible-build.ts index dbf2ec31..79f15644 100644 --- a/scripts/verify-reproducible-build.ts +++ b/scripts/verify-reproducible-build.ts @@ -4,10 +4,18 @@ import path from "node:path"; import { reproducibleBuildArtifactSchema } from "./contracts/release-artifacts.ts"; import { assertCiBuildEnvironment } from "./lib/build-environment.ts"; +import { checkoutBuildReadFailure } from "./lib/run-isolation.ts"; import { supplyChainDigest } from "./lib/supply-chain.ts"; import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts"; assertCiBuildEnvironment(process.env); +// 1106/12-02. This builds twice and digests dist where the checkout keeps it. Under RUN_OUTPUT_ROOT the build wrote elsewhere, so +// this would judge the earlier build the checkout still holds. +const outputRootFailure = checkoutBuildReadFailure("scripts/verify-reproducible-build.ts"); +if (outputRootFailure !== undefined) { + process.stderr.write(`${outputRootFailure}\n`); + process.exit(1); +} async function filesWithin(directory: string): Promise { const entries = await readdir(directory, { withFileTypes: true }); diff --git a/tests/architecture/request-fanout.test.ts b/tests/architecture/request-fanout.test.ts index c764fc96..d0df201a 100644 --- a/tests/architecture/request-fanout.test.ts +++ b/tests/architecture/request-fanout.test.ts @@ -3,7 +3,9 @@ import path from "node:path"; import { describe, expect, it } from "vitest"; import { + checkRequestFanoutFixtures, findRequestFanout, + formatFanoutViolation, scanRequestFanout, } from "../../scripts/check-request-fanout.ts"; @@ -21,6 +23,13 @@ describe("no per-row transport loop (frontend N+1 source gate)", () => { expect(new Set(forbidden.violations.map((violation) => violation.file)).size).toBe(forbidden.files); expect(forbidden.files).toBe(3); expect(allowed.violations).toEqual([]); + expect(allowed.reviewed).toEqual([ + expect.objectContaining({ + file: "tests/fixtures/architecture/request-fanout/allowed/reviewed-cursor-loop.ts", + call: "gateway.page()", + reason: "cursor pages, bounded by maxPages", + }), + ]); }); it("names the dispatch and the repeating construct", async () => { @@ -40,4 +49,24 @@ describe("no per-row transport loop (frontend N+1 source gate)", () => { ); expect(violations).toEqual([]); }); + + // 0929/12-01. The failure line named only the call, so a product team saw + // neither the cursor runtime nor the reviewed-exception comment, and the + // result line hid how many calls a comment let through. + it("tells the author both ways out on the failure line", async () => { + const [violation] = await findRequestFanout( + "inline.ts", + "export const load = (api: { get(id: string): Promise }, ids: string[]) =>\n Promise.all(ids.map((id) => api.get(id)));\n", + ); + const line = formatFanoutViolation(violation!); + expect(line).toMatch(/^FAIL inline\.ts:2 api\.get\(\) inside \.map\(\) callback: /u); + expect(line).toContain("createCursorPaginationRuntime (src/adapters/query-cache)"); + expect(line).toContain("`// fanout-reviewed: ` on the call's line"); + }); + + it("counts and lists the reviewed exceptions in the fixture result line", async () => { + const result = await checkRequestFanoutFixtures(path.resolve(".")); + expect(result.ok).toBe(true); + expect(result.line).toMatch(/allowed 0 violations and 1 reviewed exception \(tests\/fixtures\/architecture\/request-fanout\/allowed\/reviewed-cursor-loop\.ts:\d+ cursor pages, bounded by maxPages\) PASS/u); + }); }); diff --git a/tests/browser-capabilities/opfs-runtime.spec.ts b/tests/browser-capabilities/opfs-runtime.spec.ts index 59d42d64..3a4849f9 100644 --- a/tests/browser-capabilities/opfs-runtime.spec.ts +++ b/tests/browser-capabilities/opfs-runtime.spec.ts @@ -337,6 +337,7 @@ async function removeOpfsAuthority( for (const committedObjects of [0, 3]) { test(`05-01 an approved purge in another tab waits for a live put and takes it with the partition (${committedObjects} committed objects)`, async ({ page, + browserName, }) => { const secondPage = await page.context().newPage(); const suffix = crypto.randomUUID().replaceAll("-", ""); @@ -349,7 +350,18 @@ for (const committedObjects of [0, 3]) { type Tab = import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts").BrowserOpfsRuntime; try { await Promise.all([page.goto("/"), secondPage.goto("/")]); - if (!(await installOpfsTab(page, scope))) return; + // 1112/03-02. An early return here passed with no assertion on an engine + // the support check refused, while the evidence said zero skipped. + // Playwright's WebKit is refused (the first test asserts its UNSUPPORTED + // result), so there this asserts the refusal; every other engine must + // open the runtime. A skip is no way out: the capability evidence + // refuses skipped cases (scripts/verify-browser-capability-evidence.ts). + const installed = await installOpfsTab(page, scope); + if (browserName === "webkit") { + expect(installed, "the OPFS support check refuses Playwright's WebKit").toBe(false); + return; + } + expect(installed, "OPFS runtime supported in this engine").toBe(true); expect(await installOpfsTab(secondPage, scope, true)).toBe(true); const seeded = await secondPage.evaluate(async (count) => { @@ -455,6 +467,7 @@ for (const committedObjects of [0, 3]) { test("05-01 a reconcile in another tab skips a live put and the put stays readable", async ({ page, + browserName, }) => { const secondPage = await page.context().newPage(); const suffix = crypto.randomUUID().replaceAll("-", ""); @@ -467,7 +480,13 @@ test("05-01 a reconcile in another tab skips a live put and the put stays readab type Tab = import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts").BrowserOpfsRuntime; try { await Promise.all([page.goto("/"), secondPage.goto("/")]); - if (!(await installOpfsTab(page, scope))) return; + // 1112/03-02. Asserted, not returned early: see the purge test above. + const installed = await installOpfsTab(page, scope); + if (browserName === "webkit") { + expect(installed, "the OPFS support check refuses Playwright's WebKit").toBe(false); + return; + } + expect(installed, "OPFS runtime supported in this engine").toBe(true); expect(await installOpfsTab(secondPage, scope)).toBe(true); // Tab A starts a put whose source stalls, so its journal row stays diff --git a/tests/e2e/preload-use.spec.ts b/tests/e2e/preload-use.spec.ts index 7f614f83..d748b098 100644 --- a/tests/e2e/preload-use.spec.ts +++ b/tests/e2e/preload-use.spec.ts @@ -298,6 +298,25 @@ recordedTest.describe("the strict fixture's console check", () => { ); }); +// 1112/03-05. The fixture watched its own page only, so a console error in +// a second tab of the same context (the cross-tab capability specs) passed. +recordedTest.describe("the strict fixture's other tabs", () => { + recordedTest.use({ + fixtureReports: { lines: [/^console:error:\[preload-use\] second tab$/u] }, + }); + + recordedTest( + "records a console error in another tab of the context", + async ({ page }) => { + await openHome(page); + const second = await page.context().newPage(); + await openHome(second); + + await second.evaluate(() => console.error("[preload-use] second tab")); + }, + ); +}); + /** * 1105/11-01. The verdict every other spec gets: a recorded failure fails the * test. The record is proven above; this proves the verdict. diff --git a/tests/e2e/service-worker-recovery.spec.ts b/tests/e2e/service-worker-recovery.spec.ts index 400ec503..561330fc 100644 --- a/tests/e2e/service-worker-recovery.spec.ts +++ b/tests/e2e/service-worker-recovery.spec.ts @@ -1,6 +1,11 @@ import path from "node:path"; -import { expect, test, type Page } from "../support/browser/strict-browser-test.ts"; +import { + expect, + test, + watchBrowserFailures, + type Page, +} from "../support/browser/strict-browser-test.ts"; import { prepareServiceWorkerBuilds, removeServiceWorkerBuilds, @@ -91,7 +96,7 @@ test("a REMOVE_REGISTRATION release unregisters a worker that blocks boot", asyn const server = await startBuildServer(builds.dist); // The broken phase fails requests on purpose, so this test drives its own // context instead of the strict `page` fixture and checks the recovered - // load for browser errors itself. + // load for browser errors with the fixture's own watcher (1112/03-05). const context = await browser.newContext(); try { const page = await context.newPage(); @@ -133,22 +138,16 @@ test("a REMOVE_REGISTRATION release unregisters a worker that blocks boot", asyn ) .toBe(true); - const failures: string[] = []; - page.on("console", (message) => { - if (["error", "warning"].includes(message.type())) { - failures.push(`console:${message.type()}:${message.text()}`); - } - }); - page.on("pageerror", (error) => failures.push(`pageerror:${error.name}`)); - page.on("requestfailed", (request) => - failures.push(`requestfailed:${new URL(request.url()).pathname}`), - ); + // No preload check here: this context loaded the origin twice already, + // and WebKit answers the recovered load's scripts from its cache, past + // the tracker's route, so every preload reads as unused (16 in a run). + const recovered = await watchBrowserFailures(page); await page.goto(`${server.origin}/`); expect(await startOutcome(page)).toEqual({ kind: "DISABLED" }); expect( await page.evaluate(() => navigator.serviceWorker.controller === null), ).toBe(true); - expect(failures).toEqual([]); + expect(await recovered.stop()).toEqual([]); } finally { await context.close(); await server.close(); diff --git a/tests/e2e/support/fixture-builds.ts b/tests/e2e/support/fixture-builds.ts index 23a95e3f..933fa022 100644 --- a/tests/e2e/support/fixture-builds.ts +++ b/tests/e2e/support/fixture-builds.ts @@ -65,17 +65,41 @@ function run( } } -export async function prepareFixtureBuilds( - prefix: string, - builds: readonly FixtureBuild[], -): Promise> { - // An edit's source text names the build (two specs never share a prefix). - const fingerprint = JSON.stringify( - builds.map((build) => [build.name, build.env ?? {}, build.edit?.toString() ?? ""]), - ); +export type EditedCopyBuild = Readonly<{ + name: Name; + /** VITE_BUILD_ID and RELEASE_ID of this build. */ + buildId: string; + edit?: (root: string) => Promise; + env?: Readonly>; +}>; + +/** + * 0929/11-07. The one way an e2e spec gets real production builds of an + * edited copy of the checkout; prepareFixtureBuilds and + * prepareServiceWorkerBuilds (service-worker-builds.ts) are its two shapes. + * It copies the checkout (without `checkout-copy-skip`), links node_modules, + * runs `prepare` once, then for each build runs `edit`, builds, moves `dist` + * to `dist-` and asks `collect` what the spec needs from that build. + * `key` and `fingerprint` name the result in the e2e run + * (shared-fixture-builds.ts), which removes the copy when preparing fails. + */ +export async function buildEditedCopies( + input: Readonly<{ + key: string; + fingerprint: string; + prepare?: (root: string) => Promise; + builds: readonly EditedCopyBuild[]; + collect?: (name: Name, root: string) => Promise; + }>, +): Promise>; + collected: Readonly>; + shared: boolean; +}>> { const { root, value, shared } = await sharedFixtureBuild( - `fixture-${prefix}`, - fingerprint, + input.key, + input.fingerprint, async (root) => { await cp(repoRoot, root, { recursive: true, @@ -83,19 +107,36 @@ export async function prepareFixtureBuilds( !skipped.has(path.relative(repoRoot, source).split(path.sep)[0]!), }); await linkFixtureNodeModules(root, repoRoot); + await input.prepare?.(root); const dist = {} as Record; - for (const build of builds) { + const collected = {} as Record; + for (const build of input.builds) { await build.edit?.(root); - const buildId = `${prefix}-${build.name}`; - run(root, "scripts/generate-product-feature-index.ts", buildId, build.env); - run(root, "scripts/build-frontend.ts", buildId, build.env); + run(root, "scripts/generate-product-feature-index.ts", build.buildId, build.env); + run(root, "scripts/build-frontend.ts", build.buildId, build.env); dist[build.name] = path.join(root, `dist-${build.name}`); await rename(path.join(root, "dist"), dist[build.name]); + collected[build.name] = input.collect ? await input.collect(build.name, root) : (null as Collected); } - return dist; + return { dist, collected }; }, ); - return { root, dist: value, shared }; + return { root, ...value, shared }; +} + +export async function prepareFixtureBuilds( + prefix: string, + builds: readonly FixtureBuild[], +): Promise> { + // An edit's source text names the build (two specs never share a prefix). + const { root, dist, shared } = await buildEditedCopies({ + key: `fixture-${prefix}`, + fingerprint: JSON.stringify( + builds.map((build) => [build.name, build.env ?? {}, build.edit?.toString() ?? ""]), + ), + builds: builds.map((build) => ({ ...build, buildId: `${prefix}-${build.name}` })), + }); + return { root, dist, shared }; } export async function removeFixtureBuilds( diff --git a/tests/e2e/support/service-worker-builds.ts b/tests/e2e/support/service-worker-builds.ts index c96b1961..ef4bc723 100644 --- a/tests/e2e/support/service-worker-builds.ts +++ b/tests/e2e/support/service-worker-builds.ts @@ -1,12 +1,5 @@ -import { spawnSync } from "node:child_process"; import { existsSync } from "node:fs"; -import { - cp, - readFile, - rename, - rm, - writeFile, -} from "node:fs/promises"; +import { readFile, rm, writeFile } from "node:fs/promises"; import { createServer, type Server } from "node:http"; import type { AddressInfo } from "node:net"; import path from "node:path"; @@ -14,11 +7,8 @@ import { pathToFileURL } from "node:url"; import { gzipSync } from "node:zlib"; import { SERVICE_WORKER_ASSETS_OUTPUT } from "../../../scripts/generate-service-worker-assets.ts"; -import { localBuildEnvironment } from "../../../scripts/lib/build-environment.ts"; -import { linkFixtureNodeModules } from "../../../scripts/lib/fixture-node-modules.ts"; -import { selectRepositoryPaths } from "../../../scripts/lib/repository-layout.ts"; import type { StaticAssetManifestV1 } from "../../../src/contracts/service-worker.ts"; -import { sharedFixtureBuild } from "./shared-fixture-builds.ts"; +import { buildEditedCopies, editOnce } from "./fixture-builds.ts"; /** * R3-6 test helper: two real production builds with the Service Worker on, @@ -54,11 +44,6 @@ import { sharedFixtureBuild } from "./shared-fixture-builds.ts"; * text it expected. */ -const repoRoot = path.resolve(import.meta.dirname, "../../.."); -// 1105/12-04. `.tmp` holds other runs' builds and results (.tmp/runs/), -// which change while this copy is made; fixture-builds.ts skips it too. -const skipped = new Set(selectRepositoryPaths("checkout-copy-skip")); - export type BuildName = "a" | "b"; export type ServiceWorkerBuilds = Readonly<{ root: string; @@ -101,17 +86,8 @@ export function createServiceWorkerRuntimeHost( } `; -export async function replaceOnce( - file: string, - search: string, - replacement: string, -): Promise { - const source = await readFile(file, "utf8"); - if (source.split(search).length !== 2) { - throw new Error(`Expected exactly one "${search}" in ${file}.`); - } - await writeFile(file, source.replace(search, replacement)); -} +/** 0929/11-07. The same one-occurrence edit as fixture-builds.ts, not a copy of it. */ +export const replaceOnce = editOnce; /** * Premise 1. Replaces the value of the one `SERVICE_WORKER` declaration in @@ -167,25 +143,12 @@ async function precachedUrls(root: string, name: BuildName): Promise { return SERVICE_WORKER_ASSETS.assets.map((asset) => asset.url).sort(); } -function run(cwd: string, script: string, buildId: string): void { - const result = spawnSync("node", [script], { - cwd, - encoding: "utf8", - // 1103/11-08. A local build, not the CI job's release build. - env: { ...localBuildEnvironment(), VITE_BUILD_ID: buildId, RELEASE_ID: buildId }, - }); - if (result.status !== 0) { - throw new Error( - `${script} failed for ${buildId}:\n${result.stdout}\n${result.stderr}`, - ); - } -} - /** * `beforeBuild` edits the temp copy right before build `name` (06-03: a * broken worker A, then a `REMOVE_REGISTRATION` release B). Edits persist * into the later build. `key` names this pair of builds in the e2e run - * (shared-fixture-builds.ts); each spec passes its own. + * (shared-fixture-builds.ts); each spec passes its own. The copy and the + * builds are fixture-builds.ts's `buildEditedCopies` (0929/11-07). */ export async function prepareServiceWorkerBuilds( options: Readonly<{ @@ -193,38 +156,27 @@ export async function prepareServiceWorkerBuilds( beforeBuild?: (name: BuildName, root: string) => Promise; }>, ): Promise { - const { root, value, shared } = await sharedFixtureBuild( - `sw-${options.key}`, - options.beforeBuild?.toString() ?? "", - async (root) => { - await cp(repoRoot, root, { - recursive: true, - filter: (source) => - !skipped.has(path.relative(repoRoot, source).split(path.sep)[0]!), - }); - await linkFixtureNodeModules(root, repoRoot); + const { root, dist, collected, shared } = await buildEditedCopies({ + key: `sw-${options.key}`, + fingerprint: options.beforeBuild?.toString() ?? "", + prepare: async (root) => { await turnOnServiceWorker(root); const hostFile = path.join(root, "src/bootstrap/register-service-worker.ts"); - await replaceOnce( + await editOnce( hostFile, "export function createServiceWorkerRuntimeHost(", "function createServiceWorkerRuntimeHostUnobserved(", ); await writeFile(hostFile, (await readFile(hostFile, "utf8")) + HOST_PROBE); - - const dist = { a: path.join(root, "dist-a"), b: path.join(root, "dist-b") }; - const precached: Record = { a: [], b: [] }; - for (const name of ["a", "b"] as const) { - await options.beforeBuild?.(name, root); - run(root, "scripts/generate-product-feature-index.ts", `sw-e2e-${name}`); - run(root, "scripts/build-frontend.ts", `sw-e2e-${name}`); - await rename(path.join(root, "dist"), dist[name]); - precached[name] = await precachedUrls(root, name); - } - return { dist, precached }; }, - ); - return { root, ...value, shared }; + builds: (["a", "b"] as const).map((name) => ({ + name, + buildId: `sw-e2e-${name}`, + edit: async (root: string) => options.beforeBuild?.(name, root), + })), + collect: (name, root) => precachedUrls(root, name), + }); + return { root, dist, precached: collected, shared }; } export async function removeServiceWorkerBuilds( diff --git a/tests/fixtures/architecture/request-fanout/allowed/reviewed-cursor-loop.ts b/tests/fixtures/architecture/request-fanout/allowed/reviewed-cursor-loop.ts new file mode 100644 index 00000000..aa3d5341 --- /dev/null +++ b/tests/fixtures/architecture/request-fanout/allowed/reviewed-cursor-loop.ts @@ -0,0 +1,18 @@ +// Allowed: a cursor loop someone reviewed. Each page needs the cursor the +// previous page returned, so the requests cannot be one bulk read; the count +// is bounded by the page cap. Feature code normally gets this from +// createCursorPaginationRuntime (src/adapters/query-cache). +type Page = Readonly<{ items: readonly string[]; nextCursor: string | null }>; +type Gateway = { page(cursor: string | null): Promise }; + +export async function loadAll(gateway: Gateway, maxPages: number) { + const items: string[] = []; + let cursor: string | null = null; + for (let page = 0; page < maxPages; page += 1) { + const next: Page = await gateway.page(cursor); // fanout-reviewed: cursor pages, bounded by maxPages + items.push(...next.items); + if (next.nextCursor === null) break; + cursor = next.nextCursor; + } + return items; +} diff --git a/tests/fixtures/test-evidence/playwright-config/missing-forbid-only.fixture b/tests/fixtures/test-evidence/playwright-config/missing-forbid-only.fixture new file mode 100644 index 00000000..ce5ed861 --- /dev/null +++ b/tests/fixtures/test-evidence/playwright-config/missing-forbid-only.fixture @@ -0,0 +1,8 @@ +// A Playwright config that would let a committed focused test run alone: +// check-test-evidence.ts --playwright-config must reject it (1112/03-03). +import { defineConfig } from "@playwright/test"; + +export default defineConfig({ + testDir: "tests/e2e", + reporter: [["junit", { outputFile: "artifacts/tests/e2e/results.xml" }]], +}); diff --git a/tests/support/browser/strict-browser-test.ts b/tests/support/browser/strict-browser-test.ts index 1c0b5a94..fdda5bc7 100644 --- a/tests/support/browser/strict-browser-test.ts +++ b/tests/support/browser/strict-browser-test.ts @@ -20,6 +20,48 @@ function ignoredConsole(message: ConsoleMessage) { ); } +/** + * 1112/03-05. Records a page's browser failures as `console:…`, `pageerror:…` + * and `requestfailed:…` lines, with the same rules the strict fixture judges + * by. The fixture watches its `page` and every other tab opened in the same + * context; a spec that drives a context of its own watches its pages with + * this. `preloads` also checks that every modulepreload ran (preload-use.ts) + * in the documents open when `stop` runs; attach it before the navigation + * that loads them. + */ +export async function watchBrowserFailures( + page: Page, + options: Readonly<{ preloads?: boolean }> = {}, +): Promise }>> { + const failures: string[] = []; + const onConsole = (message: ConsoleMessage) => { + if (["error", "warning"].includes(message.type()) && !ignoredConsole(message)) { + failures.push(`console:${message.type()}:${message.text()}`); + } + }; + const onPageError = (error: Error) => { + failures.push(`pageerror:${error.name}`); + }; + const onRequestFailed = (request: import("@playwright/test").Request) => { + failures.push( + `requestfailed:${request.method()}:${new URL(request.url()).pathname}:${request.failure()?.errorText ?? "unknown"}`, + ); + }; + page.on("console", onConsole); + page.on("pageerror", onPageError); + page.on("requestfailed", onRequestFailed); + const preloads = options.preloads ? await trackPreloadUse(page) : null; + return Object.freeze({ + async stop() { + if (preloads) failures.push(...(await preloads.check()).failures); + page.off("console", onConsole); + page.off("pageerror", onPageError); + page.off("requestfailed", onRequestFailed); + return [...failures]; + }, + }); +} + /** * 1105/11-01. What the strict `page` fixture does with every failure it * recorded, once the test has ended. @@ -62,32 +104,19 @@ export const test = base.extend<{ ); }, page: async ({ page, browserFailureVerdict }, use) => { - const preloads = await trackPreloadUse(page); - const failures: string[] = []; - const onConsole = (message: ConsoleMessage) => { - if ( - ["error", "warning"].includes(message.type()) && - !ignoredConsole(message) - ) { - failures.push(`console:${message.type()}:${message.text()}`); - } + const watched = [await watchBrowserFailures(page, { preloads: true })]; + // 1112/03-05. A second tab of the same context (the cross-tab specs) + // went unwatched. Its listeners are attached as the tab opens, before + // the spec can navigate it; preload use is checked on the first page only. + const pending: Promise[] = []; + const onPage = (opened: Page) => { + pending.push(watchBrowserFailures(opened).then((watcher) => void watched.push(watcher))); }; - const onPageError = (error: Error) => { - failures.push(`pageerror:${error.name}`); - }; - const onRequestFailed = (request: import("@playwright/test").Request) => { - failures.push( - `requestfailed:${request.method()}:${new URL(request.url()).pathname}:${request.failure()?.errorText ?? "unknown"}`, - ); - }; - page.on("console", onConsole); - page.on("pageerror", onPageError); - page.on("requestfailed", onRequestFailed); + page.context().on("page", onPage); await use(page); - failures.push(...(await preloads.check()).failures); - page.off("console", onConsole); - page.off("pageerror", onPageError); - page.off("requestfailed", onRequestFailed); + page.context().off("page", onPage); + await Promise.all(pending); + const failures = (await Promise.all(watched.map((watcher) => watcher.stop()))).flat(); browserFailureVerdict.judge(failures); }, }); diff --git a/tests/unit/browser-rpc/browser-rpc-remediation.test.ts b/tests/unit/browser-rpc/browser-rpc-remediation.test.ts index 7167788f..8048d718 100644 --- a/tests/unit/browser-rpc/browser-rpc-remediation.test.ts +++ b/tests/unit/browser-rpc/browser-rpc-remediation.test.ts @@ -377,6 +377,19 @@ describe("RPC-RR-04 transport results and frames are exactly decoded", () => { }); }); +/** + * 1101/10-07. The system clock with every wait capped at 30 ms. Without a + * positive receipt the stream cleanup waits out its 1 s bound; capped, a test + * passes that bound instead of sleeping through it. The operations' own 25 ms + * deadlines are below the cap and unchanged. Every stream test that ends + * without a positive receipt takes it, so none sleeps a real second. + */ +const cappedClock: ClockPort = Object.freeze({ + now: () => systemClock.now(), + sleep: (milliseconds: number, signal?: AbortSignal) => + systemClock.sleep(Math.min(milliseconds, 30), signal), +}); + /** * RPC-RR-01. A bare `AsyncIterable` gave the runtime no way to cancel the * physical stream or to learn when it actually closed, so a timed-out call left @@ -395,6 +408,7 @@ describe("RPC-RR-01 server stream leases and the DRAINING fence", () => { mappers: MAPPERS, requestEncoders: { RpcResourceStreamRequestEncoder: STREAM_ENCODER }, transports: { CONNECT_REFERENCE_STREAM: probe.transport }, + clock: cappedClock, }); const results = await collect( @@ -418,6 +432,7 @@ describe("RPC-RR-01 server stream leases and the DRAINING fence", () => { mappers: MAPPERS, requestEncoders: { RpcResourceStreamRequestEncoder: STREAM_ENCODER }, transports: { CONNECT_REFERENCE_STREAM: probe.transport }, + clock: cappedClock, }); const stream = runtime.bindServerStream( "WATCH_RPC_RESOURCES", @@ -541,18 +556,6 @@ describe("RPC-01 only a positive receipt confirms physical closure", () => { { label: "never settles", waitClosed: () => new Promise(() => {}) }, ]; - /** - * 1101/10-07. The system clock with every wait capped at 30 ms. Without a - * positive receipt the stream cleanup waits out its 1 s bound; capped, the - * test passes that bound instead of sleeping through it. The operation's own - * 25 ms deadlines are below the cap and unchanged. - */ - const cappedClock: ClockPort = Object.freeze({ - now: () => systemClock.now(), - sleep: (milliseconds: number, signal?: AbortSignal) => - systemClock.sleep(Math.min(milliseconds, 30), signal), - }); - for (const { label, waitClosed } of negativeReceipts) { it(`keeps the operation DRAINING when waitClosed ${label}`, async () => { const probe = leaseProbeWith(waitClosed); @@ -598,6 +601,7 @@ describe("RPC-01 only a positive receipt confirms physical closure", () => { mappers: MAPPERS, requestEncoders: { RpcResourceStreamRequestEncoder: STREAM_ENCODER }, transports: { CONNECT_REFERENCE_STREAM: probe.transport }, + clock: cappedClock, }); const stream = runtime.bindServerStream( "WATCH_RPC_RESOURCES", diff --git a/tests/unit/browser-spec-imports.test.ts b/tests/unit/browser-spec-imports.test.ts new file mode 100644 index 00000000..596db74f --- /dev/null +++ b/tests/unit/browser-spec-imports.test.ts @@ -0,0 +1,51 @@ +import path from "node:path"; + +import { ESLint } from "eslint"; +import { describe, expect, it } from "vitest"; + +import { childProcessTimeout } from "../helpers/child-process-timeout.ts"; + +const FOLDERS = ["e2e", "browser-capabilities", "visual", "storybook"] as const; + +const BARE = 'import { expect, test } from "@playwright/test";\n\ntest("probe", async () => { expect(1).toBe(1); });\n'; +const STRICT = + 'import { expect, test } from "../support/browser/strict-browser-test.ts";\n' + + 'import type { Page } from "@playwright/test";\n\n' + + 'test("probe", async ({ page }: { page: Page }) => { expect(page).toBeTruthy(); });\n'; +const DEFAULT_IMPORT = 'import test from "@playwright/test";\n\ntest("probe", async () => {});\n'; +const BASE_TEST = 'import { test } from "../support/browser/browser-test.ts";\n\ntest("probe", async () => {});\n'; + +async function restrictedImports(code: string, file: string): Promise { + const linter = new ESLint({ cwd: process.cwd() }); + const [result] = await linter.lintText(code, { filePath: path.join(process.cwd(), file), warnIgnored: true }); + return (result?.messages ?? []) + .filter((message) => message.ruleId === "no-restricted-imports") + .map((message) => message.message); +} + +// 1112/03-04. Strict fixture use was a text search for its path in +// tests/e2e/*.spec.ts: a path in a comment passed, a file could mix in the +// bare test, and tests/browser-capabilities, tests/visual and tests/storybook +// were not looked at. It is an import rule in each of the four folders now. +describe("browser specs take test from the strict fixture (1112/03-04)", () => { + it.each(FOLDERS)("rejects the bare Playwright test in tests/%s", async (folder) => { + const messages = await restrictedImports(BARE, `tests/${folder}/lint-probe.spec.ts`); + expect(messages).toEqual([expect.stringContaining("tests/support/browser/strict-browser-test.ts")]); + }, childProcessTimeout(683)); + + it("rejects the default import, the base test the strict fixture extends, and accepts the strict test", async () => { + expect(await restrictedImports(DEFAULT_IMPORT, "tests/visual/lint-probe.spec.ts")).toEqual([ + expect.stringContaining("tests/support/browser/strict-browser-test.ts"), + ]); + expect(await restrictedImports(BASE_TEST, "tests/e2e/lint-probe.spec.ts")).toEqual([ + expect.stringContaining("the base test does not check browser errors"), + ]); + expect(await restrictedImports(STRICT, "tests/browser-capabilities/lint-probe.spec.ts")).toEqual([]); + }, childProcessTimeout(683)); + + it("lets only the two specs that need the plain test import it", async () => { + for (const file of ["tests/e2e/preload-use.spec.ts", "tests/browser-capabilities/chunk-load-error.spec.ts"]) { + expect(await restrictedImports(BARE, file), file).toEqual([]); + } + }, childProcessTimeout(683)); +}); diff --git a/tests/unit/documentation-readiness.test.ts b/tests/unit/documentation-readiness.test.ts index dbc15222..9374e83e 100644 --- a/tests/unit/documentation-readiness.test.ts +++ b/tests/unit/documentation-readiness.test.ts @@ -250,11 +250,29 @@ describe("verify:documentation commands and sibling links (1103/12-05)", () => { const result = await verify({ [CI_DOC]: (text) => `${text}\n\`corepack pnpm exec vitest run tests/unit/documentation-readiness.test.ts -t "x"\`\n` + "```bash\ncorepack pnpm drill:runbook -- FE-RB-005 --release-root \n```\n" + - "Any script is run as `node scripts/…`.\n", + "Any script is run as `node scripts/…`.\n" + + "`scripts/verify.sh exec -- node scripts/run-vitest.ts run tests/unit/documentation-readiness.test.ts`, " + + "`scripts/verify.sh script verify:documentation lint -- --x`, `scripts/verify.sh script <이름>…`, " + + "`python3 scripts/check-work-docs.py --history <경로>`\n", }); expect(result.status, result.stderr).toBe(0); }, VERIFY_TIMEOUT); + // 1112/08-06. Commands through this repository's entry point, and python3 + // commands, were not read, so a stale path or script name in them passed. + it("reads commands run through scripts/verify.sh and python3", async () => { + const result = await verify({ + [CI_DOC]: (text) => `${text}\n\`scripts/verify.sh exec -- node scripts/no-such.ts\`\n` + + "```bash\nVERIFY_SNAPSHOT=1 scripts/verify.sh exec -- node tests/unit/no-such-shell.test.ts\n```\n" + + "`python3 scripts/no-such.py --history docs`\n`scripts/verify.sh script no:such-script -- --flag`\n", + }); + expect(result.status).toBe(1); + expect(result.stderr).toContain( + `${CI_DOC} names paths that do not exist: scripts/no-such.py, scripts/no-such.ts, tests/unit/no-such-shell.test.ts`, + ); + expect(result.stderr).toContain(`${CI_DOC} names package scripts that do not exist: no:such-script`); + }, VERIFY_TIMEOUT); + it("checks a link or a path into a paired repository checked out beside this one", async () => { const naming = (target: string) => (text: string) => `${text}\n[contract](../../../${target}) and \`${target}\`\n`; diff --git a/tests/unit/e2e-build-helpers.test.ts b/tests/unit/e2e-build-helpers.test.ts new file mode 100644 index 00000000..0a57b4bf --- /dev/null +++ b/tests/unit/e2e-build-helpers.test.ts @@ -0,0 +1,36 @@ +import { readFile, readdir } from "node:fs/promises"; +import path from "node:path"; + +import { describe, expect, it } from "vitest"; + +const SUPPORT = "tests/e2e/support"; + +// 0929/11-07. The Service Worker helper kept its own copy of the checkout +// copy, the build runner and the one-occurrence edit, so a fix to one (the +// local build env of 1103/11-08, the copy skip list of 1105/12-04) had to be +// made twice. Both helpers now go through fixture-builds.ts. +describe("e2e build helpers have one copy-and-build path (0929/11-07)", () => { + it("builds Service Worker fixtures through buildEditedCopies", async () => { + const source = await readFile(path.join(SUPPORT, "service-worker-builds.ts"), "utf8"); + + expect(source).toMatch(/import \{[^}]*\bbuildEditedCopies\b[^}]*\} from "\.\/fixture-builds\.ts";/u); + expect(source).toContain("buildEditedCopies({"); + for (const copy of ["spawnSync", "sharedFixtureBuild", "linkFixtureNodeModules", "await cp(", "function run("]) { + expect(source, copy).not.toContain(copy); + } + }); + + it("keeps the build runner and the one-occurrence edit in fixture-builds.ts only", async () => { + const files = (await readdir(SUPPORT)).filter((file) => file.endsWith(".ts")); + const defining = async (pattern: RegExp) => { + const found: string[] = []; + for (const file of files) { + if (pattern.test(await readFile(path.join(SUPPORT, file), "utf8"))) found.push(file); + } + return found; + }; + + expect(await defining(/^function run\(/mu)).toEqual(["fixture-builds.ts"]); + expect(await defining(/split\(search\)\.length !== 2/u)).toEqual(["fixture-builds.ts"]); + }); +}); diff --git a/tests/unit/hosting-probe.test.ts b/tests/unit/hosting-probe.test.ts index 7d82e34d..3dda020b 100644 --- a/tests/unit/hosting-probe.test.ts +++ b/tests/unit/hosting-probe.test.ts @@ -73,11 +73,18 @@ describe("live hosting evidence target", () => { }); }); + // 0929/13-06. A release built for VITE_ROUTER_BASE_PATH=/app/ is served + // under /app/; the probe refused any URL but the root, so it could not be used. + it("accepts the sub-path a release is built for", () => { + expect(classifyLiveHostingBaseUrl("https://frontend.example.test/app/")).toMatchObject({ passed: true }); + }); + it.each([ ["http://frontend.example.test/", "requires HTTPS"], ["https://localhost:4173/", "not live deployment evidence"], ["https://127.0.0.1/", "not live deployment evidence"], - ["https://frontend.example.test/app/", "canonical root URL"], + ["https://frontend.example.test/app", "path ending in \"/\""], + ["https://frontend.example.test/?preview=1", "without query or fragment"], ["https://user:secret@frontend.example.test/", "must not contain credentials"], ])("rejects %s", (url, reason) => { expect(classifyLiveHostingBaseUrl(url)).toMatchObject({ @@ -193,6 +200,62 @@ describe("SPA fallback and worker script surfaces (1101/13-04)", () => { expect(await failed({ ...RELEASE, "/__probe/deep-link": [200, INDEX] })).toEqual([]); }); + // 0929/13-06. Under a base path the surfaces are requested below it. + it("probes a release built for /app/ below /app/ and passes it", async () => { + const appIndex = ''; + const requested: string[] = []; + const routes: Record = { + "/app/": [200, appIndex], + "/app/config.json": [200, CONFIG], + "/app/release-manifest.json": [200, JSON.stringify({ schemaVersion: 2, ...IDENTITY })], + "/app/__probe/deep-link": [200, appIndex], + "/app/assets/index-A1.js": [200, ""], + }; + const { results } = await probeLiveRelease( + new URL("https://frontend.example.test/app/"), + PATHS, + { ...CANDIDATE, indexHtml: appIndex }, + { + fetcher: async (url) => { + requested.push(url.pathname); + return host(routes)(url); + }, + negotiateAlpn: async () => "h2", + alpnProtocols: ["h2"], + }, + ); + + expect(requested).toEqual([ + "/app/", + "/app/config.json", + "/app/release-manifest.json", + "/app/__probe/deep-link", + "/app/assets/index-A1.js", + ]); + expect(results.filter((result) => !result.passed)).toEqual([]); + }); + + it("fails a HOSTING_BASE_URL path the candidate was not built for", async () => { + expect( + (await probeLiveRelease(new URL("https://frontend.example.test/app/"), PATHS, CANDIDATE, { + fetcher: host({}), + negotiateAlpn: async () => "h2", + alpnProtocols: ["h2"], + })).results.find((result) => result.header === "base-path"), + ).toMatchObject({ expected: "/app/assets/", observed: "/assets/index-A1.js", passed: false }); + // And the other way: a release built for /app/ probed at the root. + expect( + (await probeLiveRelease(base, PATHS, { + ...CANDIDATE, + indexHtml: '', + }, { + fetcher: host({}), + negotiateAlpn: async () => "h2", + alpnProtocols: ["h2"], + })).results.find((result) => result.header === "base-path"), + ).toMatchObject({ expected: "/assets/", observed: "/app/assets/index-A1.js", passed: false }); + }); + it("fails a host without the fallback", async () => { expect(await failed(RELEASE)).toEqual(["appRoute http-status", "appRoute entry-script"]); }); diff --git a/tests/unit/route-chunks.test.ts b/tests/unit/route-chunks.test.ts new file mode 100644 index 00000000..99142cad --- /dev/null +++ b/tests/unit/route-chunks.test.ts @@ -0,0 +1,74 @@ +import { readFile } from "node:fs/promises"; + +import { describe, expect, it } from "vitest"; + +import { findRouteChunk } from "../../scripts/lib/build-manifest-outputs.ts"; + +// 0929/13-03. The route chunk was found by the chunk's `name`, so naming route +// chunks in vite.config.ts (the way tech-log did) stopped the build with +// `Missing built route chunk: APP_HOME` and no reason. It is found by the +// page's source file now, which is what the route runtime's moduleId names. +describe("route chunk lookup (0929/13-03)", () => { + const manifest = { + "index.html": { file: "assets/index-a.js", isEntry: true }, + "src/presentation/pages/home-page.tsx": { + file: "assets/route-home-page-b.js", + name: "route-home-page", + src: "src/presentation/pages/home-page.tsx", + isDynamicEntry: true, + }, + "src/features/order-feature/presentation/order-feature-list-page.tsx": { + file: "assets/order-feature-list-page-c.js", + name: "order-feature-list-page", + isDynamicEntry: true, + }, + "_home-page-shared.js": { file: "assets/home-page-d.js", name: "home-page" }, + }; + + it("finds the chunk by the page's source file even when the chunk has another name", () => { + expect(findRouteChunk(manifest, "APP_HOME", "home-page")).toEqual({ file: "assets/route-home-page-b.js" }); + expect(findRouteChunk(manifest, "ORDER_FEATURE_LIST", "order-feature-list-page")).toEqual({ + file: "assets/order-feature-list-page-c.js", + }); + }); + + it("names the reason and the rule when no lazily imported page has that file name", () => { + const lookup = findRouteChunk(manifest, "NOT_FOUND", "not-found-page"); + expect(lookup).toEqual({ + error: + "Missing built route chunk: NOT_FOUND: no lazily imported module in the Vite manifest has the source " + + "file not-found-page.tsx (or .ts). A route runtime's moduleId must be the file name of the page it imports.", + }); + }); + + it("refuses two lazily imported pages with the same file name", () => { + const lookup = findRouteChunk( + { ...manifest, "src/other/home-page.tsx": { file: "assets/home-page-e.js", isDynamicEntry: true } }, + "APP_HOME", + "home-page", + ); + expect(lookup).toEqual({ + error: + "Ambiguous built route chunk: APP_HOME: the lazily imported modules " + + "src/presentation/pages/home-page.tsx, src/other/home-page.tsx are all named home-page.", + }); + }); + + it("names a route without a runtime contract", () => { + expect(findRouteChunk(manifest, "GHOST", undefined)).toEqual({ + error: "Missing built route chunk: GHOST has no route runtime contract (ROUTE_RUNTIME_CONTRACT)", + }); + }); + + it("is the one lookup the manifest writer and both release checks use", async () => { + for (const file of [ + "scripts/generate-build-manifest.ts", + "scripts/verify-release.ts", + "scripts/lib/local-release-evidence.ts", + ]) { + const source = await readFile(file, "utf8"); + expect(source, file).toContain("findRouteChunk("); + expect(source, file).not.toMatch(/\.name === runtime\?\.moduleId/u); + } + }); +}); diff --git a/tests/unit/run-isolation.test.ts b/tests/unit/run-isolation.test.ts index b4bd7d94..873bb860 100644 --- a/tests/unit/run-isolation.test.ts +++ b/tests/unit/run-isolation.test.ts @@ -1,3 +1,4 @@ +import { spawnSync } from "node:child_process"; import { createServer, type Server } from "node:net"; import { afterEach, describe, expect, it, vi } from "vitest"; @@ -12,6 +13,7 @@ import { testRunPath, testRunRoot, } from "../../scripts/lib/run-isolation.ts"; +import { childProcessTimeout } from "../helpers/child-process-timeout.ts"; const servers: Server[] = []; @@ -110,6 +112,28 @@ describe("build output root (RUN_OUTPUT_ROOT)", () => { /^check-bundle: RUN_OUTPUT_ROOT must be/u, ); }); + + // 1106/12-02, the part the gates did not cover. Run alone, these four read + // the checkout's dist/ or artifacts/ and did not refuse another root: + // `RUN_OUTPUT_ROOT=.tmp/x corepack pnpm verify:reproducible-build` wrote + // `passed: true` about the earlier build. + it.each([ + ["scripts/verify-reproducible-build.ts"], + ["scripts/check-optional-recipes.ts", "--require-dist"], + ["scripts/check-app-build-input.ts", "--require-bundle"], + ["scripts/create-release-candidate.ts"], + ])("%s refuses to read the checkout's build under another RUN_OUTPUT_ROOT", (script, ...args) => { + const { CI: _ci, ...environment } = process.env; + const result = spawnSync(process.execPath, [script, ...args], { + encoding: "utf8", + env: { ...environment, RUN_OUTPUT_ROOT: ".tmp/run-output-refusal" }, + }); + + expect(result.status, result.stderr).toBe(1); + expect(result.stderr).toContain( + `${script} reads the build at this checkout's paths (dist/, artifacts/), but RUN_OUTPUT_ROOT=.tmp/run-output-refusal`, + ); + }, childProcessTimeout(499)); }); describe("server ports", () => { diff --git a/tests/unit/runbook-drill.test.ts b/tests/unit/runbook-drill.test.ts index 611d4598..afc78619 100644 --- a/tests/unit/runbook-drill.test.ts +++ b/tests/unit/runbook-drill.test.ts @@ -4,7 +4,8 @@ import path from "node:path"; import { afterEach, describe, expect, it } from "vitest"; -import { runRunbookDrill } from "../../scripts/drill-runbook.ts"; +import { runbookRecordArtifactSchema } from "../../scripts/contracts/release-artifacts.ts"; +import { PROVIDER_EVIDENCE_MAX_AGE_MS, runRunbookDrill } from "../../scripts/drill-runbook.ts"; import { computeContractSetDigest } from "../../src/contracts/contract-set-canonical.ts"; import type { InstalledContractPackageIdentity } from "../../src/contracts/external-contract-runtime.ts"; @@ -145,3 +146,106 @@ describe("runbook drill verdicts", () => { expect(source).not.toMatch(/verdict\(\s*true\s*\)/u); }); }); + +// 0929/11-02. The drills with provider scope took only a runbook ID and a +// release root, so no input could ever make them PASS and the production +// stage could not close. Provider evidence now answers the items the local +// drill cannot observe, for this runbook and release only, and while recent. +describe("runbook drill with provider evidence (0929/11-02)", () => { + const NOW = new Date("2026-11-13T09:00:00.000Z"); + const evidence = (overrides: Record = {}) => ({ + schemaVersion: 1, + artifactType: "provider-drill-evidence", + runbookId: "FE-RB-005", + releaseId: "release-a", + observedAt: "2026-11-13T08:30:00.000Z", + runId: "provider-run-41", + observations: [ + { item: "trigger", result: "PASS", evidence: "incident INC-7 declared the release-blocking defect" }, + { item: "핵심 smoke 시험", result: "PASS", evidence: "smoke against the prior release after the switch" }, + { item: "provider verification", result: "PASS", evidence: "provider serves release-a again" }, + ], + ...overrides, + }); + const drill = async (providerEvidence: unknown) => + runRunbookDrill("FE-RB-005", { releaseRoot: await builtDist(), contractPackages, providerEvidence, now: NOW }); + + it("passes FE-RB-005 when the provider observed every item the local drill could not", async () => { + const record = await drill(evidence()); + + expect(record.result).toBe("PASS"); + expect(record.triggerAsserted).toBe("PASS"); + expect(record.recoveryAssertions.find(({ assertion }) => assertion === "핵심 smoke 시험")).toEqual({ + assertion: "핵심 smoke 시험", + evidence: "provider evidence provider-run-41 at 2026-11-13T08:30:00.000Z: smoke against the prior release after the switch", + result: "PASS", + }); + expect(record.providerEvidence).toEqual({ + accepted: true, + runId: "provider-run-41", + observedAt: "2026-11-13T08:30:00.000Z", + providerVerification: "PASS", + }); + const { gateId: _gateId, ...artifact } = record; + expect(runbookRecordArtifactSchema.safeParse(artifact).success).toBe(true); + }); + + it("fails when the provider observed a failure", async () => { + const observations = evidence().observations.map((entry) => + entry.item === "핵심 smoke 시험" ? { ...entry, result: "FAIL" } : entry); + expect((await drill(evidence({ observations }))).result).toBe("FAIL"); + }); + + it("stays UNVERIFIED while an item the local drill could not observe has no answer", async () => { + const record = await drill(evidence({ observations: evidence().observations.slice(0, 2) })); + expect(record.result).toBe("UNVERIFIED"); + expect(record.providerEvidence?.providerVerification).toBe("UNVERIFIED"); + }); + + it.each([ + ["another release", { releaseId: "release-b" }, "evidence is for release release-b, not release-a"], + ["another runbook", { runbookId: "FE-RB-002" }, "evidence is for FE-RB-002, not FE-RB-005"], + [ + "an expired observation", + { observedAt: new Date(NOW.getTime() - PROVIDER_EVIDENCE_MAX_AGE_MS - 1_000).toISOString() }, + "is older than 7 days", + ], + ["a future observation", { observedAt: "2026-11-13T10:00:00.000Z" }, "is in the future"], + [ + "an answer for a check the local drill made", + { observations: [...evidence().observations, { item: "호환성 gate", result: "PASS", evidence: "x" }] }, + "evidence names items the local drill observed or does not have: 호환성 gate", + ], + ["a record that is not provider evidence", { artifactType: "runbook-record" }, "not provider drill evidence"], + ])("does not pass on evidence for %s", async (_case, overrides, reason) => { + const record = await drill(evidence(overrides)); + + expect(record.result).toBe("UNVERIFIED"); + expect(record.triggerAsserted).toBe("UNVERIFIED"); + expect(record.providerEvidence?.accepted).toBe(false); + expect(record.providerEvidence?.reason).toContain(reason); + const { gateId: _gateId, ...artifact } = record; + expect(runbookRecordArtifactSchema.safeParse(artifact).success).toBe(true); + }); + + it("rejects a record that claims PASS without accepted provider evidence", () => { + const record = { + schemaVersion: 2, + runbookId: "FE-RB-005", + releaseId: "release-a", + drillTimestamp: NOW.toISOString(), + triggerInjected: "RELEASE_BLOCKING_DEFECT", + triggerAsserted: "PASS", + containmentAsserted: "PASS", + escalationPathAsserted: "PASS", + recoveryAssertions: [], + negativeFixtureFailedAsExpected: "PASS", + windowObservedBucket: "TBD", + providerVerificationRequired: true, + providerEvidence: { accepted: false, reason: "expired", providerVerification: "UNVERIFIED" }, + result: "PASS", + }; + expect(runbookRecordArtifactSchema.safeParse(record).success).toBe(false); + expect(runbookRecordArtifactSchema.safeParse({ ...record, result: "UNVERIFIED" }).success).toBe(true); + }); +}); diff --git a/tests/unit/system-test-prerequisites.test.ts b/tests/unit/system-test-prerequisites.test.ts index 8821bc42..9957b620 100644 --- a/tests/unit/system-test-prerequisites.test.ts +++ b/tests/unit/system-test-prerequisites.test.ts @@ -9,33 +9,52 @@ import { NAMESPACE_PROBE_ARGUMENTS, type SystemTestHost, } from "../../scripts/check-system-test-prerequisites.ts"; +import { + systemdRunPrerequisiteProbeArguments, + systemdRunProviderArguments, +} from "../../scripts/lib/provider-cgroup.ts"; import { PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS } from "../../scripts/lib/provider-sandbox-namespaces.ts"; type SpawnCall = Readonly<{ command: string; args: readonly string[] }>; type SpawnResult = ReturnType; const RTM_NEWADDR = "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted"; +const PROBE_UNIT = "ca-provider-prerequisite-4242-0123456789abcdef01234567.scope"; -/** A host that has every prerequisite, except what `bwrap` and `files` say. */ +type HostOverrides = Readonly<{ + files?: Readonly>; + missing?: readonly string[]; + userManager?: SpawnResult; + systemdRun?: SpawnResult; +}>; + +/** A host that has every prerequisite, except what `bwrap` and the overrides say. */ function fakeHost( bwrap: SpawnResult, - files: Readonly> = {}, + options: HostOverrides = {}, ): { host: SystemTestHost; calls: SpawnCall[] } { const calls: SpawnCall[] = []; const host: SystemTestHost = { - access: async () => undefined, + access: async (target) => { + if (options.missing?.includes(target)) throw new Error(`EACCES: ${target}`); + }, readFile: async (target) => { if (target === "/sys/fs/cgroup/cgroup.controllers") return "cpuset cpu io memory pids\n"; - const content = files[target]; + const content = options.files?.[target]; if (content === undefined) throw new Error(`ENOENT: ${target}`); return content; }, spawn: (command, args) => { calls.push({ command, args }); - if (command === "/usr/bin/systemctl") return { status: 0, stderr: "" }; + if (command === "/usr/bin/systemctl" && args[0] === "--user") { + return options.userManager ?? { status: 0, stdout: "255\n", stderr: "" }; + } + if (command === "/usr/bin/systemctl") return { status: 0, stdout: "", stderr: "" }; + if (command === "/usr/bin/systemd-run") return options.systemdRun ?? { status: 0, stdout: "", stderr: "" }; if (command === "/usr/bin/bwrap") return bwrap; throw new Error(`unexpected spawn: ${command}`); }, + probeUnitName: () => PROBE_UNIT, }; return { host, calls }; } @@ -74,7 +93,7 @@ describe("system test prerequisites (1105/12-01)", () => { it("fails with the namespace restriction, bwrap's stderr and what to do when bwrap cannot set up loopback", async () => { const { host } = fakeHost( { status: 1, stderr: `${RTM_NEWADDR}\n` }, - { "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" }, + { files: { "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" } }, ); const failures = await findSystemTestPrerequisiteFailures(host); @@ -103,7 +122,7 @@ describe("system test prerequisites (1105/12-01)", () => { expect(failures[0]).toContain("did not finish: spawnSync /usr/bin/bwrap ETIMEDOUT"); }); - it("keeps the executable, cgroup v2 and systemd bus checks, and probes bwrap only when it is there", async () => { + it("keeps the executable, cgroup v2 and systemd checks, and probes bwrap only when it is there", async () => { const calls: SpawnCall[] = []; const host: SystemTestHost = { access: async (target) => { @@ -116,15 +135,98 @@ describe("system test prerequisites (1105/12-01)", () => { }, }; - expect(await findSystemTestPrerequisiteFailures(host)).toEqual([ + const failures = await findSystemTestPrerequisiteFailures(host); + expect(failures.slice(0, 2)).toEqual([ "bubblewrap is required at /usr/bin/bwrap", "cgroup v2 controllers are unavailable", - "a reachable systemd manager bus is required", ]); + expect(failures[2]).toMatch(/^a reachable systemd user manager is required/u); + expect(failures).toHaveLength(3); expect(calls.map(({ command }) => command)).toEqual(["/usr/bin/systemctl"]); }); }); +// 1112/08-03. The check looked at the system manager bus while the provider +// asks the user manager for its version and starts a `systemd-run --user +// --scope`, so a host without a user manager, with an old one, or without +// prlimit, systemd-run or delegated controllers printed PASS and then failed +// each provider test with `provider cgroup unavailable: ...`. +describe("system test prerequisites match the provider's own calls (1112/08-03)", () => { + it("fails by name when the system manager answers but the user manager does not", async () => { + const { host, calls } = fakeHost( + { status: 0, stderr: "" }, + { userManager: { status: 1, stdout: "", stderr: "Failed to connect to bus: No medium found\n" } }, + ); + + const failures = await findSystemTestPrerequisiteFailures(host); + + expect(failures).toHaveLength(1); + expect(failures[0]).toMatch(/^a reachable systemd user manager is required/u); + expect(failures[0]).toContain("/usr/bin/systemctl --user show --property=Version --value exited 1"); + expect(failures[0]).toContain("systemctl stderr: Failed to connect to bus: No medium found"); + expect(calls.some(({ command }) => command === "/usr/bin/systemd-run")).toBe(false); + }); + + it("fails when the user manager is older than the provider needs", async () => { + const { host } = fakeHost({ status: 0, stderr: "" }, { userManager: { status: 0, stdout: "253\n", stderr: "" } }); + + expect(await findSystemTestPrerequisiteFailures(host)).toEqual([ + 'systemd user manager 254 or newer is required (/usr/bin/systemctl --user show --property=Version --value reported "253")', + ]); + }); + + it("requires the prlimit and systemd-run executables the provider runs", async () => { + const { host } = fakeHost({ status: 0, stderr: "" }, { missing: ["/usr/bin/prlimit", "/usr/bin/systemd-run"] }); + + expect(await findSystemTestPrerequisiteFailures(host)).toEqual([ + "prlimit is required at /usr/bin/prlimit", + "systemd-run is required at /usr/bin/systemd-run", + ]); + }); + + it("starts the provider's own scope once and names a scope that cannot enforce its limits", async () => { + const { host, calls } = fakeHost( + { status: 0, stderr: "" }, + { systemdRun: { status: 125, stdout: "", stderr: "provider cgroup enforcement failed: cannot read memory.max\n" } }, + ); + + const failures = await findSystemTestPrerequisiteFailures(host); + + expect(failures).toHaveLength(1); + expect(failures[0]).toMatch(/^systemd-run could not start the provider's cgroup scope/u); + expect(failures[0]).toContain("exited 125"); + expect(failures[0]).toContain("systemd-run stderr: provider cgroup enforcement failed: cannot read memory.max"); + expect(failures[0]).toMatch(/memory, pids and cpu controllers delegated/u); + + const scopes = calls.filter(({ command }) => command === "/usr/bin/systemd-run"); + expect(scopes).toHaveLength(1); + expect(scopes[0]?.args).toEqual(systemdRunPrerequisiteProbeArguments(PROBE_UNIT)); + expect(scopes[0]?.args.at(-1)).toBe("/bin/true"); + }); + + it("probes with the provider scope's properties and enforcement gate, not a copy", () => { + const probe = systemdRunPrerequisiteProbeArguments(PROBE_UNIT); + const provider = systemdRunProviderArguments( + "ca-provider-vulnerability-4242-0123456789abcdef01234567.scope", + 60_000, + 60, + "/usr/bin/node", + "/opt/wrapper.ts", + "/tmp/report.json", + 1, + 1, + ); + const options = (args: readonly string[]) => + args.filter((arg) => arg.startsWith("--") && !arg.startsWith("--unit=") && !arg.startsWith("--property=RuntimeMaxSec=")); + expect(options(probe)).toEqual(options(provider)); + const gate = (args: readonly string[]) => args[args.indexOf("-c") + 1]; + expect(gate(probe)).toBe(gate(provider)); + expect(gate(probe)).toContain("require_cgroup_value memory.max"); + expect(() => systemdRunPrerequisiteProbeArguments("ca-provider-vulnerability-1-0123456789abcdef01234567.scope")) + .toThrow(/unit is invalid/u); + }); +}); + // 1101/13-03. The release provider jobs run the same sandbox on whatever // runner picks them up. The supervisor checked only that the executables // exist, so a host that refuses the namespaces failed later with bwrap's raw @@ -140,7 +242,7 @@ describe("provider sandbox namespace check (1101/13-03)", () => { it("names the restriction, bwrap's stderr and the host settings on a host that refuses them", async () => { const { host } = fakeHost( { status: 1, stderr: `${RTM_NEWADDR}\n` }, - { "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" }, + { files: { "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" } }, ); const failure = (await findProviderSandboxNamespaceFailure(host)) ?? ""; diff --git a/tests/unit/test-evidence-source-rules.test.ts b/tests/unit/test-evidence-source-rules.test.ts index 074cb705..53a13f59 100644 --- a/tests/unit/test-evidence-source-rules.test.ts +++ b/tests/unit/test-evidence-source-rules.test.ts @@ -1,9 +1,10 @@ import { spawnSync } from "node:child_process"; -import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import path from "node:path"; import { afterAll, describe, expect, it } from "vitest"; +import { selectRepositoryPaths } from "../../scripts/lib/repository-layout.ts"; import { childProcessTimeout } from "../helpers/child-process-timeout.ts"; const FORBIDDEN = "tests/fixtures/test-evidence/forbidden"; @@ -13,7 +14,7 @@ afterAll(async () => { await Promise.all(roots.map((root) => rm(root, { recursive: true, force: true }))); }); -function checkSource(sourceRoot: string) { +function checkSource(sourceRoot: string, extra: readonly string[] = []) { const result = spawnSync( process.execPath, [ @@ -23,6 +24,7 @@ function checkSource(sourceRoot: string) { sourceRoot, "--artifact", "artifacts/quality/test-evidence-source-rules.json", + ...extra, ], { encoding: "utf8" }, ); @@ -44,6 +46,44 @@ describe("test evidence source rules", () => { expect(result.stderr).toContain("quarantine expired on 2020-01-01"); }, childProcessTimeout(739)); + // 1112/03-03. The mask fixture had no test of its own: the directory run + // expects only the common prefix, so this rule could stop firing unseen. + it("rejects a screenshot mask over the whole UI", () => { + const result = checkSource(`${FORBIDDEN}/full-screen-mask.fixture`); + expect(result.status).toBe(1); + expect(result.stderr).toContain("screenshot mask may not cover the whole UI"); + }, childProcessTimeout(141)); + + // 1112/03-03. The forbidOnly rule ran only in the full check, on the + // repository's own configs, so nothing showed it still rejects a config. + it("rejects a Playwright config without forbidOnly and accepts the repository's", async () => { + const root = await mkdtemp(path.join(tmpdir(), "test-evidence-config-")); + roots.push(root); + await writeFile(path.join(root, "clean.fixture"), "export {};\n"); + + const missing = checkSource(root, [ + "--playwright-config", + "tests/fixtures/test-evidence/playwright-config/missing-forbid-only.fixture", + ]); + expect(missing.status).toBe(1); + expect(missing.stderr).toContain( + "tests/fixtures/test-evidence/playwright-config/missing-forbid-only.fixture: missing forbidOnly: true", + ); + + const present = checkSource(root, ["--playwright-config", "playwright.config.ts"]); + expect(present.stderr).toBe(""); + expect(present.status).toBe(0); + + // Without the flag the rule reads the repository's configs, every one. + const repository = checkSource(root); + expect(repository.status, repository.stderr).toBe(0); + const report = JSON.parse(await readFile("artifacts/quality/test-evidence-source-rules.json", "utf8")) as { + facts: { playwrightConfigs?: string[] }; + }; + expect(report.facts.playwrightConfigs).toEqual([...selectRepositoryPaths("playwright-configs")]); + expect(report.facts.playwrightConfigs?.length).toBeGreaterThan(0); + }, childProcessTimeout(296)); + it("rejects a skip without a quarantine", () => { const result = checkSource(`${FORBIDDEN}/unowned-skip.fixture`); expect(result.status).toBe(1); diff --git a/tests/visual/scrollbar-gutter.visual.spec.ts b/tests/visual/scrollbar-gutter.visual.spec.ts index 6ad4e93a..0df6d62b 100644 --- a/tests/visual/scrollbar-gutter.visual.spec.ts +++ b/tests/visual/scrollbar-gutter.visual.spec.ts @@ -66,7 +66,9 @@ async function stripColours( new Blob([bytes], { type: "image/png" }), ); const canvas = new OffscreenCanvas(bitmap.width, bitmap.height); - const context = canvas.getContext("2d"); + // Read back once per strip: without the hint Chromium warns on the + // console of this tab, which the strict fixture watches (1112/03-05). + const context = canvas.getContext("2d", { willReadFrequently: true }); if (context === null) throw new Error("no 2d canvas context"); context.drawImage(bitmap, 0, 0); return offsets.map(({ x, width }) => {