fix(round9): 그룹 C 검사기 진단과 시험의 빈틈을 고친다 #55
@@ -221,6 +221,15 @@ namespace 점검을 돌아, 이 조건이 없는 runner에서 이름과 host 설
|
||||
stderr, 위 sysctl 값, 고치는 방법을 적고 exit 1로
|
||||
멈춘다. 전에는 이 검사가 PASS를 낸 뒤 provider 테스트가 bwrap 원문 오류로
|
||||
실패했다(2026-11-05 desktop에서 114건 중 18건).
|
||||
- 같은 검사가 provider의 cgroup 준비도 provider와 같은 호출로 해 본다.
|
||||
`/usr/bin/prlimit`·`/usr/bin/systemd-run`이 실행 가능한지 보고,
|
||||
`systemctl --user show --property=Version --value`로 user manager와 버전(254
|
||||
이상)을 확인한 뒤, provider scope와 같은 속성·같은 cgroup 강제 확인으로
|
||||
`systemd-run --user --scope`에서 `/bin/true`를 한 번 띄운다
|
||||
(`scripts/lib/provider-cgroup.ts`). user bus가 없거나, 버전이 낮거나, memory·pids·cpu
|
||||
controller가 위임되지 않은 호스트는 그 원인을 이름으로 대며 exit 1이다. 전에는
|
||||
system manager bus만 보고 PASS를 낸 뒤 provider 테스트가 하나씩
|
||||
`provider cgroup unavailable: ...`로 실패했다.
|
||||
- 고치는 방법은 `scripts/check-system-test-prerequisites.ts`가 실패할 때
|
||||
출력한다. 호스트의 보안 설정을 바꾸는 일이므로 호스트 소유자가 고른다.
|
||||
호스트를 바꾸는 설정 파일과 명령은 runner를 만드는 쪽의 인프라 repo가 갖는다.
|
||||
@@ -443,7 +452,18 @@ provider 확인이 필요하다고 표시한 것은 모두 `UNVERIFIED`다. `FAI
|
||||
drill은 `FAIL`(exit 1)이다. 그렇지 않고 `UNVERIFIED`가 하나라도 있으면
|
||||
`UNVERIFIED`(exit 3)이고, 게이트는 `PASS`가 아니다. 그래서 provider 증거가 없으면
|
||||
production 단계는 닫히지 않는다. 의도한 결과다. 로컬 drill은 승격한 release를 rollback할
|
||||
수 있다는 것을 보여 줄 수 없다. FE-GATE-016(`rollback-drill`)은 FE-GATE-025와 같은
|
||||
수 있다는 것을 보여 줄 수 없다.
|
||||
|
||||
provider 증거는 `--provider-evidence <파일>`로 넘긴다(0929/11-02). 파일은
|
||||
`providerDrillEvidenceSchema`(`scripts/contracts/release-artifacts.ts`) 꼴이다: runbook ID,
|
||||
release ID, 관찰 시각, provider 쪽 run ID, 그리고 항목마다 `PASS`/`FAIL`과 근거다. 항목
|
||||
이름은 `UNVERIFIED` 줄이 적는 이름(`trigger`, recovery assertion 이름, `provider
|
||||
verification` 등)이다. drill은 runbook과 release가 같고, 관찰이 7일 안이고, 로컬 drill이
|
||||
보지 못한 항목만 답한 증거만 받는다. 받은 증거는 그 항목의 `UNVERIFIED`만 바꾸고, 로컬에서
|
||||
본 `PASS`·`FAIL`은 그대로 둔다. 모든 항목이 답을 얻고 `FAIL`이 없을 때만 drill이 `PASS`다.
|
||||
받지 않은 증거는 이유와 함께 record의 `providerEvidence`에 남고 drill은 `UNVERIFIED`다.
|
||||
템플릿의 production 게이트 명령은 아직 이 인자를 넘기지 않는다. provider를 연결한 제품이
|
||||
증거 파일을 만들고 drill 명령에 넘겨야 그 단계가 닫힌다. FE-GATE-016(`rollback-drill`)은 FE-GATE-025와 같은
|
||||
FE-RB-005 drill을 돌렸으므로 폐기했다(`scripts/contracts/ci-gates.ts`의
|
||||
`RETIRED_GATE_IDS`).
|
||||
|
||||
@@ -482,6 +502,14 @@ gate의 realtime source/fixture 검사는 native SSE/WebSocket/Web Push API가
|
||||
bootstrap/installed-feature 조합을 차단한다. optional recipe source gate는
|
||||
uncomposed runtime 전체를 tree-shaking 없이 합성해 catalog의 gzip 예산도
|
||||
blocking으로 검증한다.
|
||||
같은 gate의 `check:request-fanout`은 `src/features`·`src/application`에서 루프나 배열
|
||||
순회 callback 안의 요청(`fetch()`, 이름이 transport인 receiver의 method)을 거절한다.
|
||||
실패 줄은 두 탈출 방법을 함께 적는다: 기능 `adapters/`에서
|
||||
`createCursorPaginationRuntime`(`src/adapters/query-cache`)으로 페이지를 받거나, 상한이
|
||||
있고 검토한 반복이면 그 호출 줄에 `// fanout-reviewed: <이유>`를 적는다. 결과 줄은
|
||||
주석으로 넘긴 호출의 수와 위치를 적는다. 이름이 다른 receiver(`repository.load()`),
|
||||
루프가 부르는 이름 있는 함수 안의 요청, 재귀, 순회 전에 만든 promise는 보지 못한다
|
||||
(`scripts/check-request-fanout.ts` 머리 주석).
|
||||
`FE-GATE-030`(main 단계)은 reference runtime source·전용 test·catalog metadata를 제거한
|
||||
임시 repository에서도 base typecheck, architecture, test, build가 통과하는지
|
||||
검증해 skeleton의 선택성을 유지한다. browser file/storage와 realtime runtime은
|
||||
|
||||
@@ -130,9 +130,12 @@ probe하지 않고 `UNVERIFIED`로 끝난다. 슬롯을 채우는 방법은
|
||||
|
||||
**주소와 연결**
|
||||
|
||||
1. 앱은 HTTPS origin의 루트(`/`)에서 내보낸다. 인증서가 유효하고, 어느 surface도 그
|
||||
origin 밖으로 redirect하지 않는다. — FE-GATE-019가 잰다. `HOSTING_BASE_URL`은 경로 없는
|
||||
HTTPS 루트 URL이어야 하고, surface마다 최종 origin을 본다.
|
||||
1. 앱은 HTTPS origin의 루트(`/`)나 빌드한 하위 경로(`VITE_ROUTER_BASE_PATH`, 예 `/app/`)에서
|
||||
내보낸다. 인증서가 유효하고, 어느 surface도 그 origin 밖으로 redirect하지 않는다. —
|
||||
FE-GATE-019가 잰다. `HOSTING_BASE_URL`은 release의 기준 URL(루트 또는 `/`로 끝나는 그
|
||||
하위 경로, query·fragment 없음)이다. probe는 surface 경로(`config/hosting/cache-policy.json`)를
|
||||
그 아래에서 요청하고, 후보의 진입 script가 그 경로 아래에 있는지와 surface마다 최종
|
||||
origin을 본다.
|
||||
2. TLS에서 HTTP/2를 맺는다(ALPN `h2`). — FE-GATE-019가 잰다(`transport.alpnProtocols`).
|
||||
|
||||
**surface별 응답** (surface: probe가 요청하는 경로 하나)
|
||||
|
||||
@@ -937,6 +937,48 @@ export default [
|
||||
"react-hooks/rules-of-hooks": "off",
|
||||
},
|
||||
},
|
||||
{
|
||||
// 1112/03-04. A browser spec takes `test` from the strict fixture, which
|
||||
// fails it on console errors, page errors, failed requests and unused
|
||||
// preloads. FE-GATE-008 used to look for the fixture's path as text in
|
||||
// tests/e2e only, so a path in a comment passed and the other browser
|
||||
// folders were not looked at. tests/unit/browser-spec-imports.test.ts
|
||||
// lints a fixture spec in each folder against this entry.
|
||||
files: [`tests/{e2e,browser-capabilities,visual,storybook}/**/*.spec.ts`],
|
||||
rules: {
|
||||
"no-restricted-imports": [
|
||||
"error",
|
||||
{
|
||||
paths: [
|
||||
{
|
||||
name: "@playwright/test",
|
||||
importNames: ["test", "default"],
|
||||
message:
|
||||
"import test from tests/support/browser/strict-browser-test.ts; the strict fixture fails a spec on browser errors (.claude/skills/fe-qa).",
|
||||
},
|
||||
],
|
||||
patterns: [
|
||||
...deprecatedAliasImports,
|
||||
{
|
||||
group: ["**/support/browser/browser-test.ts", "**/support/browser/browser-test"],
|
||||
importNames: ["test"],
|
||||
message:
|
||||
"import test from tests/support/browser/strict-browser-test.ts; the base test does not check browser errors.",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
{
|
||||
// 1112/03-04. The two specs that need the plain test on purpose:
|
||||
// preload-use proves the strict fixture's own record, and chunk-load-error
|
||||
// provokes the 404 a missing chunk gives to classify the engine's wording.
|
||||
files: ["tests/e2e/preload-use.spec.ts", "tests/browser-capabilities/chunk-load-error.spec.ts"],
|
||||
rules: {
|
||||
"no-restricted-imports": ["error", { patterns: deprecatedAliasImports }],
|
||||
},
|
||||
},
|
||||
{
|
||||
files: [
|
||||
`tests/fixtures/browser-file-storage-boundaries/forbidden/**/*.${sourceExtensions}`,
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
createFileSystemDeclarationSource,
|
||||
type TopologyDeclarationSource,
|
||||
} from "./generate-product-feature-index.ts";
|
||||
import { checkoutBuildReadFailure } from "./lib/run-isolation.ts";
|
||||
|
||||
/**
|
||||
* scale-architecture.md §3 / §37. Build-input regression gate for deployable-app isolation.
|
||||
@@ -123,6 +124,13 @@ async function main(): Promise<void> {
|
||||
// without `--require-bundle` the gate checks the generated indexes only, so a
|
||||
// stale inventory from another build can never pass or fail it.
|
||||
const requireBundle = process.argv.includes("--require-bundle");
|
||||
// 1106/12-02. The inventory is read where the checkout keeps it; under
|
||||
// RUN_OUTPUT_ROOT the build wrote elsewhere and this would judge an earlier one.
|
||||
const outputRootFailure = requireBundle ? checkoutBuildReadFailure("scripts/check-app-build-input.ts") : undefined;
|
||||
if (outputRootFailure !== undefined) {
|
||||
process.stderr.write(`${outputRootFailure}\n`);
|
||||
process.exit(1);
|
||||
}
|
||||
const report = await inspectAppBuildInput(process.cwd(), requireBundle ? {} : { bundleInventory: null });
|
||||
if (requireBundle && !report.bundle.checked) {
|
||||
process.stderr.write("app build input: no production module inventory; run the build first\n");
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
validateRecipeCatalog,
|
||||
} from "./lib/optional-recipes.ts";
|
||||
import { assertMatchesJsonSchema } from "./lib/json-schema.ts";
|
||||
import { checkoutBuildReadFailure } from "./lib/run-isolation.ts";
|
||||
import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts";
|
||||
|
||||
type GateViolation = Readonly<{
|
||||
@@ -36,6 +37,15 @@ const catalogPath = argument(
|
||||
);
|
||||
const sourceRoot = argument("--source-root", "src");
|
||||
const distRoot = argument("--dist-root", "dist");
|
||||
// 1106/12-02. Without --dist-root this scans the checkout's dist; under
|
||||
// RUN_OUTPUT_ROOT the build wrote elsewhere, so it would judge an earlier build.
|
||||
const outputRootFailure = process.argv.includes("--dist-root")
|
||||
? undefined
|
||||
: checkoutBuildReadFailure("scripts/check-optional-recipes.ts");
|
||||
if (outputRootFailure !== undefined) {
|
||||
process.stderr.write(`${outputRootFailure}\n`);
|
||||
process.exit(1);
|
||||
}
|
||||
const artifactPath = argument(
|
||||
"--artifact",
|
||||
"artifacts/quality/optional-recipes.json",
|
||||
|
||||
@@ -17,6 +17,16 @@ import { parseAsync } from "@babel/core";
|
||||
* The HTTP runtime itself (src/adapters/http) owns its bounded retry loop and is
|
||||
* out of scope. A reviewed exception is written on the call's line as
|
||||
* `// fanout-reviewed: <reason>`.
|
||||
*
|
||||
* 0929/12-01. A failure names both ways out: fetch page after page through
|
||||
* `createCursorPaginationRuntime` (src/adapters/query-cache) assembled in the
|
||||
* feature's `adapters/`, or mark a bounded, reviewed repetition on the call's
|
||||
* line. Every reviewed exception is counted and listed in the result line.
|
||||
*
|
||||
* What it does not see: a dispatch through a receiver named otherwise
|
||||
* (`repository.load()`, a destructured `get`), a dispatch in a named function
|
||||
* that a loop calls, recursion, and promises built before an iteration and
|
||||
* awaited inside it. Reviews still look for those.
|
||||
*/
|
||||
export const REQUEST_FANOUT_ROOTS = Object.freeze(["src/features", "src/application"]);
|
||||
|
||||
@@ -44,6 +54,14 @@ const FUNCTION_TYPES = new Set(["ArrowFunctionExpression", "FunctionExpression"]
|
||||
type Node = { type: string; loc?: { start: { line: number } } } & Record<string, unknown>;
|
||||
|
||||
export type FanoutViolation = Readonly<{ file: string; line: number; call: string; context: string }>;
|
||||
export type FanoutReviewedException = Readonly<{ file: string; line: number; call: string; reason: string }>;
|
||||
|
||||
const REVIEWED = /fanout-reviewed:\s*(\S.*?)\s*$/u;
|
||||
|
||||
/** How to fix a failure; both ways the source gate accepts. */
|
||||
export const FANOUT_REMEDY =
|
||||
"load one page per request with createCursorPaginationRuntime (src/adapters/query-cache) assembled in the " +
|
||||
"feature's adapters/, or, for a bounded repetition someone reviewed, write `// fanout-reviewed: <reason>` on the call's line";
|
||||
|
||||
function receiverName(node: Node | undefined): string | null {
|
||||
if (!node) return null;
|
||||
@@ -81,6 +99,14 @@ export async function findRequestFanout(
|
||||
file: string,
|
||||
sourceText: string,
|
||||
): Promise<readonly FanoutViolation[]> {
|
||||
return (await inspectRequestFanout(file, sourceText)).violations;
|
||||
}
|
||||
|
||||
/** Per-row dispatches, and the ones a `fanout-reviewed:` comment let through. */
|
||||
export async function inspectRequestFanout(
|
||||
file: string,
|
||||
sourceText: string,
|
||||
): Promise<Readonly<{ violations: readonly FanoutViolation[]; reviewed: readonly FanoutReviewedException[] }>> {
|
||||
const plugins: ("jsx" | ["typescript", { dts: boolean }])[] = [["typescript", { dts: false }]];
|
||||
if (file.endsWith(".tsx")) plugins.push("jsx");
|
||||
const tree = (await parseAsync(sourceText, {
|
||||
@@ -93,6 +119,7 @@ export async function findRequestFanout(
|
||||
if (!tree) throw new Error(`cannot parse ${file}`);
|
||||
const lines = sourceText.split("\n");
|
||||
const violations: FanoutViolation[] = [];
|
||||
const reviewed: FanoutReviewedException[] = [];
|
||||
|
||||
const visit = (node: Node, parent: Node | undefined, repeating: string | null): void => {
|
||||
let context = repeating;
|
||||
@@ -106,9 +133,9 @@ export async function findRequestFanout(
|
||||
if (node.type === "CallExpression" && context) {
|
||||
const call = dispatchDescription(node);
|
||||
const line = node.loc?.start.line ?? 0;
|
||||
if (call && !/fanout-reviewed:\s*\S/u.test(lines[line - 1] ?? "")) {
|
||||
violations.push({ file, line, call, context });
|
||||
}
|
||||
const reason = REVIEWED.exec(lines[line - 1] ?? "")?.[1];
|
||||
if (call && reason !== undefined) reviewed.push({ file, line, call, reason });
|
||||
else if (call) violations.push({ file, line, call, context });
|
||||
}
|
||||
for (const [key, value] of Object.entries(node)) {
|
||||
if (key === "loc" || key === "start" || key === "end" || key === "extra") continue;
|
||||
@@ -121,7 +148,7 @@ export async function findRequestFanout(
|
||||
}
|
||||
};
|
||||
visit(tree, undefined, null);
|
||||
return violations;
|
||||
return { violations, reviewed };
|
||||
}
|
||||
|
||||
async function sourceFiles(directory: string): Promise<string[]> {
|
||||
@@ -143,39 +170,62 @@ async function sourceFiles(directory: string): Promise<string[]> {
|
||||
export async function scanRequestFanout(
|
||||
root: string,
|
||||
roots: readonly string[] = REQUEST_FANOUT_ROOTS,
|
||||
): Promise<Readonly<{ files: number; violations: readonly FanoutViolation[] }>> {
|
||||
): Promise<Readonly<{
|
||||
files: number;
|
||||
violations: readonly FanoutViolation[];
|
||||
reviewed: readonly FanoutReviewedException[];
|
||||
}>> {
|
||||
const files = (await Promise.all(roots.map((entry) => sourceFiles(path.join(root, entry))))).flat();
|
||||
const violations: FanoutViolation[] = [];
|
||||
const reviewed: FanoutReviewedException[] = [];
|
||||
for (const file of files) {
|
||||
violations.push(
|
||||
...(await findRequestFanout(path.relative(root, file), await readFile(file, "utf8"))),
|
||||
);
|
||||
const found = await inspectRequestFanout(path.relative(root, file), await readFile(file, "utf8"));
|
||||
violations.push(...found.violations);
|
||||
reviewed.push(...found.reviewed);
|
||||
}
|
||||
return { files: files.length, violations };
|
||||
return { files: files.length, violations, reviewed };
|
||||
}
|
||||
|
||||
export function formatFanoutViolation(violation: FanoutViolation): string {
|
||||
return `FAIL ${violation.file}:${violation.line} ${violation.call} inside ${violation.context}: ${FANOUT_REMEDY}`;
|
||||
}
|
||||
|
||||
function formatReviewed(reviewed: readonly FanoutReviewedException[]): string {
|
||||
if (reviewed.length === 0) return "0 reviewed exceptions";
|
||||
return `${reviewed.length} reviewed exception${reviewed.length === 1 ? "" : "s"} (` +
|
||||
`${reviewed.map((entry) => `${entry.file}:${entry.line} ${entry.reason}`).join("; ")})`;
|
||||
}
|
||||
|
||||
/**
|
||||
* The forbidden fixture must be rejected; the allowed fixture must pass, with
|
||||
* its reviewed cursor loop counted as a reviewed exception.
|
||||
*/
|
||||
export async function checkRequestFanoutFixtures(root: string): Promise<Readonly<{ ok: boolean; line: string }>> {
|
||||
const forbidden = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/forbidden"]);
|
||||
const allowed = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/allowed"]);
|
||||
const ok = forbidden.violations.length >= forbidden.files && forbidden.files > 0 &&
|
||||
allowed.violations.length === 0 && allowed.reviewed.length > 0;
|
||||
return {
|
||||
ok,
|
||||
line: `request fan-out fixtures: forbidden ${forbidden.violations.length} rejected in ${forbidden.files} files, ` +
|
||||
`allowed ${allowed.violations.length} violations and ${formatReviewed(allowed.reviewed)} ${ok ? "PASS" : "FAIL"}`,
|
||||
};
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const fixture = process.argv.includes("--fixture");
|
||||
const root = process.cwd();
|
||||
if (fixture) {
|
||||
// The forbidden fixture must be rejected; the allowed fixture must pass.
|
||||
const forbidden = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/forbidden"]);
|
||||
const allowed = await scanRequestFanout(root, ["tests/fixtures/architecture/request-fanout/allowed"]);
|
||||
const ok = forbidden.violations.length >= forbidden.files && forbidden.files > 0 && allowed.violations.length === 0;
|
||||
process.stdout.write(
|
||||
`request fan-out fixtures: forbidden ${forbidden.violations.length} rejected in ${forbidden.files} files, allowed ${allowed.violations.length} violations ${ok ? "PASS" : "FAIL"}\n`,
|
||||
);
|
||||
if (!ok) process.exitCode = 1;
|
||||
const result = await checkRequestFanoutFixtures(root);
|
||||
process.stdout.write(`${result.line}\n`);
|
||||
if (!result.ok) process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const result = await scanRequestFanout(root);
|
||||
for (const violation of result.violations) {
|
||||
process.stderr.write(
|
||||
`FAIL ${violation.file}:${violation.line} ${violation.call} inside ${violation.context}\n`,
|
||||
);
|
||||
}
|
||||
for (const violation of result.violations) process.stderr.write(`${formatFanoutViolation(violation)}\n`);
|
||||
process.stdout.write(
|
||||
`request fan-out: ${result.files} files scanned, ${result.violations.length} per-row dispatches ${result.violations.length === 0 ? "PASS" : "FAIL"}\n`,
|
||||
`request fan-out: ${result.files} files scanned, ${result.violations.length} per-row dispatches, ` +
|
||||
`${formatReviewed(result.reviewed)} ${result.violations.length === 0 ? "PASS" : "FAIL"}\n`,
|
||||
);
|
||||
if (result.violations.length > 0) process.exitCode = 1;
|
||||
}
|
||||
|
||||
@@ -1,9 +1,16 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { constants } from "node:fs";
|
||||
import { access, readFile } from "node:fs/promises";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import {
|
||||
formatProviderPrerequisiteUnitName,
|
||||
MINIMUM_PROVIDER_SYSTEMD_VERSION,
|
||||
systemdRunPrerequisiteProbeArguments,
|
||||
USER_MANAGER_VERSION_ARGUMENTS,
|
||||
} from "./lib/provider-cgroup.ts";
|
||||
import { PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS } from "./lib/provider-sandbox-namespaces.ts";
|
||||
|
||||
/** What the check reads from the host; a test passes a fake. */
|
||||
@@ -15,7 +22,9 @@ export type SystemTestHost = Readonly<{
|
||||
command: string,
|
||||
args: readonly string[],
|
||||
timeoutMs: number,
|
||||
): Readonly<{ status: number | null; stderr: string; error?: Error | undefined }>;
|
||||
): Readonly<{ status: number | null; stdout?: string; stderr: string; error?: Error | undefined }>;
|
||||
/** Names the provider-shaped scope the probe starts; a test passes a fixed one. */
|
||||
probeUnitName?: () => string;
|
||||
}>;
|
||||
|
||||
export const nodeSystemTestHost: SystemTestHost = Object.freeze({
|
||||
@@ -23,12 +32,18 @@ export const nodeSystemTestHost: SystemTestHost = Object.freeze({
|
||||
readFile: async (target: string) => readFile(target, "utf8"),
|
||||
spawn: (command: string, args: readonly string[], timeoutMs: number) => {
|
||||
const result = spawnSync(command, [...args], { encoding: "utf8", timeout: timeoutMs });
|
||||
return { status: result.status, stderr: result.stderr ?? "", error: result.error };
|
||||
return { status: result.status, stdout: result.stdout ?? "", stderr: result.stderr ?? "", error: result.error };
|
||||
},
|
||||
probeUnitName: defaultProbeUnitName,
|
||||
});
|
||||
|
||||
function defaultProbeUnitName(): string {
|
||||
return formatProviderPrerequisiteUnitName(process.pid, randomBytes(12).toString("hex"));
|
||||
}
|
||||
|
||||
const BWRAP = "/usr/bin/bwrap";
|
||||
const SYSTEMCTL = "/usr/bin/systemctl";
|
||||
const SYSTEMD_RUN = "/usr/bin/systemd-run";
|
||||
|
||||
/**
|
||||
* 1105/12-01. The provider sandbox's own namespace options on /bin/true. The
|
||||
@@ -50,6 +65,7 @@ const USER_NAMESPACE_SETTINGS = Object.freeze([
|
||||
] as const);
|
||||
|
||||
const PROBE_TIMEOUT_MS = 5_000;
|
||||
const SCOPE_PROBE_TIMEOUT_MS = 15_000;
|
||||
const MAX_REPORTED_STDERR_LINES = 4;
|
||||
|
||||
export async function findSystemTestPrerequisiteFailures(
|
||||
@@ -59,6 +75,8 @@ export async function findSystemTestPrerequisiteFailures(
|
||||
const executable = new Map<string, boolean>();
|
||||
for (const [target, label] of [
|
||||
[BWRAP, "bubblewrap"],
|
||||
["/usr/bin/prlimit", "prlimit"],
|
||||
[SYSTEMD_RUN, "systemd-run"],
|
||||
[SYSTEMCTL, "systemctl"],
|
||||
["/usr/bin/tar", "tar"],
|
||||
] as const) {
|
||||
@@ -80,10 +98,18 @@ export async function findSystemTestPrerequisiteFailures(
|
||||
failures.push("cgroup v2 is required at /sys/fs/cgroup/cgroup.controllers");
|
||||
}
|
||||
|
||||
// 1112/08-03. The provider asks the user manager (not the system manager)
|
||||
// for its version and then starts its sandbox in a `systemd-run --user
|
||||
// --scope` (scripts/run-and-validate-provider.ts). The check makes the same
|
||||
// calls, so a host without a user bus, with an old manager, or without
|
||||
// delegated controllers fails here by name instead of in each provider test.
|
||||
if (executable.get(SYSTEMCTL)) {
|
||||
const probe = host.spawn(SYSTEMCTL, ["show-environment"], 3_000);
|
||||
if (probe.error || probe.status !== 0) {
|
||||
failures.push("a reachable systemd manager bus is required");
|
||||
const userManager = findUserManagerFailure(host);
|
||||
if (userManager !== undefined) {
|
||||
failures.push(userManager);
|
||||
} else if (executable.get(SYSTEMD_RUN)) {
|
||||
const scope = findProviderScopeFailure(host);
|
||||
if (scope !== undefined) failures.push(scope);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -94,6 +120,52 @@ export async function findSystemTestPrerequisiteFailures(
|
||||
return failures;
|
||||
}
|
||||
|
||||
function findUserManagerFailure(host: SystemTestHost): string | undefined {
|
||||
const probe = host.spawn(SYSTEMCTL, USER_MANAGER_VERSION_ARGUMENTS, 3_000);
|
||||
const command = `${SYSTEMCTL} ${USER_MANAGER_VERSION_ARGUMENTS.join(" ")}`;
|
||||
if (probe.error || probe.status !== 0) {
|
||||
return [
|
||||
`a reachable systemd user manager is required (${command} ${outcomeOf(probe)}).`,
|
||||
...stderrLines(probe.stderr, "systemctl"),
|
||||
" The provider runs in a `systemd-run --user --scope`; give the runner user a user",
|
||||
" manager (for example `loginctl enable-linger <user>`) and a user bus in XDG_RUNTIME_DIR.",
|
||||
].join("\n");
|
||||
}
|
||||
const reported = (probe.stdout ?? "").trim();
|
||||
const match = /^([0-9]+)/u.exec(reported);
|
||||
const version = match ? Number(match[1]) : Number.NaN;
|
||||
if (!Number.isSafeInteger(version) || version < MINIMUM_PROVIDER_SYSTEMD_VERSION) {
|
||||
return `systemd user manager ${MINIMUM_PROVIDER_SYSTEMD_VERSION} or newer is required (${command} reported ${JSON.stringify(reported)})`;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function findProviderScopeFailure(host: SystemTestHost): string | undefined {
|
||||
const args = systemdRunPrerequisiteProbeArguments((host.probeUnitName ?? defaultProbeUnitName)());
|
||||
const probe = host.spawn(SYSTEMD_RUN, args, SCOPE_PROBE_TIMEOUT_MS);
|
||||
if (!probe.error && probe.status === 0) return undefined;
|
||||
return [
|
||||
"systemd-run could not start the provider's cgroup scope around /bin/true " +
|
||||
`(${SYSTEMD_RUN} --user --scope ... ${outcomeOf(probe)}).`,
|
||||
...stderrLines(probe.stderr, "systemd-run"),
|
||||
" The scope needs the memory, pids and cpu controllers delegated to the user manager",
|
||||
" (Delegate= in user@.service) and cgroup v2. Runner requirements: docs/operations/ci-quality-gates.md.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function outcomeOf(probe: ReturnType<SystemTestHost["spawn"]>): string {
|
||||
return probe.error ? `did not finish: ${probe.error.message}` : `exited ${String(probe.status)}`;
|
||||
}
|
||||
|
||||
function stderrLines(stderr: string, label: string): string[] {
|
||||
const lines = stderr
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter((line) => line.length > 0)
|
||||
.slice(0, MAX_REPORTED_STDERR_LINES);
|
||||
return lines.length > 0 ? lines.map((line) => ` ${label} stderr: ${line}`) : [` ${label} stderr: (empty)`];
|
||||
}
|
||||
|
||||
/**
|
||||
* 1101/13-03. Whether this host lets the provider sandbox create its
|
||||
* namespaces, and if not, why and what to do. The release provider jobs run
|
||||
@@ -118,19 +190,11 @@ async function namespaceFailure(
|
||||
// Absent on this kernel; the others still say what the host allows.
|
||||
}
|
||||
}
|
||||
const stderr = probe.stderr
|
||||
.split("\n")
|
||||
.map((line) => line.trim())
|
||||
.filter((line) => line.length > 0)
|
||||
.slice(0, MAX_REPORTED_STDERR_LINES);
|
||||
const outcome = probe.error
|
||||
? `did not finish: ${probe.error.message}`
|
||||
: `exited ${String(probe.status)}`;
|
||||
return [
|
||||
"this host does not allow unprivileged user and network namespaces, which the provider " +
|
||||
`sandbox needs (bwrap ${PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS.join(" ")}).`,
|
||||
` probe: ${BWRAP} ${NAMESPACE_PROBE_ARGUMENTS.join(" ")} ${outcome}`,
|
||||
...(stderr.length > 0 ? stderr.map((line) => ` bwrap stderr: ${line}`) : [" bwrap stderr: (empty)"]),
|
||||
` probe: ${BWRAP} ${NAMESPACE_PROBE_ARGUMENTS.join(" ")} ${outcomeOf(probe)}`,
|
||||
...stderrLines(probe.stderr, "bwrap"),
|
||||
...(settings.length > 0 ? [` host settings: ${settings.join(", ")}`] : []),
|
||||
" To fix, do one of these:",
|
||||
" - give /usr/bin/bwrap an AppArmor profile that allows `userns` (Ubuntu 23.10 and later),",
|
||||
|
||||
@@ -64,6 +64,12 @@ const policyPath = argumentValue(
|
||||
"config/testing/test-evidence.json",
|
||||
);
|
||||
const explicitCatalogPath = optionalArgumentValue("--catalog");
|
||||
// 1112/03-03. The forbidOnly rule read only the repository's configs and ran
|
||||
// only in the full check, so no fixture could show it still fires.
|
||||
// `--playwright-config <file>` (repeatable) checks those files in any mode.
|
||||
const explicitPlaywrightConfigs = process.argv.flatMap((argument, index) =>
|
||||
argument === "--playwright-config" && process.argv[index + 1] ? [process.argv[index + 1]!] : [],
|
||||
);
|
||||
const explicitReceiptPath = optionalArgumentValue("--receipt");
|
||||
const fixtureMode = sourceRoot !== "tests";
|
||||
const sourceOnly = process.argv.includes("--source-only");
|
||||
@@ -78,6 +84,7 @@ const facts = {
|
||||
sharedScenarios: 0,
|
||||
declaredScenarioExecutions: 0,
|
||||
executedScenarioExecutions: 0,
|
||||
playwrightConfigs: [] as string[],
|
||||
};
|
||||
|
||||
async function filesBelow(target: string): Promise<string[]> {
|
||||
@@ -365,6 +372,26 @@ if (parsedPolicy) {
|
||||
}
|
||||
}
|
||||
|
||||
async function missingForbidOnly(configs: readonly string[]): Promise<string[]> {
|
||||
const missing: string[] = [];
|
||||
for (const config of configs) {
|
||||
if (!(await readFile(config, "utf8")).includes("forbidOnly: true")) {
|
||||
missing.push(`${config}: missing forbidOnly: true, so a committed test.only passes the gate`);
|
||||
}
|
||||
}
|
||||
return missing;
|
||||
}
|
||||
|
||||
// The repository's configs unless the run names others; every mode but the
|
||||
// scenario check reads them, and the report lists what it read.
|
||||
if (!scenarioOnly) {
|
||||
// playwright.dev.config.ts spreads playwright.config.ts.
|
||||
facts.playwrightConfigs = explicitPlaywrightConfigs.length > 0
|
||||
? explicitPlaywrightConfigs
|
||||
: [...selectRepositoryPaths("playwright-configs")];
|
||||
failures.push(...(await missingForbidOnly(facts.playwrightConfigs)));
|
||||
}
|
||||
|
||||
if (!fixtureMode && !scenarioOnly) {
|
||||
const e2eConfig = await readFile("playwright.config.ts", "utf8");
|
||||
for (const token of [
|
||||
@@ -381,22 +408,9 @@ if (!fixtureMode && !scenarioOnly) {
|
||||
failures.push(`playwright.config.ts missing release evidence token ${token}`);
|
||||
}
|
||||
}
|
||||
// playwright.dev.config.ts spreads playwright.config.ts.
|
||||
for (const config of selectRepositoryPaths("playwright-configs")) {
|
||||
if (!(await readFile(config, "utf8")).includes("forbidOnly: true")) {
|
||||
failures.push(`${config}: missing forbidOnly: true, so a committed test.only passes the gate`);
|
||||
}
|
||||
}
|
||||
|
||||
const e2eFiles = (await filesBelow("tests/e2e")).filter((file) =>
|
||||
/\.spec\.ts$/.test(file),
|
||||
);
|
||||
for (const file of e2eFiles) {
|
||||
const source = await readFile(file, "utf8");
|
||||
if (!source.includes("support/browser/strict-browser-test")) {
|
||||
failures.push(`${file}: bypasses strict browser fixture`);
|
||||
}
|
||||
}
|
||||
// 1112/03-04. That a browser spec takes the strict fixture's `test` is an
|
||||
// ESLint rule now (eslint.config.ts, tests/unit/browser-spec-imports.test.ts),
|
||||
// which reads the import instead of looking for the path as text.
|
||||
|
||||
const baselineFiles = (await filesBelow("tests/visual/__snapshots__")).filter(
|
||||
(file) => file.endsWith(".png"),
|
||||
|
||||
@@ -746,16 +746,52 @@ const drillVerdict = z.enum(["PASS", "FAIL", "UNVERIFIED"]);
|
||||
|
||||
export type DrillVerdict = z.infer<typeof drillVerdict>;
|
||||
|
||||
/** FAIL beats UNVERIFIED beats PASS; provider scope alone keeps a drill UNVERIFIED. */
|
||||
/**
|
||||
* FAIL beats UNVERIFIED beats PASS. A drill with provider scope stays
|
||||
* UNVERIFIED until accepted provider evidence says the provider side passed
|
||||
* (0929/11-02).
|
||||
*/
|
||||
export function combineDrillVerdicts(
|
||||
verdicts: readonly DrillVerdict[],
|
||||
providerVerificationRequired: boolean,
|
||||
providerVerification: DrillVerdict = "UNVERIFIED",
|
||||
): DrillVerdict {
|
||||
if (verdicts.includes("FAIL")) return "FAIL";
|
||||
if (verdicts.includes("UNVERIFIED") || providerVerificationRequired) return "UNVERIFIED";
|
||||
const all = providerVerificationRequired ? [...verdicts, providerVerification] : verdicts;
|
||||
if (all.includes("FAIL")) return "FAIL";
|
||||
if (all.includes("UNVERIFIED")) return "UNVERIFIED";
|
||||
return "PASS";
|
||||
}
|
||||
|
||||
/**
|
||||
* 0929/11-02. What the hosting provider observed for one drill of one
|
||||
* release: the provider-side items a local drill cannot see (a smoke after the
|
||||
* switch, live traffic, the trigger), named as the drill's UNVERIFIED line
|
||||
* names them, plus `provider verification` itself.
|
||||
*/
|
||||
export const providerDrillEvidenceSchema = z
|
||||
.object({
|
||||
schemaVersion: z.literal(1),
|
||||
artifactType: z.literal("provider-drill-evidence"),
|
||||
runbookId: z.string().regex(/^FE-RB-00[1-5]$/u),
|
||||
releaseId: nonEmptyString,
|
||||
observedAt: timestamp,
|
||||
runId: nonEmptyString,
|
||||
observations: z
|
||||
.array(
|
||||
z
|
||||
.object({ item: nonEmptyString, result: z.enum(["PASS", "FAIL"]), evidence: nonEmptyString })
|
||||
.strict(),
|
||||
)
|
||||
.min(1)
|
||||
.max(32)
|
||||
.refine((items) => new Set(items.map(({ item }) => item)).size === items.length, {
|
||||
message: "must name each item once",
|
||||
}),
|
||||
})
|
||||
.strict();
|
||||
|
||||
export type ProviderDrillEvidence = z.infer<typeof providerDrillEvidenceSchema>;
|
||||
|
||||
export const runbookRecordArtifactSchema = z
|
||||
.object({
|
||||
schemaVersion: z.literal(2),
|
||||
@@ -778,6 +814,19 @@ export const runbookRecordArtifactSchema = z
|
||||
negativeFixtureFailedAsExpected: drillVerdict,
|
||||
windowObservedBucket: nonEmptyString,
|
||||
providerVerificationRequired: z.boolean(),
|
||||
providerEvidence: z
|
||||
.object({
|
||||
accepted: z.boolean(),
|
||||
reason: nonEmptyString.optional(),
|
||||
runId: nonEmptyString.optional(),
|
||||
observedAt: timestamp.optional(),
|
||||
providerVerification: drillVerdict,
|
||||
})
|
||||
.strict()
|
||||
.refine((evidence) => evidence.accepted || (evidence.reason !== undefined && evidence.providerVerification === "UNVERIFIED"), {
|
||||
message: "rejected provider evidence must say why and verify nothing",
|
||||
})
|
||||
.optional(),
|
||||
result: drillVerdict,
|
||||
})
|
||||
.strict()
|
||||
@@ -791,6 +840,7 @@ export const runbookRecordArtifactSchema = z
|
||||
record.negativeFixtureFailedAsExpected,
|
||||
],
|
||||
record.providerVerificationRequired,
|
||||
record.providerEvidence?.providerVerification,
|
||||
);
|
||||
if (record.result !== expected) {
|
||||
context.addIssue({
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { mkdir } from "node:fs/promises";
|
||||
|
||||
import { checkoutBuildReadFailure } from "./lib/run-isolation.ts";
|
||||
import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts";
|
||||
import { localEvidenceAssessmentArtifactSchema } from "./contracts/release-artifacts.ts";
|
||||
import { createLocalEvidenceAssessment } from "./lib/local-release-evidence.ts";
|
||||
@@ -10,6 +11,15 @@ import {
|
||||
releaseCandidateManifestSchema,
|
||||
} from "./lib/release-candidate.ts";
|
||||
|
||||
// 1106/12-02. This digests dist and the evidence under artifacts/ where the
|
||||
// checkout keeps them (scripts/lib/release-candidate.ts). Under RUN_OUTPUT_ROOT
|
||||
// the build wrote elsewhere, so the candidate would package an earlier build.
|
||||
const outputRootFailure = checkoutBuildReadFailure("scripts/create-release-candidate.ts");
|
||||
if (outputRootFailure !== undefined) {
|
||||
process.stderr.write(`${outputRootFailure}\n`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const assessment = await createLocalEvidenceAssessment();
|
||||
await mkdir("artifacts/security", { recursive: true });
|
||||
await writeValidatedJsonArtifact({
|
||||
|
||||
+142
-10
@@ -22,8 +22,10 @@ import { projectTelemetryEvent } from "../src/contracts/telemetry.ts";
|
||||
import { EXPECTED_CONTRACT_SET_PACKAGES } from "../src/features/installed-contract-contributions.ts";
|
||||
import {
|
||||
combineDrillVerdicts,
|
||||
providerDrillEvidenceSchema,
|
||||
runbookRecordArtifactSchema,
|
||||
type DrillVerdict,
|
||||
type ProviderDrillEvidence,
|
||||
} from "./contracts/release-artifacts.ts";
|
||||
import { verifyReleaseRuntimeCoherence } from "./lib/release-runtime-coherence.ts";
|
||||
import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts";
|
||||
@@ -87,8 +89,25 @@ export type DrillOptions = Readonly<{
|
||||
*/
|
||||
releaseRoot?: string;
|
||||
contractPackages?: readonly InstalledContractPackageIdentity[];
|
||||
/**
|
||||
* 0929/11-02. The provider's record for this drill (`--provider-evidence`),
|
||||
* parsed JSON. Without it a drill with provider scope cannot pass.
|
||||
*/
|
||||
providerEvidence?: unknown;
|
||||
/** The clock evidence age is judged against; a test passes a fixed one. */
|
||||
now?: Date;
|
||||
}>;
|
||||
|
||||
/** Provider evidence older than this no longer describes the provider as it is. */
|
||||
export const PROVIDER_EVIDENCE_MAX_AGE_MS = 7 * 24 * 60 * 60 * 1_000;
|
||||
/** How far ahead of this clock a provider's timestamp may be. */
|
||||
const PROVIDER_EVIDENCE_CLOCK_SKEW_MS = 5 * 60 * 1_000;
|
||||
/** The item names a drill's UNVERIFIED line uses, and provider evidence answers. */
|
||||
const ITEM_TRIGGER = "trigger";
|
||||
const ITEM_CONTAINMENT = "containment";
|
||||
const ITEM_NEGATIVE_FIXTURE = "negative fixture";
|
||||
export const ITEM_PROVIDER_VERIFICATION = "provider verification";
|
||||
|
||||
type DrillContext = Readonly<{
|
||||
root: string;
|
||||
contractPackages: readonly InstalledContractPackageIdentity[];
|
||||
@@ -490,14 +509,29 @@ export async function runRunbookDrill(runbookId: string, options: DrillOptions =
|
||||
const release = parseReleaseArtifact(
|
||||
await rootedReader(context.root)(DEFAULT_ROLLBACK_PATHS.release),
|
||||
);
|
||||
const result = await drill(context);
|
||||
const local = await drill(context);
|
||||
const now = options.now ?? new Date();
|
||||
const judged = options.providerEvidence === undefined
|
||||
? undefined
|
||||
: judgeProviderEvidence(options.providerEvidence, runbookId, release.releaseId, openItems(local), now);
|
||||
const result = judged?.accepted ? applyProviderEvidence(local, judged.evidence) : local;
|
||||
const providerEvidence = judged === undefined
|
||||
? undefined
|
||||
: judged.accepted
|
||||
? {
|
||||
accepted: true as const,
|
||||
runId: judged.evidence.runId,
|
||||
observedAt: judged.evidence.observedAt,
|
||||
providerVerification: observed(judged.evidence, ITEM_PROVIDER_VERIFICATION) ?? "UNVERIFIED",
|
||||
}
|
||||
: { accepted: false as const, reason: judged.reason, providerVerification: "UNVERIFIED" as const };
|
||||
const escalationPathAsserted = verdict(specification.escalation.length >= 2);
|
||||
return {
|
||||
schemaVersion: 2 as const,
|
||||
runbookId,
|
||||
gateId: specification.gateId,
|
||||
releaseId: release.releaseId,
|
||||
drillTimestamp: new Date().toISOString(),
|
||||
drillTimestamp: now.toISOString(),
|
||||
triggerInjected: specification.triggerKinds[0] ?? "",
|
||||
triggerAsserted: result.triggerAsserted,
|
||||
containmentAsserted: result.containmentAsserted,
|
||||
@@ -506,6 +540,7 @@ export async function runRunbookDrill(runbookId: string, options: DrillOptions =
|
||||
negativeFixtureFailedAsExpected: result.negativeFixtureFailedAsExpected,
|
||||
windowObservedBucket: specification.window,
|
||||
providerVerificationRequired: result.providerVerificationRequired,
|
||||
...(providerEvidence === undefined ? {} : { providerEvidence }),
|
||||
result: combineDrillVerdicts(
|
||||
[
|
||||
result.triggerAsserted,
|
||||
@@ -515,24 +550,117 @@ export async function runRunbookDrill(runbookId: string, options: DrillOptions =
|
||||
result.negativeFixtureFailedAsExpected,
|
||||
],
|
||||
result.providerVerificationRequired,
|
||||
providerEvidence?.providerVerification,
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/** The items the local drill left UNVERIFIED, by the names provider evidence uses. */
|
||||
function openItems(result: DrillResult): string[] {
|
||||
return [
|
||||
...(result.triggerAsserted === "UNVERIFIED" ? [ITEM_TRIGGER] : []),
|
||||
...(result.containmentAsserted === "UNVERIFIED" ? [ITEM_CONTAINMENT] : []),
|
||||
...result.recoveryAssertions.filter((item) => item.result === "UNVERIFIED").map((item) => item.assertion),
|
||||
...(result.negativeFixtureFailedAsExpected === "UNVERIFIED" ? [ITEM_NEGATIVE_FIXTURE] : []),
|
||||
...(result.providerVerificationRequired ? [ITEM_PROVIDER_VERIFICATION] : []),
|
||||
];
|
||||
}
|
||||
|
||||
type JudgedProviderEvidence =
|
||||
| Readonly<{ accepted: true; evidence: ProviderDrillEvidence }>
|
||||
| Readonly<{ accepted: false; reason: string }>;
|
||||
|
||||
/**
|
||||
* 0929/11-02. Provider evidence counts only for this runbook, this release,
|
||||
* a recent observation, and the items the local drill could not observe.
|
||||
*/
|
||||
export function judgeProviderEvidence(
|
||||
value: unknown,
|
||||
runbookId: string,
|
||||
releaseId: string,
|
||||
open: readonly string[],
|
||||
now: Date,
|
||||
): JudgedProviderEvidence {
|
||||
const parsed = providerDrillEvidenceSchema.safeParse(value);
|
||||
if (!parsed.success) {
|
||||
const issue = parsed.error.issues[0];
|
||||
return { accepted: false, reason: `not provider drill evidence (${issue?.path.join(".") || "root"}: ${issue?.message ?? "invalid"})` };
|
||||
}
|
||||
const evidence = parsed.data;
|
||||
if (evidence.runbookId !== runbookId) {
|
||||
return { accepted: false, reason: `evidence is for ${evidence.runbookId}, not ${runbookId}` };
|
||||
}
|
||||
if (evidence.releaseId !== releaseId) {
|
||||
return { accepted: false, reason: `evidence is for release ${evidence.releaseId}, not ${releaseId}` };
|
||||
}
|
||||
const age = now.getTime() - Date.parse(evidence.observedAt);
|
||||
if (age > PROVIDER_EVIDENCE_MAX_AGE_MS) {
|
||||
return { accepted: false, reason: `evidence observed at ${evidence.observedAt} is older than ${PROVIDER_EVIDENCE_MAX_AGE_MS / 86_400_000} days` };
|
||||
}
|
||||
if (age < -PROVIDER_EVIDENCE_CLOCK_SKEW_MS) {
|
||||
return { accepted: false, reason: `evidence observed at ${evidence.observedAt} is in the future` };
|
||||
}
|
||||
const extra = evidence.observations.map(({ item }) => item).filter((item) => !open.includes(item));
|
||||
if (extra.length > 0) {
|
||||
return { accepted: false, reason: `evidence names items the local drill observed or does not have: ${extra.join(", ")}` };
|
||||
}
|
||||
return { accepted: true, evidence };
|
||||
}
|
||||
|
||||
function observed(evidence: ProviderDrillEvidence, item: string): DrillVerdict | undefined {
|
||||
return evidence.observations.find((observation) => observation.item === item)?.result;
|
||||
}
|
||||
|
||||
/** Replaces each UNVERIFIED item the evidence answers; local PASS and FAIL stay. */
|
||||
function applyProviderEvidence(result: DrillResult, evidence: ProviderDrillEvidence): DrillResult {
|
||||
const answer = (item: string, local: DrillVerdict): DrillVerdict =>
|
||||
local === "UNVERIFIED" ? observed(evidence, item) ?? local : local;
|
||||
const source = (item: string) =>
|
||||
`provider evidence ${evidence.runId} at ${evidence.observedAt}: ${
|
||||
evidence.observations.find((observation) => observation.item === item)?.evidence ?? ""
|
||||
}`;
|
||||
return {
|
||||
...result,
|
||||
triggerAsserted: answer(ITEM_TRIGGER, result.triggerAsserted),
|
||||
containmentAsserted: answer(ITEM_CONTAINMENT, result.containmentAsserted),
|
||||
recoveryAssertions: result.recoveryAssertions.map((item) =>
|
||||
item.result === "UNVERIFIED" && observed(evidence, item.assertion) !== undefined
|
||||
? { ...item, evidence: source(item.assertion), result: observed(evidence, item.assertion)! }
|
||||
: item,
|
||||
),
|
||||
negativeFixtureFailedAsExpected: answer(ITEM_NEGATIVE_FIXTURE, result.negativeFixtureFailedAsExpected),
|
||||
};
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const runbookId = process.argv
|
||||
.slice(2)
|
||||
.find((argument) => /^FE-RB-00[1-5]$/.test(argument));
|
||||
const releaseRootFlag = process.argv.indexOf("--release-root");
|
||||
const releaseRoot = releaseRootFlag >= 0 ? process.argv[releaseRootFlag + 1] : undefined;
|
||||
if (!runbookId || (releaseRootFlag >= 0 && !releaseRoot)) {
|
||||
const evidenceFlag = process.argv.indexOf("--provider-evidence");
|
||||
const evidencePath = evidenceFlag >= 0 ? process.argv[evidenceFlag + 1] : undefined;
|
||||
if (!runbookId || (releaseRootFlag >= 0 && !releaseRoot) || (evidenceFlag >= 0 && !evidencePath)) {
|
||||
process.stderr.write(
|
||||
"Usage: drill:runbook -- FE-RB-001..FE-RB-005 [--release-root <extracted release archive>]\n",
|
||||
"Usage: drill:runbook -- FE-RB-001..FE-RB-005 [--release-root <extracted release archive>]" +
|
||||
" [--provider-evidence <provider drill evidence JSON>]\n",
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
let providerEvidence: unknown;
|
||||
if (evidencePath !== undefined) {
|
||||
try {
|
||||
providerEvidence = JSON.parse(await readFile(evidencePath, "utf8"));
|
||||
} catch (error) {
|
||||
process.stderr.write(`${runbookId} drill: cannot read provider evidence ${evidencePath}: ${String(error)}\n`);
|
||||
process.exit(2);
|
||||
}
|
||||
}
|
||||
|
||||
const { gateId, ...record } = await runRunbookDrill(runbookId, releaseRoot ? { releaseRoot } : {});
|
||||
const { gateId, ...record } = await runRunbookDrill(runbookId, {
|
||||
...(releaseRoot ? { releaseRoot } : {}),
|
||||
...(evidencePath !== undefined ? { providerEvidence } : {}),
|
||||
});
|
||||
const artifactDirectory = `artifacts/runbooks/${runbookId}`;
|
||||
await mkdir(artifactDirectory, { recursive: true });
|
||||
await writeValidatedJsonArtifact({
|
||||
@@ -547,14 +675,18 @@ async function main(): Promise<void> {
|
||||
if (record.result === "UNVERIFIED") {
|
||||
// Not a pass: the gate stays red until provider evidence exists (D-14).
|
||||
const open = [
|
||||
["trigger", record.triggerAsserted],
|
||||
["containment", record.containmentAsserted],
|
||||
[ITEM_TRIGGER, record.triggerAsserted],
|
||||
[ITEM_CONTAINMENT, record.containmentAsserted],
|
||||
...record.recoveryAssertions.map((item) => [item.assertion, item.result] as const),
|
||||
["negative fixture", record.negativeFixtureFailedAsExpected],
|
||||
["provider verification", record.providerVerificationRequired ? "UNVERIFIED" : "PASS"],
|
||||
[ITEM_NEGATIVE_FIXTURE, record.negativeFixtureFailedAsExpected],
|
||||
[
|
||||
ITEM_PROVIDER_VERIFICATION,
|
||||
record.providerVerificationRequired ? record.providerEvidence?.providerVerification ?? "UNVERIFIED" : "PASS",
|
||||
],
|
||||
].filter(([, result]) => result === "UNVERIFIED").map(([name]) => name);
|
||||
const rejected = record.providerEvidence?.accepted === false ? `; provider evidence rejected: ${record.providerEvidence.reason}` : "";
|
||||
process.stderr.write(
|
||||
`${runbookId} drill: UNVERIFIED (${gateId}; no local check failed; not observed locally: ${open.join(", ")})\n`,
|
||||
`${runbookId} drill: UNVERIFIED (${gateId}; no local check failed; not observed locally: ${open.join(", ")}${rejected})\n`,
|
||||
);
|
||||
process.exit(3);
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ import {
|
||||
buildDate,
|
||||
} from "./lib/build-environment.ts";
|
||||
import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts";
|
||||
import { CANONICAL_VITE_MANIFEST_PATH } from "./lib/build-manifest-outputs.ts";
|
||||
import { CANONICAL_VITE_MANIFEST_PATH, findRouteChunk } from "./lib/build-manifest-outputs.ts";
|
||||
import { runOutputPath } from "./lib/run-isolation.ts";
|
||||
|
||||
assertCiBuildEnvironment(process.env);
|
||||
@@ -68,14 +68,13 @@ const routeChunks: Record<string, string> = {};
|
||||
const runtimeContracts: Readonly<Record<string, { moduleId: string }>> =
|
||||
ROUTE_RUNTIME_CONTRACT;
|
||||
for (const definition of Object.values(ROUTE_REGISTRY)) {
|
||||
const runtime = runtimeContracts[definition.routeId];
|
||||
const asset = Object.values(viteManifestObject).find(
|
||||
(entry) => entry.name === runtime?.moduleId && entry.isDynamicEntry,
|
||||
const chunk = findRouteChunk(
|
||||
viteManifestObject,
|
||||
definition.routeId,
|
||||
runtimeContracts[definition.routeId]?.moduleId,
|
||||
);
|
||||
if (!runtime || !asset?.file) {
|
||||
throw new Error(`Missing built route chunk: ${definition.routeId}`);
|
||||
}
|
||||
routeChunks[definition.chunkId] = asset.file;
|
||||
if ("error" in chunk) throw new Error(chunk.error);
|
||||
routeChunks[definition.chunkId] = chunk.file;
|
||||
}
|
||||
const runtimeConfigJsonSchema = z.toJSONSchema(runtimeConfigV2ArtifactSchema);
|
||||
|
||||
|
||||
@@ -186,3 +186,52 @@ export async function verifyBuildManifestOutputs(
|
||||
}
|
||||
return mismatches;
|
||||
}
|
||||
|
||||
/**
|
||||
* 0929/13-03. The built chunk of a route's lazy page, for the build manifest
|
||||
* and the two release checks that compare its `routeChunks`. A route runtime's
|
||||
* `moduleId` is the file name of the page it imports
|
||||
* (`src/presentation/routes/route-runtime.tsx`, a generated feature's
|
||||
* `<id>-route-runtime.ts`), so the chunk is the dynamic entry whose source
|
||||
* file — the Vite manifest key — has that name. The chunk's own `name` used to
|
||||
* be matched instead, so a chunk-naming change in vite.config.ts stopped the
|
||||
* build with a reason-less `Missing built route chunk`.
|
||||
*/
|
||||
export type RouteChunkLookup = Readonly<{ file: string }> | Readonly<{ error: string }>;
|
||||
|
||||
const SOURCE_EXTENSION = /\.(?:tsx|ts|jsx|js)$/u;
|
||||
|
||||
export function findRouteChunk(
|
||||
manifest: Readonly<Record<string, unknown>>,
|
||||
routeId: string,
|
||||
moduleId: string | undefined,
|
||||
): RouteChunkLookup {
|
||||
if (moduleId === undefined) {
|
||||
return { error: `Missing built route chunk: ${routeId} has no route runtime contract (ROUTE_RUNTIME_CONTRACT)` };
|
||||
}
|
||||
const matches: { source: string; file: string }[] = [];
|
||||
for (const [key, entry] of Object.entries(manifest)) {
|
||||
if (!isManifestRecord(entry) || entry["isDynamicEntry"] !== true || typeof entry["file"] !== "string") continue;
|
||||
const source = typeof entry["src"] === "string" ? entry["src"] : key;
|
||||
if (path.posix.basename(source).replace(SOURCE_EXTENSION, "") === moduleId) {
|
||||
matches.push({ source, file: entry["file"] });
|
||||
}
|
||||
}
|
||||
if (matches.length === 1) return { file: matches[0]!.file };
|
||||
if (matches.length === 0) {
|
||||
return {
|
||||
error:
|
||||
`Missing built route chunk: ${routeId}: no lazily imported module in the Vite manifest has the source ` +
|
||||
`file ${moduleId}.tsx (or .ts). A route runtime's moduleId must be the file name of the page it imports.`,
|
||||
};
|
||||
}
|
||||
return {
|
||||
error:
|
||||
`Ambiguous built route chunk: ${routeId}: the lazily imported modules ` +
|
||||
`${matches.map(({ source }) => source).join(", ")} are all named ${moduleId}.`,
|
||||
};
|
||||
}
|
||||
|
||||
function isManifestRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
@@ -142,9 +142,9 @@ export async function probeLiveRelease(
|
||||
const negotiateAlpn = transport.negotiateAlpn ?? negotiatedAlpnProtocol;
|
||||
const responses: ResponseHeaders = {};
|
||||
const results: ProbeResult[] = [];
|
||||
const probe = async (surface: string, pathname: string): Promise<string | null> => {
|
||||
const probe = async (surface: string, target: URL): Promise<string | null> => {
|
||||
try {
|
||||
const response = await fetcher(new URL(pathname, base));
|
||||
const response = await fetcher(target);
|
||||
const finalOrigin = new URL(response.url).origin;
|
||||
results.push(
|
||||
{
|
||||
@@ -191,11 +191,12 @@ export async function probeLiveRelease(
|
||||
|
||||
const bodies: Record<string, string | null> = {};
|
||||
for (const [surface, pathname] of Object.entries(paths)) {
|
||||
bodies[surface] = await probe(surface, pathname);
|
||||
bodies[surface] = await probe(surface, surfaceUrl(pathname, base));
|
||||
}
|
||||
const entryScript = servedEntryScript(bodies["index"] ?? "");
|
||||
if (entryScript) {
|
||||
await probe("hashedAsset", entryScript);
|
||||
// The served index.html names it as the build wrote it (under the base path).
|
||||
await probe("hashedAsset", new URL(entryScript, base));
|
||||
} else {
|
||||
results.push({
|
||||
surface: "hashedAsset",
|
||||
@@ -226,6 +227,21 @@ export async function probeLiveRelease(
|
||||
});
|
||||
}
|
||||
const candidateEntryScript = servedEntryScript(candidate.indexHtml);
|
||||
// 0929/13-06. The candidate was built for one base path; a HOSTING_BASE_URL
|
||||
// with another path probes surfaces the release does not serve.
|
||||
// The build writes its scripts to `<base>assets/`, so the entry script's
|
||||
// folder says which base the candidate was built for, in both directions.
|
||||
const builtFor = candidateEntryScript?.startsWith(`${base.pathname}${BUILD_ASSET_DIRECTORY}`) === true;
|
||||
results.push({
|
||||
surface: "deployment",
|
||||
header: "base-path",
|
||||
expected: `${base.pathname}${BUILD_ASSET_DIRECTORY}`,
|
||||
observed: candidateEntryScript,
|
||||
...(builtFor
|
||||
? {}
|
||||
: { reason: "the candidate was built for another base path than the HOSTING_BASE_URL path (VITE_ROUTER_BASE_PATH of the build)" }),
|
||||
passed: builtFor,
|
||||
});
|
||||
results.push({
|
||||
surface: "index",
|
||||
header: "candidate-entry-script",
|
||||
@@ -405,10 +421,19 @@ export function checkPublicServiceWorkers(
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 0929/13-06. The surfaces in config/hosting/cache-policy.json are written
|
||||
* from the release's root (`/config.json`); a release served under a base
|
||||
* path (`VITE_ROUTER_BASE_PATH=/app/`) serves them under that path.
|
||||
*/
|
||||
export function surfaceUrl(pathname: string, base: URL): URL {
|
||||
return new URL(`.${pathname.startsWith("/") ? pathname : `/${pathname}`}`, base);
|
||||
}
|
||||
|
||||
/**
|
||||
* A release gate must not promote a local preview server as live hosting
|
||||
* evidence.
|
||||
*
|
||||
* evidence. 0929/13-06: the URL is the release's base URL, the root or the
|
||||
* sub-path the build was made for (`/app/`).
|
||||
*/
|
||||
export function classifyLiveHostingBaseUrl(value: string):
|
||||
| {
|
||||
@@ -467,10 +492,10 @@ export function classifyLiveHostingBaseUrl(value: string):
|
||||
observedOrigin,
|
||||
};
|
||||
}
|
||||
if (url.pathname !== "/" || url.search || url.hash) {
|
||||
if (!url.pathname.endsWith("/") || url.search || url.hash) {
|
||||
return {
|
||||
passed: false,
|
||||
reason: "HOSTING_BASE_URL must be the canonical root URL",
|
||||
reason: "HOSTING_BASE_URL must be the release's base URL: a path ending in \"/\", without query or fragment",
|
||||
url,
|
||||
observedOrigin,
|
||||
};
|
||||
|
||||
@@ -26,6 +26,7 @@ import {
|
||||
} from "../contracts/release-artifacts.ts";
|
||||
import {
|
||||
CANONICAL_VITE_MANIFEST_PATH,
|
||||
findRouteChunk,
|
||||
verifyBuildManifestOutputs,
|
||||
} from "./build-manifest-outputs.ts";
|
||||
import { assertMatchesJsonSchema } from "./json-schema.ts";
|
||||
@@ -1255,14 +1256,12 @@ async function verifyReleaseOutputs(
|
||||
const runtimeContracts: Readonly<Record<string, { moduleId: string }>> =
|
||||
ROUTE_RUNTIME_CONTRACT;
|
||||
for (const definition of Object.values(ROUTE_REGISTRY)) {
|
||||
const runtime = runtimeContracts[definition.routeId];
|
||||
const viteEntry = Object.values(viteManifest).find(
|
||||
(entry) =>
|
||||
isRecord(entry) &&
|
||||
entry.name === runtime?.moduleId &&
|
||||
entry.isDynamicEntry === true,
|
||||
const chunk = findRouteChunk(
|
||||
viteManifest,
|
||||
definition.routeId,
|
||||
runtimeContracts[definition.routeId]?.moduleId,
|
||||
);
|
||||
const file = isRecord(viteEntry) ? viteEntry.file : null;
|
||||
const file = "error" in chunk ? null : chunk.file;
|
||||
if (
|
||||
typeof file !== "string" ||
|
||||
release.routeChunks[definition.chunkId] !== file ||
|
||||
|
||||
@@ -1,10 +1,24 @@
|
||||
export type ProviderKind = "vulnerability" | "provenance";
|
||||
|
||||
/** The oldest systemd user manager whose `--scope` properties the provider relies on. */
|
||||
export const MINIMUM_PROVIDER_SYSTEMD_VERSION = 254;
|
||||
|
||||
/** `systemctl` arguments that ask the user manager its version; the provider and the prerequisite check both use them. */
|
||||
export const USER_MANAGER_VERSION_ARGUMENTS: readonly string[] = Object.freeze([
|
||||
"--user",
|
||||
"show",
|
||||
"--property=Version",
|
||||
"--value",
|
||||
]);
|
||||
|
||||
const MEMORY_MAX = 1_073_741_824;
|
||||
const TASKS_MAX = 64;
|
||||
const STOP_TIMEOUT_MS = 5_000;
|
||||
const RUNTIME_GRACE_MS = 10_000;
|
||||
const UNIT_NAME = /^ca-provider-(?:vulnerability|provenance)-[1-9][0-9]*-[0-9a-f]{24}\.scope$/u;
|
||||
const PROBE_UNIT_NAME = /^ca-provider-prerequisite-[1-9][0-9]*-[0-9a-f]{24}\.scope$/u;
|
||||
/** How long the prerequisite probe's scope may live; it runs /bin/true. */
|
||||
const PROBE_RUNTIME_MS = 5_000;
|
||||
const UNIT_NONCE = /^[0-9a-f]{24}$/u;
|
||||
const ENVIRONMENT_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/u;
|
||||
const ENFORCEMENT_GATE = [
|
||||
@@ -79,6 +93,37 @@ export function systemdRunProviderArguments(
|
||||
) {
|
||||
throw new TypeError("provider scope wrapper path is invalid");
|
||||
}
|
||||
return [
|
||||
...scopeArguments(unit, timeoutMs + RUNTIME_GRACE_MS),
|
||||
nodeExecutable,
|
||||
wrapperScript,
|
||||
String(cpuSeconds),
|
||||
reportPath,
|
||||
String(reportDev),
|
||||
String(reportIno),
|
||||
];
|
||||
}
|
||||
|
||||
export function formatProviderPrerequisiteUnitName(supervisorPid: number, nonce: string): string {
|
||||
if (!Number.isSafeInteger(supervisorPid) || supervisorPid <= 0 || !UNIT_NONCE.test(nonce)) {
|
||||
throw new TypeError("provider cgroup unit identity is invalid");
|
||||
}
|
||||
return `ca-provider-prerequisite-${supervisorPid}-${nonce}.scope`;
|
||||
}
|
||||
|
||||
/**
|
||||
* 1112/08-03. The provider's own scope — same properties, same check that the
|
||||
* unit's cgroup enforces them — around /bin/true. A host whose user manager
|
||||
* cannot start the scope, or does not delegate the memory, pids and cpu
|
||||
* controllers, fails here instead of in each provider test.
|
||||
*/
|
||||
export function systemdRunPrerequisiteProbeArguments(unit: string): string[] {
|
||||
if (!PROBE_UNIT_NAME.test(unit)) throw new TypeError("provider cgroup unit is invalid");
|
||||
return [...scopeArguments(unit, PROBE_RUNTIME_MS), "/bin/true"];
|
||||
}
|
||||
|
||||
/** `systemd-run` options and the enforcement gate up to its `$0`; the caller appends what the gate execs. */
|
||||
function scopeArguments(unit: string, runtimeMaxMs: number): string[] {
|
||||
return [
|
||||
"--user",
|
||||
"--scope",
|
||||
@@ -94,19 +139,13 @@ export function systemdRunProviderArguments(
|
||||
"--property=KillMode=control-group",
|
||||
"--property=SendSIGKILL=yes",
|
||||
`--property=TimeoutStopSec=${STOP_TIMEOUT_MS}ms`,
|
||||
`--property=RuntimeMaxSec=${timeoutMs + RUNTIME_GRACE_MS}ms`,
|
||||
`--property=RuntimeMaxSec=${runtimeMaxMs}ms`,
|
||||
"--",
|
||||
"/bin/sh",
|
||||
"-eu",
|
||||
"-c",
|
||||
ENFORCEMENT_GATE,
|
||||
unit,
|
||||
nodeExecutable,
|
||||
wrapperScript,
|
||||
String(cpuSeconds),
|
||||
reportPath,
|
||||
String(reportDev),
|
||||
String(reportIno),
|
||||
];
|
||||
}
|
||||
|
||||
|
||||
@@ -12,6 +12,8 @@ export const testEvidenceReportSchema = z
|
||||
sharedScenarios: z.number().int().nonnegative(),
|
||||
declaredScenarioExecutions: z.number().int().nonnegative(),
|
||||
executedScenarioExecutions: z.number().int().nonnegative(),
|
||||
// 1112/03-03. The Playwright configs the forbidOnly rule read.
|
||||
playwrightConfigs: z.array(z.string().min(1)).optional(),
|
||||
})
|
||||
.strict(),
|
||||
failures: z.array(z.string()),
|
||||
|
||||
@@ -20,8 +20,10 @@ import {
|
||||
encodeProviderBwrapInput,
|
||||
encodeProviderScopeFrame,
|
||||
formatProviderCgroupUnitName,
|
||||
MINIMUM_PROVIDER_SYSTEMD_VERSION,
|
||||
systemctlKillProviderArguments,
|
||||
systemdRunProviderArguments,
|
||||
USER_MANAGER_VERSION_ARGUMENTS,
|
||||
} from "./lib/provider-cgroup.ts";
|
||||
import {
|
||||
assertProviderGuardianLeasePaths,
|
||||
@@ -49,7 +51,6 @@ const PROVIDER_REAP_TIMEOUT_MS = 5_000;
|
||||
const PROVIDER_CONTROL_TIMEOUT_MS = 5_000;
|
||||
const PROVIDER_CONTROL_MAX_OUTPUT_BYTES = 65_536;
|
||||
const PROVIDER_CONTROL_POLL_MS = 25;
|
||||
const MINIMUM_PROVIDER_SYSTEMD_VERSION = 254;
|
||||
if (kind !== "vulnerability" && kind !== "provenance") {
|
||||
process.stderr.write("Usage: run-and-validate-provider --kind vulnerability|provenance\n");
|
||||
process.exit(2);
|
||||
@@ -501,7 +502,7 @@ async function waitForProviderClose(
|
||||
|
||||
async function assertProviderCgroupManagerAvailable(): Promise<void> {
|
||||
const output = await runBoundedSystemctl(
|
||||
["--user", "show", "--property=Version", "--value"],
|
||||
USER_MANAGER_VERSION_ARGUMENTS,
|
||||
"provider cgroup user manager probe",
|
||||
);
|
||||
const match = /^([0-9]+)/u.exec(output.trim());
|
||||
|
||||
@@ -304,16 +304,44 @@ function linkedPath(target: string, document: string): string | null {
|
||||
// never looked up. Leading `NAME=value` assignments are skipped, a
|
||||
// `--flag=value` argument is read as its value, and placeholders (`<id>`,
|
||||
// globs, `$VAR`) are left alone by namedPath.
|
||||
const COMMAND_HEADS = new Set(["corepack", "pnpm", "npx", "node", "vitest", "playwright", "tsc", "eslint", "bash", "sh"]);
|
||||
function commandPaths(command: string, document: string): (string | null)[] {
|
||||
const COMMAND_HEADS = new Set([
|
||||
"corepack", "pnpm", "npx", "node", "vitest", "playwright", "tsc", "eslint", "bash", "sh", "python3",
|
||||
]);
|
||||
// 1112/08-06. `scripts/verify.sh` is this repository's main entry and most
|
||||
// documented commands go through it: the entry is itself a path, `exec --`
|
||||
// runs a command whose paths are looked up like any other, and `script`
|
||||
// names package scripts (verifyScripts).
|
||||
const VERIFY_ENTRY = /^(?:\.\/)?scripts\/verify\.sh$/u;
|
||||
function commandWords(command: string): Readonly<{ words: string[]; head: number }> {
|
||||
const words = command.trim().replace(/^\$\s+/u, "").split(/\s+/u);
|
||||
let head = 0;
|
||||
while (/^[A-Z_][A-Z0-9_]*=/u.test(words[head] ?? "")) head += 1;
|
||||
if (!COMMAND_HEADS.has(words[head] ?? "")) return [];
|
||||
return { words, head };
|
||||
}
|
||||
function commandPaths(command: string, document: string): (string | null)[] {
|
||||
const { words, head } = commandWords(command);
|
||||
const first = words[head] ?? "";
|
||||
if (VERIFY_ENTRY.test(first)) {
|
||||
const separator = words.indexOf("--", head + 1);
|
||||
const executed = words[head + 1] === "exec" && separator > head ? words.slice(separator + 1).join(" ") : "";
|
||||
return [namedPath(first, document), ...(executed === "" ? [] : commandPaths(executed, document))];
|
||||
}
|
||||
if (!COMMAND_HEADS.has(first)) return [];
|
||||
return words
|
||||
.slice(head + 1)
|
||||
.map((word) => namedPath(word.replace(/^--?[\w.-]+=/u, "").replace(/^["']|["']$/gu, ""), document));
|
||||
}
|
||||
/** The package scripts `scripts/verify.sh script <name>… [-- args]` runs; placeholders are left alone. */
|
||||
function verifyScripts(command: string): string[] {
|
||||
const { words, head } = commandWords(command);
|
||||
if (!VERIFY_ENTRY.test(words[head] ?? "") || words[head + 1] !== "script") return [];
|
||||
const names: string[] = [];
|
||||
for (const word of words.slice(head + 2)) {
|
||||
if (word.startsWith("-")) break;
|
||||
names.push(word);
|
||||
}
|
||||
return names.filter((name) => /^[a-z](?:[\w:-]*\w)?$/u.test(name));
|
||||
}
|
||||
// Lines of fenced shell blocks (```bash, ```sh, ```shell, ```console or no
|
||||
// language), which is where most documented commands are.
|
||||
function fencedCommands(text: string): string[] {
|
||||
@@ -398,6 +426,10 @@ for (const document of contractDocuments) {
|
||||
for (const [, name = ""] of text.matchAll(/`([a-z][\w-]*:[\w:-]*\w)(?=[` ])/gu)) {
|
||||
if (scriptPrefixes.has(name.split(":")[0]) && missingScript(name)) missingScripts.add(name);
|
||||
}
|
||||
const commands = [...[...text.matchAll(/`([^`\n]+)`/gu)].map(([, token = ""]) => token), ...fencedCommands(text)];
|
||||
for (const name of commands.flatMap(verifyScripts)) {
|
||||
if (missingScript(name)) missingScripts.add(name);
|
||||
}
|
||||
documentReferences.push({
|
||||
path: document,
|
||||
missingPaths: [...missingPaths].sort(),
|
||||
|
||||
@@ -121,7 +121,7 @@ if (liveTarget?.passed && !slotEmpty) {
|
||||
probeResults.push({
|
||||
surface: "deployment",
|
||||
header: "base-url",
|
||||
expected: "canonical non-loopback HTTPS root URL",
|
||||
expected: "non-loopback HTTPS base URL of the release",
|
||||
observed: liveTarget.observedOrigin,
|
||||
reason: liveTarget.reason,
|
||||
passed: false,
|
||||
|
||||
@@ -24,6 +24,7 @@ import {
|
||||
import { assertMatchesJsonSchema } from "./lib/json-schema.ts";
|
||||
import {
|
||||
CANONICAL_VITE_MANIFEST_PATH,
|
||||
findRouteChunk,
|
||||
verifyBuildManifestOutputs,
|
||||
} from "./lib/build-manifest-outputs.ts";
|
||||
import { verifyReleaseRuntimeCoherence } from "./lib/release-runtime-coherence.ts";
|
||||
@@ -168,12 +169,13 @@ for (const chunkId of actualChunkIds) {
|
||||
const runtimeContracts: Readonly<Record<string, { moduleId: string }>> =
|
||||
ROUTE_RUNTIME_CONTRACT;
|
||||
for (const definition of Object.values(ROUTE_REGISTRY)) {
|
||||
const runtime = runtimeContracts[definition.routeId];
|
||||
const viteEntry = Object.values(viteManifestObject).find(
|
||||
(entry) => entry.name === runtime?.moduleId && entry.isDynamicEntry,
|
||||
const viteEntry = findRouteChunk(
|
||||
viteManifestObject,
|
||||
definition.routeId,
|
||||
runtimeContracts[definition.routeId]?.moduleId,
|
||||
);
|
||||
const routeAsset = release.routeChunks[definition.chunkId];
|
||||
if (!runtime || !viteEntry || routeAsset !== viteEntry.file) {
|
||||
if ("error" in viteEntry || routeAsset !== viteEntry.file) {
|
||||
artifactMismatches.push(`routeChunk:mismatch:${definition.chunkId}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -4,10 +4,18 @@ import path from "node:path";
|
||||
|
||||
import { reproducibleBuildArtifactSchema } from "./contracts/release-artifacts.ts";
|
||||
import { assertCiBuildEnvironment } from "./lib/build-environment.ts";
|
||||
import { checkoutBuildReadFailure } from "./lib/run-isolation.ts";
|
||||
import { supplyChainDigest } from "./lib/supply-chain.ts";
|
||||
import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts";
|
||||
|
||||
assertCiBuildEnvironment(process.env);
|
||||
// 1106/12-02. This builds twice and digests dist where the checkout keeps it. Under RUN_OUTPUT_ROOT the build wrote elsewhere, so
|
||||
// this would judge the earlier build the checkout still holds.
|
||||
const outputRootFailure = checkoutBuildReadFailure("scripts/verify-reproducible-build.ts");
|
||||
if (outputRootFailure !== undefined) {
|
||||
process.stderr.write(`${outputRootFailure}\n`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
async function filesWithin(directory: string): Promise<string[]> {
|
||||
const entries = await readdir(directory, { withFileTypes: true });
|
||||
|
||||
@@ -3,7 +3,9 @@ import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
checkRequestFanoutFixtures,
|
||||
findRequestFanout,
|
||||
formatFanoutViolation,
|
||||
scanRequestFanout,
|
||||
} from "../../scripts/check-request-fanout.ts";
|
||||
|
||||
@@ -21,6 +23,13 @@ describe("no per-row transport loop (frontend N+1 source gate)", () => {
|
||||
expect(new Set(forbidden.violations.map((violation) => violation.file)).size).toBe(forbidden.files);
|
||||
expect(forbidden.files).toBe(3);
|
||||
expect(allowed.violations).toEqual([]);
|
||||
expect(allowed.reviewed).toEqual([
|
||||
expect.objectContaining({
|
||||
file: "tests/fixtures/architecture/request-fanout/allowed/reviewed-cursor-loop.ts",
|
||||
call: "gateway.page()",
|
||||
reason: "cursor pages, bounded by maxPages",
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it("names the dispatch and the repeating construct", async () => {
|
||||
@@ -40,4 +49,24 @@ describe("no per-row transport loop (frontend N+1 source gate)", () => {
|
||||
);
|
||||
expect(violations).toEqual([]);
|
||||
});
|
||||
|
||||
// 0929/12-01. The failure line named only the call, so a product team saw
|
||||
// neither the cursor runtime nor the reviewed-exception comment, and the
|
||||
// result line hid how many calls a comment let through.
|
||||
it("tells the author both ways out on the failure line", async () => {
|
||||
const [violation] = await findRequestFanout(
|
||||
"inline.ts",
|
||||
"export const load = (api: { get(id: string): Promise<unknown> }, ids: string[]) =>\n Promise.all(ids.map((id) => api.get(id)));\n",
|
||||
);
|
||||
const line = formatFanoutViolation(violation!);
|
||||
expect(line).toMatch(/^FAIL inline\.ts:2 api\.get\(\) inside \.map\(\) callback: /u);
|
||||
expect(line).toContain("createCursorPaginationRuntime (src/adapters/query-cache)");
|
||||
expect(line).toContain("`// fanout-reviewed: <reason>` on the call's line");
|
||||
});
|
||||
|
||||
it("counts and lists the reviewed exceptions in the fixture result line", async () => {
|
||||
const result = await checkRequestFanoutFixtures(path.resolve("."));
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.line).toMatch(/allowed 0 violations and 1 reviewed exception \(tests\/fixtures\/architecture\/request-fanout\/allowed\/reviewed-cursor-loop\.ts:\d+ cursor pages, bounded by maxPages\) PASS/u);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -337,6 +337,7 @@ async function removeOpfsAuthority(
|
||||
for (const committedObjects of [0, 3]) {
|
||||
test(`05-01 an approved purge in another tab waits for a live put and takes it with the partition (${committedObjects} committed objects)`, async ({
|
||||
page,
|
||||
browserName,
|
||||
}) => {
|
||||
const secondPage = await page.context().newPage();
|
||||
const suffix = crypto.randomUUID().replaceAll("-", "");
|
||||
@@ -349,7 +350,18 @@ for (const committedObjects of [0, 3]) {
|
||||
type Tab = import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts").BrowserOpfsRuntime;
|
||||
try {
|
||||
await Promise.all([page.goto("/"), secondPage.goto("/")]);
|
||||
if (!(await installOpfsTab(page, scope))) return;
|
||||
// 1112/03-02. An early return here passed with no assertion on an engine
|
||||
// the support check refused, while the evidence said zero skipped.
|
||||
// Playwright's WebKit is refused (the first test asserts its UNSUPPORTED
|
||||
// result), so there this asserts the refusal; every other engine must
|
||||
// open the runtime. A skip is no way out: the capability evidence
|
||||
// refuses skipped cases (scripts/verify-browser-capability-evidence.ts).
|
||||
const installed = await installOpfsTab(page, scope);
|
||||
if (browserName === "webkit") {
|
||||
expect(installed, "the OPFS support check refuses Playwright's WebKit").toBe(false);
|
||||
return;
|
||||
}
|
||||
expect(installed, "OPFS runtime supported in this engine").toBe(true);
|
||||
expect(await installOpfsTab(secondPage, scope, true)).toBe(true);
|
||||
|
||||
const seeded = await secondPage.evaluate(async (count) => {
|
||||
@@ -455,6 +467,7 @@ for (const committedObjects of [0, 3]) {
|
||||
|
||||
test("05-01 a reconcile in another tab skips a live put and the put stays readable", async ({
|
||||
page,
|
||||
browserName,
|
||||
}) => {
|
||||
const secondPage = await page.context().newPage();
|
||||
const suffix = crypto.randomUUID().replaceAll("-", "");
|
||||
@@ -467,7 +480,13 @@ test("05-01 a reconcile in another tab skips a live put and the put stays readab
|
||||
type Tab = import("../../src/adapters/storage/opfs/browser-opfs-runtime.ts").BrowserOpfsRuntime;
|
||||
try {
|
||||
await Promise.all([page.goto("/"), secondPage.goto("/")]);
|
||||
if (!(await installOpfsTab(page, scope))) return;
|
||||
// 1112/03-02. Asserted, not returned early: see the purge test above.
|
||||
const installed = await installOpfsTab(page, scope);
|
||||
if (browserName === "webkit") {
|
||||
expect(installed, "the OPFS support check refuses Playwright's WebKit").toBe(false);
|
||||
return;
|
||||
}
|
||||
expect(installed, "OPFS runtime supported in this engine").toBe(true);
|
||||
expect(await installOpfsTab(secondPage, scope)).toBe(true);
|
||||
|
||||
// Tab A starts a put whose source stalls, so its journal row stays
|
||||
|
||||
@@ -298,6 +298,25 @@ recordedTest.describe("the strict fixture's console check", () => {
|
||||
);
|
||||
});
|
||||
|
||||
// 1112/03-05. The fixture watched its own page only, so a console error in
|
||||
// a second tab of the same context (the cross-tab capability specs) passed.
|
||||
recordedTest.describe("the strict fixture's other tabs", () => {
|
||||
recordedTest.use({
|
||||
fixtureReports: { lines: [/^console:error:\[preload-use\] second tab$/u] },
|
||||
});
|
||||
|
||||
recordedTest(
|
||||
"records a console error in another tab of the context",
|
||||
async ({ page }) => {
|
||||
await openHome(page);
|
||||
const second = await page.context().newPage();
|
||||
await openHome(second);
|
||||
|
||||
await second.evaluate(() => console.error("[preload-use] second tab"));
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
/**
|
||||
* 1105/11-01. The verdict every other spec gets: a recorded failure fails the
|
||||
* test. The record is proven above; this proves the verdict.
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import path from "node:path";
|
||||
|
||||
import { expect, test, type Page } from "../support/browser/strict-browser-test.ts";
|
||||
import {
|
||||
expect,
|
||||
test,
|
||||
watchBrowserFailures,
|
||||
type Page,
|
||||
} from "../support/browser/strict-browser-test.ts";
|
||||
import {
|
||||
prepareServiceWorkerBuilds,
|
||||
removeServiceWorkerBuilds,
|
||||
@@ -91,7 +96,7 @@ test("a REMOVE_REGISTRATION release unregisters a worker that blocks boot", asyn
|
||||
const server = await startBuildServer(builds.dist);
|
||||
// The broken phase fails requests on purpose, so this test drives its own
|
||||
// context instead of the strict `page` fixture and checks the recovered
|
||||
// load for browser errors itself.
|
||||
// load for browser errors with the fixture's own watcher (1112/03-05).
|
||||
const context = await browser.newContext();
|
||||
try {
|
||||
const page = await context.newPage();
|
||||
@@ -133,22 +138,16 @@ test("a REMOVE_REGISTRATION release unregisters a worker that blocks boot", asyn
|
||||
)
|
||||
.toBe(true);
|
||||
|
||||
const failures: string[] = [];
|
||||
page.on("console", (message) => {
|
||||
if (["error", "warning"].includes(message.type())) {
|
||||
failures.push(`console:${message.type()}:${message.text()}`);
|
||||
}
|
||||
});
|
||||
page.on("pageerror", (error) => failures.push(`pageerror:${error.name}`));
|
||||
page.on("requestfailed", (request) =>
|
||||
failures.push(`requestfailed:${new URL(request.url()).pathname}`),
|
||||
);
|
||||
// No preload check here: this context loaded the origin twice already,
|
||||
// and WebKit answers the recovered load's scripts from its cache, past
|
||||
// the tracker's route, so every preload reads as unused (16 in a run).
|
||||
const recovered = await watchBrowserFailures(page);
|
||||
await page.goto(`${server.origin}/`);
|
||||
expect(await startOutcome(page)).toEqual({ kind: "DISABLED" });
|
||||
expect(
|
||||
await page.evaluate(() => navigator.serviceWorker.controller === null),
|
||||
).toBe(true);
|
||||
expect(failures).toEqual([]);
|
||||
expect(await recovered.stop()).toEqual([]);
|
||||
} finally {
|
||||
await context.close();
|
||||
await server.close();
|
||||
|
||||
@@ -65,17 +65,41 @@ function run(
|
||||
}
|
||||
}
|
||||
|
||||
export async function prepareFixtureBuilds<Name extends string>(
|
||||
prefix: string,
|
||||
builds: readonly FixtureBuild<Name>[],
|
||||
): Promise<FixtureBuilds<Name>> {
|
||||
// An edit's source text names the build (two specs never share a prefix).
|
||||
const fingerprint = JSON.stringify(
|
||||
builds.map((build) => [build.name, build.env ?? {}, build.edit?.toString() ?? ""]),
|
||||
);
|
||||
export type EditedCopyBuild<Name extends string> = Readonly<{
|
||||
name: Name;
|
||||
/** VITE_BUILD_ID and RELEASE_ID of this build. */
|
||||
buildId: string;
|
||||
edit?: (root: string) => Promise<void>;
|
||||
env?: Readonly<Record<string, string>>;
|
||||
}>;
|
||||
|
||||
/**
|
||||
* 0929/11-07. The one way an e2e spec gets real production builds of an
|
||||
* edited copy of the checkout; prepareFixtureBuilds and
|
||||
* prepareServiceWorkerBuilds (service-worker-builds.ts) are its two shapes.
|
||||
* It copies the checkout (without `checkout-copy-skip`), links node_modules,
|
||||
* runs `prepare` once, then for each build runs `edit`, builds, moves `dist`
|
||||
* to `dist-<name>` and asks `collect` what the spec needs from that build.
|
||||
* `key` and `fingerprint` name the result in the e2e run
|
||||
* (shared-fixture-builds.ts), which removes the copy when preparing fails.
|
||||
*/
|
||||
export async function buildEditedCopies<Name extends string, Collected = null>(
|
||||
input: Readonly<{
|
||||
key: string;
|
||||
fingerprint: string;
|
||||
prepare?: (root: string) => Promise<void>;
|
||||
builds: readonly EditedCopyBuild<Name>[];
|
||||
collect?: (name: Name, root: string) => Promise<Collected>;
|
||||
}>,
|
||||
): Promise<Readonly<{
|
||||
root: string;
|
||||
dist: Readonly<Record<Name, string>>;
|
||||
collected: Readonly<Record<Name, Collected>>;
|
||||
shared: boolean;
|
||||
}>> {
|
||||
const { root, value, shared } = await sharedFixtureBuild(
|
||||
`fixture-${prefix}`,
|
||||
fingerprint,
|
||||
input.key,
|
||||
input.fingerprint,
|
||||
async (root) => {
|
||||
await cp(repoRoot, root, {
|
||||
recursive: true,
|
||||
@@ -83,19 +107,36 @@ export async function prepareFixtureBuilds<Name extends string>(
|
||||
!skipped.has(path.relative(repoRoot, source).split(path.sep)[0]!),
|
||||
});
|
||||
await linkFixtureNodeModules(root, repoRoot);
|
||||
await input.prepare?.(root);
|
||||
const dist = {} as Record<Name, string>;
|
||||
for (const build of builds) {
|
||||
const collected = {} as Record<Name, Collected>;
|
||||
for (const build of input.builds) {
|
||||
await build.edit?.(root);
|
||||
const buildId = `${prefix}-${build.name}`;
|
||||
run(root, "scripts/generate-product-feature-index.ts", buildId, build.env);
|
||||
run(root, "scripts/build-frontend.ts", buildId, build.env);
|
||||
run(root, "scripts/generate-product-feature-index.ts", build.buildId, build.env);
|
||||
run(root, "scripts/build-frontend.ts", build.buildId, build.env);
|
||||
dist[build.name] = path.join(root, `dist-${build.name}`);
|
||||
await rename(path.join(root, "dist"), dist[build.name]);
|
||||
collected[build.name] = input.collect ? await input.collect(build.name, root) : (null as Collected);
|
||||
}
|
||||
return dist;
|
||||
return { dist, collected };
|
||||
},
|
||||
);
|
||||
return { root, dist: value, shared };
|
||||
return { root, ...value, shared };
|
||||
}
|
||||
|
||||
export async function prepareFixtureBuilds<Name extends string>(
|
||||
prefix: string,
|
||||
builds: readonly FixtureBuild<Name>[],
|
||||
): Promise<FixtureBuilds<Name>> {
|
||||
// An edit's source text names the build (two specs never share a prefix).
|
||||
const { root, dist, shared } = await buildEditedCopies({
|
||||
key: `fixture-${prefix}`,
|
||||
fingerprint: JSON.stringify(
|
||||
builds.map((build) => [build.name, build.env ?? {}, build.edit?.toString() ?? ""]),
|
||||
),
|
||||
builds: builds.map((build) => ({ ...build, buildId: `${prefix}-${build.name}` })),
|
||||
});
|
||||
return { root, dist, shared };
|
||||
}
|
||||
|
||||
export async function removeFixtureBuilds(
|
||||
|
||||
@@ -1,12 +1,5 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { existsSync } from "node:fs";
|
||||
import {
|
||||
cp,
|
||||
readFile,
|
||||
rename,
|
||||
rm,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { createServer, type Server } from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import path from "node:path";
|
||||
@@ -14,11 +7,8 @@ import { pathToFileURL } from "node:url";
|
||||
import { gzipSync } from "node:zlib";
|
||||
|
||||
import { SERVICE_WORKER_ASSETS_OUTPUT } from "../../../scripts/generate-service-worker-assets.ts";
|
||||
import { localBuildEnvironment } from "../../../scripts/lib/build-environment.ts";
|
||||
import { linkFixtureNodeModules } from "../../../scripts/lib/fixture-node-modules.ts";
|
||||
import { selectRepositoryPaths } from "../../../scripts/lib/repository-layout.ts";
|
||||
import type { StaticAssetManifestV1 } from "../../../src/contracts/service-worker.ts";
|
||||
import { sharedFixtureBuild } from "./shared-fixture-builds.ts";
|
||||
import { buildEditedCopies, editOnce } from "./fixture-builds.ts";
|
||||
|
||||
/**
|
||||
* R3-6 test helper: two real production builds with the Service Worker on,
|
||||
@@ -54,11 +44,6 @@ import { sharedFixtureBuild } from "./shared-fixture-builds.ts";
|
||||
* text it expected.
|
||||
*/
|
||||
|
||||
const repoRoot = path.resolve(import.meta.dirname, "../../..");
|
||||
// 1105/12-04. `.tmp` holds other runs' builds and results (.tmp/runs/<id>),
|
||||
// which change while this copy is made; fixture-builds.ts skips it too.
|
||||
const skipped = new Set(selectRepositoryPaths("checkout-copy-skip"));
|
||||
|
||||
export type BuildName = "a" | "b";
|
||||
export type ServiceWorkerBuilds = Readonly<{
|
||||
root: string;
|
||||
@@ -101,17 +86,8 @@ export function createServiceWorkerRuntimeHost(
|
||||
}
|
||||
`;
|
||||
|
||||
export async function replaceOnce(
|
||||
file: string,
|
||||
search: string,
|
||||
replacement: string,
|
||||
): Promise<void> {
|
||||
const source = await readFile(file, "utf8");
|
||||
if (source.split(search).length !== 2) {
|
||||
throw new Error(`Expected exactly one "${search}" in ${file}.`);
|
||||
}
|
||||
await writeFile(file, source.replace(search, replacement));
|
||||
}
|
||||
/** 0929/11-07. The same one-occurrence edit as fixture-builds.ts, not a copy of it. */
|
||||
export const replaceOnce = editOnce;
|
||||
|
||||
/**
|
||||
* Premise 1. Replaces the value of the one `SERVICE_WORKER` declaration in
|
||||
@@ -167,25 +143,12 @@ async function precachedUrls(root: string, name: BuildName): Promise<string[]> {
|
||||
return SERVICE_WORKER_ASSETS.assets.map((asset) => asset.url).sort();
|
||||
}
|
||||
|
||||
function run(cwd: string, script: string, buildId: string): void {
|
||||
const result = spawnSync("node", [script], {
|
||||
cwd,
|
||||
encoding: "utf8",
|
||||
// 1103/11-08. A local build, not the CI job's release build.
|
||||
env: { ...localBuildEnvironment(), VITE_BUILD_ID: buildId, RELEASE_ID: buildId },
|
||||
});
|
||||
if (result.status !== 0) {
|
||||
throw new Error(
|
||||
`${script} failed for ${buildId}:\n${result.stdout}\n${result.stderr}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* `beforeBuild` edits the temp copy right before build `name` (06-03: a
|
||||
* broken worker A, then a `REMOVE_REGISTRATION` release B). Edits persist
|
||||
* into the later build. `key` names this pair of builds in the e2e run
|
||||
* (shared-fixture-builds.ts); each spec passes its own.
|
||||
* (shared-fixture-builds.ts); each spec passes its own. The copy and the
|
||||
* builds are fixture-builds.ts's `buildEditedCopies` (0929/11-07).
|
||||
*/
|
||||
export async function prepareServiceWorkerBuilds(
|
||||
options: Readonly<{
|
||||
@@ -193,38 +156,27 @@ export async function prepareServiceWorkerBuilds(
|
||||
beforeBuild?: (name: BuildName, root: string) => Promise<void>;
|
||||
}>,
|
||||
): Promise<ServiceWorkerBuilds> {
|
||||
const { root, value, shared } = await sharedFixtureBuild(
|
||||
`sw-${options.key}`,
|
||||
options.beforeBuild?.toString() ?? "",
|
||||
async (root) => {
|
||||
await cp(repoRoot, root, {
|
||||
recursive: true,
|
||||
filter: (source) =>
|
||||
!skipped.has(path.relative(repoRoot, source).split(path.sep)[0]!),
|
||||
});
|
||||
await linkFixtureNodeModules(root, repoRoot);
|
||||
const { root, dist, collected, shared } = await buildEditedCopies({
|
||||
key: `sw-${options.key}`,
|
||||
fingerprint: options.beforeBuild?.toString() ?? "",
|
||||
prepare: async (root) => {
|
||||
await turnOnServiceWorker(root);
|
||||
const hostFile = path.join(root, "src/bootstrap/register-service-worker.ts");
|
||||
await replaceOnce(
|
||||
await editOnce(
|
||||
hostFile,
|
||||
"export function createServiceWorkerRuntimeHost(",
|
||||
"function createServiceWorkerRuntimeHostUnobserved(",
|
||||
);
|
||||
await writeFile(hostFile, (await readFile(hostFile, "utf8")) + HOST_PROBE);
|
||||
|
||||
const dist = { a: path.join(root, "dist-a"), b: path.join(root, "dist-b") };
|
||||
const precached: Record<BuildName, readonly string[]> = { a: [], b: [] };
|
||||
for (const name of ["a", "b"] as const) {
|
||||
await options.beforeBuild?.(name, root);
|
||||
run(root, "scripts/generate-product-feature-index.ts", `sw-e2e-${name}`);
|
||||
run(root, "scripts/build-frontend.ts", `sw-e2e-${name}`);
|
||||
await rename(path.join(root, "dist"), dist[name]);
|
||||
precached[name] = await precachedUrls(root, name);
|
||||
}
|
||||
return { dist, precached };
|
||||
},
|
||||
);
|
||||
return { root, ...value, shared };
|
||||
builds: (["a", "b"] as const).map((name) => ({
|
||||
name,
|
||||
buildId: `sw-e2e-${name}`,
|
||||
edit: async (root: string) => options.beforeBuild?.(name, root),
|
||||
})),
|
||||
collect: (name, root) => precachedUrls(root, name),
|
||||
});
|
||||
return { root, dist, precached: collected, shared };
|
||||
}
|
||||
|
||||
export async function removeServiceWorkerBuilds(
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
// Allowed: a cursor loop someone reviewed. Each page needs the cursor the
|
||||
// previous page returned, so the requests cannot be one bulk read; the count
|
||||
// is bounded by the page cap. Feature code normally gets this from
|
||||
// createCursorPaginationRuntime (src/adapters/query-cache).
|
||||
type Page = Readonly<{ items: readonly string[]; nextCursor: string | null }>;
|
||||
type Gateway = { page(cursor: string | null): Promise<Page> };
|
||||
|
||||
export async function loadAll(gateway: Gateway, maxPages: number) {
|
||||
const items: string[] = [];
|
||||
let cursor: string | null = null;
|
||||
for (let page = 0; page < maxPages; page += 1) {
|
||||
const next: Page = await gateway.page(cursor); // fanout-reviewed: cursor pages, bounded by maxPages
|
||||
items.push(...next.items);
|
||||
if (next.nextCursor === null) break;
|
||||
cursor = next.nextCursor;
|
||||
}
|
||||
return items;
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
// A Playwright config that would let a committed focused test run alone:
|
||||
// check-test-evidence.ts --playwright-config must reject it (1112/03-03).
|
||||
import { defineConfig } from "@playwright/test";
|
||||
|
||||
export default defineConfig({
|
||||
testDir: "tests/e2e",
|
||||
reporter: [["junit", { outputFile: "artifacts/tests/e2e/results.xml" }]],
|
||||
});
|
||||
@@ -20,6 +20,48 @@ function ignoredConsole(message: ConsoleMessage) {
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* 1112/03-05. Records a page's browser failures as `console:…`, `pageerror:…`
|
||||
* and `requestfailed:…` lines, with the same rules the strict fixture judges
|
||||
* by. The fixture watches its `page` and every other tab opened in the same
|
||||
* context; a spec that drives a context of its own watches its pages with
|
||||
* this. `preloads` also checks that every modulepreload ran (preload-use.ts)
|
||||
* in the documents open when `stop` runs; attach it before the navigation
|
||||
* that loads them.
|
||||
*/
|
||||
export async function watchBrowserFailures(
|
||||
page: Page,
|
||||
options: Readonly<{ preloads?: boolean }> = {},
|
||||
): Promise<Readonly<{ stop(): Promise<string[]> }>> {
|
||||
const failures: string[] = [];
|
||||
const onConsole = (message: ConsoleMessage) => {
|
||||
if (["error", "warning"].includes(message.type()) && !ignoredConsole(message)) {
|
||||
failures.push(`console:${message.type()}:${message.text()}`);
|
||||
}
|
||||
};
|
||||
const onPageError = (error: Error) => {
|
||||
failures.push(`pageerror:${error.name}`);
|
||||
};
|
||||
const onRequestFailed = (request: import("@playwright/test").Request) => {
|
||||
failures.push(
|
||||
`requestfailed:${request.method()}:${new URL(request.url()).pathname}:${request.failure()?.errorText ?? "unknown"}`,
|
||||
);
|
||||
};
|
||||
page.on("console", onConsole);
|
||||
page.on("pageerror", onPageError);
|
||||
page.on("requestfailed", onRequestFailed);
|
||||
const preloads = options.preloads ? await trackPreloadUse(page) : null;
|
||||
return Object.freeze({
|
||||
async stop() {
|
||||
if (preloads) failures.push(...(await preloads.check()).failures);
|
||||
page.off("console", onConsole);
|
||||
page.off("pageerror", onPageError);
|
||||
page.off("requestfailed", onRequestFailed);
|
||||
return [...failures];
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* 1105/11-01. What the strict `page` fixture does with every failure it
|
||||
* recorded, once the test has ended.
|
||||
@@ -62,32 +104,19 @@ export const test = base.extend<{
|
||||
);
|
||||
},
|
||||
page: async ({ page, browserFailureVerdict }, use) => {
|
||||
const preloads = await trackPreloadUse(page);
|
||||
const failures: string[] = [];
|
||||
const onConsole = (message: ConsoleMessage) => {
|
||||
if (
|
||||
["error", "warning"].includes(message.type()) &&
|
||||
!ignoredConsole(message)
|
||||
) {
|
||||
failures.push(`console:${message.type()}:${message.text()}`);
|
||||
}
|
||||
const watched = [await watchBrowserFailures(page, { preloads: true })];
|
||||
// 1112/03-05. A second tab of the same context (the cross-tab specs)
|
||||
// went unwatched. Its listeners are attached as the tab opens, before
|
||||
// the spec can navigate it; preload use is checked on the first page only.
|
||||
const pending: Promise<void>[] = [];
|
||||
const onPage = (opened: Page) => {
|
||||
pending.push(watchBrowserFailures(opened).then((watcher) => void watched.push(watcher)));
|
||||
};
|
||||
const onPageError = (error: Error) => {
|
||||
failures.push(`pageerror:${error.name}`);
|
||||
};
|
||||
const onRequestFailed = (request: import("@playwright/test").Request) => {
|
||||
failures.push(
|
||||
`requestfailed:${request.method()}:${new URL(request.url()).pathname}:${request.failure()?.errorText ?? "unknown"}`,
|
||||
);
|
||||
};
|
||||
page.on("console", onConsole);
|
||||
page.on("pageerror", onPageError);
|
||||
page.on("requestfailed", onRequestFailed);
|
||||
page.context().on("page", onPage);
|
||||
await use(page);
|
||||
failures.push(...(await preloads.check()).failures);
|
||||
page.off("console", onConsole);
|
||||
page.off("pageerror", onPageError);
|
||||
page.off("requestfailed", onRequestFailed);
|
||||
page.context().off("page", onPage);
|
||||
await Promise.all(pending);
|
||||
const failures = (await Promise.all(watched.map((watcher) => watcher.stop()))).flat();
|
||||
browserFailureVerdict.judge(failures);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -377,6 +377,19 @@ describe("RPC-RR-04 transport results and frames are exactly decoded", () => {
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* 1101/10-07. The system clock with every wait capped at 30 ms. Without a
|
||||
* positive receipt the stream cleanup waits out its 1 s bound; capped, a test
|
||||
* passes that bound instead of sleeping through it. The operations' own 25 ms
|
||||
* deadlines are below the cap and unchanged. Every stream test that ends
|
||||
* without a positive receipt takes it, so none sleeps a real second.
|
||||
*/
|
||||
const cappedClock: ClockPort = Object.freeze({
|
||||
now: () => systemClock.now(),
|
||||
sleep: (milliseconds: number, signal?: AbortSignal) =>
|
||||
systemClock.sleep(Math.min(milliseconds, 30), signal),
|
||||
});
|
||||
|
||||
/**
|
||||
* RPC-RR-01. A bare `AsyncIterable` gave the runtime no way to cancel the
|
||||
* physical stream or to learn when it actually closed, so a timed-out call left
|
||||
@@ -395,6 +408,7 @@ describe("RPC-RR-01 server stream leases and the DRAINING fence", () => {
|
||||
mappers: MAPPERS,
|
||||
requestEncoders: { RpcResourceStreamRequestEncoder: STREAM_ENCODER },
|
||||
transports: { CONNECT_REFERENCE_STREAM: probe.transport },
|
||||
clock: cappedClock,
|
||||
});
|
||||
|
||||
const results = await collect(
|
||||
@@ -418,6 +432,7 @@ describe("RPC-RR-01 server stream leases and the DRAINING fence", () => {
|
||||
mappers: MAPPERS,
|
||||
requestEncoders: { RpcResourceStreamRequestEncoder: STREAM_ENCODER },
|
||||
transports: { CONNECT_REFERENCE_STREAM: probe.transport },
|
||||
clock: cappedClock,
|
||||
});
|
||||
const stream = runtime.bindServerStream(
|
||||
"WATCH_RPC_RESOURCES",
|
||||
@@ -541,18 +556,6 @@ describe("RPC-01 only a positive receipt confirms physical closure", () => {
|
||||
{ label: "never settles", waitClosed: () => new Promise<void>(() => {}) },
|
||||
];
|
||||
|
||||
/**
|
||||
* 1101/10-07. The system clock with every wait capped at 30 ms. Without a
|
||||
* positive receipt the stream cleanup waits out its 1 s bound; capped, the
|
||||
* test passes that bound instead of sleeping through it. The operation's own
|
||||
* 25 ms deadlines are below the cap and unchanged.
|
||||
*/
|
||||
const cappedClock: ClockPort = Object.freeze({
|
||||
now: () => systemClock.now(),
|
||||
sleep: (milliseconds: number, signal?: AbortSignal) =>
|
||||
systemClock.sleep(Math.min(milliseconds, 30), signal),
|
||||
});
|
||||
|
||||
for (const { label, waitClosed } of negativeReceipts) {
|
||||
it(`keeps the operation DRAINING when waitClosed ${label}`, async () => {
|
||||
const probe = leaseProbeWith(waitClosed);
|
||||
@@ -598,6 +601,7 @@ describe("RPC-01 only a positive receipt confirms physical closure", () => {
|
||||
mappers: MAPPERS,
|
||||
requestEncoders: { RpcResourceStreamRequestEncoder: STREAM_ENCODER },
|
||||
transports: { CONNECT_REFERENCE_STREAM: probe.transport },
|
||||
clock: cappedClock,
|
||||
});
|
||||
const stream = runtime.bindServerStream(
|
||||
"WATCH_RPC_RESOURCES",
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
import path from "node:path";
|
||||
|
||||
import { ESLint } from "eslint";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { childProcessTimeout } from "../helpers/child-process-timeout.ts";
|
||||
|
||||
const FOLDERS = ["e2e", "browser-capabilities", "visual", "storybook"] as const;
|
||||
|
||||
const BARE = 'import { expect, test } from "@playwright/test";\n\ntest("probe", async () => { expect(1).toBe(1); });\n';
|
||||
const STRICT =
|
||||
'import { expect, test } from "../support/browser/strict-browser-test.ts";\n' +
|
||||
'import type { Page } from "@playwright/test";\n\n' +
|
||||
'test("probe", async ({ page }: { page: Page }) => { expect(page).toBeTruthy(); });\n';
|
||||
const DEFAULT_IMPORT = 'import test from "@playwright/test";\n\ntest("probe", async () => {});\n';
|
||||
const BASE_TEST = 'import { test } from "../support/browser/browser-test.ts";\n\ntest("probe", async () => {});\n';
|
||||
|
||||
async function restrictedImports(code: string, file: string): Promise<string[]> {
|
||||
const linter = new ESLint({ cwd: process.cwd() });
|
||||
const [result] = await linter.lintText(code, { filePath: path.join(process.cwd(), file), warnIgnored: true });
|
||||
return (result?.messages ?? [])
|
||||
.filter((message) => message.ruleId === "no-restricted-imports")
|
||||
.map((message) => message.message);
|
||||
}
|
||||
|
||||
// 1112/03-04. Strict fixture use was a text search for its path in
|
||||
// tests/e2e/*.spec.ts: a path in a comment passed, a file could mix in the
|
||||
// bare test, and tests/browser-capabilities, tests/visual and tests/storybook
|
||||
// were not looked at. It is an import rule in each of the four folders now.
|
||||
describe("browser specs take test from the strict fixture (1112/03-04)", () => {
|
||||
it.each(FOLDERS)("rejects the bare Playwright test in tests/%s", async (folder) => {
|
||||
const messages = await restrictedImports(BARE, `tests/${folder}/lint-probe.spec.ts`);
|
||||
expect(messages).toEqual([expect.stringContaining("tests/support/browser/strict-browser-test.ts")]);
|
||||
}, childProcessTimeout(683));
|
||||
|
||||
it("rejects the default import, the base test the strict fixture extends, and accepts the strict test", async () => {
|
||||
expect(await restrictedImports(DEFAULT_IMPORT, "tests/visual/lint-probe.spec.ts")).toEqual([
|
||||
expect.stringContaining("tests/support/browser/strict-browser-test.ts"),
|
||||
]);
|
||||
expect(await restrictedImports(BASE_TEST, "tests/e2e/lint-probe.spec.ts")).toEqual([
|
||||
expect.stringContaining("the base test does not check browser errors"),
|
||||
]);
|
||||
expect(await restrictedImports(STRICT, "tests/browser-capabilities/lint-probe.spec.ts")).toEqual([]);
|
||||
}, childProcessTimeout(683));
|
||||
|
||||
it("lets only the two specs that need the plain test import it", async () => {
|
||||
for (const file of ["tests/e2e/preload-use.spec.ts", "tests/browser-capabilities/chunk-load-error.spec.ts"]) {
|
||||
expect(await restrictedImports(BARE, file), file).toEqual([]);
|
||||
}
|
||||
}, childProcessTimeout(683));
|
||||
});
|
||||
@@ -250,11 +250,29 @@ describe("verify:documentation commands and sibling links (1103/12-05)", () => {
|
||||
const result = await verify({
|
||||
[CI_DOC]: (text) => `${text}\n\`corepack pnpm exec vitest run tests/unit/documentation-readiness.test.ts -t "x"\`\n` +
|
||||
"```bash\ncorepack pnpm drill:runbook -- FE-RB-005 --release-root <release dir>\n```\n" +
|
||||
"Any script is run as `node scripts/…`.\n",
|
||||
"Any script is run as `node scripts/…`.\n" +
|
||||
"`scripts/verify.sh exec -- node scripts/run-vitest.ts run tests/unit/documentation-readiness.test.ts`, " +
|
||||
"`scripts/verify.sh script verify:documentation lint -- --x`, `scripts/verify.sh script <이름>…`, " +
|
||||
"`python3 scripts/check-work-docs.py --history <경로>`\n",
|
||||
});
|
||||
expect(result.status, result.stderr).toBe(0);
|
||||
}, VERIFY_TIMEOUT);
|
||||
|
||||
// 1112/08-06. Commands through this repository's entry point, and python3
|
||||
// commands, were not read, so a stale path or script name in them passed.
|
||||
it("reads commands run through scripts/verify.sh and python3", async () => {
|
||||
const result = await verify({
|
||||
[CI_DOC]: (text) => `${text}\n\`scripts/verify.sh exec -- node scripts/no-such.ts\`\n` +
|
||||
"```bash\nVERIFY_SNAPSHOT=1 scripts/verify.sh exec -- node tests/unit/no-such-shell.test.ts\n```\n" +
|
||||
"`python3 scripts/no-such.py --history docs`\n`scripts/verify.sh script no:such-script -- --flag`\n",
|
||||
});
|
||||
expect(result.status).toBe(1);
|
||||
expect(result.stderr).toContain(
|
||||
`${CI_DOC} names paths that do not exist: scripts/no-such.py, scripts/no-such.ts, tests/unit/no-such-shell.test.ts`,
|
||||
);
|
||||
expect(result.stderr).toContain(`${CI_DOC} names package scripts that do not exist: no:such-script`);
|
||||
}, VERIFY_TIMEOUT);
|
||||
|
||||
it("checks a link or a path into a paired repository checked out beside this one", async () => {
|
||||
const naming = (target: string) => (text: string) =>
|
||||
`${text}\n[contract](../../../${target}) and \`${target}\`\n`;
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
import { readFile, readdir } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
const SUPPORT = "tests/e2e/support";
|
||||
|
||||
// 0929/11-07. The Service Worker helper kept its own copy of the checkout
|
||||
// copy, the build runner and the one-occurrence edit, so a fix to one (the
|
||||
// local build env of 1103/11-08, the copy skip list of 1105/12-04) had to be
|
||||
// made twice. Both helpers now go through fixture-builds.ts.
|
||||
describe("e2e build helpers have one copy-and-build path (0929/11-07)", () => {
|
||||
it("builds Service Worker fixtures through buildEditedCopies", async () => {
|
||||
const source = await readFile(path.join(SUPPORT, "service-worker-builds.ts"), "utf8");
|
||||
|
||||
expect(source).toMatch(/import \{[^}]*\bbuildEditedCopies\b[^}]*\} from "\.\/fixture-builds\.ts";/u);
|
||||
expect(source).toContain("buildEditedCopies({");
|
||||
for (const copy of ["spawnSync", "sharedFixtureBuild", "linkFixtureNodeModules", "await cp(", "function run("]) {
|
||||
expect(source, copy).not.toContain(copy);
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps the build runner and the one-occurrence edit in fixture-builds.ts only", async () => {
|
||||
const files = (await readdir(SUPPORT)).filter((file) => file.endsWith(".ts"));
|
||||
const defining = async (pattern: RegExp) => {
|
||||
const found: string[] = [];
|
||||
for (const file of files) {
|
||||
if (pattern.test(await readFile(path.join(SUPPORT, file), "utf8"))) found.push(file);
|
||||
}
|
||||
return found;
|
||||
};
|
||||
|
||||
expect(await defining(/^function run\(/mu)).toEqual(["fixture-builds.ts"]);
|
||||
expect(await defining(/split\(search\)\.length !== 2/u)).toEqual(["fixture-builds.ts"]);
|
||||
});
|
||||
});
|
||||
@@ -73,11 +73,18 @@ describe("live hosting evidence target", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// 0929/13-06. A release built for VITE_ROUTER_BASE_PATH=/app/ is served
|
||||
// under /app/; the probe refused any URL but the root, so it could not be used.
|
||||
it("accepts the sub-path a release is built for", () => {
|
||||
expect(classifyLiveHostingBaseUrl("https://frontend.example.test/app/")).toMatchObject({ passed: true });
|
||||
});
|
||||
|
||||
it.each([
|
||||
["http://frontend.example.test/", "requires HTTPS"],
|
||||
["https://localhost:4173/", "not live deployment evidence"],
|
||||
["https://127.0.0.1/", "not live deployment evidence"],
|
||||
["https://frontend.example.test/app/", "canonical root URL"],
|
||||
["https://frontend.example.test/app", "path ending in \"/\""],
|
||||
["https://frontend.example.test/?preview=1", "without query or fragment"],
|
||||
["https://user:secret@frontend.example.test/", "must not contain credentials"],
|
||||
])("rejects %s", (url, reason) => {
|
||||
expect(classifyLiveHostingBaseUrl(url)).toMatchObject({
|
||||
@@ -193,6 +200,62 @@ describe("SPA fallback and worker script surfaces (1101/13-04)", () => {
|
||||
expect(await failed({ ...RELEASE, "/__probe/deep-link": [200, INDEX] })).toEqual([]);
|
||||
});
|
||||
|
||||
// 0929/13-06. Under a base path the surfaces are requested below it.
|
||||
it("probes a release built for /app/ below /app/ and passes it", async () => {
|
||||
const appIndex = '<script type="module" src="/app/assets/index-A1.js"></script>';
|
||||
const requested: string[] = [];
|
||||
const routes: Record<string, [number, string]> = {
|
||||
"/app/": [200, appIndex],
|
||||
"/app/config.json": [200, CONFIG],
|
||||
"/app/release-manifest.json": [200, JSON.stringify({ schemaVersion: 2, ...IDENTITY })],
|
||||
"/app/__probe/deep-link": [200, appIndex],
|
||||
"/app/assets/index-A1.js": [200, ""],
|
||||
};
|
||||
const { results } = await probeLiveRelease(
|
||||
new URL("https://frontend.example.test/app/"),
|
||||
PATHS,
|
||||
{ ...CANDIDATE, indexHtml: appIndex },
|
||||
{
|
||||
fetcher: async (url) => {
|
||||
requested.push(url.pathname);
|
||||
return host(routes)(url);
|
||||
},
|
||||
negotiateAlpn: async () => "h2",
|
||||
alpnProtocols: ["h2"],
|
||||
},
|
||||
);
|
||||
|
||||
expect(requested).toEqual([
|
||||
"/app/",
|
||||
"/app/config.json",
|
||||
"/app/release-manifest.json",
|
||||
"/app/__probe/deep-link",
|
||||
"/app/assets/index-A1.js",
|
||||
]);
|
||||
expect(results.filter((result) => !result.passed)).toEqual([]);
|
||||
});
|
||||
|
||||
it("fails a HOSTING_BASE_URL path the candidate was not built for", async () => {
|
||||
expect(
|
||||
(await probeLiveRelease(new URL("https://frontend.example.test/app/"), PATHS, CANDIDATE, {
|
||||
fetcher: host({}),
|
||||
negotiateAlpn: async () => "h2",
|
||||
alpnProtocols: ["h2"],
|
||||
})).results.find((result) => result.header === "base-path"),
|
||||
).toMatchObject({ expected: "/app/assets/", observed: "/assets/index-A1.js", passed: false });
|
||||
// And the other way: a release built for /app/ probed at the root.
|
||||
expect(
|
||||
(await probeLiveRelease(base, PATHS, {
|
||||
...CANDIDATE,
|
||||
indexHtml: '<script type="module" src="/app/assets/index-A1.js"></script>',
|
||||
}, {
|
||||
fetcher: host({}),
|
||||
negotiateAlpn: async () => "h2",
|
||||
alpnProtocols: ["h2"],
|
||||
})).results.find((result) => result.header === "base-path"),
|
||||
).toMatchObject({ expected: "/assets/", observed: "/app/assets/index-A1.js", passed: false });
|
||||
});
|
||||
|
||||
it("fails a host without the fallback", async () => {
|
||||
expect(await failed(RELEASE)).toEqual(["appRoute http-status", "appRoute entry-script"]);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { findRouteChunk } from "../../scripts/lib/build-manifest-outputs.ts";
|
||||
|
||||
// 0929/13-03. The route chunk was found by the chunk's `name`, so naming route
|
||||
// chunks in vite.config.ts (the way tech-log did) stopped the build with
|
||||
// `Missing built route chunk: APP_HOME` and no reason. It is found by the
|
||||
// page's source file now, which is what the route runtime's moduleId names.
|
||||
describe("route chunk lookup (0929/13-03)", () => {
|
||||
const manifest = {
|
||||
"index.html": { file: "assets/index-a.js", isEntry: true },
|
||||
"src/presentation/pages/home-page.tsx": {
|
||||
file: "assets/route-home-page-b.js",
|
||||
name: "route-home-page",
|
||||
src: "src/presentation/pages/home-page.tsx",
|
||||
isDynamicEntry: true,
|
||||
},
|
||||
"src/features/order-feature/presentation/order-feature-list-page.tsx": {
|
||||
file: "assets/order-feature-list-page-c.js",
|
||||
name: "order-feature-list-page",
|
||||
isDynamicEntry: true,
|
||||
},
|
||||
"_home-page-shared.js": { file: "assets/home-page-d.js", name: "home-page" },
|
||||
};
|
||||
|
||||
it("finds the chunk by the page's source file even when the chunk has another name", () => {
|
||||
expect(findRouteChunk(manifest, "APP_HOME", "home-page")).toEqual({ file: "assets/route-home-page-b.js" });
|
||||
expect(findRouteChunk(manifest, "ORDER_FEATURE_LIST", "order-feature-list-page")).toEqual({
|
||||
file: "assets/order-feature-list-page-c.js",
|
||||
});
|
||||
});
|
||||
|
||||
it("names the reason and the rule when no lazily imported page has that file name", () => {
|
||||
const lookup = findRouteChunk(manifest, "NOT_FOUND", "not-found-page");
|
||||
expect(lookup).toEqual({
|
||||
error:
|
||||
"Missing built route chunk: NOT_FOUND: no lazily imported module in the Vite manifest has the source " +
|
||||
"file not-found-page.tsx (or .ts). A route runtime's moduleId must be the file name of the page it imports.",
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses two lazily imported pages with the same file name", () => {
|
||||
const lookup = findRouteChunk(
|
||||
{ ...manifest, "src/other/home-page.tsx": { file: "assets/home-page-e.js", isDynamicEntry: true } },
|
||||
"APP_HOME",
|
||||
"home-page",
|
||||
);
|
||||
expect(lookup).toEqual({
|
||||
error:
|
||||
"Ambiguous built route chunk: APP_HOME: the lazily imported modules " +
|
||||
"src/presentation/pages/home-page.tsx, src/other/home-page.tsx are all named home-page.",
|
||||
});
|
||||
});
|
||||
|
||||
it("names a route without a runtime contract", () => {
|
||||
expect(findRouteChunk(manifest, "GHOST", undefined)).toEqual({
|
||||
error: "Missing built route chunk: GHOST has no route runtime contract (ROUTE_RUNTIME_CONTRACT)",
|
||||
});
|
||||
});
|
||||
|
||||
it("is the one lookup the manifest writer and both release checks use", async () => {
|
||||
for (const file of [
|
||||
"scripts/generate-build-manifest.ts",
|
||||
"scripts/verify-release.ts",
|
||||
"scripts/lib/local-release-evidence.ts",
|
||||
]) {
|
||||
const source = await readFile(file, "utf8");
|
||||
expect(source, file).toContain("findRouteChunk(");
|
||||
expect(source, file).not.toMatch(/\.name === runtime\?\.moduleId/u);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { createServer, type Server } from "node:net";
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
@@ -12,6 +13,7 @@ import {
|
||||
testRunPath,
|
||||
testRunRoot,
|
||||
} from "../../scripts/lib/run-isolation.ts";
|
||||
import { childProcessTimeout } from "../helpers/child-process-timeout.ts";
|
||||
|
||||
const servers: Server[] = [];
|
||||
|
||||
@@ -110,6 +112,28 @@ describe("build output root (RUN_OUTPUT_ROOT)", () => {
|
||||
/^check-bundle: RUN_OUTPUT_ROOT must be/u,
|
||||
);
|
||||
});
|
||||
|
||||
// 1106/12-02, the part the gates did not cover. Run alone, these four read
|
||||
// the checkout's dist/ or artifacts/ and did not refuse another root:
|
||||
// `RUN_OUTPUT_ROOT=.tmp/x corepack pnpm verify:reproducible-build` wrote
|
||||
// `passed: true` about the earlier build.
|
||||
it.each([
|
||||
["scripts/verify-reproducible-build.ts"],
|
||||
["scripts/check-optional-recipes.ts", "--require-dist"],
|
||||
["scripts/check-app-build-input.ts", "--require-bundle"],
|
||||
["scripts/create-release-candidate.ts"],
|
||||
])("%s refuses to read the checkout's build under another RUN_OUTPUT_ROOT", (script, ...args) => {
|
||||
const { CI: _ci, ...environment } = process.env;
|
||||
const result = spawnSync(process.execPath, [script, ...args], {
|
||||
encoding: "utf8",
|
||||
env: { ...environment, RUN_OUTPUT_ROOT: ".tmp/run-output-refusal" },
|
||||
});
|
||||
|
||||
expect(result.status, result.stderr).toBe(1);
|
||||
expect(result.stderr).toContain(
|
||||
`${script} reads the build at this checkout's paths (dist/, artifacts/), but RUN_OUTPUT_ROOT=.tmp/run-output-refusal`,
|
||||
);
|
||||
}, childProcessTimeout(499));
|
||||
});
|
||||
|
||||
describe("server ports", () => {
|
||||
|
||||
@@ -4,7 +4,8 @@ import path from "node:path";
|
||||
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
|
||||
import { runRunbookDrill } from "../../scripts/drill-runbook.ts";
|
||||
import { runbookRecordArtifactSchema } from "../../scripts/contracts/release-artifacts.ts";
|
||||
import { PROVIDER_EVIDENCE_MAX_AGE_MS, runRunbookDrill } from "../../scripts/drill-runbook.ts";
|
||||
import { computeContractSetDigest } from "../../src/contracts/contract-set-canonical.ts";
|
||||
import type { InstalledContractPackageIdentity } from "../../src/contracts/external-contract-runtime.ts";
|
||||
|
||||
@@ -145,3 +146,106 @@ describe("runbook drill verdicts", () => {
|
||||
expect(source).not.toMatch(/verdict\(\s*true\s*\)/u);
|
||||
});
|
||||
});
|
||||
|
||||
// 0929/11-02. The drills with provider scope took only a runbook ID and a
|
||||
// release root, so no input could ever make them PASS and the production
|
||||
// stage could not close. Provider evidence now answers the items the local
|
||||
// drill cannot observe, for this runbook and release only, and while recent.
|
||||
describe("runbook drill with provider evidence (0929/11-02)", () => {
|
||||
const NOW = new Date("2026-11-13T09:00:00.000Z");
|
||||
const evidence = (overrides: Record<string, unknown> = {}) => ({
|
||||
schemaVersion: 1,
|
||||
artifactType: "provider-drill-evidence",
|
||||
runbookId: "FE-RB-005",
|
||||
releaseId: "release-a",
|
||||
observedAt: "2026-11-13T08:30:00.000Z",
|
||||
runId: "provider-run-41",
|
||||
observations: [
|
||||
{ item: "trigger", result: "PASS", evidence: "incident INC-7 declared the release-blocking defect" },
|
||||
{ item: "핵심 smoke 시험", result: "PASS", evidence: "smoke against the prior release after the switch" },
|
||||
{ item: "provider verification", result: "PASS", evidence: "provider serves release-a again" },
|
||||
],
|
||||
...overrides,
|
||||
});
|
||||
const drill = async (providerEvidence: unknown) =>
|
||||
runRunbookDrill("FE-RB-005", { releaseRoot: await builtDist(), contractPackages, providerEvidence, now: NOW });
|
||||
|
||||
it("passes FE-RB-005 when the provider observed every item the local drill could not", async () => {
|
||||
const record = await drill(evidence());
|
||||
|
||||
expect(record.result).toBe("PASS");
|
||||
expect(record.triggerAsserted).toBe("PASS");
|
||||
expect(record.recoveryAssertions.find(({ assertion }) => assertion === "핵심 smoke 시험")).toEqual({
|
||||
assertion: "핵심 smoke 시험",
|
||||
evidence: "provider evidence provider-run-41 at 2026-11-13T08:30:00.000Z: smoke against the prior release after the switch",
|
||||
result: "PASS",
|
||||
});
|
||||
expect(record.providerEvidence).toEqual({
|
||||
accepted: true,
|
||||
runId: "provider-run-41",
|
||||
observedAt: "2026-11-13T08:30:00.000Z",
|
||||
providerVerification: "PASS",
|
||||
});
|
||||
const { gateId: _gateId, ...artifact } = record;
|
||||
expect(runbookRecordArtifactSchema.safeParse(artifact).success).toBe(true);
|
||||
});
|
||||
|
||||
it("fails when the provider observed a failure", async () => {
|
||||
const observations = evidence().observations.map((entry) =>
|
||||
entry.item === "핵심 smoke 시험" ? { ...entry, result: "FAIL" } : entry);
|
||||
expect((await drill(evidence({ observations }))).result).toBe("FAIL");
|
||||
});
|
||||
|
||||
it("stays UNVERIFIED while an item the local drill could not observe has no answer", async () => {
|
||||
const record = await drill(evidence({ observations: evidence().observations.slice(0, 2) }));
|
||||
expect(record.result).toBe("UNVERIFIED");
|
||||
expect(record.providerEvidence?.providerVerification).toBe("UNVERIFIED");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["another release", { releaseId: "release-b" }, "evidence is for release release-b, not release-a"],
|
||||
["another runbook", { runbookId: "FE-RB-002" }, "evidence is for FE-RB-002, not FE-RB-005"],
|
||||
[
|
||||
"an expired observation",
|
||||
{ observedAt: new Date(NOW.getTime() - PROVIDER_EVIDENCE_MAX_AGE_MS - 1_000).toISOString() },
|
||||
"is older than 7 days",
|
||||
],
|
||||
["a future observation", { observedAt: "2026-11-13T10:00:00.000Z" }, "is in the future"],
|
||||
[
|
||||
"an answer for a check the local drill made",
|
||||
{ observations: [...evidence().observations, { item: "호환성 gate", result: "PASS", evidence: "x" }] },
|
||||
"evidence names items the local drill observed or does not have: 호환성 gate",
|
||||
],
|
||||
["a record that is not provider evidence", { artifactType: "runbook-record" }, "not provider drill evidence"],
|
||||
])("does not pass on evidence for %s", async (_case, overrides, reason) => {
|
||||
const record = await drill(evidence(overrides));
|
||||
|
||||
expect(record.result).toBe("UNVERIFIED");
|
||||
expect(record.triggerAsserted).toBe("UNVERIFIED");
|
||||
expect(record.providerEvidence?.accepted).toBe(false);
|
||||
expect(record.providerEvidence?.reason).toContain(reason);
|
||||
const { gateId: _gateId, ...artifact } = record;
|
||||
expect(runbookRecordArtifactSchema.safeParse(artifact).success).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects a record that claims PASS without accepted provider evidence", () => {
|
||||
const record = {
|
||||
schemaVersion: 2,
|
||||
runbookId: "FE-RB-005",
|
||||
releaseId: "release-a",
|
||||
drillTimestamp: NOW.toISOString(),
|
||||
triggerInjected: "RELEASE_BLOCKING_DEFECT",
|
||||
triggerAsserted: "PASS",
|
||||
containmentAsserted: "PASS",
|
||||
escalationPathAsserted: "PASS",
|
||||
recoveryAssertions: [],
|
||||
negativeFixtureFailedAsExpected: "PASS",
|
||||
windowObservedBucket: "TBD",
|
||||
providerVerificationRequired: true,
|
||||
providerEvidence: { accepted: false, reason: "expired", providerVerification: "UNVERIFIED" },
|
||||
result: "PASS",
|
||||
};
|
||||
expect(runbookRecordArtifactSchema.safeParse(record).success).toBe(false);
|
||||
expect(runbookRecordArtifactSchema.safeParse({ ...record, result: "UNVERIFIED" }).success).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -9,33 +9,52 @@ import {
|
||||
NAMESPACE_PROBE_ARGUMENTS,
|
||||
type SystemTestHost,
|
||||
} from "../../scripts/check-system-test-prerequisites.ts";
|
||||
import {
|
||||
systemdRunPrerequisiteProbeArguments,
|
||||
systemdRunProviderArguments,
|
||||
} from "../../scripts/lib/provider-cgroup.ts";
|
||||
import { PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS } from "../../scripts/lib/provider-sandbox-namespaces.ts";
|
||||
|
||||
type SpawnCall = Readonly<{ command: string; args: readonly string[] }>;
|
||||
type SpawnResult = ReturnType<SystemTestHost["spawn"]>;
|
||||
|
||||
const RTM_NEWADDR = "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted";
|
||||
const PROBE_UNIT = "ca-provider-prerequisite-4242-0123456789abcdef01234567.scope";
|
||||
|
||||
/** A host that has every prerequisite, except what `bwrap` and `files` say. */
|
||||
type HostOverrides = Readonly<{
|
||||
files?: Readonly<Record<string, string>>;
|
||||
missing?: readonly string[];
|
||||
userManager?: SpawnResult;
|
||||
systemdRun?: SpawnResult;
|
||||
}>;
|
||||
|
||||
/** A host that has every prerequisite, except what `bwrap` and the overrides say. */
|
||||
function fakeHost(
|
||||
bwrap: SpawnResult,
|
||||
files: Readonly<Record<string, string>> = {},
|
||||
options: HostOverrides = {},
|
||||
): { host: SystemTestHost; calls: SpawnCall[] } {
|
||||
const calls: SpawnCall[] = [];
|
||||
const host: SystemTestHost = {
|
||||
access: async () => undefined,
|
||||
access: async (target) => {
|
||||
if (options.missing?.includes(target)) throw new Error(`EACCES: ${target}`);
|
||||
},
|
||||
readFile: async (target) => {
|
||||
if (target === "/sys/fs/cgroup/cgroup.controllers") return "cpuset cpu io memory pids\n";
|
||||
const content = files[target];
|
||||
const content = options.files?.[target];
|
||||
if (content === undefined) throw new Error(`ENOENT: ${target}`);
|
||||
return content;
|
||||
},
|
||||
spawn: (command, args) => {
|
||||
calls.push({ command, args });
|
||||
if (command === "/usr/bin/systemctl") return { status: 0, stderr: "" };
|
||||
if (command === "/usr/bin/systemctl" && args[0] === "--user") {
|
||||
return options.userManager ?? { status: 0, stdout: "255\n", stderr: "" };
|
||||
}
|
||||
if (command === "/usr/bin/systemctl") return { status: 0, stdout: "", stderr: "" };
|
||||
if (command === "/usr/bin/systemd-run") return options.systemdRun ?? { status: 0, stdout: "", stderr: "" };
|
||||
if (command === "/usr/bin/bwrap") return bwrap;
|
||||
throw new Error(`unexpected spawn: ${command}`);
|
||||
},
|
||||
probeUnitName: () => PROBE_UNIT,
|
||||
};
|
||||
return { host, calls };
|
||||
}
|
||||
@@ -74,7 +93,7 @@ describe("system test prerequisites (1105/12-01)", () => {
|
||||
it("fails with the namespace restriction, bwrap's stderr and what to do when bwrap cannot set up loopback", async () => {
|
||||
const { host } = fakeHost(
|
||||
{ status: 1, stderr: `${RTM_NEWADDR}\n` },
|
||||
{ "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" },
|
||||
{ files: { "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" } },
|
||||
);
|
||||
|
||||
const failures = await findSystemTestPrerequisiteFailures(host);
|
||||
@@ -103,7 +122,7 @@ describe("system test prerequisites (1105/12-01)", () => {
|
||||
expect(failures[0]).toContain("did not finish: spawnSync /usr/bin/bwrap ETIMEDOUT");
|
||||
});
|
||||
|
||||
it("keeps the executable, cgroup v2 and systemd bus checks, and probes bwrap only when it is there", async () => {
|
||||
it("keeps the executable, cgroup v2 and systemd checks, and probes bwrap only when it is there", async () => {
|
||||
const calls: SpawnCall[] = [];
|
||||
const host: SystemTestHost = {
|
||||
access: async (target) => {
|
||||
@@ -116,15 +135,98 @@ describe("system test prerequisites (1105/12-01)", () => {
|
||||
},
|
||||
};
|
||||
|
||||
expect(await findSystemTestPrerequisiteFailures(host)).toEqual([
|
||||
const failures = await findSystemTestPrerequisiteFailures(host);
|
||||
expect(failures.slice(0, 2)).toEqual([
|
||||
"bubblewrap is required at /usr/bin/bwrap",
|
||||
"cgroup v2 controllers are unavailable",
|
||||
"a reachable systemd manager bus is required",
|
||||
]);
|
||||
expect(failures[2]).toMatch(/^a reachable systemd user manager is required/u);
|
||||
expect(failures).toHaveLength(3);
|
||||
expect(calls.map(({ command }) => command)).toEqual(["/usr/bin/systemctl"]);
|
||||
});
|
||||
});
|
||||
|
||||
// 1112/08-03. The check looked at the system manager bus while the provider
|
||||
// asks the user manager for its version and starts a `systemd-run --user
|
||||
// --scope`, so a host without a user manager, with an old one, or without
|
||||
// prlimit, systemd-run or delegated controllers printed PASS and then failed
|
||||
// each provider test with `provider cgroup unavailable: ...`.
|
||||
describe("system test prerequisites match the provider's own calls (1112/08-03)", () => {
|
||||
it("fails by name when the system manager answers but the user manager does not", async () => {
|
||||
const { host, calls } = fakeHost(
|
||||
{ status: 0, stderr: "" },
|
||||
{ userManager: { status: 1, stdout: "", stderr: "Failed to connect to bus: No medium found\n" } },
|
||||
);
|
||||
|
||||
const failures = await findSystemTestPrerequisiteFailures(host);
|
||||
|
||||
expect(failures).toHaveLength(1);
|
||||
expect(failures[0]).toMatch(/^a reachable systemd user manager is required/u);
|
||||
expect(failures[0]).toContain("/usr/bin/systemctl --user show --property=Version --value exited 1");
|
||||
expect(failures[0]).toContain("systemctl stderr: Failed to connect to bus: No medium found");
|
||||
expect(calls.some(({ command }) => command === "/usr/bin/systemd-run")).toBe(false);
|
||||
});
|
||||
|
||||
it("fails when the user manager is older than the provider needs", async () => {
|
||||
const { host } = fakeHost({ status: 0, stderr: "" }, { userManager: { status: 0, stdout: "253\n", stderr: "" } });
|
||||
|
||||
expect(await findSystemTestPrerequisiteFailures(host)).toEqual([
|
||||
'systemd user manager 254 or newer is required (/usr/bin/systemctl --user show --property=Version --value reported "253")',
|
||||
]);
|
||||
});
|
||||
|
||||
it("requires the prlimit and systemd-run executables the provider runs", async () => {
|
||||
const { host } = fakeHost({ status: 0, stderr: "" }, { missing: ["/usr/bin/prlimit", "/usr/bin/systemd-run"] });
|
||||
|
||||
expect(await findSystemTestPrerequisiteFailures(host)).toEqual([
|
||||
"prlimit is required at /usr/bin/prlimit",
|
||||
"systemd-run is required at /usr/bin/systemd-run",
|
||||
]);
|
||||
});
|
||||
|
||||
it("starts the provider's own scope once and names a scope that cannot enforce its limits", async () => {
|
||||
const { host, calls } = fakeHost(
|
||||
{ status: 0, stderr: "" },
|
||||
{ systemdRun: { status: 125, stdout: "", stderr: "provider cgroup enforcement failed: cannot read memory.max\n" } },
|
||||
);
|
||||
|
||||
const failures = await findSystemTestPrerequisiteFailures(host);
|
||||
|
||||
expect(failures).toHaveLength(1);
|
||||
expect(failures[0]).toMatch(/^systemd-run could not start the provider's cgroup scope/u);
|
||||
expect(failures[0]).toContain("exited 125");
|
||||
expect(failures[0]).toContain("systemd-run stderr: provider cgroup enforcement failed: cannot read memory.max");
|
||||
expect(failures[0]).toMatch(/memory, pids and cpu controllers delegated/u);
|
||||
|
||||
const scopes = calls.filter(({ command }) => command === "/usr/bin/systemd-run");
|
||||
expect(scopes).toHaveLength(1);
|
||||
expect(scopes[0]?.args).toEqual(systemdRunPrerequisiteProbeArguments(PROBE_UNIT));
|
||||
expect(scopes[0]?.args.at(-1)).toBe("/bin/true");
|
||||
});
|
||||
|
||||
it("probes with the provider scope's properties and enforcement gate, not a copy", () => {
|
||||
const probe = systemdRunPrerequisiteProbeArguments(PROBE_UNIT);
|
||||
const provider = systemdRunProviderArguments(
|
||||
"ca-provider-vulnerability-4242-0123456789abcdef01234567.scope",
|
||||
60_000,
|
||||
60,
|
||||
"/usr/bin/node",
|
||||
"/opt/wrapper.ts",
|
||||
"/tmp/report.json",
|
||||
1,
|
||||
1,
|
||||
);
|
||||
const options = (args: readonly string[]) =>
|
||||
args.filter((arg) => arg.startsWith("--") && !arg.startsWith("--unit=") && !arg.startsWith("--property=RuntimeMaxSec="));
|
||||
expect(options(probe)).toEqual(options(provider));
|
||||
const gate = (args: readonly string[]) => args[args.indexOf("-c") + 1];
|
||||
expect(gate(probe)).toBe(gate(provider));
|
||||
expect(gate(probe)).toContain("require_cgroup_value memory.max");
|
||||
expect(() => systemdRunPrerequisiteProbeArguments("ca-provider-vulnerability-1-0123456789abcdef01234567.scope"))
|
||||
.toThrow(/unit is invalid/u);
|
||||
});
|
||||
});
|
||||
|
||||
// 1101/13-03. The release provider jobs run the same sandbox on whatever
|
||||
// runner picks them up. The supervisor checked only that the executables
|
||||
// exist, so a host that refuses the namespaces failed later with bwrap's raw
|
||||
@@ -140,7 +242,7 @@ describe("provider sandbox namespace check (1101/13-03)", () => {
|
||||
it("names the restriction, bwrap's stderr and the host settings on a host that refuses them", async () => {
|
||||
const { host } = fakeHost(
|
||||
{ status: 1, stderr: `${RTM_NEWADDR}\n` },
|
||||
{ "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" },
|
||||
{ files: { "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" } },
|
||||
);
|
||||
|
||||
const failure = (await findProviderSandboxNamespaceFailure(host)) ?? "";
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import { afterAll, describe, expect, it } from "vitest";
|
||||
import { selectRepositoryPaths } from "../../scripts/lib/repository-layout.ts";
|
||||
import { childProcessTimeout } from "../helpers/child-process-timeout.ts";
|
||||
|
||||
const FORBIDDEN = "tests/fixtures/test-evidence/forbidden";
|
||||
@@ -13,7 +14,7 @@ afterAll(async () => {
|
||||
await Promise.all(roots.map((root) => rm(root, { recursive: true, force: true })));
|
||||
});
|
||||
|
||||
function checkSource(sourceRoot: string) {
|
||||
function checkSource(sourceRoot: string, extra: readonly string[] = []) {
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
@@ -23,6 +24,7 @@ function checkSource(sourceRoot: string) {
|
||||
sourceRoot,
|
||||
"--artifact",
|
||||
"artifacts/quality/test-evidence-source-rules.json",
|
||||
...extra,
|
||||
],
|
||||
{ encoding: "utf8" },
|
||||
);
|
||||
@@ -44,6 +46,44 @@ describe("test evidence source rules", () => {
|
||||
expect(result.stderr).toContain("quarantine expired on 2020-01-01");
|
||||
}, childProcessTimeout(739));
|
||||
|
||||
// 1112/03-03. The mask fixture had no test of its own: the directory run
|
||||
// expects only the common prefix, so this rule could stop firing unseen.
|
||||
it("rejects a screenshot mask over the whole UI", () => {
|
||||
const result = checkSource(`${FORBIDDEN}/full-screen-mask.fixture`);
|
||||
expect(result.status).toBe(1);
|
||||
expect(result.stderr).toContain("screenshot mask may not cover the whole UI");
|
||||
}, childProcessTimeout(141));
|
||||
|
||||
// 1112/03-03. The forbidOnly rule ran only in the full check, on the
|
||||
// repository's own configs, so nothing showed it still rejects a config.
|
||||
it("rejects a Playwright config without forbidOnly and accepts the repository's", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "test-evidence-config-"));
|
||||
roots.push(root);
|
||||
await writeFile(path.join(root, "clean.fixture"), "export {};\n");
|
||||
|
||||
const missing = checkSource(root, [
|
||||
"--playwright-config",
|
||||
"tests/fixtures/test-evidence/playwright-config/missing-forbid-only.fixture",
|
||||
]);
|
||||
expect(missing.status).toBe(1);
|
||||
expect(missing.stderr).toContain(
|
||||
"tests/fixtures/test-evidence/playwright-config/missing-forbid-only.fixture: missing forbidOnly: true",
|
||||
);
|
||||
|
||||
const present = checkSource(root, ["--playwright-config", "playwright.config.ts"]);
|
||||
expect(present.stderr).toBe("");
|
||||
expect(present.status).toBe(0);
|
||||
|
||||
// Without the flag the rule reads the repository's configs, every one.
|
||||
const repository = checkSource(root);
|
||||
expect(repository.status, repository.stderr).toBe(0);
|
||||
const report = JSON.parse(await readFile("artifacts/quality/test-evidence-source-rules.json", "utf8")) as {
|
||||
facts: { playwrightConfigs?: string[] };
|
||||
};
|
||||
expect(report.facts.playwrightConfigs).toEqual([...selectRepositoryPaths("playwright-configs")]);
|
||||
expect(report.facts.playwrightConfigs?.length).toBeGreaterThan(0);
|
||||
}, childProcessTimeout(296));
|
||||
|
||||
it("rejects a skip without a quarantine", () => {
|
||||
const result = checkSource(`${FORBIDDEN}/unowned-skip.fixture`);
|
||||
expect(result.status).toBe(1);
|
||||
|
||||
@@ -66,7 +66,9 @@ async function stripColours(
|
||||
new Blob([bytes], { type: "image/png" }),
|
||||
);
|
||||
const canvas = new OffscreenCanvas(bitmap.width, bitmap.height);
|
||||
const context = canvas.getContext("2d");
|
||||
// Read back once per strip: without the hint Chromium warns on the
|
||||
// console of this tab, which the strict fixture watches (1112/03-05).
|
||||
const context = canvas.getContext("2d", { willReadFrequently: true });
|
||||
if (context === null) throw new Error("no 2d canvas context");
|
||||
context.drawImage(bitmap, 0, 0);
|
||||
return offsets.map(({ x, width }) => {
|
||||
|
||||
Reference in New Issue
Block a user