fix(test): 파일 존재로 정하던 run 격리·빌드 입력 검사를 소유자로 정한다

문제: 두 테스트가 검사 대상을 "파일이 있으면"으로 정한다.
- `playwright-run-isolation.test.ts`는 capabilities 설정이 있을 때만
  그 설정의 검사 8개를 정의한다(`existsSync`). browser-file-storage
  드릴 사본에서 33개가 25개로 줄었는데 skip 보고가 없다. 템플릿에서
  설정 이름을 바꾸고 테스트의 문자열만 놓쳐도 같이 빠진다.
- `vite-build-inputs.test.ts`는 목록 소비자 둘을 `existsSync`로 거른다.
  어느 제거 목록에도 없는 `product-feature-selection-build.test.ts`의
  이름이 바뀌어도 검사가 빈 반복으로 통과한다.
- 주석의 skip 차단 ID가 0929/10-04로 틀렸다. 1101/10-04다.

고른 방법(리뷰 문서 ⓐ):
- capabilities 검사 8개를 `playwright-capabilities-run-isolation.test.ts`로
  옮기고 설정을 이름으로 import한다. browser-file-storage 드릴이 설정과
  이 파일을 이름으로 함께 지운다. 같은 드릴이 `ci-workflow-generation`
  테스트를 지우는 방식이다. `force` 없이 지워, 이름이 바뀐 파일은 드릴이
  그 자리에서 멈춘다.
- 두 파일이 같은 검사를 쓰도록 불러오기·포트·결과 폴더 검사를
  `tests/helpers/playwright-run-isolation.ts`로 뺐다. 단언은 그대로다.
- `vite-build-inputs`는 빌드 테스트를 늘 읽는다. 측정 스크립트는 그
  파일을 가진 제거 가능 기능(`removable-template-features.ts`)이 설치돼
  있을 때(feature-module.ts가 있을 때, verify:documentation과 같은 규칙)
  읽는다. 그 소유 기능이 없으면 실패한다. 기능 ID를 테스트에 쓰지 않는
  것은 샘플 드릴의 residue 검사가 `src`·`tests`의 샘플 이름을 잡기 때문이다.
- 틀린 ID 주석은 옮긴 검사와 함께 없어지고, 새 파일 머리말에 1101/10-04로
  적었다.
버린 대안: 드릴이 표시를 넘기고 테스트가 읽는 방식(ⓑ). 드릴과 테스트
사이 계약이 늘고, capability를 지우는 제품이 테스트를 고쳐야 한다. 주석
ID만 고치기(ⓒ)는 공백이 남는다.

위험·한계:
- 테스트 수는 템플릿 33개(25 + 8)로 같다. browser-file-storage 사본에서는
  25개가 돌고 8개는 파일째 지워진다(드릴 코드에 적힘).
- 측정 스크립트의 조건은 여전히 파일 존재(소유 기능의 feature-module.ts)다.
  대상이 지울 수 있다고 등록된 소유자로 바뀌었다.
- 이 노트북에서는 실행하지 못했다.

검증: desktop batch에서 확인.
node scripts/run-vitest.ts run tests/unit/playwright-run-isolation.test.ts tests/unit/playwright-capabilities-run-isolation.test.ts tests/unit/vite-build-inputs.test.ts
이름 바꾸기 변이 둘(capabilities 설정, product-feature-selection-build
테스트), check:types, lint, ci:gate FE-GATE-020

Refs: 1106/10-01
This commit is contained in:
DongHyeonka
2026-11-06 05:27:21 +09:00
parent 302d3a5359
commit a135fa8a09
5 changed files with 310 additions and 201 deletions
@@ -0,0 +1,51 @@
import { afterEach, describe, expect, it } from "vitest";
import {
configRuns,
describeResultFolders,
describeServerPorts,
hold,
itExitsWhenItsPortIsTaken,
itKeepsTheViteCacheInTheRun,
portOf,
releaseRuns,
} from "../helpers/playwright-run-isolation.ts";
/**
* 1105/12-04 for the capabilities config; why each check exists is in
* tests/unit/playwright-run-isolation.test.ts.
*
* 1106/10-01. This config goes with the browser file/storage runtime:
* test:browser-file-storage-removal deletes it, and deletes this file by
* name with it. Its eight checks used to sit with the other configs and were
* defined only while `existsSync` found the config. test:unit fails on a
* skipped test (1101/10-04), but a test that is never defined is not
* reported: the checks left that drill's copy unnoticed, and a renamed config
* would have taken them out of the template the same way. Imported by name, a
* renamed config fails check:types and this file instead.
*/
const runs = configRuns({
capabilities: () => import("../../playwright.capabilities.config.ts"),
visual: () => import("../../playwright.visual.config.ts"),
});
afterEach(releaseRuns);
describeResultFolders(runs, [["capabilities", "browser-capabilities"]]);
describe("the build a run serves", () => {
itKeepsTheViteCacheInTheRun(runs, ["capabilities"]);
});
describeServerPorts(runs, ["capabilities"]);
describe("vite servers", () => {
itExitsWhenItsPortIsTaken(runs, ["capabilities"]);
it("no longer share one port between the visual preview and the capability dev server", async () => {
const visual = portOf(await runs.load("visual"));
await hold(visual);
expect(portOf(await runs.load("capabilities"))).not.toBe(visual);
});
});
+31 -192
View File
@@ -1,9 +1,16 @@
import { existsSync, readFileSync } from "node:fs";
import { createServer, type Server } from "node:net";
import path from "node:path";
import { readFileSync } from "node:fs";
import type { PlaywrightTestConfig } from "@playwright/test";
import { afterEach, describe, expect, it, vi } from "vitest";
import { afterEach, describe, expect, it } from "vitest";
import {
configRuns,
describeResultFolders,
describeServerPorts,
itExitsWhenItsPortIsTaken,
itKeepsTheViteCacheInTheRun,
releaseRuns,
webServer,
} from "../helpers/playwright-run-isolation.ts";
/**
* 1105/12-04. Every Playwright config wrote its results, report and JUnit
@@ -20,151 +27,36 @@ import { afterEach, describe, expect, it, vi } from "vitest";
*
* Each load evaluates the config again with the environment of one process.
*
* test:browser-file-storage-removal deletes the capabilities config with the
* capability it runs, so a project without it checks the other four. Its path
* is a variable so that typecheck does not need the file.
* 1106/10-01. The capabilities config is checked in
* tests/unit/playwright-capabilities-run-isolation.test.ts: it goes with the
* browser file/storage runtime, and test:browser-file-storage-removal deletes
* the config and that file by name. This file has the four configs every
* project has.
*/
const CAPABILITIES_CONFIG = "playwright.capabilities.config.ts";
const HAS_CAPABILITIES = existsSync(CAPABILITIES_CONFIG);
const LOADERS = {
const runs = configRuns({
e2e: () => import("../../playwright.config.ts"),
visual: () => import("../../playwright.visual.config.ts"),
capabilities: () =>
import(/* @vite-ignore */ `../../${CAPABILITIES_CONFIG}`) as Promise<{
default: PlaywrightTestConfig;
}>,
storybook: () => import("../../playwright.storybook.config.ts"),
dev: () => import("../../playwright.dev.config.ts"),
} as const;
type ConfigName = keyof typeof LOADERS;
/** The names among `names` whose config file this checkout has. */
function installed<T extends ConfigName>(names: readonly T[]): T[] {
return names.filter((name) => name !== "capabilities" || HAS_CAPABILITIES);
}
const CONFIG_NAMES = installed(Object.keys(LOADERS) as ConfigName[]);
const PORT_VARIABLES: Readonly<Record<ConfigName, string>> = {
e2e: "TEST_E2E_PORT",
visual: "TEST_VISUAL_PORT",
capabilities: "TEST_CAPABILITIES_PORT",
storybook: "TEST_STORYBOOK_PORT",
dev: "TEST_DEV_PORT",
};
const RUN_VARIABLES = ["TEST_RUN_ID", "RUN_OUTPUT_ROOT", ...Object.values(PORT_VARIABLES)];
const held: Server[] = [];
afterEach(async () => {
vi.resetModules();
await Promise.all(
held.splice(0).map((server) => new Promise<void>((resolve) => server.close(() => resolve()))),
);
});
/** The config as one process sees it, and what it left in that environment. */
async function loadProcess(
name: ConfigName,
environment: Readonly<Record<string, string>> = {},
): Promise<{ config: PlaywrightTestConfig; after: Readonly<Record<string, string | undefined>> }> {
const saved = RUN_VARIABLES.map((variable) => [variable, process.env[variable]] as const);
for (const variable of RUN_VARIABLES) delete process.env[variable];
Object.assign(process.env, environment);
vi.resetModules();
try {
const config = (await LOADERS[name]()).default;
return {
config,
after: Object.fromEntries(RUN_VARIABLES.map((variable) => [variable, process.env[variable]])),
};
} finally {
for (const [variable, value] of saved) {
if (value === undefined) delete process.env[variable];
else process.env[variable] = value;
}
}
}
afterEach(releaseRuns);
async function load(
name: ConfigName,
environment: Readonly<Record<string, string>> = {},
): Promise<PlaywrightTestConfig> {
return (await loadProcess(name, environment)).config;
}
function portOf(config: PlaywrightTestConfig): number {
return Number(new URL(String(config.use?.baseURL)).port);
}
/** Listens on `port` the way the first run's server does. */
async function hold(port: number): Promise<void> {
const server = createServer();
held.push(server);
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(port, "127.0.0.1", () => resolve());
});
}
function reporterOption(config: PlaywrightTestConfig, reporter: string, option: string): string {
const reporters: readonly unknown[] = Array.isArray(config.reporter) ? config.reporter : [];
const entry = reporters.find(
(candidate) => Array.isArray(candidate) && candidate[0] === reporter,
) as readonly [string, Record<string, unknown>] | undefined;
return String(entry?.[1][option]);
}
function outputs(config: PlaywrightTestConfig) {
return {
results: path.resolve(String(config.outputDir)),
report: path.resolve(reporterOption(config, "html", "outputFolder")),
junit: path.resolve(reporterOption(config, "junit", "outputFile")),
};
}
function webServer(config: PlaywrightTestConfig) {
const server = config.webServer;
if (!server || Array.isArray(server)) throw new TypeError("expected one web server");
return server;
}
const RESULT_FOLDERS: Readonly<Record<Exclude<ConfigName, "dev">, string>> = {
e2e: "e2e",
visual: "visual",
capabilities: "browser-capabilities",
storybook: "storybook",
};
const INSTALLED_RESULT_FOLDERS = installed(
Object.keys(RESULT_FOLDERS) as Exclude<ConfigName, "dev">[],
).map((name) => [name, RESULT_FOLDERS[name]] as const);
describe.each(INSTALLED_RESULT_FOLDERS)("the %s config", (name, folder) => {
const expected = (root: string) => ({
results: path.resolve(root, `artifacts/tests/${folder}/results`),
report: path.resolve(root, `artifacts/tests/${folder}/report`),
junit: path.resolve(root, `artifacts/tests/${folder}/results.xml`),
});
it("writes where the gates read without a run ID", async () => {
expect(outputs(await load(name as ConfigName))).toEqual(expected("."));
});
it("gives each run ID its own results, report and JUnit file", async () => {
const a = outputs(await load(name as ConfigName, { TEST_RUN_ID: "a" }));
const b = outputs(await load(name as ConfigName, { TEST_RUN_ID: "b" }));
expect(a).toEqual(expected(".tmp/runs/a"));
expect(b).toEqual(expected(".tmp/runs/b"));
});
});
describeResultFolders(runs, [
["e2e", "e2e"],
["visual", "visual"],
["storybook", "storybook"],
]);
describe("the build a run serves", () => {
it.each(["e2e", "visual"] as const)(
"is the checkout's dist without a run ID, and the run's own with one (%s)",
async (name) => {
const checkout = webServer(await load(name));
const checkout = webServer(await runs.load(name));
expect(checkout.env?.["RUN_OUTPUT_ROOT"]).toBe(".");
expect(checkout.command).toMatch(/ --outDir dist(?: |$)/u);
const run = webServer(await load(name, { TEST_RUN_ID: "a" }));
const run = webServer(await runs.load(name, { TEST_RUN_ID: "a" }));
// The build writes under RUN_OUTPUT_ROOT and the preview serves it.
expect(run.env?.["RUN_OUTPUT_ROOT"]).toBe(".tmp/runs/a");
expect(run.command).toMatch(/^corepack pnpm build && corepack pnpm preview /u);
@@ -172,23 +64,15 @@ describe("the build a run serves", () => {
},
);
it.each(installed(["capabilities", "dev"] as const))(
"keeps the dev server's Vite cache in the run's folder (%s)",
async (name) => {
expect(webServer(await load(name)).env?.["RUN_OUTPUT_ROOT"]).toBe(".");
expect(webServer(await load(name, { TEST_RUN_ID: "a" })).env?.["RUN_OUTPUT_ROOT"]).toBe(
".tmp/runs/a",
);
},
);
itKeepsTheViteCacheInTheRun(runs, ["dev"]);
it("builds the workshop into the run's folder the way build:storybook does", async () => {
const checkout = webServer(await load("storybook")).command;
const checkout = webServer(await runs.load("storybook")).command;
expect(checkout).toMatch(
/^corepack pnpm build:storybook && node scripts\/serve-static\.ts artifacts\/storybook\/static /u,
);
const run = webServer(await load("storybook", { TEST_RUN_ID: "a" })).command;
const run = webServer(await runs.load("storybook", { TEST_RUN_ID: "a" })).command;
const script = (
JSON.parse(readFileSync("package.json", "utf8")) as { scripts: Record<string, string> }
).scripts["build:storybook"]!;
@@ -201,53 +85,8 @@ describe("the build a run serves", () => {
});
});
describe.each(CONFIG_NAMES)("the %s server port", (name) => {
it("is a free port the runner hands to its workers", async () => {
const runner = await loadProcess(name);
const port = portOf(runner.config);
const server = webServer(runner.config);
// Playwright evaluates the config again in each worker, which inherits
// the runner's environment: the port is there, so baseURL agrees.
expect(runner.after[PORT_VARIABLES[name]]).toBe(String(port));
expect(portOf(await load(name, { [PORT_VARIABLES[name]]: String(port) }))).toBe(port);
expect(server.url).toBe(`http://127.0.0.1:${port}`);
expect(server.command).toMatch(new RegExp(` (?:--port )?${port}(?: |$)`, "u"));
expect(server.reuseExistingServer).toBe(false);
});
it("moves for a second run while the first run's server holds its port", async () => {
const first = portOf(await load(name));
await hold(first);
expect(portOf(await load(name))).not.toBe(first);
});
it("is the one its variable pins", async () => {
const pinned = await load(name, { [PORT_VARIABLES[name]]: "40123" });
expect(portOf(pinned)).toBe(40123);
expect(webServer(pinned).url).toBe("http://127.0.0.1:40123");
});
});
describeServerPorts(runs, ["e2e", "visual", "storybook", "dev"]);
describe("vite servers", () => {
it.each(installed(["e2e", "visual", "capabilities", "dev"] as const))(
"exit instead of moving to the next port when theirs is taken (%s)",
async (name) => {
// Without --strictPort vite moved on, and Playwright, waiting on the
// configured URL, tested whatever already listened there.
expect(webServer(await load(name)).command).toMatch(/ --port \d+ --strictPort(?: |$)/u);
},
);
// Defined only where the capabilities config exists, not skipped:
// test:unit fails on a skipped test (0929/10-04).
if (HAS_CAPABILITIES) {
it("no longer share one port between the visual preview and the capability dev server", async () => {
const visual = portOf(await load("visual"));
await hold(visual);
expect(portOf(await load("capabilities"))).not.toBe(visual);
});
}
itExitsWhenItsPortIsTaken(runs, ["e2e", "visual", "dev"]);
});
+21 -6
View File
@@ -3,6 +3,8 @@ import path from "node:path";
import { describe, expect, it } from "vitest";
import { REMOVABLE_TEMPLATE_FEATURES } from "../../scripts/lib/removable-template-features.ts";
/**
* 1105/15-03. Two copies of the list of files `vite build` reads from the
* root (the initial JavaScript measurement and the
@@ -12,14 +14,25 @@ import { describe, expect, it } from "vitest";
* now, and it has to cover what vite.config.ts imports.
*/
const LIST_MODULE = "scripts/lib/vite-build-inputs.ts";
// The measurement goes with the reference feature it copies
// (test:sample-removal), so a project without that feature checks the build
// test alone. In the template, tests/unit/feature-shape-growth.test.ts
// imports the measurement and fails if the file is gone.
const MEASUREMENT = "scripts/scale/measure-feature-shape-growth.ts";
// 1106/10-01. The measurement copies the reference sample and is one of its
// files (scripts/lib/removable-template-features.ts), so test:sample-removal
// deletes it with the sample. It is left out only where its owner is not
// installed: the owner's feature-module.ts is gone, the rule
// verify:documentation uses. The build test is a template file and is always
// read. Filtering both on "the file exists" let a renamed consumer drop out of
// the check unnoticed. The owner is found by its files, not written here: the
// sample drill fails on any test that still names the sample.
const MEASUREMENT_OWNER = REMOVABLE_TEMPLATE_FEATURES.find((feature) =>
feature.extraPaths.includes(MEASUREMENT),
);
const OWNER_INSTALLED =
MEASUREMENT_OWNER !== undefined &&
existsSync(`src/features/${MEASUREMENT_OWNER.featureId}/feature-module.ts`);
const CONSUMERS = [
"scripts/scale/measure-feature-shape-growth.ts",
"tests/unit/product-feature-selection-build.test.ts",
].filter((file) => existsSync(file));
...(OWNER_INSTALLED ? [MEASUREMENT] : []),
];
const LOCAL_IMPORT =
/\bfrom\s+["'](\.{1,2}\/[^"']+)["']|\bimport\s+["'](\.{1,2}\/[^"']+)["']/gu;
@@ -47,6 +60,8 @@ function localImportGraph(entry: string): string[] {
describe("vite build inputs (1105/15-03)", () => {
it("defines the list in one module that both builds import", () => {
// The measurement may be missing only because a removable feature owns it.
expect(MEASUREMENT_OWNER, `no removable template feature owns ${MEASUREMENT}`).toBeDefined();
for (const consumer of CONSUMERS) {
const source = readFileSync(consumer, "utf8");
expect(source, consumer).not.toMatch(/\bconst BUILD_INPUTS\s*=/u);