From 68fde0277715e65311c4cfad63d2bb35e8286833 Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Fri, 6 Nov 2026 05:35:33 +0900 Subject: [PATCH] =?UTF-8?q?fix(test-run):=20RUN=5FOUTPUT=5FROOT=EA=B0=80?= =?UTF-8?q?=20=EC=9E=88=EC=9C=BC=EB=A9=B4=20=EB=B9=8C=EB=93=9C=20=EB=92=A4?= =?UTF-8?q?=20=EA=B2=80=EC=82=AC=EA=B0=80=20=EB=A9=88=EC=B6=98=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 문제: 1105/12-04 뒤 빌드는 `RUN_OUTPUT_ROOT` 아래에 쓰는데, check:bundle의 두 단계(`generate-supply-chain.ts`, `check-bundle.ts`)는 이 변수를 읽지 않고 checkout의 `dist`와 `artifacts/`를 읽는다. 셸에 이 변수를 둔 채 빌드와 check:bundle을 돌리면 예산을 3,504 B 넘는 새 빌드 대신 이전 빌드를 재서 PASS한다(리뷰 12-R2). ci:gate의 신선도 검사도 bundle.json을 check:bundle이 다시 쓰므로 막지 못한다. 고른 방법: `scripts/lib/run-isolation.ts`에 `checkoutBuildReadFailure`를 두고, 두 스크립트가 무엇을 읽기 전에 부른다. 출력 루트가 `.`이 아니면 변수 값과 이유를 적고 exit 1로 멈춘다. `.`(Playwright 설정이 run ID 없이 web server에 넘기는 값)과 빈 값은 그대로 돈다. 그래서 `RUN_OUTPUT_ROOT=.tmp/x corepack pnpm ci:gate -- FE-GATE-012`도 check-bundle 단계에서 변수 이름을 적고 실패한다. 버린 대안: ci:gate가 시작 전에 거절하기(리뷰 문서 (b)). 12-R2처럼 check:bundle을 직접 돌리는 경우를 막지 못한다. 두 스크립트가 같은 루트를 읽게 하기(c)는 supply-chain 산출물 전부와 CI upload 경로를 함께 옮겨야 한다. 위험·한계: - FE-GATE-012는 빌드 단계를 끝까지 돈 뒤 check-bundle에서 실패한다. 빌드 시간만큼 늦게 알린다. - 같은 모양일 수 있는 `check:app-build-input:bundle`, `check:optional-recipes --require-dist`, release candidate 조립은 보지 않았다. FE-GATE-012에서는 check-bundle이 먼저 실패해 거기까지 가지 않는다. - 디버깅으로 변수를 둔 사람은 check:bundle 전에 지워야 한다(문서에 적음). - 이 노트북에서는 실행하지 못했다. 검증: desktop batch에서 확인. node scripts/run-vitest.ts run tests/unit/check-bundle-output.test.ts tests/unit/run-isolation.test.ts 리뷰 12-R2 명령(stale-bundle-rc=1, stderr에 RUN_OUTPUT_ROOT), check:types, lint, verify:documentation, ci:gate FE-GATE-012 Refs: 1106/12-02 --- .../frontend-platform-testing-strategy.md | 5 +- scripts/check-bundle.ts | 11 +++ scripts/generate-supply-chain.ts | 10 +++ scripts/lib/run-isolation.ts | 28 +++++++ tests/unit/check-bundle-output.test.ts | 83 ++++++++++++++++--- tests/unit/run-isolation.test.ts | 19 +++++ 6 files changed, 143 insertions(+), 13 deletions(-) diff --git a/docs/testing/frontend-platform-testing-strategy.md b/docs/testing/frontend-platform-testing-strategy.md index f9b15724..bd1443f8 100644 --- a/docs/testing/frontend-platform-testing-strategy.md +++ b/docs/testing/frontend-platform-testing-strategy.md @@ -1026,7 +1026,7 @@ TEST_RUN_ID=a playwright test | 변수 | 누가 읽나 | 없을 때 | |---|---|---| | `TEST_RUN_ID` | 다섯 Playwright 설정, `tests/e2e/preload-use.spec.ts` | `dist`, `artifacts/tests/<이름>/` | -| `RUN_OUTPUT_ROOT` | 빌드 단계 전부(`scripts/build-frontend.ts`가 부르는 것), `vite.config.ts`. 설정이 web server에만 넘긴다 | `.` | +| `RUN_OUTPUT_ROOT` | 빌드 단계 전부(`scripts/build-frontend.ts`가 부르는 것), `vite.config.ts`. 설정이 web server에만 넘긴다. 빌드 뒤 검사(`scripts/generate-supply-chain.ts`, `scripts/check-bundle.ts`)는 `.`이 아니면 멈춘다 | `.` | | `TEST_E2E_PORT`, `TEST_VISUAL_PORT`, `TEST_CAPABILITIES_PORT`, `TEST_STORYBOOK_PORT`, `TEST_DEV_PORT` | 각 설정 | 빈 포트 | - run ID는 글자·숫자·`.`·`_`·`-` 64자 이하이고 글자나 숫자로 시작한다. @@ -1035,6 +1035,9 @@ TEST_RUN_ID=a playwright test - `RUN_OUTPUT_ROOT`는 직접 export하지 않는다. 설정이 run ID에서 만든다. 빌드 코드는 `TEST_RUN_ID`를 읽지 않는다. e2e가 저장소 복사본을 빌드할 때 그 빌드는 복사본 자기 `dist`에 써야 하기 때문이다. +- 빌드 뒤 검사는 checkout의 `dist`와 `artifacts/`만 읽는다. 그래서 + `RUN_OUTPUT_ROOT`가 셸에 있으면 `check:bundle`은 변수 이름을 적고 실패한다. + 빌드는 `.tmp/` 아래에 쓰고 검사는 이전 빌드를 재서 통과했기 때문이다(1106/12-02). - 규칙은 `scripts/lib/run-isolation.ts` 한 곳에 있다. 두 e2e 실행을 동시에 돌리는 예: diff --git a/scripts/check-bundle.ts b/scripts/check-bundle.ts index bbd9fb9e..8a7294e5 100644 --- a/scripts/check-bundle.ts +++ b/scripts/check-bundle.ts @@ -9,6 +9,7 @@ import { bundlePerformanceArtifactSchema, } from "./contracts/release-artifacts.ts"; import { classifyViteJavascript } from "./lib/classify-vite-bundle.ts"; +import { checkoutBuildReadFailure } from "./lib/run-isolation.ts"; import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts"; type ViteManifest = Record< @@ -21,6 +22,16 @@ type BundleBudgets = { lazyChunkGzipBytes: number; }; +// 1106/12-02. This reads dist/.vite/manifest.json and the inventory +// scripts/generate-supply-chain.ts made of dist. Under RUN_OUTPUT_ROOT the +// build wrote elsewhere, and this passed a build over budget on the earlier +// one the checkout still held. +const outputRootFailure = checkoutBuildReadFailure("scripts/check-bundle.ts"); +if (outputRootFailure !== undefined) { + process.stderr.write(`${outputRootFailure}\n`); + process.exit(1); +} + const report = bundleOutputInventoryArtifactSchema.parse( JSON.parse(await readFile("artifacts/performance/bundle.json", "utf8")) as unknown, ); diff --git a/scripts/generate-supply-chain.ts b/scripts/generate-supply-chain.ts index 505f63d7..14b06a74 100644 --- a/scripts/generate-supply-chain.ts +++ b/scripts/generate-supply-chain.ts @@ -39,8 +39,18 @@ import { parseRepositoryFileInventoryPolicy, } from "./lib/repository-file-inventory.ts"; import { collectDistOutputs, distSha256 } from "./lib/release-candidate.ts"; +import { checkoutBuildReadFailure } from "./lib/run-isolation.ts"; import { deterministicSupplyChainGeneratedAt } from "./lib/supply-chain-time.ts"; +// 1106/12-02. This reads artifacts/release/build-manifest.json and dist where +// the checkout keeps them. Under RUN_OUTPUT_ROOT the build wrote elsewhere, +// and this recorded the earlier build the checkout still held. +const outputRootFailure = checkoutBuildReadFailure("scripts/generate-supply-chain.ts"); +if (outputRootFailure !== undefined) { + process.stderr.write(`${outputRootFailure}\n`); + process.exit(1); +} + type Document = Record; function isRecord(value: unknown): value is Document { diff --git a/scripts/lib/run-isolation.ts b/scripts/lib/run-isolation.ts index 6475532d..d51807bd 100644 --- a/scripts/lib/run-isolation.ts +++ b/scripts/lib/run-isolation.ts @@ -82,6 +82,34 @@ export function runOutputPath( return path.posix.join(runOutputRoot(environment), relativePath); } +/** + * 1106/12-02. Why `step`, which reads the build where the checkout keeps it + * (`dist/`, the build manifest and the evidence under `artifacts/`), must not + * run in this environment; `undefined` when it may. With RUN_OUTPUT_ROOT set + * the build writes under that folder, so such a step would judge whatever + * build the checkout still holds: check:bundle passed an over-budget build + * that way, on the earlier one. `.` is the checkout itself, which is what a + * Playwright config hands its web server without a run ID. + */ +export function checkoutBuildReadFailure( + step: string, + environment: Environment = process.env, +): string | undefined { + let root: string; + try { + root = runOutputRoot(environment); + } catch (error) { + return `${step}: ${error instanceof Error ? error.message : String(error)}`; + } + if (root === ".") return undefined; + return ( + `${step} reads the build at this checkout's paths (dist/, artifacts/), but ` + + `${RUN_OUTPUT_ROOT_VARIABLE}=${root} makes the build write under ${root}/, so it ` + + `would judge an earlier build. Unset ${RUN_OUTPUT_ROOT_VARIABLE}, build again and ` + + `run ${step} again.` + ); +} + /** * The port variable of each Playwright config's web server. Unset, the config * picks a free port; set, it uses that one (to pin a port while debugging). diff --git a/tests/unit/check-bundle-output.test.ts b/tests/unit/check-bundle-output.test.ts index f5dd42ee..6eec892d 100644 --- a/tests/unit/check-bundle-output.test.ts +++ b/tests/unit/check-bundle-output.test.ts @@ -24,6 +24,8 @@ async function checkBundle( outputs: readonly Output[]; manifest: Record; budgets: Record; + /** RUN_OUTPUT_ROOT for the run, and the build written under it. */ + runOutput?: Readonly<{ root: string; outputs: readonly Output[] }>; }>, ) { const root = await mkdtemp(path.join(tmpdir(), "check-bundle-output-")); @@ -32,22 +34,28 @@ async function checkBundle( await mkdir(path.dirname(path.join(root, file)), { recursive: true }); await writeFile(path.join(root, file), `${JSON.stringify(value, null, 2)}\n`); }; + const writeBuild = async (buildRoot: string, outputs: readonly Output[]) => { + await write(path.posix.join(buildRoot, "dist/.vite/manifest.json"), options.manifest); + await write(path.posix.join(buildRoot, "artifacts/performance/bundle.json"), { + schemaVersion: 1, + generatedAt: "2026-11-04T00:00:00.000Z", + context: { nodeVersion: "v24.14.0", packageManager: "pnpm@11.17.0", runnerImage: "fixture" }, + outputs: outputs.map((output) => ({ + path: `dist/${output.path}`, + bytes: output.gzipBytes * 3, + gzipBytes: output.gzipBytes, + sha256: "a".repeat(64), + })), + }); + }; await write("config/performance/budgets.json", { schemaVersion: 1, bundle: options.budgets }); - await write("dist/.vite/manifest.json", options.manifest); - await write("artifacts/performance/bundle.json", { - schemaVersion: 1, - generatedAt: "2026-11-04T00:00:00.000Z", - context: { nodeVersion: "v24.14.0", packageManager: "pnpm@11.17.0", runnerImage: "fixture" }, - outputs: options.outputs.map((output) => ({ - path: `dist/${output.path}`, - bytes: output.gzipBytes * 3, - gzipBytes: output.gzipBytes, - sha256: "a".repeat(64), - })), - }); + await writeBuild(".", options.outputs); + if (options.runOutput) await writeBuild(options.runOutput.root, options.runOutput.outputs); const result = spawnSync(process.execPath, [path.resolve("scripts/check-bundle.ts")], { cwd: root, encoding: "utf8", + // Empty is unset (scripts/lib/run-isolation.ts), whatever the runner has. + env: { ...process.env, RUN_OUTPUT_ROOT: options.runOutput?.root ?? "" }, }); return { status: result.status, stdout: result.stdout, stderr: result.stderr }; } @@ -126,3 +134,54 @@ describe("check:bundle failure output (1103/15-03)", () => { expect(result.stderr).not.toContain("Bundle budget failed"); }); }); + +/** + * 1106/12-02. Under RUN_OUTPUT_ROOT scripts/build-frontend.ts writes the build + * below that folder, and check:bundle (scripts/generate-supply-chain.ts, then + * scripts/check-bundle.ts) read the checkout's dist/ and artifacts/ anyway: a + * build over budget under .tmp/ passed on the checkout's earlier build + * (review run 12-R2). Both stop and name the variable instead. + */ +describe("check:bundle with RUN_OUTPUT_ROOT (1106/12-02)", () => { + it("does not pass the checkout's earlier build while the build writes elsewhere", async () => { + const result = await checkBundle({ + outputs: OUTPUTS, + manifest: MANIFEST, + budgets: BUDGETS, + runOutput: { + root: ".tmp/runs/other", + outputs: OUTPUTS.map((output) => ({ ...output, gzipBytes: output.gzipBytes * 4 })), + }, + }); + expect(result.stdout).not.toContain("Bundle budget: PASS"); + expect(result.status).toBe(1); + expect(result.stderr).toContain("RUN_OUTPUT_ROOT=.tmp/runs/other"); + }); + + it("measures the checkout's build when the output root is the checkout", async () => { + const result = await checkBundle({ + outputs: OUTPUTS, + manifest: MANIFEST, + budgets: BUDGETS, + runOutput: { root: ".", outputs: OUTPUTS }, + }); + expect(result.stderr).toBe(""); + expect(result.status).toBe(0); + expect(result.stdout).toContain("initial JS 900 / 1000 gzip bytes"); + }); + + it("stops scripts/generate-supply-chain.ts before it reads the checkout's build", async () => { + const root = await mkdtemp(path.join(tmpdir(), "generate-supply-chain-output-root-")); + roots.push(root); + const result = spawnSync(process.execPath, [path.resolve("scripts/generate-supply-chain.ts")], { + cwd: root, + encoding: "utf8", + env: { ...process.env, RUN_OUTPUT_ROOT: ".tmp/runs/other" }, + }); + expect(result.status).toBe(1); + expect(result.stderr).toContain( + "scripts/generate-supply-chain.ts reads the build at this checkout's paths", + ); + expect(result.stderr).toContain("RUN_OUTPUT_ROOT=.tmp/runs/other"); + }); +}); diff --git a/tests/unit/run-isolation.test.ts b/tests/unit/run-isolation.test.ts index ddc33a5f..b4bd7d94 100644 --- a/tests/unit/run-isolation.test.ts +++ b/tests/unit/run-isolation.test.ts @@ -3,6 +3,7 @@ import { createServer, type Server } from "node:net"; import { afterEach, describe, expect, it, vi } from "vitest"; import { + checkoutBuildReadFailure, findFreePort, moduleSpecifier, reservePort, @@ -91,6 +92,24 @@ describe("build output root (RUN_OUTPUT_ROOT)", () => { expect(() => runOutputRoot({ RUN_OUTPUT_ROOT: root }), root).toThrow(/RUN_OUTPUT_ROOT/u); } }); + + // 1106/12-02. The steps after the build read the checkout's dist/ and + // artifacts/; under another root they judged an earlier build. + it("lets a step that reads the checkout's build run only when the build writes there", () => { + for (const environment of [{}, { RUN_OUTPUT_ROOT: "" }, { RUN_OUTPUT_ROOT: "." }, { RUN_OUTPUT_ROOT: "./" }]) { + expect(checkoutBuildReadFailure("check-bundle", environment), JSON.stringify(environment)) + .toBeUndefined(); + } + expect(checkoutBuildReadFailure("check-bundle", { RUN_OUTPUT_ROOT: "./.tmp/runs/a/" })).toBe( + "check-bundle reads the build at this checkout's paths (dist/, artifacts/), but " + + "RUN_OUTPUT_ROOT=.tmp/runs/a makes the build write under .tmp/runs/a/, so it would " + + "judge an earlier build. Unset RUN_OUTPUT_ROOT, build again and run check-bundle again.", + ); + // A root the build itself refuses is no reason to read the checkout's. + expect(checkoutBuildReadFailure("check-bundle", { RUN_OUTPUT_ROOT: "dist" })).toMatch( + /^check-bundle: RUN_OUTPUT_ROOT must be/u, + ); + }); }); describe("server ports", () => {