fix(test-run): RUN_OUTPUT_ROOT가 있으면 빌드 뒤 검사가 멈춘다

문제: 1105/12-04 뒤 빌드는 `RUN_OUTPUT_ROOT` 아래에 쓰는데,
check:bundle의 두 단계(`generate-supply-chain.ts`, `check-bundle.ts`)는 이
변수를 읽지 않고 checkout의 `dist`와 `artifacts/`를 읽는다. 셸에 이 변수를
둔 채 빌드와 check:bundle을 돌리면 예산을 3,504 B 넘는 새 빌드 대신 이전
빌드를 재서 PASS한다(리뷰 12-R2). ci:gate의 신선도 검사도 bundle.json을
check:bundle이 다시 쓰므로 막지 못한다.

고른 방법: `scripts/lib/run-isolation.ts`에 `checkoutBuildReadFailure`를
두고, 두 스크립트가 무엇을 읽기 전에 부른다. 출력 루트가 `.`이 아니면 변수
값과 이유를 적고 exit 1로 멈춘다. `.`(Playwright 설정이 run ID 없이 web
server에 넘기는 값)과 빈 값은 그대로 돈다. 그래서
`RUN_OUTPUT_ROOT=.tmp/x corepack pnpm ci:gate -- FE-GATE-012`도 check-bundle
단계에서 변수 이름을 적고 실패한다.
버린 대안: ci:gate가 시작 전에 거절하기(리뷰 문서 (b)). 12-R2처럼
check:bundle을 직접 돌리는 경우를 막지 못한다. 두 스크립트가 같은 루트를
읽게 하기(c)는 supply-chain 산출물 전부와 CI upload 경로를 함께 옮겨야 한다.

위험·한계:
- FE-GATE-012는 빌드 단계를 끝까지 돈 뒤 check-bundle에서 실패한다. 빌드
  시간만큼 늦게 알린다.
- 같은 모양일 수 있는 `check:app-build-input:bundle`,
  `check:optional-recipes --require-dist`, release candidate 조립은 보지
  않았다. FE-GATE-012에서는 check-bundle이 먼저 실패해 거기까지 가지 않는다.
- 디버깅으로 변수를 둔 사람은 check:bundle 전에 지워야 한다(문서에 적음).
- 이 노트북에서는 실행하지 못했다.

검증: desktop batch에서 확인.
node scripts/run-vitest.ts run tests/unit/check-bundle-output.test.ts tests/unit/run-isolation.test.ts
리뷰 12-R2 명령(stale-bundle-rc=1, stderr에 RUN_OUTPUT_ROOT), check:types,
lint, verify:documentation, ci:gate FE-GATE-012

Refs: 1106/12-02
This commit is contained in:
DongHyeonka
2026-11-06 05:35:48 +09:00
parent b8535f8ce7
commit 68fde02777
6 changed files with 143 additions and 13 deletions
@@ -1026,7 +1026,7 @@ TEST_RUN_ID=a playwright test
| 변수 | 누가 읽나 | 없을 때 |
|---|---|---|
| `TEST_RUN_ID` | 다섯 Playwright 설정, `tests/e2e/preload-use.spec.ts` | `dist`, `artifacts/tests/<이름>/` |
| `RUN_OUTPUT_ROOT` | 빌드 단계 전부(`scripts/build-frontend.ts`가 부르는 것), `vite.config.ts`. 설정이 web server에만 넘긴다 | `.` |
| `RUN_OUTPUT_ROOT` | 빌드 단계 전부(`scripts/build-frontend.ts`가 부르는 것), `vite.config.ts`. 설정이 web server에만 넘긴다. 빌드 뒤 검사(`scripts/generate-supply-chain.ts`, `scripts/check-bundle.ts`)는 `.`이 아니면 멈춘다 | `.` |
| `TEST_E2E_PORT`, `TEST_VISUAL_PORT`, `TEST_CAPABILITIES_PORT`, `TEST_STORYBOOK_PORT`, `TEST_DEV_PORT` | 각 설정 | 빈 포트 |
- run ID는 글자·숫자·`.`·`_`·`-` 64자 이하이고 글자나 숫자로 시작한다.
@@ -1035,6 +1035,9 @@ TEST_RUN_ID=a playwright test
- `RUN_OUTPUT_ROOT`는 직접 export하지 않는다. 설정이 run ID에서 만든다. 빌드
코드는 `TEST_RUN_ID`를 읽지 않는다. e2e가 저장소 복사본을 빌드할 때 그
빌드는 복사본 자기 `dist`에 써야 하기 때문이다.
- 빌드 뒤 검사는 checkout의 `dist`와 `artifacts/`만 읽는다. 그래서
`RUN_OUTPUT_ROOT`가 셸에 있으면 `check:bundle`은 변수 이름을 적고 실패한다.
빌드는 `.tmp/` 아래에 쓰고 검사는 이전 빌드를 재서 통과했기 때문이다(1106/12-02).
- 규칙은 `scripts/lib/run-isolation.ts` 한 곳에 있다.
두 e2e 실행을 동시에 돌리는 예:
+11
View File
@@ -9,6 +9,7 @@ import {
bundlePerformanceArtifactSchema,
} from "./contracts/release-artifacts.ts";
import { classifyViteJavascript } from "./lib/classify-vite-bundle.ts";
import { checkoutBuildReadFailure } from "./lib/run-isolation.ts";
import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts";
type ViteManifest = Record<
@@ -21,6 +22,16 @@ type BundleBudgets = {
lazyChunkGzipBytes: number;
};
// 1106/12-02. This reads dist/.vite/manifest.json and the inventory
// scripts/generate-supply-chain.ts made of dist. Under RUN_OUTPUT_ROOT the
// build wrote elsewhere, and this passed a build over budget on the earlier
// one the checkout still held.
const outputRootFailure = checkoutBuildReadFailure("scripts/check-bundle.ts");
if (outputRootFailure !== undefined) {
process.stderr.write(`${outputRootFailure}\n`);
process.exit(1);
}
const report = bundleOutputInventoryArtifactSchema.parse(
JSON.parse(await readFile("artifacts/performance/bundle.json", "utf8")) as unknown,
);
+10
View File
@@ -39,8 +39,18 @@ import {
parseRepositoryFileInventoryPolicy,
} from "./lib/repository-file-inventory.ts";
import { collectDistOutputs, distSha256 } from "./lib/release-candidate.ts";
import { checkoutBuildReadFailure } from "./lib/run-isolation.ts";
import { deterministicSupplyChainGeneratedAt } from "./lib/supply-chain-time.ts";
// 1106/12-02. This reads artifacts/release/build-manifest.json and dist where
// the checkout keeps them. Under RUN_OUTPUT_ROOT the build wrote elsewhere,
// and this recorded the earlier build the checkout still held.
const outputRootFailure = checkoutBuildReadFailure("scripts/generate-supply-chain.ts");
if (outputRootFailure !== undefined) {
process.stderr.write(`${outputRootFailure}\n`);
process.exit(1);
}
type Document = Record<string, unknown>;
function isRecord(value: unknown): value is Document {
+28
View File
@@ -82,6 +82,34 @@ export function runOutputPath(
return path.posix.join(runOutputRoot(environment), relativePath);
}
/**
* 1106/12-02. Why `step`, which reads the build where the checkout keeps it
* (`dist/`, the build manifest and the evidence under `artifacts/`), must not
* run in this environment; `undefined` when it may. With RUN_OUTPUT_ROOT set
* the build writes under that folder, so such a step would judge whatever
* build the checkout still holds: check:bundle passed an over-budget build
* that way, on the earlier one. `.` is the checkout itself, which is what a
* Playwright config hands its web server without a run ID.
*/
export function checkoutBuildReadFailure(
step: string,
environment: Environment = process.env,
): string | undefined {
let root: string;
try {
root = runOutputRoot(environment);
} catch (error) {
return `${step}: ${error instanceof Error ? error.message : String(error)}`;
}
if (root === ".") return undefined;
return (
`${step} reads the build at this checkout's paths (dist/, artifacts/), but ` +
`${RUN_OUTPUT_ROOT_VARIABLE}=${root} makes the build write under ${root}/, so it ` +
`would judge an earlier build. Unset ${RUN_OUTPUT_ROOT_VARIABLE}, build again and ` +
`run ${step} again.`
);
}
/**
* The port variable of each Playwright config's web server. Unset, the config
* picks a free port; set, it uses that one (to pin a port while debugging).
+71 -12
View File
@@ -24,6 +24,8 @@ async function checkBundle(
outputs: readonly Output[];
manifest: Record<string, unknown>;
budgets: Record<string, number>;
/** RUN_OUTPUT_ROOT for the run, and the build written under it. */
runOutput?: Readonly<{ root: string; outputs: readonly Output[] }>;
}>,
) {
const root = await mkdtemp(path.join(tmpdir(), "check-bundle-output-"));
@@ -32,22 +34,28 @@ async function checkBundle(
await mkdir(path.dirname(path.join(root, file)), { recursive: true });
await writeFile(path.join(root, file), `${JSON.stringify(value, null, 2)}\n`);
};
const writeBuild = async (buildRoot: string, outputs: readonly Output[]) => {
await write(path.posix.join(buildRoot, "dist/.vite/manifest.json"), options.manifest);
await write(path.posix.join(buildRoot, "artifacts/performance/bundle.json"), {
schemaVersion: 1,
generatedAt: "2026-11-04T00:00:00.000Z",
context: { nodeVersion: "v24.14.0", packageManager: "pnpm@11.17.0", runnerImage: "fixture" },
outputs: outputs.map((output) => ({
path: `dist/${output.path}`,
bytes: output.gzipBytes * 3,
gzipBytes: output.gzipBytes,
sha256: "a".repeat(64),
})),
});
};
await write("config/performance/budgets.json", { schemaVersion: 1, bundle: options.budgets });
await write("dist/.vite/manifest.json", options.manifest);
await write("artifacts/performance/bundle.json", {
schemaVersion: 1,
generatedAt: "2026-11-04T00:00:00.000Z",
context: { nodeVersion: "v24.14.0", packageManager: "pnpm@11.17.0", runnerImage: "fixture" },
outputs: options.outputs.map((output) => ({
path: `dist/${output.path}`,
bytes: output.gzipBytes * 3,
gzipBytes: output.gzipBytes,
sha256: "a".repeat(64),
})),
});
await writeBuild(".", options.outputs);
if (options.runOutput) await writeBuild(options.runOutput.root, options.runOutput.outputs);
const result = spawnSync(process.execPath, [path.resolve("scripts/check-bundle.ts")], {
cwd: root,
encoding: "utf8",
// Empty is unset (scripts/lib/run-isolation.ts), whatever the runner has.
env: { ...process.env, RUN_OUTPUT_ROOT: options.runOutput?.root ?? "" },
});
return { status: result.status, stdout: result.stdout, stderr: result.stderr };
}
@@ -126,3 +134,54 @@ describe("check:bundle failure output (1103/15-03)", () => {
expect(result.stderr).not.toContain("Bundle budget failed");
});
});
/**
* 1106/12-02. Under RUN_OUTPUT_ROOT scripts/build-frontend.ts writes the build
* below that folder, and check:bundle (scripts/generate-supply-chain.ts, then
* scripts/check-bundle.ts) read the checkout's dist/ and artifacts/ anyway: a
* build over budget under .tmp/ passed on the checkout's earlier build
* (review run 12-R2). Both stop and name the variable instead.
*/
describe("check:bundle with RUN_OUTPUT_ROOT (1106/12-02)", () => {
it("does not pass the checkout's earlier build while the build writes elsewhere", async () => {
const result = await checkBundle({
outputs: OUTPUTS,
manifest: MANIFEST,
budgets: BUDGETS,
runOutput: {
root: ".tmp/runs/other",
outputs: OUTPUTS.map((output) => ({ ...output, gzipBytes: output.gzipBytes * 4 })),
},
});
expect(result.stdout).not.toContain("Bundle budget: PASS");
expect(result.status).toBe(1);
expect(result.stderr).toContain("RUN_OUTPUT_ROOT=.tmp/runs/other");
});
it("measures the checkout's build when the output root is the checkout", async () => {
const result = await checkBundle({
outputs: OUTPUTS,
manifest: MANIFEST,
budgets: BUDGETS,
runOutput: { root: ".", outputs: OUTPUTS },
});
expect(result.stderr).toBe("");
expect(result.status).toBe(0);
expect(result.stdout).toContain("initial JS 900 / 1000 gzip bytes");
});
it("stops scripts/generate-supply-chain.ts before it reads the checkout's build", async () => {
const root = await mkdtemp(path.join(tmpdir(), "generate-supply-chain-output-root-"));
roots.push(root);
const result = spawnSync(process.execPath, [path.resolve("scripts/generate-supply-chain.ts")], {
cwd: root,
encoding: "utf8",
env: { ...process.env, RUN_OUTPUT_ROOT: ".tmp/runs/other" },
});
expect(result.status).toBe(1);
expect(result.stderr).toContain(
"scripts/generate-supply-chain.ts reads the build at this checkout's paths",
);
expect(result.stderr).toContain("RUN_OUTPUT_ROOT=.tmp/runs/other");
});
});
+19
View File
@@ -3,6 +3,7 @@ import { createServer, type Server } from "node:net";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
checkoutBuildReadFailure,
findFreePort,
moduleSpecifier,
reservePort,
@@ -91,6 +92,24 @@ describe("build output root (RUN_OUTPUT_ROOT)", () => {
expect(() => runOutputRoot({ RUN_OUTPUT_ROOT: root }), root).toThrow(/RUN_OUTPUT_ROOT/u);
}
});
// 1106/12-02. The steps after the build read the checkout's dist/ and
// artifacts/; under another root they judged an earlier build.
it("lets a step that reads the checkout's build run only when the build writes there", () => {
for (const environment of [{}, { RUN_OUTPUT_ROOT: "" }, { RUN_OUTPUT_ROOT: "." }, { RUN_OUTPUT_ROOT: "./" }]) {
expect(checkoutBuildReadFailure("check-bundle", environment), JSON.stringify(environment))
.toBeUndefined();
}
expect(checkoutBuildReadFailure("check-bundle", { RUN_OUTPUT_ROOT: "./.tmp/runs/a/" })).toBe(
"check-bundle reads the build at this checkout's paths (dist/, artifacts/), but " +
"RUN_OUTPUT_ROOT=.tmp/runs/a makes the build write under .tmp/runs/a/, so it would " +
"judge an earlier build. Unset RUN_OUTPUT_ROOT, build again and run check-bundle again.",
);
// A root the build itself refuses is no reason to read the checkout's.
expect(checkoutBuildReadFailure("check-bundle", { RUN_OUTPUT_ROOT: "dist" })).toMatch(
/^check-bundle: RUN_OUTPUT_ROOT must be/u,
);
});
});
describe("server ports", () => {