frontend-quality-gates / merge / static checks (pull_request) Canceled after 0s
frontend-quality-gates / merge / tests (pull_request) Canceled after 0s
frontend-quality-gates / merge / browser tests (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-014 / config-compatibility (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-026 / lab-performance (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-028 / manual-accessibility-review (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-015 / immutable-release-candidate (pull_request) Canceled after 0s
frontend-quality-gates / external-vulnerability-provider (pull_request) Canceled after 0s
frontend-quality-gates / external-provenance-provider (pull_request) Canceled after 0s
frontend-quality-gates / promote-verified-immutable-candidate (pull_request) Canceled after 0s
frontend-quality-gates / deploy-promoted-release (pull_request) Canceled after 0s
frontend-quality-gates / main / self-tests and removal drills (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-019 / hosting-header (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-021 / runbook-boot-config (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-022 / runbook-chunk-mismatch (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-023 / runbook-api-degradation (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-024 / runbook-telemetry (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-025 / runbook-release-rollback (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-018 / field-web-vitals (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-017 / diagram-review (pull_request) Canceled after 0s
문제: 자식 프로세스 시험의 timeout은 `childProcessTimeout(<잰 ms>)`로
정한다(docs/work/reference/testing.md 2절). 그런데 기본값이 아니라서
`tests/setup.ts` 검사에 걸리지 않는 손 숫자가 남았다. 리뷰가 센 54곳 중
둘(`browser-soak.contract.test.ts`의 `evaluateSoak` 인자)은 timeout이 아니고,
AST로 다시 세면 80곳이다: 시험·describe·hook에 쓴 숫자, `VERIFY_TIMEOUT`·
`PROCESS_HEAVY_TIMEOUT_MS`(파일 넷에 따로)·`PROVIDER_BASE_FIXTURE_TIMEOUT_MS`
같은 상수, `PROCESS_WATCHDOG_MS * 3` 같은 식. taxonomy는 옛 상수를 예로
들었다. 실측으로 보면 손 숫자 하나가 모자랐다: system 시험의 provider 묶음
30초는 desktop 여섯 번 중 두 번 넘겼다.
고친 것:
- 80곳을 desktop에서 잰 가장 긴 실행으로 `childProcessTimeout(<ms>)`에
넘긴다. coverage 묶음(maxWorkers=4)·test:unit 각 9회, integration 3회,
scale(traffic·session·composition·global-local) 3회, 시스템 컨테이너의
tests/system 6회, JSON reporter로 받았다. hook은 파일 시간에서 시험 합을
뺀 값이다. 시간 초과로 끝난 실행은 그 시간을 하한으로 넣었다.
- 상수 사본은 지운다. 한 파일 안에서 여럿이 쓰는 값만
`const X = childProcessTimeout(...)`로 둔다(`VERIFY_TIMEOUT`). guardian
시험의 lease는 시험 timeout이 아니라 `childProcessTimeout(0)`(helper의 최소
30초)로 둔다.
- `tests/unit/test-timeout-declarations.test.ts`가 tests/의 Vitest 파일을
Babel로 읽어 시험·describe·hook의 timeout 인자가 helper 호출(또는 그것을
담은 파일 상수)이 아니면 파일과 줄로 실패한다. 프로세스 안에서 도는 긴
시험(scale tier)도 같은 helper를 쓴다.
- taxonomy 예시, testing.md 2절, `.claude/rules/tests.md` 9번, helper 머리
주석을 같은 방법으로 맞춘다.
버린 대안:
- `tests/setup.ts` 검사를 「helper가 만든 값이 아닌 timeout」까지 넓히기.
helper 값은 30초 단위 숫자라 손으로 쓴 60_000과 가릴 수 없고, hook과
프로세스 안 시험은 보지 못한다.
- 프로세스 안 시험은 손 숫자 그대로 두기. 근거 없는 숫자가 남고 검사가
예외를 가져야 한다.
위험·한계:
- 값이 커진 곳이 있다(15배 규칙): `manual-code-blocks` 300초 → 900초,
provider 묶음 30초 → 480초 등. hang을 늦게 잡는다. 줄어든 곳도 있다
(scale traffic 60초 → 30초 등, 잰 값의 15배 안).
- 측정은 desktop 부하 6~35에서 했다. runner가 더 느리면 다시 잰다.
- 검사 시험은 Vitest 파일 전부를 파싱해 coverage 묶음에서 최대 13.5초
걸렸고, 그 값으로 자기 timeout을 준다. 이름 있는 함수를 콜백으로 넘긴
시험(`it("x", run, 30_000)`)은 못 찾는다.
- 다운스트림: 손 숫자 timeout이 있는 제품 시험은 test:unit에서 이 검사에
걸린다. 메시지가 helper를 알린다.
검증(desktop):
- RED: 검사 시험이 고치기 전 80곳을 파일·줄로 내고 실패. GREEN: 검사·
child-process-timeout 6개 통과. check:types·eslint(바뀐 34파일) rc=0,
verify:documentation PASS_SCOPED, 노트북 `scripts/verify.sh docs` OK.
- 측정 원자료: /tmp/claude-gates/w2fe2-t08m, -t08m2, -t08m3.
- 게이트 결과는 PR 본문에 적는다.
Refs: 1112/03-08
189 lines
6.5 KiB
TypeScript
189 lines
6.5 KiB
TypeScript
import { spawn } from "node:child_process";
|
|
import { readFile } from "node:fs/promises";
|
|
import { createServer, type Server } from "node:http";
|
|
import type { AddressInfo } from "node:net";
|
|
import path from "node:path";
|
|
|
|
import { afterEach, describe, expect, it } from "vitest";
|
|
|
|
import {
|
|
assertPortUnused,
|
|
assertServedRelease,
|
|
labBrowserLaunchOptions,
|
|
labFeatureRoute,
|
|
previewArguments,
|
|
summarizeSamples,
|
|
} from "../../scripts/lib/lab-performance.ts";
|
|
import {
|
|
NAVIGATION_ROUTES,
|
|
ROUTE_FEATURE_OWNER,
|
|
ROUTE_REGISTRY,
|
|
} from "../../src/features/installed-feature-contracts.ts";
|
|
import { childProcessTimeout } from "../helpers/child-process-timeout.ts";
|
|
|
|
/**
|
|
* 11-07. The lab performance gate measured once, and measured whatever server
|
|
* held its port: `vite preview` without `--strictPort` moved elsewhere and the
|
|
* script never checked which build answered.
|
|
*/
|
|
const servers: Server[] = [];
|
|
|
|
afterEach(async () => {
|
|
await Promise.all(
|
|
servers.splice(0).map(
|
|
(server) => new Promise<void>((resolve) => server.close(() => resolve())),
|
|
),
|
|
);
|
|
});
|
|
|
|
async function foreignServer(manifest: unknown): Promise<string> {
|
|
const server = createServer((_request, response) => {
|
|
response
|
|
.writeHead(200, { "content-type": "application/json" })
|
|
.end(JSON.stringify(manifest));
|
|
});
|
|
servers.push(server);
|
|
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", () => resolve()));
|
|
return `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
|
}
|
|
|
|
describe("lab performance gate (11-07)", () => {
|
|
it("starts the preview server on exactly the requested port", () => {
|
|
expect(previewArguments(4173)).toEqual([
|
|
"pnpm",
|
|
"preview",
|
|
"--host",
|
|
"127.0.0.1",
|
|
"--port",
|
|
"4173",
|
|
"--strictPort",
|
|
]);
|
|
});
|
|
|
|
it("summarizes repeated runs by their median and keeps every sample", () => {
|
|
expect(summarizeSamples([326, 1103, 410, 390, 505])).toEqual({
|
|
median: 410,
|
|
min: 326,
|
|
max: 1103,
|
|
samples: [326, 1103, 410, 390, 505],
|
|
});
|
|
expect(summarizeSamples([1, 4, 2, 3]).median).toBe(2.5);
|
|
expect(() => summarizeSamples([])).toThrow(RangeError);
|
|
});
|
|
|
|
it("refuses a port that another server already answers on", async () => {
|
|
const baseUrl = await foreignServer({});
|
|
await expect(assertPortUnused(baseUrl)).rejects.toThrow(/another server/u);
|
|
await expect(assertPortUnused("http://127.0.0.1:9")).resolves.toBeUndefined();
|
|
});
|
|
|
|
it("refuses a server that serves a different build", async () => {
|
|
const baseUrl = await foreignServer({ buildId: "other", releaseId: "r1" });
|
|
await expect(
|
|
assertServedRelease(baseUrl, { buildId: "measured", releaseId: "r1" }),
|
|
).rejects.toThrow(/serves build other\/r1, not the measured measured\/r1/u);
|
|
await expect(
|
|
assertServedRelease(baseUrl, { buildId: "other", releaseId: "r1" }),
|
|
).resolves.toBeUndefined();
|
|
});
|
|
|
|
it("fails the gate before measuring when its port is taken", async () => {
|
|
const baseUrl = await foreignServer({ buildId: "other", releaseId: "r1" });
|
|
const child = spawn(
|
|
process.execPath,
|
|
[path.resolve("scripts/test-performance.ts")],
|
|
{
|
|
env: { ...process.env, LAB_PERFORMANCE_PORT: new URL(baseUrl).port },
|
|
stdio: ["ignore", "ignore", "pipe"],
|
|
},
|
|
);
|
|
let stderr = "";
|
|
child.stderr.on("data", (chunk: Buffer) => {
|
|
stderr += chunk.toString("utf8");
|
|
});
|
|
const code = await new Promise<number | null>((resolve) => {
|
|
const timer = setTimeout(() => child.kill("SIGKILL"), 20_000);
|
|
child.on("exit", (exitCode) => {
|
|
clearTimeout(timer);
|
|
resolve(exitCode);
|
|
});
|
|
});
|
|
|
|
expect(code).not.toBe(0);
|
|
expect(stderr).toMatch(/another server holds the port/u);
|
|
}, childProcessTimeout(1_715));
|
|
});
|
|
|
|
/**
|
|
* 1105/15-05. The home page's first render added a vertical scrollbar and
|
|
* pushed the header's right side 15 px (CLS 0.0002 at 1280x720 and
|
|
* 1366x657). Every gate's Chromium hid scrollbars, the lab included, so no
|
|
* gate could see it. The page reserves the gutter, and the lab measures with
|
|
* the scrollbar a Windows or Linux desktop draws.
|
|
*/
|
|
describe("lab performance with real scrollbars (1105/15-05)", () => {
|
|
it("starts Chromium without Playwright's --hide-scrollbars", () => {
|
|
expect(labBrowserLaunchOptions().ignoreDefaultArgs).toContain("--hide-scrollbars");
|
|
});
|
|
|
|
it("reserves the root scrollbar gutter before the first route renders", async () => {
|
|
const base = await readFile("src/presentation/styles/theme/base.css", "utf8");
|
|
const root = base.match(/:root\s*\{([^}]*)\}/u)?.[1] ?? "";
|
|
|
|
expect(root).toMatch(/scrollbar-gutter:\s*stable\s*;/u);
|
|
});
|
|
});
|
|
|
|
/**
|
|
* 1105/15-04. The lab timed only the platform screen, which loads no feature
|
|
* code; a feature route's first entry (two request rounds, 406 ms under the
|
|
* lab conditions) had no gate. The lab now times the first public navigation
|
|
* route an installed feature owns, with the same clock.
|
|
*/
|
|
describe("lab feature route (1105/15-04)", () => {
|
|
const route = (
|
|
routeId: string,
|
|
navigationOrder: number | null,
|
|
access: "public" | "session-required" = "public",
|
|
) => ({
|
|
routeId,
|
|
path: `/${routeId.toLowerCase()}`,
|
|
access,
|
|
title: routeId,
|
|
navigationLabel: navigationOrder === null ? null : routeId,
|
|
navigationOrder,
|
|
});
|
|
|
|
it("takes the first public navigation route a feature owns", () => {
|
|
const routes = [
|
|
route("APP_HOME", 10),
|
|
route("EXAMPLES_PLATFORM", 15),
|
|
route("ORDERS_LIST", 50, "session-required"),
|
|
route("ORDERS_DETAIL", null),
|
|
route("GALLERY", 30),
|
|
route("CATALOG", 20),
|
|
];
|
|
const owners = {
|
|
ORDERS_LIST: "orders-feature",
|
|
ORDERS_DETAIL: "orders-feature",
|
|
GALLERY: "gallery-feature",
|
|
CATALOG: "catalog-feature",
|
|
};
|
|
|
|
expect(labFeatureRoute(routes, owners)?.routeId).toBe("CATALOG");
|
|
expect(labFeatureRoute(routes, { ORDERS_LIST: "orders-feature" })).toBeNull();
|
|
expect(labFeatureRoute(routes, {})).toBeNull();
|
|
});
|
|
|
|
it("finds one in the installed registry while a feature has a public screen", () => {
|
|
const selected = labFeatureRoute(Object.values(ROUTE_REGISTRY), ROUTE_FEATURE_OWNER);
|
|
const publicFeatureRoutes = NAVIGATION_ROUTES.filter(
|
|
(definition) =>
|
|
definition.access === "public" && ROUTE_FEATURE_OWNER[definition.routeId] !== undefined,
|
|
);
|
|
|
|
expect(selected?.routeId ?? null).toBe(publicFeatureRoutes[0]?.routeId ?? null);
|
|
if (selected) expect(ROUTE_FEATURE_OWNER[selected.routeId]).toEqual(expect.any(String));
|
|
});
|
|
});
|