문제: lab(FE-GATE-026)은 플랫폼 구성 화면(EXAMPLES_PLATFORM) 하나만 잰다.
이 화면은 기능 코드를 받지 않아 기능을 더해도 수치가 거의 그대로다
(146 → 154 ms). 기능 라우트는 처음 들어갈 때 context 라우트 모듈과 기능
실행부를 받고, 그다음 페이지 청크를 받는 두 번의 요청 묶음을 거친다.
lab 조건에서 EXAMPLES_UI 첫 진입은 중앙값 406 ms(403~410, 리뷰 15-R1)로
대상 화면의 2.7배인데 어느 게이트도 재지 않는다. 다운스트림의 화면은
모두 기능 라우트다.
고른 방법:
- labFeatureRoute(scripts/lib/lab-performance.ts): 설치된 기능이 가진,
세션 없이 열리는 첫 내비게이션 라우트. 템플릿에서는 EXAMPLES_UI다.
없으면 null이고 lab은 measured: false로 남기고 판정하지 않는다.
- lab은 run마다 `/`에서 그 라우트의 내비게이션 링크를 눌러 heading까지를
같은 페이지 시계로 잰다(새 cold context). metrics.featureRouteInteractionMs
(중앙값), context.sampling.featureRoute(표본, owner)에 남긴다. 기존 대상
라우트와 그 예산(200 ms)은 그대로라 회귀 비교가 이어진다.
- 예산 config/performance/budgets.json lab.featureRouteInteractionMs = 500 ms.
측정 406 ms(8a44226, i5-10400F, lab 조건)를 통과시키고, 지금 둘째 요청
묶음 크기(약 100 ms, 클릭 뒤 98~204 ms)의 왕복이 하나 더 붙으면 넘는다.
lab 자가 검사 fixture에 feature-route-over-threshold를 더해 이 예산이
실패할 수 있음을 매번 확인한다.
- 단위 테스트: 라우트 선택 규칙, 설치된 레지스트리에서의 선택, 커밋된
예산이 406 ms는 통과·516 ms는 실패·null은 판정 제외.
버린 대안:
- 두 요청 묶음을 하나로 줄이기(진입한 URL의 페이지 청크를 context 라우트
모듈과 같이 받기): 페이지 청크 이름이 context 라우트 모듈 안의 lazy()에만
있어, 첫 묶음에서 받으려면 초기 청크의 라우트 선언이 페이지 로더를 가져야
한다. 선언 모양·생성기·초기 JS를 바꾸는 일이라 이 커밋에서 하지 않았다
(결정 대기, performance-evidence.md에 적음).
- 처음에는 기록만 하기: 기록만 하면 늘어도 아무도 모른다. 측정값이 있으니
예산을 같이 둔다.
- 라우트별 예산 표: 다운스트림이 채워야 하는 표가 는다.
위험·한계: 예산 여유가 94 ms(23%)라 lab 호스트가 지금보다 20% 이상 느리면
기능 라우트 판정이 흔들릴 수 있다(0929/11-04 lab 부하 가드는 여전히 없음).
FE-GATE-026 시간이 run마다 cold navigation 하나(약 2~3초) 늘어 5회면
+10~15초다. 생성 기능의 목록 라우트는 session-required라 lab 대상이 되지
않는다. 기능 라우트의 heading은 계약 `title`과 같은 글자로 찾는다(대상
라우트와 같은 방식). CHANGELOG의 "기능 라우트 첫 진입은 어느 게이트도
재지 않는다" 제한 문장은 이 커밋에서 고치지 않았다(오케스트레이터 몫).
검증: desktop batch에서 확인.
node scripts/run-vitest.ts run tests/unit/lab-performance.test.ts
tests/unit/performance-budgets.test.ts
corepack pnpm build && corepack pnpm test:performance -- --runs=5
(featureRoute EXAMPLES_UI 중앙값이 500 ms 안인지, 표본 기록)
Refs: 1105/15-04
436 lines
16 KiB
TypeScript
436 lines
16 KiB
TypeScript
import { spawn } from "node:child_process";
|
|
import { mkdir, readFile } from "node:fs/promises";
|
|
import { cpus, loadavg } from "node:os";
|
|
import process from "node:process";
|
|
|
|
import { chromium, type Browser, type Page } from "@playwright/test";
|
|
|
|
import {
|
|
assertPortUnused,
|
|
assertServedRelease,
|
|
DEFAULT_LAB_PORT,
|
|
DEFAULT_LAB_RUNS,
|
|
labBrowserLaunchOptions,
|
|
labFeatureRoute,
|
|
previewArguments,
|
|
summarizeSamples,
|
|
} from "./lib/lab-performance.ts";
|
|
import { evaluateLabBudget } from "./lib/performance-budgets.ts";
|
|
import {
|
|
ROUTE_FEATURE_OWNER,
|
|
ROUTE_REGISTRY,
|
|
} from "../src/features/installed-feature-contracts.ts";
|
|
import { labPerformanceArtifactSchema } from "./contracts/release-artifacts.ts";
|
|
import { writeValidatedJsonArtifact } from "./lib/validated-json-artifact.ts";
|
|
|
|
type ContractPerformanceEvidence = {
|
|
lcpMs: number;
|
|
cls: number;
|
|
clickAt: number | null;
|
|
headingAt: number | null;
|
|
eventDurationMs: number | null;
|
|
};
|
|
|
|
type ContractPerformanceWindow = Window & {
|
|
__contractPerformance?: ContractPerformanceEvidence;
|
|
};
|
|
|
|
type LayoutShiftEntry = PerformanceEntry & {
|
|
hadRecentInput: boolean;
|
|
value: number;
|
|
};
|
|
|
|
const argument = (name: string) =>
|
|
process.argv.find((entry) => entry.startsWith(`--${name}=`))?.slice(name.length + 3);
|
|
const port = Number(process.env.LAB_PERFORMANCE_PORT ?? DEFAULT_LAB_PORT);
|
|
const runs = Number(argument("runs") ?? DEFAULT_LAB_RUNS);
|
|
if (!Number.isSafeInteger(port) || port <= 0 || !Number.isSafeInteger(runs) || runs < 1) {
|
|
throw new TypeError("LAB_PERFORMANCE_PORT must be a port and --runs a positive integer.");
|
|
}
|
|
const baseUrl = `http://127.0.0.1:${port}`;
|
|
const target = ROUTE_REGISTRY.EXAMPLES_PLATFORM;
|
|
const targetLabel = target.navigationLabel;
|
|
if (!targetLabel) {
|
|
throw new Error("Performance route must be present in navigation.");
|
|
}
|
|
// 1105/15-04. A feature route's first entry, timed like the target.
|
|
const featureRoute = labFeatureRoute(Object.values(ROUTE_REGISTRY), ROUTE_FEATURE_OWNER);
|
|
|
|
// 11-07. A server already on the port used to be measured in place of this
|
|
// build: vite moved to another port and the gate never noticed.
|
|
await assertPortUnused(baseUrl);
|
|
// Its own process group: `corepack` starts pnpm, which starts vite, and a
|
|
// signal to `corepack` alone left vite serving the port after the gate ended.
|
|
const server = spawn("corepack", [...previewArguments(port)], {
|
|
stdio: "ignore",
|
|
detached: true,
|
|
});
|
|
let serverExit: string | null = null;
|
|
server.on("exit", (code, signal) => {
|
|
serverExit = `preview server exited (${signal ?? code})`;
|
|
});
|
|
|
|
async function waitForServer() {
|
|
for (let attempt = 0; attempt < 100; attempt += 1) {
|
|
if (serverExit) throw new Error(serverExit);
|
|
try {
|
|
const response = await fetch(baseUrl);
|
|
if (response.ok) return;
|
|
} catch {
|
|
// The bounded retry loop handles startup races.
|
|
}
|
|
await new Promise((resolve) => setTimeout(resolve, 100));
|
|
}
|
|
throw new Error("Preview server did not become ready.");
|
|
}
|
|
|
|
type RunMetrics = {
|
|
lcpMs: number;
|
|
cls: number;
|
|
namedInteractionMs: number;
|
|
eventDurationMs: number | null;
|
|
/** 1105/15-01. CLS of a direct visit to the target route. */
|
|
targetDeepLinkCls: number;
|
|
/** 1105/15-04. Click to heading on `featureRoute`, null without one. */
|
|
featureRouteInteractionMs: number | null;
|
|
};
|
|
|
|
type ClickMetrics = Omit<RunMetrics, "targetDeepLinkCls" | "featureRouteInteractionMs">;
|
|
|
|
/**
|
|
* A cold, isolated context with the lab's network and CPU throttling and the
|
|
* page-side evidence recorder. `title` is the heading whose arrival after a
|
|
* click ends the named interaction.
|
|
*/
|
|
async function openMeasuredPage(browser: Browser, title: string) {
|
|
const context = await browser.newContext({
|
|
viewport: { width: 1280, height: 720 },
|
|
});
|
|
try {
|
|
const page = await context.newPage();
|
|
const cdp = await context.newCDPSession(page);
|
|
await cdp.send("Network.enable");
|
|
await cdp.send("Network.emulateNetworkConditions", {
|
|
offline: false,
|
|
latency: 40,
|
|
downloadThroughput: 200_000,
|
|
uploadThroughput: 93_750,
|
|
connectionType: "cellular4g",
|
|
});
|
|
await cdp.send("Emulation.setCPUThrottlingRate", { rate: 4 });
|
|
await page.addInitScript((headingTitle: string) => {
|
|
const evidence: ContractPerformanceEvidence = {
|
|
lcpMs: 0,
|
|
cls: 0,
|
|
clickAt: null,
|
|
headingAt: null,
|
|
eventDurationMs: null,
|
|
};
|
|
(window as ContractPerformanceWindow).__contractPerformance = evidence;
|
|
new PerformanceObserver((list) => {
|
|
for (const entry of list.getEntries()) evidence.lcpMs = entry.startTime;
|
|
}).observe({ type: "largest-contentful-paint", buffered: true });
|
|
new PerformanceObserver((list) => {
|
|
for (const entry of list.getEntries()) {
|
|
const layoutShift = entry as LayoutShiftEntry;
|
|
if (!layoutShift.hadRecentInput) {
|
|
evidence.cls += layoutShift.value;
|
|
}
|
|
}
|
|
}).observe({ type: "layout-shift", buffered: true });
|
|
// 11-07. The interaction is timed by the page's own clock, from the
|
|
// click event to the target heading entering the DOM, so Playwright's
|
|
// round trips are not in it. The Event Timing entry of the click (the
|
|
// INP input) is recorded next to it.
|
|
new PerformanceObserver((list) => {
|
|
for (const entry of list.getEntries()) {
|
|
if (entry.name === "click") {
|
|
evidence.eventDurationMs = Math.max(evidence.eventDurationMs ?? 0, entry.duration);
|
|
}
|
|
}
|
|
}).observe({ type: "event", buffered: true, durationThreshold: 16 } as PerformanceObserverInit);
|
|
document.addEventListener(
|
|
"click",
|
|
(event) => {
|
|
evidence.clickAt ??= event.timeStamp;
|
|
},
|
|
{ capture: true },
|
|
);
|
|
new MutationObserver((_records, observer) => {
|
|
if (evidence.clickAt === null) return;
|
|
const found = [...document.querySelectorAll("h1, h2")].some(
|
|
(heading) => heading.textContent?.trim() === headingTitle,
|
|
);
|
|
if (found) {
|
|
evidence.headingAt = performance.now();
|
|
observer.disconnect();
|
|
}
|
|
}).observe(document, { childList: true, subtree: true, characterData: true });
|
|
}, title);
|
|
return { context, page };
|
|
} catch (error) {
|
|
await context.close();
|
|
throw error;
|
|
}
|
|
}
|
|
|
|
/** The evidence once the layout-shift observer has delivered its entries. */
|
|
async function settledEvidence(page: Page) {
|
|
return page.evaluate(async () => {
|
|
// Observer callbacks run as tasks after the frame that recorded them.
|
|
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
|
await new Promise((resolve) => setTimeout(resolve, 0));
|
|
return (window as ContractPerformanceWindow).__contractPerformance;
|
|
});
|
|
}
|
|
|
|
/** From `/`, a click on `route`'s navigation link to its heading. */
|
|
async function measureClick(
|
|
browser: Browser,
|
|
route: Readonly<{ title: string; navigationLabel: string }>,
|
|
): Promise<ClickMetrics> {
|
|
// Every run is a cold, isolated context.
|
|
const { context, page } = await openMeasuredPage(browser, route.title);
|
|
try {
|
|
await page.goto(baseUrl, { waitUntil: "networkidle" });
|
|
// Playwright matches accessible names by substring, so the navigation entry
|
|
// "플랫폼 구성" also matched the home page's "플랫폼 구성 보기" call to
|
|
// action; `exact` keeps the locator on the navigation link.
|
|
await page.getByRole("link", { name: route.navigationLabel, exact: true }).click();
|
|
await page.getByRole("heading", { name: route.title }).waitFor();
|
|
await page.waitForFunction(
|
|
() => (window as ContractPerformanceWindow).__contractPerformance?.headingAt !== null,
|
|
);
|
|
const paint = await page.evaluate(
|
|
() => (window as ContractPerformanceWindow).__contractPerformance,
|
|
);
|
|
if (!paint || paint.clickAt === null || paint.headingAt === null) {
|
|
throw new Error("Browser performance evidence was not initialized.");
|
|
}
|
|
return {
|
|
lcpMs: Math.round(paint.lcpMs),
|
|
cls: Number(paint.cls.toFixed(4)),
|
|
namedInteractionMs: Math.round(paint.headingAt - paint.clickAt),
|
|
eventDurationMs:
|
|
paint.eventDurationMs === null ? null : Math.round(paint.eventDurationMs),
|
|
};
|
|
} finally {
|
|
await context.close();
|
|
}
|
|
}
|
|
|
|
/**
|
|
* 1105/15-01. The click run cannot see a shift the target screen makes while
|
|
* it fills in: a shift within 500 ms of the click has `hadRecentInput` and
|
|
* is left out of CLS by definition, and the lab only opened `/`. The target
|
|
* screen used to push its content 21 px after its title (CLS 0.0046) and the
|
|
* lab reported 0. A direct visit has no input, so every shift until the
|
|
* network is idle after the heading counts.
|
|
*/
|
|
async function measureTargetDeepLink(browser: Browser): Promise<number> {
|
|
const { context, page } = await openMeasuredPage(browser, target.title);
|
|
try {
|
|
await page.goto(new URL(target.path, baseUrl).href, { waitUntil: "networkidle" });
|
|
await page.getByRole("heading", { name: target.title, exact: true }).waitFor();
|
|
await page.waitForLoadState("networkidle");
|
|
const evidence = await settledEvidence(page);
|
|
if (!evidence) throw new Error("Browser performance evidence was not initialized.");
|
|
return Number(evidence.cls.toFixed(4));
|
|
} finally {
|
|
await context.close();
|
|
}
|
|
}
|
|
|
|
const load = () => loadavg().map((value) => Math.round(value * 100) / 100);
|
|
|
|
try {
|
|
await waitForServer();
|
|
const release = JSON.parse(
|
|
await readFile("dist/release-manifest.json", "utf8"),
|
|
);
|
|
await assertServedRelease(baseUrl, release);
|
|
const thresholds = JSON.parse(
|
|
await readFile("config/performance/budgets.json", "utf8"),
|
|
).lab;
|
|
const browser = await chromium.launch(labBrowserLaunchOptions());
|
|
try {
|
|
const loadBefore = load();
|
|
const samples: RunMetrics[] = [];
|
|
for (let run = 0; run < runs; run += 1) {
|
|
const click = await measureClick(browser, { title: target.title, navigationLabel: targetLabel });
|
|
const targetDeepLinkCls = await measureTargetDeepLink(browser);
|
|
const featureClick =
|
|
featureRoute === null
|
|
? null
|
|
: await measureClick(browser, {
|
|
title: featureRoute.title,
|
|
navigationLabel: featureRoute.navigationLabel!,
|
|
});
|
|
samples.push({
|
|
...click,
|
|
targetDeepLinkCls,
|
|
featureRouteInteractionMs: featureClick?.namedInteractionMs ?? null,
|
|
});
|
|
}
|
|
const loadAfter = load();
|
|
// The served build must not have changed under the runs either.
|
|
await assertServedRelease(baseUrl, release);
|
|
const lcp = summarizeSamples(samples.map((sample) => sample.lcpMs));
|
|
const cls = summarizeSamples(samples.map((sample) => sample.cls));
|
|
const interaction = summarizeSamples(
|
|
samples.map((sample) => sample.namedInteractionMs),
|
|
);
|
|
const deepLinkCls = summarizeSamples(
|
|
samples.map((sample) => sample.targetDeepLinkCls),
|
|
);
|
|
const featureSamples = samples.flatMap((sample) =>
|
|
sample.featureRouteInteractionMs === null ? [] : [sample.featureRouteInteractionMs],
|
|
);
|
|
const featureInteraction = featureSamples.length ? summarizeSamples(featureSamples) : null;
|
|
const eventDurations = samples.flatMap((sample) =>
|
|
sample.eventDurationMs === null ? [] : [sample.eventDurationMs],
|
|
);
|
|
const contextMetadata = {
|
|
runner: {
|
|
platform: process.platform,
|
|
architecture: process.arch,
|
|
nodeVersion: process.version,
|
|
cpuModel: cpus()[0]?.model.trim() ?? "unknown",
|
|
logicalCpus: cpus().length,
|
|
loadAverage: { before: loadBefore, after: loadAfter },
|
|
},
|
|
// 1105/15-05. Classic scrollbars, not Playwright's hidden ones.
|
|
browser: {
|
|
name: "chromium",
|
|
version: await browser.version(),
|
|
scrollbars: "visible",
|
|
},
|
|
viewport: { width: 1280, height: 720 },
|
|
network: {
|
|
profile: "contract-fast-4g",
|
|
latencyMs: 40,
|
|
downloadBytesPerSecond: 200_000,
|
|
uploadBytesPerSecond: 93_750,
|
|
},
|
|
cpu: { throttlingRate: 4 },
|
|
cache: { state: "cold", isolation: "new-browser-context" },
|
|
build: {
|
|
buildId: release.buildId,
|
|
releaseId: release.releaseId,
|
|
served: `${baseUrl}/release-manifest.json`,
|
|
},
|
|
// 11-07. The budget is judged on the median of cold runs; every run and
|
|
// the spread stay in the evidence.
|
|
sampling: {
|
|
runs,
|
|
statistic: "median",
|
|
interactionClock: "page: click event to target heading in DOM",
|
|
lcpMs: lcp,
|
|
cls,
|
|
namedInteractionMs: interaction,
|
|
clickEventDurationMs: eventDurations.length
|
|
? summarizeSamples(eventDurations)
|
|
: null,
|
|
// 1105/15-01. A cold direct visit to the target route in each run.
|
|
targetDeepLink: {
|
|
path: target.path,
|
|
clock: "navigation start to network idle after the target heading",
|
|
cls: deepLinkCls,
|
|
},
|
|
// 1105/15-04. The first public navigation route a feature owns, from
|
|
// `/` by its navigation link, with the same clock.
|
|
featureRoute:
|
|
featureRoute === null || featureInteraction === null
|
|
? { measured: false, reason: "no installed feature owns a public navigation route" }
|
|
: {
|
|
measured: true,
|
|
routeId: featureRoute.routeId,
|
|
path: featureRoute.path,
|
|
owner: ROUTE_FEATURE_OWNER[featureRoute.routeId] ?? null,
|
|
interactionMs: featureInteraction,
|
|
},
|
|
},
|
|
};
|
|
const metrics = {
|
|
lcpMs: lcp.median,
|
|
cls: cls.median,
|
|
namedInteractionMs: interaction.median,
|
|
targetDeepLinkCls: deepLinkCls.median,
|
|
featureRouteInteractionMs: featureInteraction?.median ?? null,
|
|
};
|
|
const result = evaluateLabBudget(
|
|
{ context: contextMetadata, metrics },
|
|
thresholds,
|
|
);
|
|
const fixtures = [
|
|
{
|
|
name: "missing-context",
|
|
passed: !evaluateLabBudget({ metrics }, thresholds).passed,
|
|
},
|
|
{
|
|
name: "lcp-over-threshold",
|
|
passed: !evaluateLabBudget(
|
|
{
|
|
context: contextMetadata,
|
|
metrics: { ...metrics, lcpMs: thresholds.lcpMs + 1 },
|
|
},
|
|
thresholds,
|
|
).passed,
|
|
},
|
|
// 1105/15-04. The feature route budget can fail.
|
|
{
|
|
name: "feature-route-over-threshold",
|
|
passed: !evaluateLabBudget(
|
|
{
|
|
context: contextMetadata,
|
|
metrics: {
|
|
...metrics,
|
|
featureRouteInteractionMs: thresholds.featureRouteInteractionMs + 1,
|
|
},
|
|
},
|
|
thresholds,
|
|
).passed,
|
|
},
|
|
];
|
|
const passed = result.passed && fixtures.every((fixture) => fixture.passed);
|
|
await mkdir("artifacts/performance", { recursive: true });
|
|
await writeValidatedJsonArtifact({
|
|
path: "artifacts/performance/lab.json",
|
|
schema: labPerformanceArtifactSchema,
|
|
value: {
|
|
schemaVersion: 1,
|
|
generatedAt: new Date().toISOString(),
|
|
context: contextMetadata,
|
|
metrics,
|
|
thresholds,
|
|
fixtures,
|
|
passed,
|
|
},
|
|
});
|
|
if (!passed) {
|
|
throw new Error(
|
|
`Lab performance failed (median of ${runs}): ${JSON.stringify(metrics)}; ` +
|
|
`interaction samples ${interaction.samples.join(",")}` +
|
|
(featureInteraction === null
|
|
? ""
|
|
: `; feature route samples ${featureInteraction.samples.join(",")}`),
|
|
);
|
|
}
|
|
process.stdout.write(
|
|
`Lab performance: PASS, median of ${runs} (LCP ${metrics.lcpMs}ms, CLS ${metrics.cls}, interaction ${metrics.namedInteractionMs}ms ` +
|
|
`[${interaction.min}-${interaction.max}], ${target.path} direct CLS ${metrics.targetDeepLinkCls} ` +
|
|
`[${deepLinkCls.min}-${deepLinkCls.max}], ` +
|
|
(featureRoute === null || featureInteraction === null
|
|
? "no feature route measured)\n"
|
|
: `feature route ${featureRoute.routeId} ${featureInteraction.median}ms ` +
|
|
`[${featureInteraction.min}-${featureInteraction.max}])\n`),
|
|
);
|
|
} finally {
|
|
await browser.close();
|
|
}
|
|
} finally {
|
|
if (server.pid !== undefined && serverExit === null) {
|
|
process.kill(-server.pid, "SIGTERM");
|
|
}
|
|
}
|