frontend-quality-gates / merge / static checks (pull_request) Canceled after 0s
frontend-quality-gates / merge / tests (pull_request) Canceled after 0s
frontend-quality-gates / merge / browser tests (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-014 / config-compatibility (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-026 / lab-performance (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-028 / manual-accessibility-review (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-015 / immutable-release-candidate (pull_request) Canceled after 0s
frontend-quality-gates / external-vulnerability-provider (pull_request) Canceled after 0s
frontend-quality-gates / external-provenance-provider (pull_request) Canceled after 0s
frontend-quality-gates / promote-verified-immutable-candidate (pull_request) Canceled after 0s
frontend-quality-gates / deploy-promoted-release (pull_request) Canceled after 0s
frontend-quality-gates / main / self-tests and removal drills (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-019 / hosting-header (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-021 / runbook-boot-config (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-022 / runbook-chunk-mismatch (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-023 / runbook-api-degradation (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-024 / runbook-telemetry (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-025 / runbook-release-rollback (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-018 / field-web-vitals (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-017 / diagram-review (pull_request) Canceled after 0s
문제: 자식 프로세스 시험의 timeout은 `childProcessTimeout(<잰 ms>)`로
정한다(docs/work/reference/testing.md 2절). 그런데 기본값이 아니라서
`tests/setup.ts` 검사에 걸리지 않는 손 숫자가 남았다. 리뷰가 센 54곳 중
둘(`browser-soak.contract.test.ts`의 `evaluateSoak` 인자)은 timeout이 아니고,
AST로 다시 세면 80곳이다: 시험·describe·hook에 쓴 숫자, `VERIFY_TIMEOUT`·
`PROCESS_HEAVY_TIMEOUT_MS`(파일 넷에 따로)·`PROVIDER_BASE_FIXTURE_TIMEOUT_MS`
같은 상수, `PROCESS_WATCHDOG_MS * 3` 같은 식. taxonomy는 옛 상수를 예로
들었다. 실측으로 보면 손 숫자 하나가 모자랐다: system 시험의 provider 묶음
30초는 desktop 여섯 번 중 두 번 넘겼다.
고친 것:
- 80곳을 desktop에서 잰 가장 긴 실행으로 `childProcessTimeout(<ms>)`에
넘긴다. coverage 묶음(maxWorkers=4)·test:unit 각 9회, integration 3회,
scale(traffic·session·composition·global-local) 3회, 시스템 컨테이너의
tests/system 6회, JSON reporter로 받았다. hook은 파일 시간에서 시험 합을
뺀 값이다. 시간 초과로 끝난 실행은 그 시간을 하한으로 넣었다.
- 상수 사본은 지운다. 한 파일 안에서 여럿이 쓰는 값만
`const X = childProcessTimeout(...)`로 둔다(`VERIFY_TIMEOUT`). guardian
시험의 lease는 시험 timeout이 아니라 `childProcessTimeout(0)`(helper의 최소
30초)로 둔다.
- `tests/unit/test-timeout-declarations.test.ts`가 tests/의 Vitest 파일을
Babel로 읽어 시험·describe·hook의 timeout 인자가 helper 호출(또는 그것을
담은 파일 상수)이 아니면 파일과 줄로 실패한다. 프로세스 안에서 도는 긴
시험(scale tier)도 같은 helper를 쓴다.
- taxonomy 예시, testing.md 2절, `.claude/rules/tests.md` 9번, helper 머리
주석을 같은 방법으로 맞춘다.
버린 대안:
- `tests/setup.ts` 검사를 「helper가 만든 값이 아닌 timeout」까지 넓히기.
helper 값은 30초 단위 숫자라 손으로 쓴 60_000과 가릴 수 없고, hook과
프로세스 안 시험은 보지 못한다.
- 프로세스 안 시험은 손 숫자 그대로 두기. 근거 없는 숫자가 남고 검사가
예외를 가져야 한다.
위험·한계:
- 값이 커진 곳이 있다(15배 규칙): `manual-code-blocks` 300초 → 900초,
provider 묶음 30초 → 480초 등. hang을 늦게 잡는다. 줄어든 곳도 있다
(scale traffic 60초 → 30초 등, 잰 값의 15배 안).
- 측정은 desktop 부하 6~35에서 했다. runner가 더 느리면 다시 잰다.
- 검사 시험은 Vitest 파일 전부를 파싱해 coverage 묶음에서 최대 13.5초
걸렸고, 그 값으로 자기 timeout을 준다. 이름 있는 함수를 콜백으로 넘긴
시험(`it("x", run, 30_000)`)은 못 찾는다.
- 다운스트림: 손 숫자 timeout이 있는 제품 시험은 test:unit에서 이 검사에
걸린다. 메시지가 helper를 알린다.
검증(desktop):
- RED: 검사 시험이 고치기 전 80곳을 파일·줄로 내고 실패. GREEN: 검사·
child-process-timeout 6개 통과. check:types·eslint(바뀐 34파일) rc=0,
verify:documentation PASS_SCOPED, 노트북 `scripts/verify.sh docs` OK.
- 측정 원자료: /tmp/claude-gates/w2fe2-t08m, -t08m2, -t08m3.
- 게이트 결과는 PR 본문에 적는다.
Refs: 1112/03-08
249 lines
11 KiB
TypeScript
249 lines
11 KiB
TypeScript
import { readFile } from "node:fs/promises";
|
|
|
|
import { describe, expect, it } from "vitest";
|
|
|
|
import {
|
|
FEATURE_SHAPE_BASELINE_PATH,
|
|
FEATURE_SHAPE_CEILINGS_PATH,
|
|
FEATURES_PER_CONTEXT,
|
|
initialJavascriptRecordDrift,
|
|
measureInitialJavascriptCosts,
|
|
planShapeFeatures,
|
|
SHAPE_FEATURE_COMMAND,
|
|
shapeFeature,
|
|
type InitialJavascriptCosts,
|
|
} from "../../scripts/scale/measure-feature-shape-growth.ts";
|
|
import { APP_MAX_CONTEXTS } from "../../src/apps/template-app/app-module.ts";
|
|
import { childProcessTimeout } from "../helpers/child-process-timeout.ts";
|
|
|
|
/**
|
|
* 11-04 (D-18). The scale evidence used to come from stub features that only
|
|
* differ by name, so the cost of one app growing towards its declared ceiling
|
|
* (20 contexts x 200 features) was never measured. The committed baseline is a
|
|
* measurement with copies of a real feature: since 2026-11-07 the one
|
|
* `create:feature` writes, before that the template's reference sample.
|
|
*/
|
|
type Run = Readonly<{
|
|
addedFeatures: number;
|
|
routes: number;
|
|
tsc: Readonly<{ passed: boolean; instantiations: number; wallMs: number }>;
|
|
build: Readonly<{
|
|
wallMs: number;
|
|
initialJavascriptGzipBytes: number;
|
|
lazyJavascriptFiles: number;
|
|
}>;
|
|
loadBefore: readonly number[];
|
|
}>;
|
|
|
|
describe("feature-shape growth measurement (11-04)", () => {
|
|
it("plans each copy with create:feature, 200 to a context", () => {
|
|
const copies = FEATURES_PER_CONTEXT + 1;
|
|
const files = planShapeFeatures(copies);
|
|
const first = shapeFeature(0);
|
|
const last = shapeFeature(copies - 1);
|
|
|
|
expect([...files.keys()].filter((file) => file.endsWith("/context-module.ts"))).toEqual([
|
|
`src/contexts/${first.context.id}/context-module.ts`,
|
|
`src/contexts/${last.context.id}/context-module.ts`,
|
|
]);
|
|
expect(last.context.id).not.toBe(first.context.id);
|
|
for (const index of [0, FEATURES_PER_CONTEXT - 1, FEATURES_PER_CONTEXT]) {
|
|
const { id, context } = shapeFeature(index);
|
|
const module = files.get(`src/features/${id}/feature-module.ts`);
|
|
expect(module, id).toContain(`FEATURE_CONTEXT_ID = "${context.id}"`);
|
|
// The write slice: its detail and create pages are planned too.
|
|
expect([...files.keys()].filter((file) => file.startsWith(`src/features/${id}/`))).toEqual(
|
|
expect.arrayContaining([
|
|
expect.stringMatching(/-detail-page\.tsx$/u),
|
|
expect.stringMatching(/-create-page\.tsx$/u),
|
|
]),
|
|
);
|
|
}
|
|
// Only what a build reads: no tests, no accessibility records.
|
|
expect([...files.keys()].every((file) => file.startsWith("src/"))).toBe(true);
|
|
});
|
|
|
|
// 1101/07-04. Assembly refuses the template fixture source in a product
|
|
// feature, so every copy pins an external package of its own, and has a
|
|
// topic of its own.
|
|
it("pins a package and a topic of its own for every copy", () => {
|
|
const files = planShapeFeatures(3);
|
|
const contracts = [0, 1, 2].map((index) => {
|
|
const { id } = shapeFeature(index);
|
|
const contract = [...files].find(([file]) =>
|
|
file.startsWith(`src/features/${id}/contracts/`) && file.endsWith("-feature-contract.ts"),
|
|
)?.[1];
|
|
expect(contract, id).toContain(`packageId: "@feature-shape/${id}"`);
|
|
return contract ?? "";
|
|
});
|
|
const topics = contracts.map((contract) => /"(qinv\.[0-9a-z]{26})"/u.exec(contract)?.[1]);
|
|
expect(new Set(topics).size).toBe(3);
|
|
expect(contracts.join("\n")).not.toContain("PLACEHOLDER_CONTRACT_PACKAGE");
|
|
});
|
|
|
|
it("commits a baseline measured at 25 and 100 real-shape features, twice each", async () => {
|
|
const baseline = JSON.parse(
|
|
await readFile(FEATURE_SHAPE_BASELINE_PATH, "utf8"),
|
|
) as Readonly<{
|
|
environment: Readonly<Record<string, unknown>>;
|
|
source: unknown;
|
|
sourceFeature: unknown;
|
|
runs: readonly Run[];
|
|
}>;
|
|
// 2026-11-07. Measured with the generated feature, not the deleted sample.
|
|
expect(baseline.sourceFeature).toBe(SHAPE_FEATURE_COMMAND);
|
|
expect(baseline.environment).toMatchObject({
|
|
cpuModel: expect.any(String),
|
|
logicalCpus: expect.any(Number),
|
|
});
|
|
// 11-05 (2026-09-29). Without the commit it measured, nobody can tell
|
|
// whether the baseline still describes the code or compare a new run.
|
|
expect(baseline.source).toEqual({
|
|
commit: expect.stringMatching(/^[0-9a-f]{40}$/u),
|
|
modifiedPaths: expect.any(Array),
|
|
});
|
|
|
|
const bySize = (size: number) =>
|
|
baseline.runs.filter((run) => run.addedFeatures === size);
|
|
const [origin] = bySize(0);
|
|
expect(origin?.tsc.passed).toBe(true);
|
|
for (const size of [25, 100]) {
|
|
const runs = bySize(size);
|
|
// A failed typecheck is the result and is not repeated.
|
|
expect(runs.length).toBeGreaterThanOrEqual(runs[0]?.tsc.passed ? 2 : 1);
|
|
for (const run of runs) {
|
|
// Every copy contributes the generated feature's own routes.
|
|
expect((run.routes - origin!.routes) % size).toBe(0);
|
|
expect(run.routes).toBeGreaterThan(origin!.routes);
|
|
expect(run.build.lazyJavascriptFiles).toBeGreaterThan(
|
|
origin!.build.lazyJavascriptFiles,
|
|
);
|
|
expect(run.loadBefore).toHaveLength(3);
|
|
}
|
|
// Compiler counts are deterministic; only times vary with the host.
|
|
expect(new Set(runs.map((run) => run.tsc.instantiations)).size).toBe(1);
|
|
}
|
|
});
|
|
|
|
/**
|
|
* 15-01. The initial budget (config/performance/budgets.json) is a fixed
|
|
* 202 KiB for the whole app, so what one installed feature adds to the
|
|
* initial chunk decides how many features fit. Before 15-01 every feature's
|
|
* contracts, schemas, mappers, HTTP gateway and API loaded at boot: +3,922 B
|
|
* per copy, and the first added feature exceeded the budget. Only what the
|
|
* shell needs before a feature route is entered may load at boot (route
|
|
* table, navigation and page messages, invalidation identities); the rest
|
|
* loads with the feature's routes. The bound also fails if the
|
|
* feature-declarations chunk group in vite.config.ts stops matching.
|
|
*
|
|
* 1103/15-04. The first copy also adds a context (shape-context-0), so one
|
|
* number mixed a context's cost with a feature's and had 40 B left under
|
|
* the bound. A second copy in the same context is one more feature alone;
|
|
* the second copy in a context of its own is one more context. Each has its
|
|
* own bound and the failure names which one grew. 2026-11-07: with no
|
|
* feature in the template, the first copy also brings the code all features
|
|
* share into the initial chunks (`firstFeatureGzipBytes`); it is paid once
|
|
* and recorded, not bounded here: the initial budget (check:bundle) bounds
|
|
* the total.
|
|
*
|
|
* 1103/15-02. The record in config/scale/feature-shape-ceilings.json is the
|
|
* one place the documents take the template's baseline from. Written by
|
|
* hand, it went stale twice; it now has to match this build.
|
|
*/
|
|
it("bounds what one more feature and one more context add, and matches the record (15-01)", async () => {
|
|
const measured = await measureInitialJavascriptCosts();
|
|
const { initialJavascript: record } = JSON.parse(
|
|
await readFile(FEATURE_SHAPE_CEILINGS_PATH, "utf8"),
|
|
) as { initialJavascript: InitialJavascriptCosts };
|
|
const observed = `measured ${JSON.stringify(measured)}`;
|
|
|
|
expect(measured.perFeatureGzipBytes, `one more feature in an existing context: ${observed}`)
|
|
.toBeLessThanOrEqual(1_024);
|
|
expect(measured.perContextGzipBytes, `one more context: ${observed}`)
|
|
.toBeLessThanOrEqual(512);
|
|
expect(
|
|
initialJavascriptRecordDrift(measured, record),
|
|
`${FEATURE_SHAPE_CEILINGS_PATH} initialJavascript is stale (${observed}); ` +
|
|
"rewrite it with node scripts/scale/measure-feature-shape-growth.ts --initial-baseline --write",
|
|
).toEqual([]);
|
|
}, childProcessTimeout(26_359));
|
|
|
|
/**
|
|
* 1105/15-02. The total used to match within 1% (2,029 B at 202,927 B),
|
|
* more than the room left under the budget (1,873 B) and more than one
|
|
* feature (768 B), so a template change that took the room could leave the
|
|
* record. Now a few hundred bytes are enough to fail: above the build id
|
|
* spread (tens of bytes), below one feature.
|
|
*/
|
|
it("reports a record a few hundred bytes off the build, or a different file count", () => {
|
|
const measured = {
|
|
gzipBytes: 200_000,
|
|
files: 25,
|
|
firstFeatureGzipBytes: 5_000,
|
|
perFeatureGzipBytes: 700,
|
|
perContextGzipBytes: 250,
|
|
};
|
|
expect(initialJavascriptRecordDrift(measured, measured)).toEqual([]);
|
|
// The build id alone moves the total by tens of bytes.
|
|
expect(
|
|
initialJavascriptRecordDrift(measured, { ...measured, gzipBytes: 200_048 }),
|
|
).toEqual([]);
|
|
expect(
|
|
initialJavascriptRecordDrift(measured, { ...measured, gzipBytes: 199_800 }),
|
|
).toEqual([]);
|
|
// A template change that grows the initial JavaScript by 300 B.
|
|
expect(
|
|
initialJavascriptRecordDrift({ ...measured, gzipBytes: 200_300 }, measured),
|
|
).toEqual(["gzipBytes: record 200000, build 200300"]);
|
|
expect(
|
|
initialJavascriptRecordDrift(measured, { ...measured, gzipBytes: 201_500 }),
|
|
).toEqual(["gzipBytes: record 201500, build 200000"]);
|
|
expect(
|
|
initialJavascriptRecordDrift(measured, { ...measured, gzipBytes: 204_000 }),
|
|
).toEqual(["gzipBytes: record 204000, build 200000"]);
|
|
expect(initialJavascriptRecordDrift(measured, { ...measured, files: 26 })).toEqual([
|
|
"files: record 26, build 25",
|
|
]);
|
|
expect(
|
|
initialJavascriptRecordDrift(measured, { ...measured, perFeatureGzipBytes: 800 }),
|
|
).toEqual(["perFeatureGzipBytes: record 800, build 700"]);
|
|
// A cost of a few bytes keeps a floor: gzip moves it by a byte or two.
|
|
const tiny = { ...measured, perContextGzipBytes: 4 };
|
|
expect(initialJavascriptRecordDrift(tiny, { ...tiny, perContextGzipBytes: 6 })).toEqual([]);
|
|
expect(initialJavascriptRecordDrift(tiny, { ...tiny, perContextGzipBytes: 30 })).toEqual([
|
|
"perContextGzipBytes: record 30, build 4",
|
|
]);
|
|
});
|
|
|
|
it("states in the committed rationale what the ceilings rest on", async () => {
|
|
// The rationale lives next to the baseline, not in a review document: the
|
|
// history documents may be deleted after copying (README.md).
|
|
const rationale = JSON.parse(
|
|
await readFile(FEATURE_SHAPE_CEILINGS_PATH, "utf8"),
|
|
) as Readonly<{
|
|
baseline: string;
|
|
ceilings: Readonly<{ contextMaxFeatures: number; appMaxContexts: number }>;
|
|
axes: readonly Readonly<Record<string, string>>[];
|
|
}>;
|
|
|
|
expect(rationale.baseline).toBe(FEATURE_SHAPE_BASELINE_PATH);
|
|
expect(rationale.ceilings).toEqual({
|
|
contextMaxFeatures: FEATURES_PER_CONTEXT,
|
|
appMaxContexts: APP_MAX_CONTEXTS,
|
|
});
|
|
expect(rationale.axes.map((row) => row.axis)).toEqual(
|
|
expect.arrayContaining([
|
|
"Typecheck of one app",
|
|
"Production build",
|
|
"Initial JavaScript",
|
|
"Lazy JavaScript",
|
|
"Routes",
|
|
]),
|
|
);
|
|
for (const row of rationale.axes) {
|
|
for (const column of ["axis", "verified", "condition", "notVerified"])
|
|
expect(row[column], `${row.axis}.${column}`).toMatch(/\S/u);
|
|
}
|
|
});
|
|
});
|