문제: config/scale/feature-shape-growth.json에 측정 시각만 있고 어느
코드를 쟀는지가 없다. 기준선이 지금 코드와 맞는지, 다음 측정과 비교할 수
있는지 알 수 없다. 실제로 기준선은 2026-09-27 측정이고, 그 뒤 15-01이
초기 청크 구조를 바꿨다(ceilings.json도 "baseline predates 15-01"이라고
적고 있었다). 기준선 0개 초기 JS 200,001 B, 기능당 2,907 B는 지금 코드와
다르다.
고른 방법:
- 측정 스크립트가 시작할 때 `git rev-parse HEAD`와 추적 파일 변경 목록
(`git status --porcelain --untracked-files=no`)을 읽어 결과의
`source: { commit, modifiedPaths }`에 싣는다. 변경 목록이 비면 커밋만으로
재현된다는 뜻이다. 측정 중 트리가 바뀔 수 있어 시작할 때 읽는다.
- 기준선을 현재 코드로 다시 잰다(--sizes=0,25,100 --repeats=2 --write).
- ceilings.json의 수치를 새 기준선에 맞춘다. 초기 JS 여유는 리뷰 개선안대로
"예산 − 기준선 ÷ 기능당 증가"로 적는다(2,291 B ÷ 약 610 B ≈ 3개).
- 단위 테스트가 기준선에 40자 commit과 modifiedPaths 배열이 있는지 본다.
버린 대안:
- 기준선 커밋이 현재 이력의 조상인지 테스트로 확인: 이력 없이 3-way로
이식하는 다운스트림(tech-log)에서는 템플릿 커밋이 없어 항상 실패한다.
- dirty 트리면 --write를 거절: 이 커밋처럼 측정 스크립트와 기준선을 한
커밋에 넣을 수 없게 된다. 대신 변경 경로를 그대로 기록한다.
위험·한계:
- 기록된 커밋은 2be973a(이 브랜치의 직전 커밋)이고 modifiedPaths는 측정
스크립트와 그 테스트 2개다. 측정 대상(src, vite 설정)은 그 커밋과 같다.
이 브랜치를 rebase하면 기록된 SHA가 이력에서 사라진다. 그때는 다시 재야
한다.
- 시간 값은 호스트 부하(loadavg 27~29)에 따라 흔들린다. 바이트와
instantiation 수만 비교 대상이다.
- 15-01 커밋 본문의 202,361 B와 이번 0개 측정 202,509 B는 148 B 다르다.
원인은 확인하지 않았다. ceilings.json은 기준선 수치 하나로 통일했다.
검증:
- RED: node scripts/run-vitest.ts run tests/unit/feature-shape-growth.test.ts
-t 'commits a baseline' — "expected undefined to deeply equal { commit:
StringMatching…" 실패.
- 측정(flock): 0개 202,509 B·720,060 inst, 25개 218,010 B, 100개
263,119 B·12,399,171 inst, 반복 2회 바이트·inst 동일. 기능당 초기 JS
약 610 B, lazy 약 8.3 KB·8파일, inst 약 117k, tsc 메모리 약 36 MB,
route 4개.
- 재현: 같은 트리에서 --sizes=0,25 --repeats=1을 다시 돌려 0개 202,509 B,
25개 218,010 B로 같음.
- GREEN: feature-shape-growth.test.ts 5 passed(빌드 측정 테스트 포함).
- tsc(node·test), eslint(바꾼 2파일), verify-documentation-readiness
PASS_SCOPED.
Refs: docs/reviews/2026-09-29-maturity-review/11-tests-browser-release-scale.md 11-05
308 lines
13 KiB
TypeScript
308 lines
13 KiB
TypeScript
import { spawnSync } from "node:child_process";
|
|
import { cp, mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
|
|
import { availableParallelism, cpus, loadavg, tmpdir } from "node:os";
|
|
import path from "node:path";
|
|
import { pathToFileURL } from "node:url";
|
|
import { gzipSync } from "node:zlib";
|
|
|
|
import { classifyViteJavascript } from "../lib/classify-vite-bundle.ts";
|
|
import { linkFixtureNodeModules } from "../lib/fixture-node-modules.ts";
|
|
import { parseTscExtendedDiagnostics, type TscDiagnostics } from "./collect-tsc-diagnostics.ts";
|
|
|
|
/**
|
|
* 11-04 (D-18). Cost of one app as its feature count grows, measured with
|
|
* features shaped like the real one: `src/features/reference-feature` copied N
|
|
* times with every feature-owned identifier renamed (feature id, route ids and
|
|
* paths, API operations, schemas, query namespace, invalidation topic,
|
|
* messages). The copies live in contexts of their own (200 features each, the
|
|
* context ceiling) so the reference context is unchanged.
|
|
*
|
|
* For each N the script runs the product generator, `tsc --extendedDiagnostics`
|
|
* over tsconfig.app.json and `vite build`, and records compiler counts, wall
|
|
* times and initial/lazy JavaScript gzip bytes. Deterministic counts repeat
|
|
* exactly; times depend on the host, so the CPU and load average are recorded
|
|
* next to them. A typecheck that fails (for example TS2590, a union too complex
|
|
* to represent) is recorded as a result with its first error.
|
|
*
|
|
* node scripts/scale/measure-feature-shape-growth.ts [--sizes=0,25,100] [--repeats=2] [--write]
|
|
*
|
|
* `--write` stores the result as config/scale/feature-shape-growth.json.
|
|
*/
|
|
export const FEATURE_SHAPE_BASELINE_PATH = "config/scale/feature-shape-growth.json";
|
|
/** What the declared ceilings rest on, derived by hand from the baseline. */
|
|
export const FEATURE_SHAPE_CEILINGS_PATH = "config/scale/feature-shape-ceilings.json";
|
|
|
|
const SOURCE_FEATURE = "src/features/reference-feature";
|
|
const FEATURES_PER_CONTEXT = 200;
|
|
const BUILD_INPUTS = [
|
|
"src",
|
|
"public",
|
|
"index.html",
|
|
"package.json",
|
|
"vite.config.ts",
|
|
"tsconfig.base.json",
|
|
"tsconfig.app.json",
|
|
"scripts/lib/vite-module-inventory.ts",
|
|
] as const;
|
|
// Platform-owned names that only happen to contain "reference".
|
|
const PLATFORM_TOKENS = [
|
|
"REFERENCE_EXTERNAL_BEARER",
|
|
"REFERENCE_FEATURE_V1",
|
|
"supportReference",
|
|
] as const;
|
|
|
|
/** Renames every feature-owned identifier of the reference feature for copy `id`. */
|
|
export function renameReferenceFeature(
|
|
text: string,
|
|
id: string,
|
|
context: Readonly<{ id: string; routeRoot: string }>,
|
|
): string {
|
|
const upper = id.toUpperCase();
|
|
let renamed = text;
|
|
PLATFORM_TOKENS.forEach((token, index) => {
|
|
renamed = renamed.replaceAll(token, `\u0000${index}\u0000`);
|
|
});
|
|
renamed = renamed
|
|
.replaceAll("/examples/reference-resources", `/${context.routeRoot}/${id}-resources`)
|
|
.replaceAll("qinv.01k10f7m3w9p6r2c8v5n4x", `qinv.shape.${id}`)
|
|
.replaceAll("NoRequest", `${upper}NoRequest`)
|
|
.replaceAll("reference-context", context.id)
|
|
.replaceAll("REFERENCE", upper)
|
|
.replaceAll("Reference", upper)
|
|
.replaceAll("reference", id);
|
|
PLATFORM_TOKENS.forEach((token, index) => {
|
|
renamed = renamed.replaceAll(`\u0000${index}\u0000`, token);
|
|
});
|
|
return renamed;
|
|
}
|
|
|
|
async function copyRenamed(from: string, to: string, rename: (text: string) => string) {
|
|
await mkdir(to, { recursive: true });
|
|
for (const entry of await readdir(from, { withFileTypes: true })) {
|
|
const source = path.join(from, entry.name);
|
|
const target = path.join(to, rename(entry.name));
|
|
if (entry.isDirectory()) await copyRenamed(source, target, rename);
|
|
else await writeFile(target, rename(await readFile(source, "utf8")));
|
|
}
|
|
}
|
|
|
|
async function createFixture(features: number): Promise<string> {
|
|
const root = await mkdtemp(path.join(tmpdir(), `feature-shape-${features}-`));
|
|
for (const input of BUILD_INPUTS) {
|
|
await mkdir(path.dirname(path.join(root, input)), { recursive: true });
|
|
await cp(path.resolve(input), path.join(root, input), { recursive: true });
|
|
}
|
|
await rm(path.join(root, "src/.generated"), { recursive: true, force: true });
|
|
await linkFixtureNodeModules(root);
|
|
for (let index = 0; index < features; index += 1) {
|
|
const contextNumber = Math.floor(index / FEATURES_PER_CONTEXT);
|
|
const context = { id: `shape-context-${contextNumber}`, routeRoot: `shape-${contextNumber}` };
|
|
if (index % FEATURES_PER_CONTEXT === 0) {
|
|
const contextFile = path.join(root, "src/contexts", context.id, "context-module.ts");
|
|
await mkdir(path.dirname(contextFile), { recursive: true });
|
|
await writeFile(
|
|
contextFile,
|
|
`export const CONTEXT_MODULE_ID = "${context.id}";\n` +
|
|
`export const CONTEXT_APP_ID = "template-app";\n` +
|
|
`export const CONTEXT_ROUTE_PATH = "${context.routeRoot}/*";\n` +
|
|
`export const CONTEXT_MAX_FEATURES = ${FEATURES_PER_CONTEXT};\n`,
|
|
);
|
|
}
|
|
const id = `s${String(index).padStart(3, "0")}`;
|
|
await copyRenamed(
|
|
path.resolve(SOURCE_FEATURE),
|
|
path.join(root, "src/features", `${id}-feature`),
|
|
(text) => renameReferenceFeature(text, id, context),
|
|
);
|
|
}
|
|
return root;
|
|
}
|
|
|
|
function run(
|
|
command: string,
|
|
args: readonly string[],
|
|
cwd: string,
|
|
options: Readonly<{ allowFailure?: boolean }> = {},
|
|
) {
|
|
const started = performance.now();
|
|
const result = spawnSync(command, args, {
|
|
cwd,
|
|
encoding: "utf8",
|
|
maxBuffer: 64 * 1024 * 1024,
|
|
});
|
|
const ms = Math.round(performance.now() - started);
|
|
if (result.status !== 0 && !(options.allowFailure && result.stdout.includes("Instantiations:"))) {
|
|
throw new Error(
|
|
`${path.basename(command)} ${args.join(" ")} failed in ${cwd}:\n${(result.stdout + result.stderr).slice(0, 4_000)}`,
|
|
);
|
|
}
|
|
return { ms, passed: result.status === 0, stdout: result.stdout };
|
|
}
|
|
|
|
const load = () => loadavg().map((value) => Math.round(value * 100) / 100);
|
|
|
|
async function javascriptGzipBytes(root: string, files: readonly string[]) {
|
|
let bytes = 0;
|
|
for (const file of files) bytes += gzipSync(await readFile(path.join(root, "dist", file))).byteLength;
|
|
return bytes;
|
|
}
|
|
|
|
/**
|
|
* 15-01. Initial JavaScript gzip bytes of the app with `features` reference
|
|
* copies added: the generator and `vite build` only, no typecheck. The unit
|
|
* suite (merge stage) bounds what one more feature adds to the initial chunk
|
|
* with it, so the growth is caught on the PR instead of at the next release.
|
|
*/
|
|
export async function measureInitialJavascriptGzipBytes(features: number): Promise<number> {
|
|
const root = await createFixture(features);
|
|
try {
|
|
run(process.execPath, [path.resolve("scripts/generate-product-feature-index.ts"), `--root=${root}`], root);
|
|
run(path.resolve("node_modules/.bin/vite"), ["build", "--logLevel", "error"], root);
|
|
const manifest = JSON.parse(await readFile(path.join(root, "dist/.vite/manifest.json"), "utf8"));
|
|
return await javascriptGzipBytes(root, classifyViteJavascript(manifest).initialFiles);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
}
|
|
|
|
async function measureOnce(root: string, features: number) {
|
|
const loadBefore = load();
|
|
run(process.execPath, [path.resolve("scripts/generate-product-feature-index.ts"), `--root=${root}`], root);
|
|
// The composed registries throw at module evaluation on a duplicate route,
|
|
// operation, schema, namespace or message key: prove the copies compose.
|
|
const contracts = (await import(
|
|
`${pathToFileURL(path.join(root, "src/features/installed-feature-contracts.ts")).href}?features=${features}`
|
|
)) as typeof import("../../src/features/installed-feature-contracts.ts");
|
|
// 15-01. Operations, schemas and contract contributions compose from the
|
|
// runtime halves, which only this module loads.
|
|
await import(
|
|
`${pathToFileURL(path.join(root, "src/features/loaded-feature-runtimes.ts")).href}?features=${features}`
|
|
);
|
|
const routes = Object.keys(contracts.ROUTE_REGISTRY).length;
|
|
const tsc = run(
|
|
path.resolve("node_modules/.bin/tsc"),
|
|
["--project", path.join(root, "tsconfig.app.json"), "--noEmit", "--extendedDiagnostics"],
|
|
root,
|
|
// A type error is a result here (the typecheck stops scaling), not a crash.
|
|
{ allowFailure: true },
|
|
);
|
|
await rm(path.join(root, "dist"), { recursive: true, force: true });
|
|
const build = run(path.resolve("node_modules/.bin/vite"), ["build", "--logLevel", "error"], root);
|
|
const manifest = JSON.parse(await readFile(path.join(root, "dist/.vite/manifest.json"), "utf8"));
|
|
const javascript = classifyViteJavascript(manifest);
|
|
const diagnostics: TscDiagnostics = parseTscExtendedDiagnostics(tsc.stdout);
|
|
return {
|
|
addedFeatures: features,
|
|
routes,
|
|
tsc: {
|
|
passed: tsc.passed,
|
|
firstError: tsc.stdout.split("\n").find((line) => line.includes("error TS")) ?? null,
|
|
files: diagnostics.files,
|
|
types: diagnostics.types,
|
|
symbols: diagnostics.symbols,
|
|
instantiations: diagnostics.instantiations,
|
|
memoryKb: diagnostics.memoryKb ?? null,
|
|
wallMs: tsc.ms,
|
|
},
|
|
build: {
|
|
wallMs: build.ms,
|
|
initialJavascriptFiles: javascript.initialFiles.length,
|
|
initialJavascriptGzipBytes: await javascriptGzipBytes(root, javascript.initialFiles),
|
|
lazyJavascriptFiles: javascript.lazyFiles.length,
|
|
lazyJavascriptGzipBytes: await javascriptGzipBytes(root, javascript.lazyFiles),
|
|
},
|
|
loadBefore,
|
|
loadAfter: load(),
|
|
};
|
|
}
|
|
|
|
/**
|
|
* 11-05 (2026-09-29). The commit the run measured, so a later run can be
|
|
* compared with the same code and a reader can tell whether the baseline
|
|
* still describes the tree. Tracked files that differed from that commit are
|
|
* listed; an empty list means the commit alone reproduces the run.
|
|
*/
|
|
function sourceRevision(): Readonly<{ commit: string; modifiedPaths: string[] }> {
|
|
const git = (...args: string[]) => spawnSync("git", args, { encoding: "utf8" });
|
|
const head = git("rev-parse", "HEAD");
|
|
const status = git("status", "--porcelain", "--untracked-files=no");
|
|
if (head.status !== 0 || status.status !== 0) {
|
|
throw new Error("feature-shape growth: cannot read the source commit (git rev-parse HEAD, git status)");
|
|
}
|
|
return {
|
|
commit: head.stdout.trim(),
|
|
modifiedPaths: status.stdout
|
|
.split("\n")
|
|
.filter(Boolean)
|
|
.map((line) => line.slice(3)),
|
|
};
|
|
}
|
|
|
|
function version(bin: string): string {
|
|
const result = spawnSync(path.resolve("node_modules/.bin", bin), ["--version"], { encoding: "utf8" });
|
|
return (result.stdout ?? "").trim();
|
|
}
|
|
|
|
async function main(): Promise<void> {
|
|
const argument = (name: string) =>
|
|
process.argv.find((entry) => entry.startsWith(`--${name}=`))?.slice(name.length + 3);
|
|
const sizes = (argument("sizes") ?? "0,25,100").split(",").map(Number);
|
|
const repeats = Number(argument("repeats") ?? "2");
|
|
if (!sizes.every((size) => Number.isSafeInteger(size) && size >= 0) || !(repeats >= 1)) {
|
|
throw new TypeError("--sizes must be nonnegative integers and --repeats at least 1");
|
|
}
|
|
// Read before measuring: the run takes minutes and the tree may move.
|
|
const source = sourceRevision();
|
|
const runs = [];
|
|
for (const features of sizes) {
|
|
const root = await createFixture(features);
|
|
try {
|
|
for (let repeat = 1; repeat <= repeats; repeat += 1) {
|
|
const measured = await measureOnce(root, features);
|
|
runs.push({ repeat, ...measured });
|
|
process.stdout.write(
|
|
`features +${features} (repeat ${repeat}): routes=${measured.routes} ` +
|
|
`instantiations=${measured.tsc.instantiations} tsc=${measured.tsc.wallMs}ms ` +
|
|
`build=${measured.build.wallMs}ms initial=${measured.build.initialJavascriptGzipBytes}B ` +
|
|
`lazy=${measured.build.lazyJavascriptGzipBytes}B load=${measured.loadBefore.join("/")}` +
|
|
`${measured.tsc.passed ? "" : ` typecheck FAILED: ${measured.tsc.firstError}`}\n`,
|
|
);
|
|
// Compiler results repeat exactly; a failed typecheck costs minutes and
|
|
// gigabytes again for no new information.
|
|
if (!measured.tsc.passed) break;
|
|
}
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
}
|
|
const cpu = cpus();
|
|
const result = {
|
|
schemaVersion: 1,
|
|
measuredAt: new Date().toISOString(),
|
|
command: `node scripts/scale/measure-feature-shape-growth.ts --sizes=${sizes.join(",")} --repeats=${repeats}`,
|
|
environment: {
|
|
node: process.versions.node,
|
|
platform: process.platform,
|
|
arch: process.arch,
|
|
cpuModel: cpu[0]?.model.trim() ?? "unknown",
|
|
logicalCpus: cpu.length,
|
|
availableParallelism: availableParallelism(),
|
|
typescript: version("tsc"),
|
|
vite: version("vite"),
|
|
},
|
|
source,
|
|
sourceFeature: SOURCE_FEATURE,
|
|
runs,
|
|
};
|
|
const output = process.argv.includes("--write")
|
|
? FEATURE_SHAPE_BASELINE_PATH
|
|
: "artifacts/scale/feature-shape-growth.json";
|
|
await mkdir(path.dirname(output), { recursive: true });
|
|
await writeFile(output, `${JSON.stringify(result, null, 2)}\n`);
|
|
process.stdout.write(`feature-shape growth: ${output}\n`);
|
|
}
|
|
|
|
if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
|
|
await main();
|
|
}
|