Files
clean-architecture-frontend…/scripts/scale/measure-feature-shape-growth.ts
T
DongHyeonka c9e7bab2d7 test(scale): 기능·context 비용을 따로 재고 초기 JS 기준선을 빌드와 대조한다
문제: (1) 기능당 1 KiB 상한 테스트는 복사본 0개와 1개만 쟀다. 첫
복사본은 새 context(shape-context-0)도 만들어서, 잰 값에 context 비용이
섞였고 상한까지 40 B만 남았다. 넘으면 기능이 커졌는지 context가
비싸졌는지 알 수 없었다. (2) 성능 기준선 숫자는 performance-evidence.md,
static-delivery.md, feature-shape-ceilings.json에 손으로 적혀 있었고
현재 빌드와도, 같은 파일 안에서도 서로 달랐다(1101/15-02 재발).
(3) 이 테스트는 vitest 안에서 돌아 NODE_ENV=test를 물려받았다. 그래서
픽스처 vite build가 React 개발 빌드를 묶었고, 잰 합계가 운영 빌드
(약 201 KB)가 아니라 약 264 KB였다.

고른 방법: 복사본 0·1·2개를 잰다. 둘째 복사본은 같은 context에 들어가
기능 하나의 비용이고, 첫 복사본에서 그것을 뺀 나머지가 context 하나의
비용이다. 기능은 1 KiB, context는 512 B로 따로 막고 실패 메시지에 잰
값을 넣는다. 기준선은 feature-shape-ceilings.json의 initialJavascript
한 곳에 기계가 읽는 칸으로 두고, 같은 테스트가 빌드와 대조한다(합계
1%, 파일 수 정확히, 기능·context 비용 10%). 기록은
`measure-feature-shape-growth.ts --initial-baseline --write`로 다시 쓴다.
문서는 숫자 대신 그 칸을 가리킨다. 픽스처 명령은 NODE_ENV를 지우고
돈다.
버린 대안: 문서 숫자를 다시 손으로 고치기(두 번 낡았다), 문서에서
숫자를 지우고 bundle.json만 가리키기(예산을 정할 숫자를 보려면 빌드가
필요하다).

위험·한계: unit에 빌드가 하나 늘었다(약 3초). 초기 JS를 움직이는
변경은 기록도 다시 써야 test:unit이 통과한다. 그것이 목적이다.
`298201f` 출처별 크기 표와 `ec7435c` HTTP/1.1 측정 표는 날짜가 붙은
측정 기록으로 남겼다.

검증: 기록을 쓰기 전 새 테스트가 실패했다. 기록 gzipBytes를 2%
바꾸면 실패하고 되돌리면 통과한다. 지금 기록: 200,939 B, 파일 25개,
기능 +765 B, context +214 B. feature-shape-growth 7건 통과,
verify:documentation PASS_SCOPED.

Refs: docs/reviews/2026-11-03-maturity-review/15-performance.md 15-02, 15-04
2026-11-04 20:34:39 +09:00

409 lines
17 KiB
TypeScript

import { spawnSync } from "node:child_process";
import { cp, mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { availableParallelism, cpus, loadavg, tmpdir } from "node:os";
import path from "node:path";
import { pathToFileURL } from "node:url";
import { gzipSync } from "node:zlib";
import { classifyViteJavascript } from "../lib/classify-vite-bundle.ts";
import { linkFixtureNodeModules } from "../lib/fixture-node-modules.ts";
import { parseTscExtendedDiagnostics, type TscDiagnostics } from "./collect-tsc-diagnostics.ts";
/**
* 11-04 (D-18). Cost of one app as its feature count grows, measured with
* features shaped like the real one: `src/features/reference-feature` copied N
* times with every feature-owned identifier renamed (feature id, route ids and
* paths, API operations, schemas, query namespace, invalidation topic,
* messages). The copies live in contexts of their own (200 features each, the
* context ceiling) so the reference context is unchanged.
*
* For each N the script runs the product generator, `tsc --extendedDiagnostics`
* over tsconfig.app.json and `vite build`, and records compiler counts, wall
* times and initial/lazy JavaScript gzip bytes. Deterministic counts repeat
* exactly; times depend on the host, so the CPU and load average are recorded
* next to them. A typecheck that fails (for example TS2590, a union too complex
* to represent) is recorded as a result with its first error.
*
* node scripts/scale/measure-feature-shape-growth.ts [--sizes=0,25,100] [--repeats=2] [--write]
* node scripts/scale/measure-feature-shape-growth.ts --initial-baseline [--write]
*
* `--write` stores the result as config/scale/feature-shape-growth.json, or
* with `--initial-baseline` the initial JavaScript record in
* config/scale/feature-shape-ceilings.json.
*/
export const FEATURE_SHAPE_BASELINE_PATH = "config/scale/feature-shape-growth.json";
/** What the declared ceilings rest on, derived by hand from the baseline. */
export const FEATURE_SHAPE_CEILINGS_PATH = "config/scale/feature-shape-ceilings.json";
const SOURCE_FEATURE = "src/features/reference-feature";
const FEATURES_PER_CONTEXT = 200;
const BUILD_INPUTS = [
"src",
"public",
"index.html",
"package.json",
"vite.config.ts",
"tsconfig.base.json",
"tsconfig.app.json",
"scripts/lib/vite-module-inventory.ts",
] as const;
// Platform-owned names that only happen to contain "reference".
const PLATFORM_TOKENS = [
"REFERENCE_FEATURE_V1",
"supportReference",
] as const;
/** Renames every feature-owned identifier of the reference feature for copy `id`. */
export function renameReferenceFeature(
text: string,
id: string,
context: Readonly<{ id: string; routeRoot: string }>,
): string {
const upper = id.toUpperCase();
// 1101/07-04. Assembly refuses the reference fixture source in any other
// feature, so a copy pins an external package and declares it in its
// contracts, as a product feature would.
const contractPackage = [
"Object.freeze({",
` packageId: "@feature-shape/${id}",`,
' version: "1.0.0",',
' digest: `sha256:${"0".repeat(64)}`,',
" runtimeProtocolVersion: 1 as const,",
' sourceRevision: "0000000",',
" })",
].join("\n");
let renamed = text
.replace(
/source: Object\.freeze\(\{\s*kind: "TEMPLATE_FIXTURE" as const,\s*fixtureId: "REFERENCE_FEATURE_V1" as const,\s*revision: 1 as const,\s*\}\)/u,
`source: Object.freeze({\n kind: "EXTERNAL_PACKAGE" as const,\n package: ${contractPackage},\n })`,
)
.replace(
"contractPackages: Object.freeze([]),",
`contractPackages: Object.freeze([${contractPackage}]),`,
);
PLATFORM_TOKENS.forEach((token, index) => {
renamed = renamed.replaceAll(token, `\u0000${index}\u0000`);
});
renamed = renamed
.replaceAll("/examples/reference-resources", `/${context.routeRoot}/${id}-resources`)
.replaceAll("qinv.01k10f7m3w9p6r2c8v5n4x", `qinv.shape.${id}`)
.replaceAll("NoRequest", `${upper}NoRequest`)
.replaceAll("reference-context", context.id)
.replaceAll("REFERENCE", upper)
.replaceAll("Reference", upper)
.replaceAll("reference", id);
PLATFORM_TOKENS.forEach((token, index) => {
renamed = renamed.replaceAll(`\u0000${index}\u0000`, token);
});
return renamed;
}
async function copyRenamed(from: string, to: string, rename: (text: string) => string) {
await mkdir(to, { recursive: true });
for (const entry of await readdir(from, { withFileTypes: true })) {
const source = path.join(from, entry.name);
const target = path.join(to, rename(entry.name));
if (entry.isDirectory()) await copyRenamed(source, target, rename);
else await writeFile(target, rename(await readFile(source, "utf8")));
}
}
async function createFixture(features: number): Promise<string> {
const root = await mkdtemp(path.join(tmpdir(), `feature-shape-${features}-`));
for (const input of BUILD_INPUTS) {
await mkdir(path.dirname(path.join(root, input)), { recursive: true });
await cp(path.resolve(input), path.join(root, input), { recursive: true });
}
await rm(path.join(root, "src/.generated"), { recursive: true, force: true });
await linkFixtureNodeModules(root);
for (let index = 0; index < features; index += 1) {
const contextNumber = Math.floor(index / FEATURES_PER_CONTEXT);
const context = { id: `shape-context-${contextNumber}`, routeRoot: `shape-${contextNumber}` };
if (index % FEATURES_PER_CONTEXT === 0) {
const contextFile = path.join(root, "src/contexts", context.id, "context-module.ts");
await mkdir(path.dirname(contextFile), { recursive: true });
await writeFile(
contextFile,
`export const CONTEXT_MODULE_ID = "${context.id}";\n` +
`export const CONTEXT_APP_ID = "template-app";\n` +
`export const CONTEXT_ROUTE_PATH = "${context.routeRoot}/*";\n` +
`export const CONTEXT_MAX_FEATURES = ${FEATURES_PER_CONTEXT};\n`,
);
}
const id = `s${String(index).padStart(3, "0")}`;
await copyRenamed(
path.resolve(SOURCE_FEATURE),
path.join(root, "src/features", `${id}-feature`),
(text) => renameReferenceFeature(text, id, context),
);
}
return root;
}
function run(
command: string,
args: readonly string[],
cwd: string,
options: Readonly<{ allowFailure?: boolean }> = {},
) {
// 1103/15-02. Vitest sets NODE_ENV=test, and a vite build that inherits it
// bundles React's development build: the unit test measured 264 KB where
// the production build is 201 KB. Every command here measures production.
const { NODE_ENV: _inherited, ...environment } = process.env;
const started = performance.now();
const result = spawnSync(command, args, {
cwd,
encoding: "utf8",
env: environment,
maxBuffer: 64 * 1024 * 1024,
});
const ms = Math.round(performance.now() - started);
if (result.status !== 0 && !(options.allowFailure && result.stdout.includes("Instantiations:"))) {
throw new Error(
`${path.basename(command)} ${args.join(" ")} failed in ${cwd}:\n${(result.stdout + result.stderr).slice(0, 4_000)}`,
);
}
return { ms, passed: result.status === 0, stdout: result.stdout };
}
const load = () => loadavg().map((value) => Math.round(value * 100) / 100);
async function javascriptGzipBytes(root: string, files: readonly string[]) {
let bytes = 0;
for (const file of files) bytes += gzipSync(await readFile(path.join(root, "dist", file))).byteLength;
return bytes;
}
/**
* 15-01. Initial JavaScript of the app with `features` reference copies
* added: the generator and `vite build` only, no typecheck. The unit suite
* (merge stage) bounds what one more feature adds to the initial chunk with
* it, so the growth is caught on the PR instead of at the next release.
*/
async function measureInitialJavascript(
features: number,
): Promise<Readonly<{ gzipBytes: number; files: number }>> {
const root = await createFixture(features);
try {
run(process.execPath, [path.resolve("scripts/generate-product-feature-index.ts"), `--root=${root}`], root);
run(path.resolve("node_modules/.bin/vite"), ["build", "--logLevel", "error"], root);
const manifest = JSON.parse(await readFile(path.join(root, "dist/.vite/manifest.json"), "utf8"));
const { initialFiles } = classifyViteJavascript(manifest);
return { gzipBytes: await javascriptGzipBytes(root, initialFiles), files: initialFiles.length };
} finally {
await rm(root, { recursive: true, force: true });
}
}
/**
* 1103/15-02, 15-04. The template's initial JavaScript and what one more
* feature and one more context add to it. The first copy lands in a new
* context (shape-context-0) and the second in the same one, so the second
* copy is a feature alone and the rest of the first is the context.
* `initialJavascript` in config/scale/feature-shape-ceilings.json records
* these numbers; tests/unit/feature-shape-growth.test.ts compares them with
* the build on every unit run.
*/
export type InitialJavascriptCosts = Readonly<{
gzipBytes: number;
files: number;
perFeatureGzipBytes: number;
perContextGzipBytes: number;
}>;
export async function measureInitialJavascriptCosts(): Promise<InitialJavascriptCosts> {
const none = await measureInitialJavascript(0);
const one = await measureInitialJavascript(1);
const two = await measureInitialJavascript(2);
const perFeatureGzipBytes = two.gzipBytes - one.gzipBytes;
return {
gzipBytes: none.gzipBytes,
files: none.files,
perFeatureGzipBytes,
perContextGzipBytes: one.gzipBytes - none.gzipBytes - perFeatureGzipBytes,
};
}
/**
* Fields of the record that no longer describe the build. The total moves
* by tens of bytes with the build id (CI and local builds differ), so it may
* be 1% off; the per-feature and per-context costs are differences of builds
* with the same id but only a few hundred bytes, so 10%; the file count is
* exact.
*/
export function initialJavascriptRecordDrift(
measured: InitialJavascriptCosts,
record: InitialJavascriptCosts,
): string[] {
const tolerance: Readonly<Record<keyof InitialJavascriptCosts, number>> = {
gzipBytes: 0.01,
files: 0,
perFeatureGzipBytes: 0.1,
perContextGzipBytes: 0.1,
};
return (Object.keys(tolerance) as (keyof InitialJavascriptCosts)[])
.filter((field) => Math.abs(record[field] - measured[field]) > Math.abs(measured[field]) * tolerance[field])
.map((field) => `${field}: record ${record[field]}, build ${measured[field]}`);
}
async function measureOnce(root: string, features: number) {
const loadBefore = load();
run(process.execPath, [path.resolve("scripts/generate-product-feature-index.ts"), `--root=${root}`], root);
// The composed registries throw at module evaluation on a duplicate route,
// operation, schema, namespace or message key: prove the copies compose.
const contracts = (await import(
`${pathToFileURL(path.join(root, "src/features/installed-feature-contracts.ts")).href}?features=${features}`
)) as typeof import("../../src/features/installed-feature-contracts.ts");
// 15-01. Operations, schemas and contract contributions compose from the
// runtime halves, which only this module loads.
await import(
`${pathToFileURL(path.join(root, "src/features/loaded-feature-runtimes.ts")).href}?features=${features}`
);
const routes = Object.keys(contracts.ROUTE_REGISTRY).length;
const tsc = run(
path.resolve("node_modules/.bin/tsc"),
["--project", path.join(root, "tsconfig.app.json"), "--noEmit", "--extendedDiagnostics"],
root,
// A type error is a result here (the typecheck stops scaling), not a crash.
{ allowFailure: true },
);
await rm(path.join(root, "dist"), { recursive: true, force: true });
const build = run(path.resolve("node_modules/.bin/vite"), ["build", "--logLevel", "error"], root);
const manifest = JSON.parse(await readFile(path.join(root, "dist/.vite/manifest.json"), "utf8"));
const javascript = classifyViteJavascript(manifest);
const diagnostics: TscDiagnostics = parseTscExtendedDiagnostics(tsc.stdout);
return {
addedFeatures: features,
routes,
tsc: {
passed: tsc.passed,
firstError: tsc.stdout.split("\n").find((line) => line.includes("error TS")) ?? null,
files: diagnostics.files,
types: diagnostics.types,
symbols: diagnostics.symbols,
instantiations: diagnostics.instantiations,
memoryKb: diagnostics.memoryKb ?? null,
wallMs: tsc.ms,
},
build: {
wallMs: build.ms,
initialJavascriptFiles: javascript.initialFiles.length,
initialJavascriptGzipBytes: await javascriptGzipBytes(root, javascript.initialFiles),
lazyJavascriptFiles: javascript.lazyFiles.length,
lazyJavascriptGzipBytes: await javascriptGzipBytes(root, javascript.lazyFiles),
},
loadBefore,
loadAfter: load(),
};
}
/**
* 11-05 (2026-09-29). The commit the run measured, so a later run can be
* compared with the same code and a reader can tell whether the baseline
* still describes the tree. Tracked files that differed from that commit are
* listed; an empty list means the commit alone reproduces the run.
*/
function sourceRevision(): Readonly<{ commit: string; modifiedPaths: string[] }> {
const git = (...args: string[]) => spawnSync("git", args, { encoding: "utf8" });
const head = git("rev-parse", "HEAD");
const status = git("status", "--porcelain", "--untracked-files=no");
if (head.status !== 0 || status.status !== 0) {
throw new Error("feature-shape growth: cannot read the source commit (git rev-parse HEAD, git status)");
}
return {
commit: head.stdout.trim(),
modifiedPaths: status.stdout
.split("\n")
.filter(Boolean)
.map((line) => line.slice(3)),
};
}
function version(bin: string): string {
const result = spawnSync(path.resolve("node_modules/.bin", bin), ["--version"], { encoding: "utf8" });
return (result.stdout ?? "").trim();
}
/**
* 1103/15-02. `--initial-baseline` measures the record the documents cite and,
* with `--write`, stores it as `initialJavascript` in
* config/scale/feature-shape-ceilings.json. Run it in the change that moves
* the initial JavaScript.
*/
async function writeInitialBaseline(): Promise<void> {
const measured = await measureInitialJavascriptCosts();
process.stdout.write(`initial JavaScript: ${JSON.stringify(measured)}\n`);
if (!process.argv.includes("--write")) return;
const ceilings = JSON.parse(await readFile(FEATURE_SHAPE_CEILINGS_PATH, "utf8")) as Record<string, unknown>;
ceilings["initialJavascript"] = measured;
await writeFile(FEATURE_SHAPE_CEILINGS_PATH, `${JSON.stringify(ceilings, null, 2)}\n`);
process.stdout.write(`feature-shape initial baseline: ${FEATURE_SHAPE_CEILINGS_PATH}\n`);
}
async function main(): Promise<void> {
if (process.argv.includes("--initial-baseline")) {
await writeInitialBaseline();
return;
}
const argument = (name: string) =>
process.argv.find((entry) => entry.startsWith(`--${name}=`))?.slice(name.length + 3);
const sizes = (argument("sizes") ?? "0,25,100").split(",").map(Number);
const repeats = Number(argument("repeats") ?? "2");
if (!sizes.every((size) => Number.isSafeInteger(size) && size >= 0) || !(repeats >= 1)) {
throw new TypeError("--sizes must be nonnegative integers and --repeats at least 1");
}
// Read before measuring: the run takes minutes and the tree may move.
const source = sourceRevision();
const runs = [];
for (const features of sizes) {
const root = await createFixture(features);
try {
for (let repeat = 1; repeat <= repeats; repeat += 1) {
const measured = await measureOnce(root, features);
runs.push({ repeat, ...measured });
process.stdout.write(
`features +${features} (repeat ${repeat}): routes=${measured.routes} ` +
`instantiations=${measured.tsc.instantiations} tsc=${measured.tsc.wallMs}ms ` +
`build=${measured.build.wallMs}ms initial=${measured.build.initialJavascriptGzipBytes}B ` +
`lazy=${measured.build.lazyJavascriptGzipBytes}B load=${measured.loadBefore.join("/")}` +
`${measured.tsc.passed ? "" : ` typecheck FAILED: ${measured.tsc.firstError}`}\n`,
);
// Compiler results repeat exactly; a failed typecheck costs minutes and
// gigabytes again for no new information.
if (!measured.tsc.passed) break;
}
} finally {
await rm(root, { recursive: true, force: true });
}
}
const cpu = cpus();
const result = {
schemaVersion: 1,
measuredAt: new Date().toISOString(),
command: `node scripts/scale/measure-feature-shape-growth.ts --sizes=${sizes.join(",")} --repeats=${repeats}`,
environment: {
node: process.versions.node,
platform: process.platform,
arch: process.arch,
cpuModel: cpu[0]?.model.trim() ?? "unknown",
logicalCpus: cpu.length,
availableParallelism: availableParallelism(),
typescript: version("tsc"),
vite: version("vite"),
},
source,
sourceFeature: SOURCE_FEATURE,
runs,
};
const output = process.argv.includes("--write")
? FEATURE_SHAPE_BASELINE_PATH
: "artifacts/scale/feature-shape-growth.json";
await mkdir(path.dirname(output), { recursive: true });
await writeFile(output, `${JSON.stringify(result, null, 2)}\n`);
process.stdout.write(`feature-shape growth: ${output}\n`);
}
if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
await main();
}