Files
clean-architecture-frontend…/scripts/scale/scale-baseline.ts
T
DongHyeonka b26517e6f2 fix(docs): 설계 주석에 문서 이름을 붙이고 지운 뒤 끊기는 링크를 없앤다 (1101/14-01)
문제: 코드 주석 87곳(74파일, .dependency-cruiser.json 포함)이 "Spec §N"
처럼 문서 이름 없이 절 번호만 적는다(대문자 83, 소문자 "spec §" 4).
가리키는 설계서는 README가 복사 뒤 지워도 된다고 한 docs/reviews/ 안에
있었다. 지울 수 있는 기록 검사는 경로 문자열만 찾아 이 참조를 잡지
못했다. 현재 계약 문서도 지울 수 있는 파일로 링크한다:
docs/reviews/adapters/INVENTORY.md 145행이 옆의 adapter 리뷰로,
docs/operations/adapter-remediation-ledger.md가 adapter 리뷰와
docs/superpowers/plans의 계획으로.

고른 방법:
- 주석의 "Spec §"/"spec §"를 "scale-architecture.md §"로 바꾼다(앞
  커밋이 옮긴 docs/architecture/scale-architecture.md). 설명 문장과 절
  번호는 그대로다. 절 번호가 없는 "Spec §"는 없었다.
- tests/unit/deletable-history-documents.test.ts에 "spec" + 선택 공백 +
  "§"(대소문자 무시)를 거부하는 검사를 더하고, 검사 대상에
  .dependency-cruiser.json을 넣는다.
- README "Files you can delete after copying"에서 INVENTORY.md 한 파일
  대신 docs/reviews/adapters/ 폴더를 남기게 하고 이유(INVENTORY 행과
  ledger가 그 리뷰로 링크한다)를 적는다. docs/README.md §3도 같게 고친다.
  INVENTORY의 행 구조는 바꾸지 않는다.
- 남긴 adapter 리뷰와 ledger가 docs/superpowers/plans로 거는 링크 5곳은
  링크를 풀고 "작업 이력 계획, 복사본은 지울 수 있다"로 적는다.
버린 대안: INVENTORY 145칸의 링크를 그룹 이름으로 바꾸기. 템플릿에서도
상세 리뷰로 가는 길이 없어지고 수정이 145곳이다. 계획 3개도 지우기
목록에서 빼기. 지울 수 있는 폴더 안에 예외가 흩어지고, 계획이 다시
다른 이력을 가리키면 남길 목록이 계속 커진다.

위험·한계: 검사는 "HTML spec §4"처럼 다른 명세를 "spec §"로 부르는
주석도 거부한다. 그때는 "HTML Standard §4"처럼 이름을 쓴다.
src/contracts/storage-keys.ts:79의 "release-coherence plan Task 6"처럼
이름 없는 계획 참조는 이 검사가 잡지 않는다. 복사한 프로젝트가 이미
docs/reviews/adapters/를 지웠다면 링크는 계속 끊겨 있다. CHANGELOG.md의
이력 언급 4곳(:15 두 경로, :75 PRODUCT.md, :93)은 고치지 않았다.
check:remediation-ledger는 이 변경 전후 같은 4줄로 실패한다(이미 있던
실패, 게이트 밖). tests/unit/telemetry.test.ts의 describe 이름 하나가
바뀐다.

검증:
- RED: 치환 전 새 검사가 87줄을 들어 실패(1 failed | 1 passed).
- GREEN: 같은 테스트 2 passed. 변이 "// Spec §4. budgets live here"를
  src/platform/errors/app-error.ts에 넣으면 그 한 줄로 실패, 되돌림.
- 치환 diff: 바뀐 87줄 쌍이 토큰 외에 같음(스크립트 대조, 불일치 0).
- 삭제 실험(작업 트리 복사본에서 README 목록대로 지움): 끊긴 링크
  148→0, 끊긴 경로 언급 10→4(남은 4는 CHANGELOG.md), 이름 없는 설계
  참조 87→0, scale-architecture.md §N 87곳 중 없는 절 0. 같은 복사본에서
  verify-documentation-readiness PASS_SCOPED, 지울 수 있는 기록 테스트
  2 passed.
- node scripts/verify-documentation-readiness.ts: PASS_SCOPED
- node scripts/check-adapter-inventory.ts: PASS
- node scripts/run-vitest.ts run tests/unit/deletable-history-documents.test.ts
  tests/unit/documentation-readiness.test.ts tests/unit/telemetry.test.ts:
  50 passed. tests/architecture: 39 passed.
- npx tsc --noEmit (app·node·test): 모두 통과. 바꾼 TS 74파일 eslint 통과.
  .dependency-cruiser.json JSON 파싱 통과.
- 미실행: check:architecture(pnpm 필요).
2026-11-03 07:17:49 +09:00

382 lines
14 KiB
TypeScript

import { createHash } from "node:crypto";
import { cpus } from "node:os";
import { growthExponents } from "../../src/platform/cardinality/growth-regression.ts";
import type { BenchmarkResult } from "./benchmark-result.ts";
import type { TscDiagnostics } from "./collect-tsc-diagnostics.ts";
/**
* scale-architecture.md §48–§49. Stored benchmark baseline, compatibility check and relative
* regression gate.
*
* The growth gates judge the *shape* of one run. This module compares a run
* against an approved stored baseline so a change that keeps the shape but
* multiplies the cost (or silently adds compiler work) is still caught.
*
* Two kinds of metric are compared differently:
* - deterministic metrics (TypeScript types/symbols/instantiations/files) are
* exact for a given compiler, so they use a tight relative tolerance and
* require a compatible toolchain;
* - timing metrics depend on the machine, so they use a loose tolerance plus an
* absolute noise floor, and are only enforced on a compatible environment.
*/
export const SCALE_BASELINE_SCHEMA_VERSION = 1;
export type ScaleEnvironment = Readonly<{
nodeMajor: number;
platform: string;
arch: string;
cpuModel: string;
cpuCount: number;
typescript: string;
}>;
export type TimingEntry = Readonly<{
workload: string;
profile: string;
cardinality: number;
medianMs: number;
minMs: number;
}>;
export type CompilerEntry = Readonly<{
fixture: string;
files: number;
types: number;
symbols: number;
instantiations: number;
}>;
export type ScaleMetrics = Readonly<{
timing: readonly TimingEntry[];
compiler: readonly CompilerEntry[];
}>;
export type ScaleBaseline = Readonly<{
schemaVersion: typeof SCALE_BASELINE_SCHEMA_VERSION;
environment: ScaleEnvironment;
metrics: ScaleMetrics;
approval: Readonly<{
owner: string;
reason: string;
approvedAt: string;
metricsDigest: string;
}>;
}>;
export const REGRESSION_TOLERANCE = Object.freeze({
/** Deterministic compiler counts may grow at most 10% without re-approval. */
compilerRelative: 0.1,
/** Timings may grow at most 2.5x (scheduler/GC noise lives below that)... */
timingRelative: 1.5,
/** ...and never fail on differences below this absolute noise floor. */
timingNoiseFloorMs: 1,
});
/**
* Axis 1 — absolute ceilings. Machine-agnostic hard limits (roughly 50x the
* approved baseline for timings, ~2x for compiler work) that hold even with
* no comparable baseline.
*/
export const ABSOLUTE_CEILINGS = Object.freeze({
timingMinMsByProfile: Object.freeze({ S0: 5, S1: 25, S2: 150 } as Record<string, number>),
app: Object.freeze({ files: 1_500, types: 600_000, symbols: 1_600_000, instantiations: 1_500_000 }),
/**
* Generated `local-<N>` compiler fixtures: a fixed base (lib/runtime
* overhead present at any N) plus a per-local-feature allowance.
*/
localBase: Object.freeze({ files: 100, types: 5_000, symbols: 15_000, instantiations: 5_000 }),
localPerFeature: Object.freeze({ files: 2, types: 100, symbols: 250, instantiations: 150 }),
});
/** Axis 3 — growth shape (see src/platform/cardinality/growth-regression.ts). */
export const GROWTH_LIMITS = Object.freeze({
timingMaxExponent: 1.6,
timingNoiseFloorMs: 0.5,
compilerMaxExponent: 1.3,
});
export function currentEnvironment(typescript: string): ScaleEnvironment {
const cpu = cpus();
return Object.freeze({
nodeMajor: Number(process.versions.node.split(".")[0]),
platform: process.platform,
arch: process.arch,
cpuModel: cpu[0]?.model.trim() ?? "unknown",
cpuCount: cpu.length,
typescript,
});
}
function canonical(value: unknown): string {
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
if (value && typeof value === "object") {
return `{${Object.keys(value)
.sort()
.map((key) => `${JSON.stringify(key)}:${canonical((value as Record<string, unknown>)[key])}`)
.join(",")}}`;
}
return JSON.stringify(value);
}
export function metricsDigest(metrics: ScaleMetrics): string {
return createHash("sha256").update(canonical(metrics)).digest("hex");
}
export function snapshotMetrics(
composition: readonly BenchmarkResult[],
compiler: Readonly<{
app: TscDiagnostics;
local: readonly Readonly<{ features: number; diagnostics: TscDiagnostics }>[];
}>,
): ScaleMetrics {
const round = (value: number) => Math.round(value * 1_000) / 1_000;
const toCompiler = (fixture: string, diagnostics: TscDiagnostics): CompilerEntry => ({
fixture,
files: diagnostics.files,
types: diagnostics.types,
symbols: diagnostics.symbols,
instantiations: diagnostics.instantiations,
});
return Object.freeze({
timing: composition
.map((result) => ({
workload: result.name,
profile: result.profile,
cardinality: result.cardinality,
medianMs: round(result.medianMs),
minMs: round(Math.min(...result.samples)),
}))
.sort((left, right) =>
`${left.workload}/${left.profile}`.localeCompare(`${right.workload}/${right.profile}`),
),
compiler: [
toCompiler("app", compiler.app),
...compiler.local.map((entry) => toCompiler(`local-${entry.features}`, entry.diagnostics)),
].sort((left, right) => left.fixture.localeCompare(right.fixture)),
});
}
export type Compatibility = Readonly<{
/** Baseline format, digest and metric coverage are usable at all. */
structural: Readonly<{ ok: boolean; reasons: readonly string[] }>;
/** Same compiler: deterministic compiler counts are comparable. */
compiler: Readonly<{ ok: boolean; reasons: readonly string[] }>;
/** Same machine class and runtime: timings are comparable. */
timing: Readonly<{ ok: boolean; reasons: readonly string[] }>;
}>;
export function checkCompatibility(
baseline: ScaleBaseline,
environment: ScaleEnvironment,
current: ScaleMetrics,
): Compatibility {
const structural: string[] = [];
if (baseline.schemaVersion !== SCALE_BASELINE_SCHEMA_VERSION) {
structural.push(`baseline schemaVersion ${String(baseline.schemaVersion)} is not ${SCALE_BASELINE_SCHEMA_VERSION}`);
}
if (metricsDigest(baseline.metrics) !== baseline.approval?.metricsDigest) {
structural.push("baseline metrics do not match their approved digest (hand-edited?)");
}
const timingKey = (entry: TimingEntry) => `${entry.workload}/${entry.profile}/${entry.cardinality}`;
const baselineTiming = new Set(baseline.metrics.timing.map(timingKey));
const baselineCompiler = new Set(baseline.metrics.compiler.map((entry) => entry.fixture));
for (const entry of current.timing) {
if (!baselineTiming.has(timingKey(entry))) structural.push(`no baseline for timing ${timingKey(entry)}`);
}
for (const entry of current.compiler) {
if (!baselineCompiler.has(entry.fixture)) structural.push(`no baseline for compiler fixture ${entry.fixture}`);
}
const compiler: string[] = [];
if (baseline.environment.typescript !== environment.typescript) {
compiler.push(`TypeScript ${environment.typescript} differs from baseline ${baseline.environment.typescript}`);
}
const timing: string[] = [];
const base = baseline.environment;
if (base.nodeMajor !== environment.nodeMajor) timing.push(`Node ${environment.nodeMajor} vs baseline ${base.nodeMajor}`);
if (base.platform !== environment.platform || base.arch !== environment.arch) {
timing.push(`${environment.platform}/${environment.arch} vs baseline ${base.platform}/${base.arch}`);
}
if (base.cpuModel !== environment.cpuModel) timing.push(`CPU "${environment.cpuModel}" vs baseline "${base.cpuModel}"`);
return Object.freeze({
structural: { ok: structural.length === 0, reasons: structural },
compiler: { ok: compiler.length === 0, reasons: compiler },
timing: { ok: timing.length === 0, reasons: timing },
});
}
export type RegressionFinding = Readonly<{
metric: string;
baseline: number;
current: number;
limit: number;
passed: boolean;
}>;
export type AxisResult = Readonly<{
passed: boolean;
findings: readonly RegressionFinding[];
}>;
export type BaselineComparison = Readonly<{
/**
* `passed` only when every axis was evaluated and held. `incomplete` when the
* environment is not timing-compatible: timing regression was not compared,
* so it is not evidence of a pass.
*/
status: "passed" | "failed" | "incomplete";
passed: boolean;
compatibility: Compatibility;
absolute: AxisResult;
growth: AxisResult;
compiler: readonly RegressionFinding[];
/** Empty when the environment is not timing-compatible. */
timing: readonly RegressionFinding[];
failures: readonly string[];
}>;
function finding(metric: string, baseline: number, current: number, limit: number): RegressionFinding {
return { metric, baseline, current, limit: Math.round(limit * 1_000) / 1_000, passed: current <= limit };
}
export function checkAbsoluteCeilings(current: ScaleMetrics): AxisResult {
const findings: RegressionFinding[] = [];
for (const entry of current.timing) {
const limit = ABSOLUTE_CEILINGS.timingMinMsByProfile[entry.profile];
if (limit !== undefined) {
findings.push(finding(`ceiling:time:${entry.workload}/${entry.profile}`, 0, entry.minMs, limit));
}
}
for (const entry of current.compiler) {
const local = /^local-(\d+)$/u.exec(entry.fixture);
for (const metric of ["files", "types", "symbols", "instantiations"] as const) {
const limit = local
? ABSOLUTE_CEILINGS.localBase[metric] +
ABSOLUTE_CEILINGS.localPerFeature[metric] * Number(local[1])
: entry.fixture === "app"
? ABSOLUTE_CEILINGS.app[metric]
: Number.POSITIVE_INFINITY;
findings.push(finding(`ceiling:tsc:${entry.fixture}:${metric}`, 0, entry[metric], limit));
}
}
return { passed: findings.every((item) => item.passed), findings };
}
export function checkGrowthShape(current: ScaleMetrics): AxisResult {
const findings: RegressionFinding[] = [];
const byWorkload = new Map<string, { n: number; cost: number }[]>();
for (const entry of current.timing) {
const series = byWorkload.get(entry.workload) ?? [];
series.push({ n: entry.cardinality, cost: entry.minMs });
byWorkload.set(entry.workload, series);
}
for (const [workload, series] of byWorkload) {
if (series.length < 2) continue;
const worst = Math.max(...growthExponents(series, GROWTH_LIMITS.timingNoiseFloorMs));
findings.push(finding(`growth:time:${workload}`, 1, Math.round(worst * 1_000) / 1_000, GROWTH_LIMITS.timingMaxExponent));
}
const local = current.compiler
.map((entry) => ({ entry, n: Number(/^local-(\d+)$/u.exec(entry.fixture)?.[1]) }))
.filter(({ n }) => Number.isFinite(n));
if (local.length >= 2) {
for (const metric of ["files", "types", "symbols", "instantiations"] as const) {
const worst = Math.max(...growthExponents(local.map(({ entry, n }) => ({ n, cost: entry[metric] }))));
findings.push(finding(`growth:tsc:local:${metric}`, 1, Math.round(worst * 1_000) / 1_000, GROWTH_LIMITS.compilerMaxExponent));
}
}
return { passed: findings.every((item) => item.passed), findings };
}
export function compareToBaseline(
baseline: ScaleBaseline,
environment: ScaleEnvironment,
current: ScaleMetrics,
tolerance = REGRESSION_TOLERANCE,
): BaselineComparison {
const compatibility = checkCompatibility(baseline, environment, current);
const failures: string[] = [...compatibility.structural.reasons];
// A different compiler changes every deterministic count: the baseline must
// be re-approved, it is never silently compared or silently skipped.
if (!compatibility.compiler.ok) {
failures.push(
...compatibility.compiler.reasons.map((reason) => `${reason}; re-approve with update:scale-baseline`),
);
}
const absolute = checkAbsoluteCeilings(current);
const growth = checkGrowthShape(current);
for (const item of [...absolute.findings, ...growth.findings]) {
if (!item.passed) failures.push(`${item.metric} ${item.current} exceeds ${item.limit}`);
}
const compiler: RegressionFinding[] = [];
if (compatibility.structural.ok && compatibility.compiler.ok) {
const byFixture = new Map(baseline.metrics.compiler.map((entry) => [entry.fixture, entry]));
for (const entry of current.compiler) {
const reference = byFixture.get(entry.fixture);
if (!reference) continue;
for (const metric of ["files", "types", "symbols", "instantiations"] as const) {
const item = finding(
`tsc:${entry.fixture}:${metric}`,
reference[metric],
entry[metric],
Math.floor(reference[metric] * (1 + tolerance.compilerRelative)),
);
compiler.push(item);
if (!item.passed) {
failures.push(`${item.metric} ${item.current} exceeds ${item.limit} (baseline ${item.baseline})`);
}
}
}
}
const timing: RegressionFinding[] = [];
if (compatibility.structural.ok && compatibility.timing.ok) {
const key = (entry: TimingEntry) => `${entry.workload}/${entry.profile}`;
const byKey = new Map(baseline.metrics.timing.map((entry) => [key(entry), entry]));
for (const entry of current.timing) {
const reference = byKey.get(key(entry));
if (!reference) continue;
// Best-observed samples: contention only ever adds time.
const item = finding(
`time:${key(entry)}`,
reference.minMs,
entry.minMs,
Math.max(reference.minMs * (1 + tolerance.timingRelative), reference.minMs + tolerance.timingNoiseFloorMs),
);
timing.push(item);
if (!item.passed) {
failures.push(`${item.metric} ${item.current}ms exceeds ${item.limit}ms (baseline ${item.baseline}ms)`);
}
}
}
const status =
failures.length > 0 ? "failed" : compatibility.timing.ok ? "passed" : "incomplete";
return Object.freeze({
status,
passed: status === "passed",
compatibility,
absolute,
growth,
compiler,
timing,
failures,
});
}
export function approveBaseline(
environment: ScaleEnvironment,
metrics: ScaleMetrics,
approval: Readonly<{ owner: string; reason: string; approvedAt: string }>,
): ScaleBaseline {
return Object.freeze({
schemaVersion: SCALE_BASELINE_SCHEMA_VERSION,
environment,
metrics,
approval: { ...approval, metricsDigest: metricsDigest(metrics) },
});
}