Files
clean-architecture-frontend…/scripts/test-sample-removal.ts
T
DongHyeonka 5d064b8ace test(features): 기능을 더해도 기능 밖 테스트와 시나리오 하네스가 깨지지 않게 한다 (16-02)
문제: 기능 하나를 더하면(리뷰 16 실험 E) 기능 밖 테스트 3개가 깨진다.
① installed-context-routing은 context 목록이 정확히
["reference-context"]이고 첫 context가 참조 context라고 단정한다.
② product-feature-switch는 정렬 첫 기능(INSTALLED_PRODUCT_FEATURE_IDS[0])을
끄고 참조 라우트를 본다. 새 기능이 앞에 정렬되면 깨진다.
③ release-manifest fixture가 contractSet.packages: []로 고정이다.
EXTERNAL_PACKAGE를 쓰는 첫 기능에서 반드시 깨진다.
또 시나리오 하네스의 operation ID 타입이 테스트 계약 3개 리터럴이고
시나리오 응답 함수가 /api/test-entities 핸들러 안에 묶여 있다. 새 기능
테스트가 자기 operation에 실패 시나리오를 쓰려면 카탈로그를 고쳐야 한다
(안 고치면 TS2345, 실행하면 catalog.ts:469 TypeError).

방법: 리뷰의 (b) 최소 개선.
① "목록에 reference-context가 있다"로 바꾸고, 라우트 검사는 id로 찾은
  context에서 한다.
② REFERENCE_FEATURE_ID를 쓴다.
③ fixture의 packages와 setDigest를 EXPECTED_CONTRACT_SET_PACKAGES에서
  만든다.
하네스: HttpScenarioOperationId를 string으로 넓혔다. 시나리오 사실(명령
여부, 응답 상한, 성공 status)은 httpScenarioOperationFacts가 설치된 V3
descriptor에서 읽는다. 시나리오 응답은 tests/mocks/handlers/
http-scenarios.ts의 createHttpScenarioHandler로 옮겼다. 경로와 method도
descriptor에서 만든다. test-entities 핸들러는 이 함수를 쓰고 카탈로그
확인(assertOperationScenario)을 그대로 거친다. 카탈로그 기대값 57개는
계속 테스트 계약 3개에만 있고, 그 타입도 3개로 둔다. 카탈로그에 없는
operation을 assertOperationScenario에 넘기면 TypeError 대신 그 사실을
말하는 오류가 난다. test-entities 핸들러는 요청의 이름·id를 되돌려 주던
응답을 카탈로그 payload로 고정했다(기대값과 같은 값이다).
새 기능 테스트의 MSW 시나리오: server.use(createHttpScenarioHandler(
<자기 설치 계약>, { scenario: "server-terminal-503", payload }))
버린 대안: (a) 현상 유지는 기능마다 4곳을 고친다. (c) 하네스를 기능별로
나누면 변경이 크고 test-evidence.json의 증거 경로가 바뀐다. 기능
operation을 카탈로그에 더하는 방식은 기능 추가를 공유 파일 수정으로
만든다.
test-evidence.json은 바꾸지 않았다(sourceContracts 토큰이 test-entities.ts에
그대로 있다). 제거 드릴의 플랫폼 보존 목록에 http-scenarios.ts를 더했고,
테스트 전략 문서 11절에 사용법을 적었다.

위험·한계:
- 앱 전체를 렌더하는 테스트에서 response-too-large는 끝나지 않는다.
  실행기가 MSW가 넘긴 본문을 cancel하는 promise가 끝나지 않는 기존
  제약이다. 카탈로그 통합 테스트는 프록시 fetcher로 우회한다.
- 같은 실험에서 기능 밖 테스트가 더 깨진다. 이 발견 범위 밖이라 두었다.
  http-operation-v2-v3-parity는 스키마 ID "NoRequest"를 센티널로 고정한다.
  스키마 ID는 전역이라 두 번째 기능은 다른 이름을 써야 하고, 그러면 이
  테스트가 깨진다. manual-a11y-evidence·documentation-readiness(라우트별
  a11y 기록, 16-01)와 second-api-feature-registries(무효화 그래프 추가가
  breaking, 16-01)도 깨진다.
- test:sample-removal, test:http-scenario-evidence는 pnpm이 필요해 돌리지
  않았다.

검증:
RED: 참조 기능을 토큰 경계로 order로 바꾼 복사본(order-context,
EXTERNAL_PACKAGE 출처 포함, 커밋 안 함)을 넣고 tests/features
tests/component tests/runtime-schema를 돌렸다. 5파일 8개 실패. 기능 밖은
① 2개(["order-context","reference-context"]), ② 2개(order-feature를 꺼서
참조 링크가 남음), ③ 1개(ReleaseManifestError)다. 나머지 3개는 복사한
테스트다. 하네스 probe는 TS2345 2건, 실행 시 TypeError(catalog.ts:469).
새 통합 테스트는 기준 하네스에서 모듈 없음으로 실패하고, 기준
assertOperationScenario는 미등록 operation에 TypeError를 낸다.
GREEN: 같은 복사본으로 같은 폴더와 카탈로그 통합 테스트를 돌렸다. 78파일
중 실패는 출처를 바꾼 복사본 provenance 1파일뿐이고 ①~③은 통과한다.
order 테스트가 createHttpScenarioHandler(LIST_ORDER_RESOURCES_CONTRACT)로
content-type-mismatch·schema-mismatch·server-terminal-503을 화면에서
확인했다(카탈로그 수정 없음).
복사본을 지운 뒤: tests/features·component·runtime-schema·unit·
integration(security-followup-archive 제외)·contract 250파일 3111개 통과,
카탈로그 57/57, check-test-evidence --scenario-only·--source-only PASS,
tsc app·test·node 통과, verify-documentation-readiness PASS_SCOPED,
check-ci-contract PASS, 바꾼 파일 eslint 통과.

Refs: docs/reviews/2026-11-01-maturity-review/16-structure-patterns.md 16-02
2026-11-03 01:49:11 +09:00

468 lines
17 KiB
TypeScript

import {
access,
mkdir,
readdir,
readFile,
rm,
writeFile,
} from "node:fs/promises";
import path from "node:path";
import { ROUTE_FEATURE_OWNER } from "../src/features/installed-feature-contracts.ts";
import { createFileSystemDeclarationSource } from "./generate-product-feature-index.ts";
import { isProductionModulePath } from "./lib/risk-coverage.ts";
import {
createRemovalFixtureRoot,
filesBelow,
prepareRemovalFixture,
pruneRemovalFixtureCiContract,
pruneScriptOrchestration,
regenerateRemovalFixtureWorkflow,
renderRemovalJunit,
requireRemovalFixtureEnvironment,
runRemovalFixtureCheck,
runRemovalFixturePnpm,
type RemovalCheck,
} from "./lib/removal-fixture.ts";
/**
* 09-07 (D-10). Removes one installed feature, `--feature=<id>` (default: the
* reference sample), from a copy of the repository and proves the rest still
* typechecks, tests, covers and builds. Every feature loses its directory, its
* tests/features directory, its context when no other feature is left in it,
* its runtime-profile override and its central policy rows. The sample also
* owns the extra paths and scripts listed in SAMPLE_EXTRAS.
*/
const featureId =
process.argv.find((argument) => argument.startsWith("--feature="))
?.slice("--feature=".length) ?? "reference-feature";
type FeatureExtras = Readonly<{
paths: readonly string[];
ciScripts: readonly string[];
evidencePathFragments: readonly string[];
residue: RegExp;
}>;
const SAMPLE_EXTRAS: Readonly<Record<string, FeatureExtras>> = Object.freeze({
"reference-feature": Object.freeze({
// 0929/10-02. The HTTP scenario catalog (tests/mocks) and the negative
// type fixtures of the feature APIs run on the test contracts, so they are
// platform checks and stay (commonTestPaths below).
paths: [
"tests/component/product-feature-switch.test.tsx",
"tests/e2e/reference-form.spec.ts",
"tests/e2e/reference-route.spec.ts",
"tests/e2e/reference-save.spec.ts",
// 11-04: the growth measurement copies this sample feature.
"tests/unit/feature-shape-growth.test.ts",
"scripts/scale/measure-feature-shape-growth.ts",
"config/scale/feature-shape-growth.json",
"config/scale/feature-shape-ceilings.json",
],
ciScripts: [
"test:reference-feature",
],
evidencePathFragments: [
"reference-feature.xml",
],
residue: /REFERENCE_RESOURCE|reference-feature|reference-resource/i,
}),
});
function escapeRegExp(value: string): string {
return value.replace(/[.*+?^${}()|[\]\\]/gu, "\\$&");
}
async function locateFeature(id: string) {
const source = await createFileSystemDeclarationSource(process.cwd());
for (const appId of source.appIds()) {
for (const context of source.contextsOf(appId)) {
const features = source.featuresOf(context.id);
const feature = features.find((candidate) => candidate.id === id);
if (!feature) continue;
let contextDirectory: string | null = null;
for (const entry of await readdir("src/contexts", { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const declaration = await readFile(
path.join("src/contexts", entry.name, "context-module.ts"),
"utf8",
).catch(() => "");
if (
new RegExp(`CONTEXT_MODULE_ID\\s*=\\s*["']${escapeRegExp(context.id)}["']`, "u")
.test(declaration)
) {
contextDirectory = `src/contexts/${entry.name}`;
}
}
return Object.freeze({
directory: feature.directory,
// The context goes with its last feature.
contextDirectory: features.length === 1 ? contextDirectory : null,
});
}
}
throw new Error(`No installed feature declares FEATURE_MODULE_ID "${id}"`);
}
const located = await locateFeature(featureId);
const extras: FeatureExtras = SAMPLE_EXTRAS[featureId] ?? Object.freeze({
paths: [],
ciScripts: [],
evidencePathFragments: [],
residue: new RegExp(
`${escapeRegExp(featureId)}|${escapeRegExp(located.directory)}`,
"i",
),
});
// Captured from this repository before the copy loses them.
const featureRouteIds = Object.entries(ROUTE_FEATURE_OWNER)
.filter(([, owner]) => owner === featureId)
.map(([routeId]) => routeId);
const fixtureRoot = await createRemovalFixtureRoot(`${featureId}-removal`);
const pnpmCli = requireRemovalFixtureEnvironment("npm_execpath");
const featureSource = `src/features/${located.directory}`;
const featureTests = `tests/features/${located.directory}`;
const isSample = featureId in SAMPLE_EXTRAS;
/**
* Platform tests that must survive the sample feature's removal. Asserting they
* are still present is what stops the removal fixture from "passing" by having
* quietly deleted the platform's own coverage along with the feature.
*/
const commonTestPaths = [
"tests/unit/external-contract-runtime.test.ts",
"tests/unit/http-execution-v3.test.ts",
"tests/unit/runtime-adapters.test.ts",
"tests/integration/http-execution-v3-observability.test.ts",
// 0929/10-02. HTTP scenario evidence (FE-GATE-007) and the feature API
// negative type fixtures (FE-GATE-003) run on the test contracts.
"tests/integration/http-scenario-catalog.test.ts",
"tests/mocks/scenarios/catalog.ts",
"tests/mocks/handlers/test-entities.ts",
// 16-02. Feature tests answer their own operations with it.
"tests/mocks/handlers/http-scenarios.ts",
"tests/fixtures/typecheck/fixture-feature-contract.ts",
"tests/fixtures/typecheck/invalid-feature-input.ts",
"tests/fixtures/typecheck/invalid-reference-operation.ts",
"tests/fixtures/typecheck/invalid-http-operation-input.ts",
"tests/fixtures/typecheck/invalid-http-wire-mapper.ts",
"tests/fixtures/typecheck/invalid-http-operation-id.ts",
"tests/fixtures/typecheck/invalid-http-route-id.ts",
"tests/fixtures/typecheck/invalid-feature-contribution-input.ts",
"tests/fixtures/typecheck/invalid-feature-capability-selection.ts",
];
const featureOwnedPaths = [
featureSource,
featureTests,
// The feature's bounded-context declaration when it was the last feature in
// it; the template app stays.
...(located.contextDirectory === null ? [] : [located.contextDirectory]),
...extras.paths,
];
type CoveragePolicy = {
repositoryBaseline: number;
generatedPaths: string[];
criticalModules: Array<{
path?: string;
minimum?: Record<string, number>;
}>;
highRiskPaths: string[];
waivers: Array<{ path?: string }>;
};
type EvidenceContribution = Readonly<{ owner?: string }>;
type EvidencePolicy = Record<
"scenarioCatalogs" | "sourceContracts",
unknown
>;
type GovernanceConsumer = Readonly<{ path?: string }>;
type GovernanceRegistry = Record<string, unknown> & {
consumers?: unknown;
consumerDirectories?: unknown;
};
type RemovalGovernance = { registries: GovernanceRegistry[] };
function runPnpm(script: string, extra: string[] = []): boolean {
return runRemovalFixturePnpm(fixtureRoot, pnpmCli, script, extra);
}
function check(name: string, script: string, extra: string[] = []): RemovalCheck {
return runRemovalFixtureCheck(name, fixtureRoot, pnpmCli, script, extra);
}
/** 14-08. A non-command check fails with its own findings, nothing else's. */
function outcome(name: string, findings: readonly string[], label: string): RemovalCheck {
return findings.length === 0
? { name, passed: true }
: { name, passed: false, failure: `${label}: ${findings.join(", ")}` };
}
try {
await prepareRemovalFixture(fixtureRoot);
for (const excludedFixtureTest of [
"tests/unit/ci-workflow-generation.test.ts",
"tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap",
"tests/unit/removal-fixture.test.ts",
"tests/unit/http-scenario-evidence.test.ts",
// They check README.md and CHANGELOG.md, which the fixture does not copy
// (REMOVAL_FIXTURE_COPY_TARGETS); removing a feature does not change them.
"tests/unit/changelog-review-ids.test.ts",
"tests/unit/documentation-readiness.test.ts",
]) {
await rm(path.join(fixtureRoot, excludedFixtureTest), { force: true });
}
const coveragePolicyFile = path.join(
fixtureRoot,
"config/testing/risk-coverage.json",
);
const coveragePolicy = JSON.parse(
await readFile(coveragePolicyFile, "utf8"),
) as CoveragePolicy;
const generatedProductionModules = new Set(coveragePolicy.generatedPaths);
const removedProductionModuleCount = (
await filesBelow(path.join(fixtureRoot, featureSource))
)
.map((file) => path.relative(fixtureRoot, file).split(path.sep).join("/"))
.filter(
(file) =>
isProductionModulePath(file) && !generatedProductionModules.has(file),
).length;
for (const ownedPath of featureOwnedPaths) {
await rm(path.join(fixtureRoot, ownedPath), {
recursive: true,
force: true,
});
}
if (!runPnpm("generate:feature-index")) {
throw new Error("Feature topology indexes failed to regenerate after sample removal");
}
// Runtime profiles are product configuration. Once the feature is physically
// removed, its override key must disappear too or the built config still
// advertises a source capability that no longer exists. public/config.json
// is the local profile `dev` serves and has to stay equal to it (F-5).
for (const configPath of [
"config/runtime/local.json",
"config/runtime/development.json",
"config/runtime/staging.json",
"config/runtime/production.json",
"public/config.json",
]) {
const profilePath = path.join(fixtureRoot, configPath);
const runtimeConfig = JSON.parse(
await readFile(profilePath, "utf8"),
) as {
FEATURE_OVERRIDES?: Record<string, string>;
};
if (runtimeConfig.FEATURE_OVERRIDES) {
delete runtimeConfig.FEATURE_OVERRIDES[featureId];
}
await writeFile(
profilePath,
`${JSON.stringify(runtimeConfig, null, 2)}\n`,
);
}
const retainedCriticalModules = coveragePolicy.criticalModules.filter(
(modulePolicy) => !modulePolicy.path?.startsWith(`${featureSource}/`),
);
if (
isSample &&
retainedCriticalModules.length === coveragePolicy.criticalModules.length
) {
throw new Error("Reference feature coverage policy is not registered");
}
coveragePolicy.criticalModules = retainedCriticalModules;
coveragePolicy.highRiskPaths = coveragePolicy.highRiskPaths.filter(
(modulePath) => !modulePath.startsWith(`${featureSource}/`),
);
coveragePolicy.waivers = coveragePolicy.waivers.filter(
(waiver) => !waiver.path?.startsWith(`${featureSource}/`),
);
coveragePolicy.generatedPaths = coveragePolicy.generatedPaths.filter(
(modulePath) => !modulePath.startsWith(`${featureSource}/`),
);
coveragePolicy.repositoryBaseline -= removedProductionModuleCount;
if (coveragePolicy.repositoryBaseline <= 0) {
throw new Error("Reference feature removal produced an invalid coverage baseline");
}
await writeFile(
coveragePolicyFile,
`${JSON.stringify(coveragePolicy, null, 2)}\n`,
);
const evidencePolicyFile = path.join(
fixtureRoot,
"config/testing/test-evidence.json",
);
const evidencePolicy = JSON.parse(
await readFile(evidencePolicyFile, "utf8"),
) as EvidencePolicy;
for (const policyKey of ["scenarioCatalogs", "sourceContracts"] as const) {
const contributions = evidencePolicy[policyKey];
if (!Array.isArray(contributions)) {
throw new Error(`Test evidence policy is missing ${policyKey}`);
}
// A feature's own evidence goes with it. The sample has none since
// 0929/10-02: the HTTP scenario catalog is owned by the platform.
evidencePolicy[policyKey] = contributions.filter(
(candidate: unknown) =>
(candidate as EvidenceContribution).owner !== featureId,
);
}
await writeFile(
evidencePolicyFile,
`${JSON.stringify(evidencePolicy, null, 2)}\n`,
);
const governanceFile = path.join(
fixtureRoot,
"config/contracts/registry-governance.json",
);
const removalGovernance = JSON.parse(
await readFile(governanceFile, "utf8"),
) as RemovalGovernance;
removalGovernance.registries = removalGovernance.registries.map(
(registry) => ({
...registry,
...(Array.isArray(registry.consumers)
? {
consumers: registry.consumers.filter(
(candidate: unknown) => {
const consumer = candidate as GovernanceConsumer;
return !consumer.path?.includes(featureSource);
},
),
}
: {}),
...(Array.isArray(registry.consumerDirectories)
? {
consumerDirectories: registry.consumerDirectories.filter(
(directory: unknown) =>
typeof directory !== "string" ||
!directory.includes(featureSource),
),
}
: {}),
}),
);
await writeFile(
governanceFile,
`${JSON.stringify(removalGovernance, null, 2)}\n`,
);
const removedCiScripts = new Set(extras.ciScripts);
const fixturePackagePath = path.join(fixtureRoot, "package.json");
const fixturePackage = JSON.parse(await readFile(fixturePackagePath, "utf8")) as {
scripts: Record<string, string>;
};
pruneScriptOrchestration(fixturePackage.scripts, "test:all", removedCiScripts);
await writeFile(fixturePackagePath, `${JSON.stringify(fixturePackage, null, 2)}\n`);
await pruneRemovalFixtureCiContract({
root: fixtureRoot,
removedScripts: removedCiScripts,
removedEvidencePathFragments: extras.evidencePathFragments,
});
// D-13: removing the sample is a breaking registry change. The fixture
// records the new registry baseline the way a downstream project does, and
// the canonical check:registries then runs against it.
if (!runRemovalFixturePnpm(fixtureRoot, pnpmCli, "update:registry-baseline", [], {
REGISTRY_BASELINE_OWNER: "removal-fixture",
REGISTRY_BASELINE_REASON: `${featureId} removed`,
})) {
throw new Error("Registry baseline failed to update after sample removal");
}
await regenerateRemovalFixtureWorkflow(fixtureRoot);
const residue: string[] = [];
for (const root of ["src", "tests"]) {
for (const file of await filesBelow(path.join(fixtureRoot, root))) {
const relative = path.relative(fixtureRoot, file);
const content = await readFile(file, "utf8");
if (extras.residue.test(`${relative}\n${content}`)) {
residue.push(relative);
}
}
}
const missingCommonTests: string[] = [];
for (const testPath of commonTestPaths) {
await access(path.join(fixtureRoot, testPath)).catch(() => {
missingCommonTests.push(testPath);
});
}
const checks: RemovalCheck[] = [
outcome("common-test-evidence", missingCommonTests, "platform tests missing"),
check("typecheck", "check:types"),
check("architecture", "check:architecture"),
check("registries", "check:registries"),
check("unit-integration", "test:all"),
check("coverage", "test:coverage"),
check("test-evidence-source", "check:test-evidence:source"),
check("ci-contract", "check:ci"),
check("home-smoke", "exec", [
"vitest",
"run",
"tests/component/router.test.tsx",
"--reporter=default",
]),
check("build", "build"),
];
const builtResidue: string[] = [];
const buildSucceeded =
checks.find(({ name }) => name === "build")?.passed === true;
if (buildSucceeded) {
for (const file of await filesBelow(path.join(fixtureRoot, "dist"))) {
if (!/\.(?:js|css|html|json)$/.test(file)) continue;
const content = await readFile(file, "utf8");
if (extras.residue.test(content)) {
builtResidue.push(path.relative(fixtureRoot, file));
}
}
}
const routeCatalog = await import(
`${new URL(
"../src/features/installed-feature-contracts.ts",
`file://${fixtureRoot}/scripts/`,
).href}?removed=${Date.now()}`
) as { ROUTE_REGISTRY: Readonly<Record<string, unknown>> };
const routeIds = new Set(Object.keys(routeCatalog.ROUTE_REGISTRY));
checks.push(
outcome(
"route-absent",
featureRouteIds.filter((routeId) => routeIds.has(routeId)),
"routes still registered",
),
outcome("fixture-id-residue", residue, "residue"),
outcome("built-fixture-id-residue", builtResidue, "built residue"),
);
const failed = checks.filter((entry) => !entry.passed);
await mkdir("artifacts/tests", { recursive: true });
await writeFile(
isSample
? "artifacts/tests/sample-removal.xml"
: `artifacts/tests/feature-removal-${featureId}.xml`,
renderRemovalJunit(`${featureId}-removal`, checks),
);
if (failed.length > 0) {
process.stderr.write(
`${featureId} removal failed:\n${failed
.map((entry) => ` ${entry.name}: ${entry.failure ?? "failed"}`)
.join("\n")}\n`,
);
process.exitCode = 1;
} else {
process.stdout.write(
`${featureId} removal: PASS (${checks.length} checks, no fixture IDs)\n`,
);
}
} finally {
await rm(fixtureRoot, { recursive: true, force: true });
}