refactor: align coverage counter provenance

This commit is contained in:
DongHyeonka
2026-08-02 09:31:28 +09:00
parent 6e05a35790
commit 5cecbb9820
4 changed files with 247 additions and 77 deletions
@@ -0,0 +1,133 @@
# Counter-bearing Coverage Provenance Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Align repository coverage provenance names and static classification with the counters that Vitest/V8 actually emits, without making claims about JavaScript runtime executability.
**Architecture:** The inventory parser classifies source files only by whether their top-level AST contains statements known to receive V8 counters. Coverage evaluation requires exact agreement between that static counter-bearing/counterless partition and producer rows, while policy-sensitive modules must remain counter-bearing. The JSON artifact exposes the same terminology as the inventory and diagnostics.
**Tech Stack:** TypeScript 7, Node.js 24, `@babel/eslint-parser`, Vitest 4, V8 coverage.
## Global Constraints
- `runtime.ts` declarations/initializers and direct execution statements are counter-bearing.
- Type-only modules, `import type {}`, `import {}`, bare side-effect imports, value imports, named value re-exports, and star value re-exports are counterless under the observed Vitest/V8 producer.
- Counterless does not mean non-executable; code, artifacts, diagnostics, tests, and documentation must not make that claim.
- Exact all-zero rows are accepted only for statically counterless modules.
- Critical and high-risk policy modules cannot be counterless.
- All source edits use `apply_patch` and behavior changes follow RED-GREEN TDD.
---
### Task 1: Lock the Vitest/V8 classifier contract with RED tests
**Files:**
- Modify: `tests/unit/risk-coverage.test.ts`
**Interfaces:**
- Consumes: `buildProductionModuleInventory()` and `evaluateRiskCoverage()`.
- Produces: expectations for `counterBearingModules`, `counterlessModules`, `counterBearingTotal`, `instrumentedCounterBearingTotal`, `counterlessTotal`, and `counterlessModules`.
- [x] **Step 1: Rename the test inventory helper and artifact assertions to the desired API.**
```ts
function inventory(
files: readonly string[],
generatedExclusions: readonly string[] = [],
counterlessModules: readonly string[] = [],
): ProductionModuleInventory {
return {
files,
preExclusionTotal: files.length + generatedExclusions.length,
generatedExclusions,
counterBearingModules: files.filter((file) => !counterlessModules.includes(file)),
counterlessModules,
};
}
```
- [x] **Step 2: Add a real-source inventory regression table.**
```ts
const counterlessSources = {
"import-type-empty.ts": "import type {} from './a.ts';\n",
"import-value-empty.ts": "import {} from './a.ts';\n",
"import-side-effect.ts": "import './a.ts';\n",
"import-value.ts": "import { a } from './a.ts';\n",
"reexport-named.ts": "export { a } from './a.ts';\n",
"reexport-star.ts": "export * from './a.ts';\n",
};
```
Assert every key appears in `counterlessModules`, while `export const runtimeValue = 1` and `void globalThis` appear in `counterBearingModules`.
- [x] **Step 3: Run the focused test and verify RED.**
Run: `./node_modules/.bin/vitest run tests/unit/risk-coverage.test.ts --reporter=dot`
Expected: TypeScript/test failures because the counter-bearing API fields do not exist and the current bare import classifier is executable-labelled.
### Task 2: Rename and align static coverage provenance
**Files:**
- Modify: `scripts/lib/risk-coverage.ts`
- Modify: `tests/unit/risk-coverage.test.ts`
**Interfaces:**
- Consumes: Babel `Program.body` nodes and parsed Istanbul/V8 counters.
- Produces: `hasCoverageCounterBearingStatements(source, relativePath)`, a complete `counterBearingModules`/`counterlessModules` partition, and consistently named `RiskCoverageResult` fields.
- [x] **Step 1: Implement the minimal classifier needed by the RED cases.**
`ImportDeclaration`, `ExportAllDeclaration`, and export declarations without a local declaration return `false`; `importKind === "type"` therefore remains counterless even with an empty specifier list. Runtime declarations/initializers and direct statements return `true`.
- [x] **Step 2: Rename inventory, evaluator sets, totals, diagnostics, and policy guards.**
Use these exact artifact fields: `counterBearingTotal`, `instrumentedCounterBearingTotal`, `counterlessTotal`, `counterlessModules`. Use diagnostics containing `counter-bearing`, `counterless`, and `policy-sensitive module cannot be counterless`; remove executable/non-executable terminology from the risk-coverage implementation and tests.
- [x] **Step 3: Run focused GREEN verification.**
Run: `./node_modules/.bin/vitest run tests/unit/risk-coverage.test.ts tests/unit/risk-coverage-files.test.ts tests/unit/bounded-body-reader.test.ts --reporter=dot`
Expected: all focused tests pass and both static partition directions remain fail-closed.
### Task 3: Refresh documentation, repository evidence, and removal evidence
**Files:**
- Modify: `docs/testing/frontend-platform-testing-strategy.md`
- Modify: `.superpowers/sdd/2026-08-01-quality-architecture-remediation/task-1-report.md`
- Modify: `.superpowers/sdd/2026-08-01-quality-architecture-remediation/progress.md`
**Interfaces:**
- Consumes: root and sample-removal checker output after Task 2.
- Produces: documented V8 counter-bearing semantics and current 285/285 plus 268/268 evidence.
- [x] **Step 1: Document that counterless imports/re-exports may execute at runtime but receive no file counters in the observed producer.**
- [x] **Step 2: Run relevant verification.**
```sh
./node_modules/.bin/tsc --noEmit -p tsconfig.node.json
./node_modules/.bin/tsc --noEmit -p tsconfig.test.json
./node_modules/.bin/eslint scripts/lib/risk-coverage.ts tests/unit/risk-coverage.test.ts --max-warnings=0
node scripts/check-risk-coverage.ts
corepack pnpm test:sample-removal
git diff --check
```
Expected root checker: `Risk coverage: PASS (285/285 production modules, 80 thresholds)`.
Expected removal checker: `Risk coverage: PASS (268/268 production modules, 76 thresholds)`; the already-known dependency-cruiser architecture diagnostic may remain the sole removal failure.
- [x] **Step 3: Commit the independently verified follow-up.**
```sh
git add docs/superpowers/plans/2026-08-02-counter-bearing-coverage-provenance.md docs/testing/frontend-platform-testing-strategy.md scripts/lib/risk-coverage.ts tests/unit/risk-coverage.test.ts
git commit -m "refactor: align coverage counter provenance"
```
## Self-review
- Spec coverage: terminology, import/re-export edge cases, policy diagnostics, artifact fields, root/removal evidence, report, and ledger are each assigned above.
- Placeholder scan: no deferred implementation or unspecified test step remains.
- Type consistency: inventory and result names use `counterBearing*`/`counterless*` throughout; the classifier is `hasCoverageCounterBearingStatements`.
@@ -133,6 +133,17 @@ operation/query-mutation/registry compatibility 12개 high-risk module에 52개
scoped threshold를 적용한다. critical module 누락 또는 threshold 미달 fixture는
merge gate를 실패시킨다.
Repository inventory의 정적 provenance는 runtime 실행 가능 여부가 아니라 현재
Vitest/V8 producer가 file counter를 생성하는 문장의 존재 여부를 나타낸다. 따라서
artifact는 `counterBearingTotal`, `instrumentedCounterBearingTotal`,
`counterlessTotal`, `counterlessModules`를 기록한다. 실제 microfixture에서
`import type {}`, `import {}`, bare side-effect import, value import, named/star value
re-export만 있는 모듈은 모두 exact all-zero row였고, 선언/초기화 또는 직접 실행문이
있는 모듈은 counter-bearing이었다. Counterless import/re-export도 module evaluation
과 side effect를 유발할 수 있으므로 이를 non-executable로 해석하지 않는다. Gate는
static counterless 집합과 exact all-zero row 집합의 일치를 요구하고 critical/high-risk
policy module이 counterless이면 실패시킨다.
남은 범위는 실제 device/browser farm, cloud visual approval, 외부 인증·telemetry
provider와 production field data다. 이 증거가 없을 때 저장소 내부 test를
`PRODUCTION_READY`의 대체물로 사용하지 않는다.
+52 -49
View File
@@ -63,18 +63,18 @@ export type ProductionModuleInventory = Readonly<{
files: readonly string[];
preExclusionTotal: number;
generatedExclusions: readonly string[];
executableModules: readonly string[];
nonExecutableModules: readonly string[];
counterBearingModules: readonly string[];
counterlessModules: readonly string[];
}>;
export type RiskCoverageResult = Readonly<{
status: "PASS" | "FAIL";
selectedTotal: number;
repositoryTotal: number;
executableTotal: number;
instrumentedExecutableTotal: number;
nonExecutableTotal: number;
nonExecutableModules: readonly string[];
counterBearingTotal: number;
instrumentedCounterBearingTotal: number;
counterlessTotal: number;
counterlessModules: readonly string[];
preExclusionTotal: number;
generatedExclusionCount: number;
generatedExclusions: readonly string[];
@@ -370,13 +370,12 @@ type TypeScriptAstNode = Readonly<{
type?: unknown;
body?: unknown;
declaration?: unknown;
specifiers?: unknown;
declare?: unknown;
const?: unknown;
importKind?: unknown;
}>;
function statementIsExecutable(value: unknown): boolean {
function statementIsCoverageCounterBearing(value: unknown): boolean {
if (!isRecord(value) || typeof value.type !== "string") {
throw new TypeError("TypeScript parser returned an invalid statement");
}
@@ -393,7 +392,7 @@ function statementIsExecutable(value: unknown): boolean {
return false;
}
if (statement.type === "ImportDeclaration") {
return Array.isArray(statement.specifiers) && statement.specifiers.length === 0;
return false;
}
if (
statement.type === "ExportAllDeclaration" ||
@@ -406,7 +405,7 @@ function statementIsExecutable(value: unknown): boolean {
statement.type === "ExportDefaultDeclaration"
) {
return statement.declaration !== null && statement.declaration !== undefined
? statementIsExecutable(statement.declaration)
? statementIsCoverageCounterBearing(statement.declaration)
: false;
}
if (
@@ -427,7 +426,7 @@ function statementIsExecutable(value: unknown): boolean {
return true;
}
export function hasExecutableTypeScriptStatements(
export function hasCoverageCounterBearingStatements(
source: string,
relativePath: string,
): boolean {
@@ -454,7 +453,7 @@ export function hasExecutableTypeScriptStatements(
if (!isRecord(parsed) || !Array.isArray(parsed.body)) {
throw new TypeError("TypeScript parser returned an invalid program");
}
return parsed.body.some(statementIsExecutable);
return parsed.body.some(statementIsCoverageCounterBearing);
}
export async function buildProductionModuleInventory(
@@ -481,8 +480,8 @@ export async function buildProductionModuleInventory(
}
const allModules: string[] = [];
const executableModules: string[] = [];
const nonExecutableModules: string[] = [];
const counterBearingModules: string[] = [];
const counterlessModules: string[] = [];
async function visit(relativeDirectory: string): Promise<void> {
const absoluteDirectory = path.join(repositoryRoot, relativeDirectory);
const entries = await readDirectory(absoluteDirectory);
@@ -543,10 +542,10 @@ export async function buildProductionModuleInventory(
await handle?.close();
}
allModules.push(relativeTarget);
if (hasExecutableTypeScriptStatements(source, relativeTarget)) {
executableModules.push(relativeTarget);
if (hasCoverageCounterBearingStatements(source, relativeTarget)) {
counterBearingModules.push(relativeTarget);
} else {
nonExecutableModules.push(relativeTarget);
counterlessModules.push(relativeTarget);
}
}
}
@@ -565,11 +564,11 @@ export async function buildProductionModuleInventory(
files: Object.freeze(inventory),
preExclusionTotal: allModules.length,
generatedExclusions: Object.freeze([...generatedPaths].sort()),
executableModules: Object.freeze(
executableModules.filter((file) => !generated.has(file)).sort(),
counterBearingModules: Object.freeze(
counterBearingModules.filter((file) => !generated.has(file)).sort(),
),
nonExecutableModules: Object.freeze(
nonExecutableModules.filter((file) => !generated.has(file)).sort(),
counterlessModules: Object.freeze(
counterlessModules.filter((file) => !generated.has(file)).sort(),
),
});
}
@@ -718,22 +717,22 @@ export function evaluateRiskCoverage(input: Readonly<{
if (new Set(inventory).size !== inventory.length) {
throw new TypeError("production module inventory contains a duplicate path");
}
const executableModules = input.inventory.executableModules.map((file) =>
exactSourcePath(file, "executable inventory path"),
const counterBearingModules = input.inventory.counterBearingModules.map((file) =>
exactSourcePath(file, "counter-bearing inventory path"),
);
const nonExecutableModules = input.inventory.nonExecutableModules.map((file) =>
exactSourcePath(file, "non-executable inventory path"),
const counterlessModules = input.inventory.counterlessModules.map((file) =>
exactSourcePath(file, "counterless inventory path"),
);
const executableSet = new Set(executableModules);
const nonExecutableSet = new Set(nonExecutableModules);
const partition = [...executableModules, ...nonExecutableModules].sort();
const counterBearingSet = new Set(counterBearingModules);
const counterlessSet = new Set(counterlessModules);
const partition = [...counterBearingModules, ...counterlessModules].sort();
if (
executableSet.size !== executableModules.length ||
nonExecutableSet.size !== nonExecutableModules.length ||
executableModules.some((file) => nonExecutableSet.has(file)) ||
counterBearingSet.size !== counterBearingModules.length ||
counterlessSet.size !== counterlessModules.length ||
counterBearingModules.some((file) => counterlessSet.has(file)) ||
partition.join("\n") !== [...inventory].sort().join("\n")
) {
throw new TypeError("production inventory executable provenance is inconsistent");
throw new TypeError("production inventory counter-bearing provenance is inconsistent");
}
const generatedExclusions = input.inventory.generatedExclusions.map((file) =>
exactSourcePath(file, "generated exclusion"),
@@ -780,14 +779,14 @@ export function evaluateRiskCoverage(input: Readonly<{
);
});
const zeroCoverageSet = new Set(zeroCoverageModules);
const zeroExecutableModules = zeroCoverageModules.filter((file) =>
executableSet.has(file),
const zeroCounterBearingModules = zeroCoverageModules.filter((file) =>
counterBearingSet.has(file),
);
const nonExecutableWithCounters = nonExecutableModules.filter((file) => {
const counterlessWithCounters = counterlessModules.filter((file) => {
const metrics = selected.get(file);
return metrics !== undefined && !zeroCoverageSet.has(file);
});
const instrumentedExecutableTotal = executableModules.filter((file) => {
const instrumentedCounterBearingTotal = counterBearingModules.filter((file) => {
const metrics = selected.get(file);
return metrics !== undefined && !zeroCoverageSet.has(file);
}).length;
@@ -827,18 +826,18 @@ export function evaluateRiskCoverage(input: Readonly<{
.filter(
(file) =>
!selected.has(file) ||
(executableSet.has(file) && zeroCoverageSet.has(file)),
(counterBearingSet.has(file) && zeroCoverageSet.has(file)),
)
.sort();
failures.push(
...inventory
.filter((file) => !selected.has(file))
.map((file) => `production module missing from coverage: ${file}`),
...zeroExecutableModules.map(
(file) => `production module has zero coverage totals: ${file}`,
...zeroCounterBearingModules.map(
(file) => `counter-bearing module has zero coverage totals: ${file}`,
),
...nonExecutableWithCounters.map(
(file) => `non-executable module has coverage counters: ${file}`,
...counterlessWithCounters.map(
(file) => `counterless module has coverage counters: ${file}`,
),
);
for (const modulePolicy of input.policy.criticalModules) {
@@ -846,8 +845,10 @@ export function evaluateRiskCoverage(input: Readonly<{
failures.push(`critical module is outside production inventory: ${modulePolicy.path}`);
continue;
}
if (nonExecutableSet.has(modulePolicy.path)) {
failures.push(`critical module cannot be non-executable: ${modulePolicy.path}`);
if (counterlessSet.has(modulePolicy.path)) {
failures.push(
`critical policy-sensitive module cannot be counterless: ${modulePolicy.path}`,
);
}
const actual = selected.get(modulePolicy.path);
if (!actual) {
@@ -863,8 +864,10 @@ export function evaluateRiskCoverage(input: Readonly<{
if (!inventorySet.has(highRiskPath)) {
failures.push(`high-risk module is outside production inventory: ${highRiskPath}`);
}
if (nonExecutableSet.has(highRiskPath)) {
failures.push(`high-risk module cannot be non-executable: ${highRiskPath}`);
if (counterlessSet.has(highRiskPath)) {
failures.push(
`high-risk policy-sensitive module cannot be counterless: ${highRiskPath}`,
);
}
}
for (const waiver of input.policy.waivers) {
@@ -881,10 +884,10 @@ export function evaluateRiskCoverage(input: Readonly<{
status: failures.length === 0 ? "PASS" : "FAIL",
selectedTotal: inventory.length - uncoveredModules.length,
repositoryTotal: inventory.length,
executableTotal: executableModules.length,
instrumentedExecutableTotal,
nonExecutableTotal: nonExecutableModules.length,
nonExecutableModules: Object.freeze([...nonExecutableModules].sort()),
counterBearingTotal: counterBearingModules.length,
instrumentedCounterBearingTotal,
counterlessTotal: counterlessModules.length,
counterlessModules: Object.freeze([...counterlessModules].sort()),
preExclusionTotal: input.inventory.preExclusionTotal,
generatedExclusionCount: generatedExclusions.length,
generatedExclusions: Object.freeze([...generatedExclusions].sort()),
+51 -28
View File
@@ -48,16 +48,16 @@ const fullMetrics = metrics();
function inventory(
files: readonly string[],
generatedExclusions: readonly string[] = [],
nonExecutableModules: readonly string[] = [],
counterlessModules: readonly string[] = [],
): ProductionModuleInventory {
return {
files,
preExclusionTotal: files.length + generatedExclusions.length,
generatedExclusions,
executableModules: files.filter(
(file) => !nonExecutableModules.includes(file),
counterBearingModules: files.filter(
(file) => !counterlessModules.includes(file),
),
nonExecutableModules,
counterlessModules,
};
}
@@ -160,8 +160,8 @@ describe("repository-aware risk coverage", () => {
files: ["src/a.ts", "src/nested/b.tsx"],
preExclusionTotal: 3,
generatedExclusions: ["src/generated.ts"],
executableModules: ["src/a.ts", "src/nested/b.tsx"],
nonExecutableModules: [],
counterBearingModules: ["src/a.ts", "src/nested/b.tsx"],
counterlessModules: [],
});
expect(result).toMatchObject({
status: "FAIL",
@@ -347,7 +347,7 @@ describe("repository-aware risk coverage", () => {
uncoveredModules: ["src/a.ts"],
});
expect(result.failures).toContain(
"production module has zero coverage totals: src/a.ts",
"counter-bearing module has zero coverage totals: src/a.ts",
);
expect(result.failures).toEqual(
expect.arrayContaining([
@@ -359,7 +359,7 @@ describe("repository-aware risk coverage", () => {
);
});
it("accepts exact all-zero rows only for statically non-executable modules", () => {
it("accepts exact all-zero rows only for statically counterless modules", () => {
const parsedPolicy = parseRiskCoveragePolicy(
policy({ repositoryBaseline: 2, generatedPaths: [] }),
{ now },
@@ -384,10 +384,10 @@ describe("repository-aware risk coverage", () => {
status: "PASS",
selectedTotal: 2,
repositoryTotal: 2,
executableTotal: 1,
instrumentedExecutableTotal: 1,
nonExecutableTotal: 1,
nonExecutableModules: ["src/type-only.ts"],
counterBearingTotal: 1,
instrumentedCounterBearingTotal: 1,
counterlessTotal: 1,
counterlessModules: ["src/type-only.ts"],
uncoveredModules: [],
});
@@ -402,11 +402,11 @@ describe("repository-aware risk coverage", () => {
},
});
expect(mismatch.failures).toContain(
"non-executable module has coverage counters: src/type-only.ts",
"counterless module has coverage counters: src/type-only.ts",
);
});
it("forbids critical and high-risk modules from being non-executable", () => {
it("forbids policy-sensitive modules from being counterless", () => {
const parsedPolicy = parseRiskCoveragePolicy(
policy({ repositoryBaseline: 1, generatedPaths: [] }),
{ now },
@@ -420,8 +420,8 @@ describe("repository-aware risk coverage", () => {
expect(result.failures).toEqual(
expect.arrayContaining([
"critical module cannot be non-executable: src/a.ts",
"high-risk module cannot be non-executable: src/a.ts",
"critical policy-sensitive module cannot be counterless: src/a.ts",
"high-risk policy-sensitive module cannot be counterless: src/a.ts",
]),
);
});
@@ -564,7 +564,7 @@ describe("repository-aware risk coverage", () => {
).rejects.toThrow(/stable file identity unavailable/u);
});
it("classifies type-only and barrel modules separately from runtime statements", async () => {
it("matches the observed V8 counterless import and re-export syntax", async () => {
const repositoryRoot = await repositoryFixture();
await Promise.all([
writeFile(
@@ -584,9 +584,29 @@ describe("repository-aware risk coverage", () => {
"void globalThis;\n",
),
writeFile(
path.join(repositoryRoot, "src/side-effect-import.ts"),
path.join(repositoryRoot, "src/import-type-empty.ts"),
"import type {} from './a.ts';\n",
),
writeFile(
path.join(repositoryRoot, "src/import-value-empty.ts"),
"import {} from './a.ts';\n",
),
writeFile(
path.join(repositoryRoot, "src/import-side-effect.ts"),
"import './a.ts';\n",
),
writeFile(
path.join(repositoryRoot, "src/import-value.ts"),
"import { a } from './a.ts';\n",
),
writeFile(
path.join(repositoryRoot, "src/reexport-named.ts"),
"export { a } from './a.ts';\n",
),
writeFile(
path.join(repositoryRoot, "src/reexport-star.ts"),
"export * from './a.ts';\n",
),
]);
const productionInventory = await buildProductionModuleInventory({
@@ -594,19 +614,22 @@ describe("repository-aware risk coverage", () => {
generatedPaths: ["src/generated.ts"],
});
expect(productionInventory.nonExecutableModules).toEqual([
expect(productionInventory.counterlessModules).toEqual([
"src/barrel.ts",
"src/import-side-effect.ts",
"src/import-type-empty.ts",
"src/import-value-empty.ts",
"src/import-value.ts",
"src/reexport-named.ts",
"src/reexport-star.ts",
"src/type-only.ts",
]);
expect(productionInventory.executableModules).toEqual(
expect.arrayContaining([
"src/a.ts",
"src/nested/b.tsx",
"src/runtime-export.ts",
"src/side-effect-import.ts",
"src/side-effect.ts",
]),
);
expect(productionInventory.counterBearingModules).toEqual([
"src/a.ts",
"src/nested/b.tsx",
"src/runtime-export.ts",
"src/side-effect.ts",
]);
});
it("rejects a post-lstat file identity swap even without relying on O_NOFOLLOW", async () => {