diff --git a/docs/superpowers/plans/2026-08-02-counter-bearing-coverage-provenance.md b/docs/superpowers/plans/2026-08-02-counter-bearing-coverage-provenance.md new file mode 100644 index 0000000..f7cea71 --- /dev/null +++ b/docs/superpowers/plans/2026-08-02-counter-bearing-coverage-provenance.md @@ -0,0 +1,133 @@ +# Counter-bearing Coverage Provenance Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Align repository coverage provenance names and static classification with the counters that Vitest/V8 actually emits, without making claims about JavaScript runtime executability. + +**Architecture:** The inventory parser classifies source files only by whether their top-level AST contains statements known to receive V8 counters. Coverage evaluation requires exact agreement between that static counter-bearing/counterless partition and producer rows, while policy-sensitive modules must remain counter-bearing. The JSON artifact exposes the same terminology as the inventory and diagnostics. + +**Tech Stack:** TypeScript 7, Node.js 24, `@babel/eslint-parser`, Vitest 4, V8 coverage. + +## Global Constraints + +- `runtime.ts` declarations/initializers and direct execution statements are counter-bearing. +- Type-only modules, `import type {}`, `import {}`, bare side-effect imports, value imports, named value re-exports, and star value re-exports are counterless under the observed Vitest/V8 producer. +- Counterless does not mean non-executable; code, artifacts, diagnostics, tests, and documentation must not make that claim. +- Exact all-zero rows are accepted only for statically counterless modules. +- Critical and high-risk policy modules cannot be counterless. +- All source edits use `apply_patch` and behavior changes follow RED-GREEN TDD. + +--- + +### Task 1: Lock the Vitest/V8 classifier contract with RED tests + +**Files:** +- Modify: `tests/unit/risk-coverage.test.ts` + +**Interfaces:** +- Consumes: `buildProductionModuleInventory()` and `evaluateRiskCoverage()`. +- Produces: expectations for `counterBearingModules`, `counterlessModules`, `counterBearingTotal`, `instrumentedCounterBearingTotal`, `counterlessTotal`, and `counterlessModules`. + +- [x] **Step 1: Rename the test inventory helper and artifact assertions to the desired API.** + +```ts +function inventory( + files: readonly string[], + generatedExclusions: readonly string[] = [], + counterlessModules: readonly string[] = [], +): ProductionModuleInventory { + return { + files, + preExclusionTotal: files.length + generatedExclusions.length, + generatedExclusions, + counterBearingModules: files.filter((file) => !counterlessModules.includes(file)), + counterlessModules, + }; +} +``` + +- [x] **Step 2: Add a real-source inventory regression table.** + +```ts +const counterlessSources = { + "import-type-empty.ts": "import type {} from './a.ts';\n", + "import-value-empty.ts": "import {} from './a.ts';\n", + "import-side-effect.ts": "import './a.ts';\n", + "import-value.ts": "import { a } from './a.ts';\n", + "reexport-named.ts": "export { a } from './a.ts';\n", + "reexport-star.ts": "export * from './a.ts';\n", +}; +``` + +Assert every key appears in `counterlessModules`, while `export const runtimeValue = 1` and `void globalThis` appear in `counterBearingModules`. + +- [x] **Step 3: Run the focused test and verify RED.** + +Run: `./node_modules/.bin/vitest run tests/unit/risk-coverage.test.ts --reporter=dot` + +Expected: TypeScript/test failures because the counter-bearing API fields do not exist and the current bare import classifier is executable-labelled. + +### Task 2: Rename and align static coverage provenance + +**Files:** +- Modify: `scripts/lib/risk-coverage.ts` +- Modify: `tests/unit/risk-coverage.test.ts` + +**Interfaces:** +- Consumes: Babel `Program.body` nodes and parsed Istanbul/V8 counters. +- Produces: `hasCoverageCounterBearingStatements(source, relativePath)`, a complete `counterBearingModules`/`counterlessModules` partition, and consistently named `RiskCoverageResult` fields. + +- [x] **Step 1: Implement the minimal classifier needed by the RED cases.** + +`ImportDeclaration`, `ExportAllDeclaration`, and export declarations without a local declaration return `false`; `importKind === "type"` therefore remains counterless even with an empty specifier list. Runtime declarations/initializers and direct statements return `true`. + +- [x] **Step 2: Rename inventory, evaluator sets, totals, diagnostics, and policy guards.** + +Use these exact artifact fields: `counterBearingTotal`, `instrumentedCounterBearingTotal`, `counterlessTotal`, `counterlessModules`. Use diagnostics containing `counter-bearing`, `counterless`, and `policy-sensitive module cannot be counterless`; remove executable/non-executable terminology from the risk-coverage implementation and tests. + +- [x] **Step 3: Run focused GREEN verification.** + +Run: `./node_modules/.bin/vitest run tests/unit/risk-coverage.test.ts tests/unit/risk-coverage-files.test.ts tests/unit/bounded-body-reader.test.ts --reporter=dot` + +Expected: all focused tests pass and both static partition directions remain fail-closed. + +### Task 3: Refresh documentation, repository evidence, and removal evidence + +**Files:** +- Modify: `docs/testing/frontend-platform-testing-strategy.md` +- Modify: `.superpowers/sdd/2026-08-01-quality-architecture-remediation/task-1-report.md` +- Modify: `.superpowers/sdd/2026-08-01-quality-architecture-remediation/progress.md` + +**Interfaces:** +- Consumes: root and sample-removal checker output after Task 2. +- Produces: documented V8 counter-bearing semantics and current 285/285 plus 268/268 evidence. + +- [x] **Step 1: Document that counterless imports/re-exports may execute at runtime but receive no file counters in the observed producer.** + +- [x] **Step 2: Run relevant verification.** + +```sh +./node_modules/.bin/tsc --noEmit -p tsconfig.node.json +./node_modules/.bin/tsc --noEmit -p tsconfig.test.json +./node_modules/.bin/eslint scripts/lib/risk-coverage.ts tests/unit/risk-coverage.test.ts --max-warnings=0 +node scripts/check-risk-coverage.ts +corepack pnpm test:sample-removal +git diff --check +``` + +Expected root checker: `Risk coverage: PASS (285/285 production modules, 80 thresholds)`. + +Expected removal checker: `Risk coverage: PASS (268/268 production modules, 76 thresholds)`; the already-known dependency-cruiser architecture diagnostic may remain the sole removal failure. + +- [x] **Step 3: Commit the independently verified follow-up.** + +```sh +git add docs/superpowers/plans/2026-08-02-counter-bearing-coverage-provenance.md docs/testing/frontend-platform-testing-strategy.md scripts/lib/risk-coverage.ts tests/unit/risk-coverage.test.ts +git commit -m "refactor: align coverage counter provenance" +``` + +## Self-review + +- Spec coverage: terminology, import/re-export edge cases, policy diagnostics, artifact fields, root/removal evidence, report, and ledger are each assigned above. +- Placeholder scan: no deferred implementation or unspecified test step remains. +- Type consistency: inventory and result names use `counterBearing*`/`counterless*` throughout; the classifier is `hasCoverageCounterBearingStatements`. diff --git a/docs/testing/frontend-platform-testing-strategy.md b/docs/testing/frontend-platform-testing-strategy.md index 12c4039..2319fec 100644 --- a/docs/testing/frontend-platform-testing-strategy.md +++ b/docs/testing/frontend-platform-testing-strategy.md @@ -133,6 +133,17 @@ operation/query-mutation/registry compatibility 12개 high-risk module에 52개 scoped threshold를 적용한다. critical module 누락 또는 threshold 미달 fixture는 merge gate를 실패시킨다. +Repository inventory의 정적 provenance는 runtime 실행 가능 여부가 아니라 현재 +Vitest/V8 producer가 file counter를 생성하는 문장의 존재 여부를 나타낸다. 따라서 +artifact는 `counterBearingTotal`, `instrumentedCounterBearingTotal`, +`counterlessTotal`, `counterlessModules`를 기록한다. 실제 microfixture에서 +`import type {}`, `import {}`, bare side-effect import, value import, named/star value +re-export만 있는 모듈은 모두 exact all-zero row였고, 선언/초기화 또는 직접 실행문이 +있는 모듈은 counter-bearing이었다. Counterless import/re-export도 module evaluation +과 side effect를 유발할 수 있으므로 이를 non-executable로 해석하지 않는다. Gate는 +static counterless 집합과 exact all-zero row 집합의 일치를 요구하고 critical/high-risk +policy module이 counterless이면 실패시킨다. + 남은 범위는 실제 device/browser farm, cloud visual approval, 외부 인증·telemetry provider와 production field data다. 이 증거가 없을 때 저장소 내부 test를 `PRODUCTION_READY`의 대체물로 사용하지 않는다. diff --git a/scripts/lib/risk-coverage.ts b/scripts/lib/risk-coverage.ts index 953c25f..c0b6bc2 100644 --- a/scripts/lib/risk-coverage.ts +++ b/scripts/lib/risk-coverage.ts @@ -63,18 +63,18 @@ export type ProductionModuleInventory = Readonly<{ files: readonly string[]; preExclusionTotal: number; generatedExclusions: readonly string[]; - executableModules: readonly string[]; - nonExecutableModules: readonly string[]; + counterBearingModules: readonly string[]; + counterlessModules: readonly string[]; }>; export type RiskCoverageResult = Readonly<{ status: "PASS" | "FAIL"; selectedTotal: number; repositoryTotal: number; - executableTotal: number; - instrumentedExecutableTotal: number; - nonExecutableTotal: number; - nonExecutableModules: readonly string[]; + counterBearingTotal: number; + instrumentedCounterBearingTotal: number; + counterlessTotal: number; + counterlessModules: readonly string[]; preExclusionTotal: number; generatedExclusionCount: number; generatedExclusions: readonly string[]; @@ -370,13 +370,12 @@ type TypeScriptAstNode = Readonly<{ type?: unknown; body?: unknown; declaration?: unknown; - specifiers?: unknown; declare?: unknown; const?: unknown; importKind?: unknown; }>; -function statementIsExecutable(value: unknown): boolean { +function statementIsCoverageCounterBearing(value: unknown): boolean { if (!isRecord(value) || typeof value.type !== "string") { throw new TypeError("TypeScript parser returned an invalid statement"); } @@ -393,7 +392,7 @@ function statementIsExecutable(value: unknown): boolean { return false; } if (statement.type === "ImportDeclaration") { - return Array.isArray(statement.specifiers) && statement.specifiers.length === 0; + return false; } if ( statement.type === "ExportAllDeclaration" || @@ -406,7 +405,7 @@ function statementIsExecutable(value: unknown): boolean { statement.type === "ExportDefaultDeclaration" ) { return statement.declaration !== null && statement.declaration !== undefined - ? statementIsExecutable(statement.declaration) + ? statementIsCoverageCounterBearing(statement.declaration) : false; } if ( @@ -427,7 +426,7 @@ function statementIsExecutable(value: unknown): boolean { return true; } -export function hasExecutableTypeScriptStatements( +export function hasCoverageCounterBearingStatements( source: string, relativePath: string, ): boolean { @@ -454,7 +453,7 @@ export function hasExecutableTypeScriptStatements( if (!isRecord(parsed) || !Array.isArray(parsed.body)) { throw new TypeError("TypeScript parser returned an invalid program"); } - return parsed.body.some(statementIsExecutable); + return parsed.body.some(statementIsCoverageCounterBearing); } export async function buildProductionModuleInventory( @@ -481,8 +480,8 @@ export async function buildProductionModuleInventory( } const allModules: string[] = []; - const executableModules: string[] = []; - const nonExecutableModules: string[] = []; + const counterBearingModules: string[] = []; + const counterlessModules: string[] = []; async function visit(relativeDirectory: string): Promise { const absoluteDirectory = path.join(repositoryRoot, relativeDirectory); const entries = await readDirectory(absoluteDirectory); @@ -543,10 +542,10 @@ export async function buildProductionModuleInventory( await handle?.close(); } allModules.push(relativeTarget); - if (hasExecutableTypeScriptStatements(source, relativeTarget)) { - executableModules.push(relativeTarget); + if (hasCoverageCounterBearingStatements(source, relativeTarget)) { + counterBearingModules.push(relativeTarget); } else { - nonExecutableModules.push(relativeTarget); + counterlessModules.push(relativeTarget); } } } @@ -565,11 +564,11 @@ export async function buildProductionModuleInventory( files: Object.freeze(inventory), preExclusionTotal: allModules.length, generatedExclusions: Object.freeze([...generatedPaths].sort()), - executableModules: Object.freeze( - executableModules.filter((file) => !generated.has(file)).sort(), + counterBearingModules: Object.freeze( + counterBearingModules.filter((file) => !generated.has(file)).sort(), ), - nonExecutableModules: Object.freeze( - nonExecutableModules.filter((file) => !generated.has(file)).sort(), + counterlessModules: Object.freeze( + counterlessModules.filter((file) => !generated.has(file)).sort(), ), }); } @@ -718,22 +717,22 @@ export function evaluateRiskCoverage(input: Readonly<{ if (new Set(inventory).size !== inventory.length) { throw new TypeError("production module inventory contains a duplicate path"); } - const executableModules = input.inventory.executableModules.map((file) => - exactSourcePath(file, "executable inventory path"), + const counterBearingModules = input.inventory.counterBearingModules.map((file) => + exactSourcePath(file, "counter-bearing inventory path"), ); - const nonExecutableModules = input.inventory.nonExecutableModules.map((file) => - exactSourcePath(file, "non-executable inventory path"), + const counterlessModules = input.inventory.counterlessModules.map((file) => + exactSourcePath(file, "counterless inventory path"), ); - const executableSet = new Set(executableModules); - const nonExecutableSet = new Set(nonExecutableModules); - const partition = [...executableModules, ...nonExecutableModules].sort(); + const counterBearingSet = new Set(counterBearingModules); + const counterlessSet = new Set(counterlessModules); + const partition = [...counterBearingModules, ...counterlessModules].sort(); if ( - executableSet.size !== executableModules.length || - nonExecutableSet.size !== nonExecutableModules.length || - executableModules.some((file) => nonExecutableSet.has(file)) || + counterBearingSet.size !== counterBearingModules.length || + counterlessSet.size !== counterlessModules.length || + counterBearingModules.some((file) => counterlessSet.has(file)) || partition.join("\n") !== [...inventory].sort().join("\n") ) { - throw new TypeError("production inventory executable provenance is inconsistent"); + throw new TypeError("production inventory counter-bearing provenance is inconsistent"); } const generatedExclusions = input.inventory.generatedExclusions.map((file) => exactSourcePath(file, "generated exclusion"), @@ -780,14 +779,14 @@ export function evaluateRiskCoverage(input: Readonly<{ ); }); const zeroCoverageSet = new Set(zeroCoverageModules); - const zeroExecutableModules = zeroCoverageModules.filter((file) => - executableSet.has(file), + const zeroCounterBearingModules = zeroCoverageModules.filter((file) => + counterBearingSet.has(file), ); - const nonExecutableWithCounters = nonExecutableModules.filter((file) => { + const counterlessWithCounters = counterlessModules.filter((file) => { const metrics = selected.get(file); return metrics !== undefined && !zeroCoverageSet.has(file); }); - const instrumentedExecutableTotal = executableModules.filter((file) => { + const instrumentedCounterBearingTotal = counterBearingModules.filter((file) => { const metrics = selected.get(file); return metrics !== undefined && !zeroCoverageSet.has(file); }).length; @@ -827,18 +826,18 @@ export function evaluateRiskCoverage(input: Readonly<{ .filter( (file) => !selected.has(file) || - (executableSet.has(file) && zeroCoverageSet.has(file)), + (counterBearingSet.has(file) && zeroCoverageSet.has(file)), ) .sort(); failures.push( ...inventory .filter((file) => !selected.has(file)) .map((file) => `production module missing from coverage: ${file}`), - ...zeroExecutableModules.map( - (file) => `production module has zero coverage totals: ${file}`, + ...zeroCounterBearingModules.map( + (file) => `counter-bearing module has zero coverage totals: ${file}`, ), - ...nonExecutableWithCounters.map( - (file) => `non-executable module has coverage counters: ${file}`, + ...counterlessWithCounters.map( + (file) => `counterless module has coverage counters: ${file}`, ), ); for (const modulePolicy of input.policy.criticalModules) { @@ -846,8 +845,10 @@ export function evaluateRiskCoverage(input: Readonly<{ failures.push(`critical module is outside production inventory: ${modulePolicy.path}`); continue; } - if (nonExecutableSet.has(modulePolicy.path)) { - failures.push(`critical module cannot be non-executable: ${modulePolicy.path}`); + if (counterlessSet.has(modulePolicy.path)) { + failures.push( + `critical policy-sensitive module cannot be counterless: ${modulePolicy.path}`, + ); } const actual = selected.get(modulePolicy.path); if (!actual) { @@ -863,8 +864,10 @@ export function evaluateRiskCoverage(input: Readonly<{ if (!inventorySet.has(highRiskPath)) { failures.push(`high-risk module is outside production inventory: ${highRiskPath}`); } - if (nonExecutableSet.has(highRiskPath)) { - failures.push(`high-risk module cannot be non-executable: ${highRiskPath}`); + if (counterlessSet.has(highRiskPath)) { + failures.push( + `high-risk policy-sensitive module cannot be counterless: ${highRiskPath}`, + ); } } for (const waiver of input.policy.waivers) { @@ -881,10 +884,10 @@ export function evaluateRiskCoverage(input: Readonly<{ status: failures.length === 0 ? "PASS" : "FAIL", selectedTotal: inventory.length - uncoveredModules.length, repositoryTotal: inventory.length, - executableTotal: executableModules.length, - instrumentedExecutableTotal, - nonExecutableTotal: nonExecutableModules.length, - nonExecutableModules: Object.freeze([...nonExecutableModules].sort()), + counterBearingTotal: counterBearingModules.length, + instrumentedCounterBearingTotal, + counterlessTotal: counterlessModules.length, + counterlessModules: Object.freeze([...counterlessModules].sort()), preExclusionTotal: input.inventory.preExclusionTotal, generatedExclusionCount: generatedExclusions.length, generatedExclusions: Object.freeze([...generatedExclusions].sort()), diff --git a/tests/unit/risk-coverage.test.ts b/tests/unit/risk-coverage.test.ts index 3c1d60c..3bc567d 100644 --- a/tests/unit/risk-coverage.test.ts +++ b/tests/unit/risk-coverage.test.ts @@ -48,16 +48,16 @@ const fullMetrics = metrics(); function inventory( files: readonly string[], generatedExclusions: readonly string[] = [], - nonExecutableModules: readonly string[] = [], + counterlessModules: readonly string[] = [], ): ProductionModuleInventory { return { files, preExclusionTotal: files.length + generatedExclusions.length, generatedExclusions, - executableModules: files.filter( - (file) => !nonExecutableModules.includes(file), + counterBearingModules: files.filter( + (file) => !counterlessModules.includes(file), ), - nonExecutableModules, + counterlessModules, }; } @@ -160,8 +160,8 @@ describe("repository-aware risk coverage", () => { files: ["src/a.ts", "src/nested/b.tsx"], preExclusionTotal: 3, generatedExclusions: ["src/generated.ts"], - executableModules: ["src/a.ts", "src/nested/b.tsx"], - nonExecutableModules: [], + counterBearingModules: ["src/a.ts", "src/nested/b.tsx"], + counterlessModules: [], }); expect(result).toMatchObject({ status: "FAIL", @@ -347,7 +347,7 @@ describe("repository-aware risk coverage", () => { uncoveredModules: ["src/a.ts"], }); expect(result.failures).toContain( - "production module has zero coverage totals: src/a.ts", + "counter-bearing module has zero coverage totals: src/a.ts", ); expect(result.failures).toEqual( expect.arrayContaining([ @@ -359,7 +359,7 @@ describe("repository-aware risk coverage", () => { ); }); - it("accepts exact all-zero rows only for statically non-executable modules", () => { + it("accepts exact all-zero rows only for statically counterless modules", () => { const parsedPolicy = parseRiskCoveragePolicy( policy({ repositoryBaseline: 2, generatedPaths: [] }), { now }, @@ -384,10 +384,10 @@ describe("repository-aware risk coverage", () => { status: "PASS", selectedTotal: 2, repositoryTotal: 2, - executableTotal: 1, - instrumentedExecutableTotal: 1, - nonExecutableTotal: 1, - nonExecutableModules: ["src/type-only.ts"], + counterBearingTotal: 1, + instrumentedCounterBearingTotal: 1, + counterlessTotal: 1, + counterlessModules: ["src/type-only.ts"], uncoveredModules: [], }); @@ -402,11 +402,11 @@ describe("repository-aware risk coverage", () => { }, }); expect(mismatch.failures).toContain( - "non-executable module has coverage counters: src/type-only.ts", + "counterless module has coverage counters: src/type-only.ts", ); }); - it("forbids critical and high-risk modules from being non-executable", () => { + it("forbids policy-sensitive modules from being counterless", () => { const parsedPolicy = parseRiskCoveragePolicy( policy({ repositoryBaseline: 1, generatedPaths: [] }), { now }, @@ -420,8 +420,8 @@ describe("repository-aware risk coverage", () => { expect(result.failures).toEqual( expect.arrayContaining([ - "critical module cannot be non-executable: src/a.ts", - "high-risk module cannot be non-executable: src/a.ts", + "critical policy-sensitive module cannot be counterless: src/a.ts", + "high-risk policy-sensitive module cannot be counterless: src/a.ts", ]), ); }); @@ -564,7 +564,7 @@ describe("repository-aware risk coverage", () => { ).rejects.toThrow(/stable file identity unavailable/u); }); - it("classifies type-only and barrel modules separately from runtime statements", async () => { + it("matches the observed V8 counterless import and re-export syntax", async () => { const repositoryRoot = await repositoryFixture(); await Promise.all([ writeFile( @@ -584,9 +584,29 @@ describe("repository-aware risk coverage", () => { "void globalThis;\n", ), writeFile( - path.join(repositoryRoot, "src/side-effect-import.ts"), + path.join(repositoryRoot, "src/import-type-empty.ts"), + "import type {} from './a.ts';\n", + ), + writeFile( + path.join(repositoryRoot, "src/import-value-empty.ts"), + "import {} from './a.ts';\n", + ), + writeFile( + path.join(repositoryRoot, "src/import-side-effect.ts"), "import './a.ts';\n", ), + writeFile( + path.join(repositoryRoot, "src/import-value.ts"), + "import { a } from './a.ts';\n", + ), + writeFile( + path.join(repositoryRoot, "src/reexport-named.ts"), + "export { a } from './a.ts';\n", + ), + writeFile( + path.join(repositoryRoot, "src/reexport-star.ts"), + "export * from './a.ts';\n", + ), ]); const productionInventory = await buildProductionModuleInventory({ @@ -594,19 +614,22 @@ describe("repository-aware risk coverage", () => { generatedPaths: ["src/generated.ts"], }); - expect(productionInventory.nonExecutableModules).toEqual([ + expect(productionInventory.counterlessModules).toEqual([ "src/barrel.ts", + "src/import-side-effect.ts", + "src/import-type-empty.ts", + "src/import-value-empty.ts", + "src/import-value.ts", + "src/reexport-named.ts", + "src/reexport-star.ts", "src/type-only.ts", ]); - expect(productionInventory.executableModules).toEqual( - expect.arrayContaining([ - "src/a.ts", - "src/nested/b.tsx", - "src/runtime-export.ts", - "src/side-effect-import.ts", - "src/side-effect.ts", - ]), - ); + expect(productionInventory.counterBearingModules).toEqual([ + "src/a.ts", + "src/nested/b.tsx", + "src/runtime-export.ts", + "src/side-effect.ts", + ]); }); it("rejects a post-lstat file identity swap even without relying on O_NOFOLLOW", async () => {