test: lock V8 coverage counter semantics
This commit is contained in:
DongHyeonka
@@ -0,0 +1,176 @@
|
||||
# V8 Coverage Counter Contract Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Publish risk-coverage artifact schema version 3 and continuously verify the installed Vitest/V8 producer's counter-bearing/counterless row semantics in an isolated child run.
|
||||
|
||||
**Architecture:** A real CLI contract test owns the serialized artifact assertion. A standalone producer checker copies fixed source templates into one OS-temp root, creates its child config and report there, validates an exact JSON summary, bounds child diagnostics, and removes the owned root in `finally`. `test:coverage` invokes the checker before repository coverage, which also carries it into CI and sample removal.
|
||||
|
||||
**Tech Stack:** TypeScript 7, Node.js 24 child processes and filesystem APIs, Vitest 4, V8 coverage.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Policy schema remains version 2; serialized risk-coverage artifact schema becomes version 3.
|
||||
- Child root, config, and reports directory are all below one owned OS temporary directory.
|
||||
- Main Vitest must not discover the child `.fixture.ts` file.
|
||||
- Child exit, summary absence, malformed/missing/additional rows, counterless nonzero drift, and runtime all-zero drift fail closed.
|
||||
- Child stdout/stderr included in diagnostics is bounded.
|
||||
- Cleanup uses `finally` and targets only the exact owned temporary root.
|
||||
- Source edits use `apply_patch`; behavior changes follow RED-GREEN TDD.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Version the real serialized artifact
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/unit/risk-coverage.test.ts`
|
||||
- Modify: `scripts/check-risk-coverage.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: the real `check-risk-coverage.ts` CLI, current policy structure, and an exact temporary coverage summary.
|
||||
- Produces: serialized artifact schema version 3 with `counterBearingTotal`, `instrumentedCounterBearingTotal`, `counterlessTotal`, and `counterlessModules` only.
|
||||
|
||||
- [x] **Step 1: Add the actual CLI serialization contract test.**
|
||||
|
||||
Create a temporary repository with the 19 current policy paths, write each as `export const covered = true`, set the cloned policy baseline to 19, write one full counter row per module, run the CLI with `process.execPath`, and assert:
|
||||
|
||||
```ts
|
||||
expect(artifact).toMatchObject({
|
||||
schemaVersion: 3,
|
||||
counterBearingTotal: 19,
|
||||
instrumentedCounterBearingTotal: 19,
|
||||
counterlessTotal: 0,
|
||||
counterlessModules: [],
|
||||
});
|
||||
expect(artifact).not.toHaveProperty("executableTotal");
|
||||
expect(artifact).not.toHaveProperty("instrumentedExecutableTotal");
|
||||
expect(artifact).not.toHaveProperty("nonExecutableTotal");
|
||||
expect(artifact).not.toHaveProperty("nonExecutableModules");
|
||||
```
|
||||
|
||||
- [x] **Step 2: Run the single test and verify RED.**
|
||||
|
||||
Run: `./node_modules/.bin/vitest run tests/unit/risk-coverage.test.ts -t "publishes artifact schema version 3" --reporter=dot`
|
||||
|
||||
Expected: FAIL because the actual artifact contains `schemaVersion: 2`.
|
||||
|
||||
- [x] **Step 3: Change only the serialized envelope to version 3.**
|
||||
|
||||
Change `schemaVersion: 2` to `schemaVersion: 3` in the value passed to `writeRiskCoverageArtifactAtomic`; do not change policy parsing.
|
||||
|
||||
- [x] **Step 4: Re-run the single test and verify GREEN.**
|
||||
|
||||
Run the Step 2 command and expect one passing test.
|
||||
|
||||
### Task 2: Lock actual Vitest/V8 counter semantics
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/counter-semantics.fixture.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/runtime.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/import-type-empty.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/import-empty.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/import-side-effect.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/import-value.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/reexport-named.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/reexport-star.ts`
|
||||
- Create: `tests/fixtures/v8-coverage-counter-semantics/src/type-only.ts`
|
||||
- Create: `tests/unit/v8-coverage-counter-semantics.test.ts`
|
||||
- Create: `scripts/lib/v8-coverage-counter-semantics.ts`
|
||||
- Create: `scripts/check-v8-coverage-counter-semantics.ts`
|
||||
- Modify: `vitest.config.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `assertV8CoverageCounterSemantics(summary, fixtureRoot)` and `checkV8CoverageCounterSemantics(options?)`.
|
||||
- Consumes: fixed fixture templates, owned temp paths, a shell-free Vitest child result, and `coverage-summary.json`.
|
||||
|
||||
- [x] **Step 1: Add fixture templates and failing checker tests.**
|
||||
|
||||
The fixture test imports the seven counterless modules and observes the direct/named/star runtime values. The unit tests use literal summaries to require exact rows and mutate them for missing row, extra row, counterless nonzero, and runtime all-zero failures. Runner tests inject child exit and successful-without-summary results and require bounded diagnostics plus removal of the owned root.
|
||||
|
||||
- [x] **Step 2: Run the new unit file and verify RED.**
|
||||
|
||||
Run: `./node_modules/.bin/vitest run tests/unit/v8-coverage-counter-semantics.test.ts --reporter=dot`
|
||||
|
||||
Expected: FAIL because `scripts/lib/v8-coverage-counter-semantics.ts` does not exist.
|
||||
|
||||
- [x] **Step 3: Implement exact summary validation and owned child execution.**
|
||||
|
||||
The default runner executes:
|
||||
|
||||
```ts
|
||||
execFile(process.execPath, [
|
||||
path.join(repositoryRoot, "node_modules/vitest/vitest.mjs"),
|
||||
"run",
|
||||
"--config",
|
||||
configPath,
|
||||
"--coverage",
|
||||
"--reporter=dot",
|
||||
"--no-color",
|
||||
], { cwd: ownedRoot, timeout: 30_000, maxBuffer: 256 * 1024 });
|
||||
```
|
||||
|
||||
The generated config has `root`, `include`, `coverage.reportsDirectory`, and `coverage.include` paths inside the owned root. Always remove the root in `finally`.
|
||||
|
||||
- [x] **Step 4: Add a behavioral main-discovery assertion.**
|
||||
|
||||
Run main `vitest list` filtered to the fixture directory with `--filesOnly --passWithNoTests`; require empty stdout. Add an explicit fixture-directory exclude in `vitest.config.ts`.
|
||||
|
||||
- [x] **Step 5: Run the new unit file and standalone checker for GREEN.**
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
./node_modules/.bin/vitest run tests/unit/v8-coverage-counter-semantics.test.ts --reporter=dot
|
||||
node scripts/check-v8-coverage-counter-semantics.ts
|
||||
```
|
||||
|
||||
Expected checker output: `V8 coverage counter semantics: PASS (1 counter-bearing, 7 counterless)`.
|
||||
|
||||
### Task 3: Wire coverage/CI and refresh documentation
|
||||
|
||||
**Files:**
|
||||
- Modify: `package.json`
|
||||
- Modify: `docs/testing/frontend-platform-testing-strategy.md`
|
||||
- Modify: `.superpowers/sdd/2026-08-01-quality-architecture-remediation/task-1-report.md`
|
||||
- Modify: `.superpowers/sdd/2026-08-01-quality-architecture-remediation/progress.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `check:v8-coverage-counter-semantics` and existing FE-GATE-005 `test:coverage` step.
|
||||
- Produces: package/CI/sample-removal execution and current 19-module/80-threshold documentation.
|
||||
|
||||
- [x] **Step 1: Add the package checker and prepend it to `test:coverage`.**
|
||||
|
||||
```json
|
||||
"check:v8-coverage-counter-semantics": "node scripts/check-v8-coverage-counter-semantics.ts",
|
||||
"test:coverage": "corepack pnpm check:v8-coverage-counter-semantics && vitest run ..."
|
||||
```
|
||||
|
||||
- [x] **Step 2: Synchronize documentation.**
|
||||
|
||||
Replace stale `12개 high-risk module` and `52개 scoped threshold` with `19개` and `80개`; document artifact schema 3 and policy schema 2 separately.
|
||||
|
||||
- [x] **Step 3: Run full relevant verification.**
|
||||
|
||||
```sh
|
||||
./node_modules/.bin/vitest run tests/unit/risk-coverage.test.ts tests/unit/risk-coverage-files.test.ts tests/unit/v8-coverage-counter-semantics.test.ts tests/unit/bounded-body-reader.test.ts --reporter=dot
|
||||
./node_modules/.bin/tsc --noEmit -p tsconfig.node.json
|
||||
./node_modules/.bin/tsc --noEmit -p tsconfig.test.json
|
||||
./node_modules/.bin/eslint scripts/check-risk-coverage.ts scripts/check-v8-coverage-counter-semantics.ts scripts/lib/v8-coverage-counter-semantics.ts tests/unit/risk-coverage.test.ts tests/unit/v8-coverage-counter-semantics.test.ts vitest.config.ts --max-warnings=0
|
||||
corepack pnpm check:v8-coverage-counter-semantics
|
||||
node scripts/check-risk-coverage.ts
|
||||
corepack pnpm test:sample-removal
|
||||
git diff --check
|
||||
```
|
||||
|
||||
- [x] **Step 4: Commit the verified closeout.**
|
||||
|
||||
```sh
|
||||
git add package.json vitest.config.ts scripts/check-risk-coverage.ts scripts/check-v8-coverage-counter-semantics.ts scripts/lib/v8-coverage-counter-semantics.ts tests/fixtures/v8-coverage-counter-semantics tests/unit/risk-coverage.test.ts tests/unit/v8-coverage-counter-semantics.test.ts docs/testing/frontend-platform-testing-strategy.md docs/superpowers/plans/2026-08-02-v8-coverage-counter-contract.md
|
||||
git commit -m "test: lock V8 coverage counter semantics"
|
||||
```
|
||||
|
||||
## Self-review
|
||||
|
||||
- Spec coverage: artifact versioning, actual producer rows, discovery isolation, every fail-closed path, bounded diagnostics, cleanup, coverage/CI linkage, sample-removal preservation, and documentation counts are assigned.
|
||||
- Placeholder scan: no deferred implementation remains.
|
||||
- Type consistency: parser and runner names match in tests, script, and plan.
|
||||
@@ -129,7 +129,7 @@ Chromium, Firefox, WebKit과 compact project로 실행한다. 빠른 Vite 개발
|
||||
|
||||
V8 text/JSON/LCOV를 생성하고 전체 기준과 retry/storage/telemetry/application
|
||||
composition/compatibility/performance/promotion/chunk/diagnostics/reference HTTP
|
||||
operation/query-mutation/registry compatibility 12개 high-risk module에 52개
|
||||
operation/query-mutation/registry compatibility 19개 high-risk module에 80개
|
||||
scoped threshold를 적용한다. critical module 누락 또는 threshold 미달 fixture는
|
||||
merge gate를 실패시킨다.
|
||||
|
||||
@@ -144,6 +144,11 @@ re-export만 있는 모듈은 모두 exact all-zero row였고, 선언/초기화
|
||||
static counterless 집합과 exact all-zero row 집합의 일치를 요구하고 critical/high-risk
|
||||
policy module이 counterless이면 실패시킨다.
|
||||
|
||||
Coverage policy 입력은 schema version 2를 유지한다. 반면 위 counter-bearing 필드를
|
||||
직렬화하는 risk-coverage 결과 artifact는 schema version 3이다. 두 version은 서로
|
||||
독립적인 계약이며, 실제 CLI contract test가 artifact의 새 필드와 legacy 필드 부재를
|
||||
검증한다.
|
||||
|
||||
남은 범위는 실제 device/browser farm, cloud visual approval, 외부 인증·telemetry
|
||||
provider와 production field data다. 이 증거가 없을 때 저장소 내부 test를
|
||||
`PRODUCTION_READY`의 대체물로 사용하지 않는다.
|
||||
|
||||
Reference in New Issue
Block a user