1.7 KiB
Performance Evidence Contract
A performance refactor is not complete because code looks faster. It needs a comparable before/after experiment.
Lifecycle
A performance item uses READY → BASELINING → IN_PROGRESS → VERIFYING → WAITING_APPROVAL. BASELINING is measurement-only. Source changes are forbidden until the baseline at analysisRevision has been retained and the item moves to IN_PROGRESS.
Before code changes
Freeze the measurement contract in work-item.json:
- exact command or reproducible procedure;
- cwd;
- environment record;
- dataset/fixture/load profile;
- warmup and iteration policy when relevant;
- metrics and units;
- acceptance criteria.
Capture baseline raw output under evidence/baseline/raw/ and fill evidence/baseline/metadata.json with source revision, command, cwd, exit code, dataset, metrics, and raw file references.
After code changes
Use the same measurement contract. Capture raw output under evidence/after/raw/ and the matching metadata file.
Comparison
evidence/comparison.md must state whether conditions are materially equivalent and list baseline, after, delta, acceptance result, functional regression checks, conclusion, and limitations.
Allowed conclusions: IMPROVED, NEUTRAL, REGRESSED, INCOMPARABLE.
If command/procedure, metric definition, dataset/load profile, or material environment differs enough to invalidate comparison, use INCOMPARABLE. Do not convert incomparable measurements into an improvement claim.
Evidence quality
Retain raw output. A hand-written summary alone is insufficient. Do not fabricate missing runs. Avoid secrets at command construction time rather than relying on later redaction.