90 lines
3.8 KiB
Markdown
90 lines
3.8 KiB
Markdown
# Test and evidence taxonomy
|
|
|
|
Each gate is blocking in its declared scope. Failures are not downgraded with
|
|
`continue-on-error` or warning-only scripts.
|
|
|
|
## Executable levels
|
|
|
|
| Level | Command | Ownership / prerequisite | Evidence |
|
|
| --- | --- | --- | --- |
|
|
| runtime schema | `pnpm test:runtime-schema` | pure runtime schema contracts | `artifacts/tests/runtime-schema.xml` |
|
|
| unit | `pnpm test:unit` | Node-only domain/application/pure policy/runtime units; no systemd/bwrap/cgroup prerequisite | `artifacts/tests/unit.xml` |
|
|
| capability contract | `pnpm test:contract` | reusable capability consumer contracts | `artifacts/tests/contract.xml` |
|
|
| component | `pnpm test:component` | React/hook/UI behavior | `artifacts/tests/component.xml` |
|
|
| integration | `pnpm test:integration` | HTTP/MSW, IndexedDB and composed browser-runtime boundaries | `artifacts/tests/integration.xml` |
|
|
| system / CI runner | `pnpm test:system` | compatible Linux host with systemd, bubblewrap, cgroup v2 and CI-provider process controls | `artifacts/tests/system.xml` |
|
|
| end-to-end | `pnpm test:e2e` | pinned browser engines | `artifacts/tests/e2e/` |
|
|
| accessibility | `pnpm test:a11y` | pinned browser engines | `artifacts/tests/a11y.json` |
|
|
|
|
`test:all` is the normal product-development loop. It intentionally includes
|
|
runtime-schema, unit, capability-contract, component, integration, reference
|
|
feature and recipe suites, but does not include `test:system`. CI-runner and
|
|
supply-chain assurance has different host prerequisites and is invoked
|
|
explicitly in the assurance path.
|
|
|
|
## Deterministic test process
|
|
|
|
All Vitest package scripts launch through `scripts/run-vitest.ts`.
|
|
|
|
That runner:
|
|
|
|
1. rejects Node versions outside the repository-supported
|
|
`>=24.11.0 <25.0.0` range before the suite starts,
|
|
2. owns `NODE_ENV=test` rather than trusting the parent shell,
|
|
3. removes host-specific `npm_config_userconfig`, `npm_config_prefix` and
|
|
`npm_config_globalconfig` values before Vitest starts.
|
|
|
|
`vitest.config.ts` also fixes `NODE_ENV=test` so a direct Vitest invocation
|
|
cannot accidentally select React's production behavior.
|
|
|
|
The system suite additionally runs
|
|
`scripts/check-system-test-prerequisites.ts` and fails with one prerequisite
|
|
report when the CI-runner host does not provide its required Linux facilities.
|
|
|
|
## Development paths
|
|
|
|
Product feature:
|
|
|
|
- focused feature/unit/component test
|
|
- capability contract test when a reusable boundary changes
|
|
- type/lint/architecture
|
|
- `test:all`
|
|
|
|
Reusable capability:
|
|
|
|
- focused unit tests
|
|
- capability contract tests
|
|
- integration tests
|
|
- type/lint/architecture
|
|
|
|
CI / release assurance:
|
|
|
|
- `test:system`
|
|
- supply-chain / promotion / release gates
|
|
|
|
A host-level process/sandbox test must not be placed in `tests/unit` merely
|
|
because it uses Vitest. The classification follows the system boundary and
|
|
prerequisites, not the test framework.
|
|
|
|
End-to-end and automated accessibility scenarios run on the pinned Chromium,
|
|
Firefox, and WebKit engines. The responsive contract explicitly exercises
|
|
320px reflow, a mobile navigation drawer, and a wide two-column gallery.
|
|
Theme scenarios verify persistence, operating-system changes, and dark-surface
|
|
axe results. Native modal focus and validation association are exercised in
|
|
both component and browser tests.
|
|
|
|
A control is verified only when a positive fixture passes and its deliberately
|
|
failing negative fixture is rejected. Generated evidence is retained by CI;
|
|
the repository tracks only the evidence directory structure.
|
|
|
|
Promotion is an AND graph:
|
|
|
|
1. merge gates
|
|
2. merge gates plus release gates
|
|
3. release gates plus rollback/runbook drills
|
|
4. production promotion plus eligible field Web Vitals evidence
|
|
|
|
The [frontend platform testing strategy](./frontend-platform-testing-strategy.md)
|
|
contains the broader testing design. This file is the executable taxonomy for
|
|
where a test belongs and which environment is allowed to run it.
|