docs(testing): match determinism, scenario and taxonomy text to the code (10-07)
The strategy doc promised clock/random/scheduler helpers that no test used (the helper's clearTimeout did nothing), listed three scenario IDs the catalog does not have, and the taxonomy described an older test:all and a child-process rule tests/unit does not follow. Removes the unused helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
d68e25a2ad
commit
84f2ab929f
@@ -142,8 +142,12 @@ rasterization과 실제 device farm은 프로젝트가 요구할 때 연결한
|
||||
|
||||
operation별 success/empty/slow/network/timeout/content/envelope/schema/auth/
|
||||
403/404/409/422/429/retry/terminal을 포함한 19개 scenario catalog와 strict
|
||||
unhandled-request server를 공유한다. clock/random/scheduler/storage helper와
|
||||
browser console/page/network failure 정책으로 비결정적 우회를 차단한다.
|
||||
unhandled-request server(`tests/mocks/server.ts`의
|
||||
`onUnhandledRequest: "error"`)를 공유한다. browser test는 공용 fixture
|
||||
(`tests/support/browser/strict-browser-test.ts`)가 console error/warning,
|
||||
page error, 실패한 request를 test 실패로 만든다. 시간은 각 test가
|
||||
`vi.useFakeTimers()`로 통제하고, 전역 `afterEach`가 실제 timer로 되돌린다
|
||||
(`tests/setup.ts`). 공용 clock/random/scheduler helper는 두지 않는다.
|
||||
|
||||
#### Built-dist와 compact E2E
|
||||
|
||||
@@ -738,15 +742,19 @@ tests/factories/
|
||||
| `malformed-json` | 파싱 불가능한 JSON |
|
||||
| `envelope-mismatch` | top-level contract 위반 |
|
||||
| `schema-mismatch` | data payload contract 위반 |
|
||||
| `auth-recover-once` | 첫 401 후 복구 성공 |
|
||||
| `auth-persistent-401` | 복구 뒤에도 401 |
|
||||
| `response-too-large` | 응답 본문이 크기 상한을 넘어 읽기를 멈춤 |
|
||||
| `unauthenticated-401` | 401. 명령은 적용되지 않음(`NOT_APPLIED`) |
|
||||
| `forbidden-403` | 권한 없음 |
|
||||
| `not-found-404` | 리소스 없음 |
|
||||
| `conflict-409` | mutation conflict |
|
||||
| `validation-422` | 승인된 field/global error |
|
||||
| `rate-limited-429` | Retry-After 포함 |
|
||||
| `server-retry-success` | bounded retry 뒤 성공 |
|
||||
| `server-terminal-500` | retry 소진 |
|
||||
| `server-terminal-503` | 503. 조회는 retry 2회 뒤 소진, 명령은 retry 없이 결과 불명(`MAYBE_APPLIED`) |
|
||||
|
||||
이 표는 `tests/mocks/scenarios/catalog.ts`의 `HTTP_SCENARIO_IDS`와 같은 순서다.
|
||||
401 뒤 인증 복구 흐름은 catalog가 아니라 `tests/integration/auth-recovery.test.ts`가
|
||||
다룬다.
|
||||
|
||||
### 11.3 Catalog 규칙
|
||||
|
||||
|
||||
@@ -16,9 +16,11 @@ Each gate is blocking in its declared scope. Failures are not downgraded with
|
||||
| end-to-end | `pnpm test:e2e` | pinned browser engines | `artifacts/tests/e2e/` |
|
||||
| accessibility | `pnpm test:a11y` | pinned browser engines | `artifacts/tests/a11y.json` |
|
||||
|
||||
`test:all` is the normal product-development loop. It intentionally includes
|
||||
runtime-schema, unit, capability-contract, component, integration, reference
|
||||
feature and recipe suites, but does not include `test:system`. CI-runner and
|
||||
`test:all` is the normal product-development loop. It runs, in order,
|
||||
`test:runtime-schema`, `test:unit`, `test:contract`, `test:component`,
|
||||
`test:integration`, `test:reference-feature`, `test:platform` (`tests/platform`
|
||||
and `tests/architecture`), `check:release-coherence` and `test:recipes`
|
||||
(`package.json` `test:all`). It does not include `test:system`. CI-runner and
|
||||
supply-chain assurance has different host prerequisites and is invoked
|
||||
explicitly in the assurance path.
|
||||
|
||||
@@ -82,11 +84,19 @@ CI / release assurance:
|
||||
|
||||
A host-level process/sandbox test must not be placed in `tests/unit` merely
|
||||
because it uses Vitest. The classification follows the system boundary and
|
||||
prerequisites, not the test framework. Process-heavy tests that spawn child
|
||||
processes but do not require privileged host facilities belong in
|
||||
`tests/integration`; for example the provider guardian transaction protocol
|
||||
lives at `tests/integration/provider-guardian-transaction.test.ts`. Its
|
||||
handshake timeout is a test watchdog, not a production protocol deadline.
|
||||
prerequisites, not the test framework:
|
||||
|
||||
- Tests that need systemd, bubblewrap, cgroup v2 or CI-provider process
|
||||
controls belong in `tests/system`.
|
||||
- Tests of a long-lived child-process protocol belong in `tests/integration`;
|
||||
for example the provider guardian transaction protocol lives at
|
||||
`tests/integration/provider-guardian-transaction.test.ts`. Its handshake
|
||||
timeout is a test watchdog, not a production protocol deadline.
|
||||
- A script's own tests may start that script, Node or Vitest as a short-lived
|
||||
child process to check its command-line contract, and stay in `tests/unit`
|
||||
next to the script's other tests. Such a suite declares its own longer
|
||||
timeout instead of raising the global one; for example
|
||||
`tests/unit/risk-coverage.test.ts` uses `PROCESS_HEAVY_TIMEOUT_MS`.
|
||||
|
||||
End-to-end and automated accessibility scenarios run on the pinned Chromium,
|
||||
Firefox, and WebKit engines. The responsive contract explicitly exercises
|
||||
|
||||
@@ -1,69 +0,0 @@
|
||||
import { vi } from "vitest";
|
||||
|
||||
export function createDeterministicClock(start = 0) {
|
||||
let current = start;
|
||||
return Object.freeze({
|
||||
now: () => current,
|
||||
sleep: async (milliseconds: number, signal?: AbortSignal) => {
|
||||
if (signal?.aborted) throw new DOMException("aborted", "AbortError");
|
||||
current += Math.max(0, milliseconds);
|
||||
},
|
||||
advance(milliseconds: number) {
|
||||
current += Math.max(0, milliseconds);
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function createSeededRandom(seed = 1) {
|
||||
let state = seed >>> 0;
|
||||
return () => {
|
||||
state = (state * 1_664_525 + 1_013_904_223) >>> 0;
|
||||
return state / 0x1_0000_0000;
|
||||
};
|
||||
}
|
||||
|
||||
export function createControlledScheduler() {
|
||||
const callbacks: Array<() => void> = [];
|
||||
return Object.freeze({
|
||||
callbacks,
|
||||
setTimeout: vi.fn((callback: () => void) => {
|
||||
callbacks.push(callback);
|
||||
return callbacks.length - 1;
|
||||
}),
|
||||
clearTimeout: vi.fn(),
|
||||
runNext() {
|
||||
callbacks.shift()?.();
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function createRecordingStorage() {
|
||||
const values = new Map<string, string>();
|
||||
const operations: Array<
|
||||
Readonly<{ operation: "get" | "set" | "remove"; key: string }>
|
||||
> = [];
|
||||
const storage: Storage = {
|
||||
getItem(key) {
|
||||
operations.push({ operation: "get", key });
|
||||
return values.get(key) ?? null;
|
||||
},
|
||||
setItem(key, value) {
|
||||
operations.push({ operation: "set", key });
|
||||
values.set(key, value);
|
||||
},
|
||||
removeItem(key) {
|
||||
operations.push({ operation: "remove", key });
|
||||
values.delete(key);
|
||||
},
|
||||
clear() {
|
||||
values.clear();
|
||||
},
|
||||
key(index) {
|
||||
return [...values.keys()][index] ?? null;
|
||||
},
|
||||
get length() {
|
||||
return values.size;
|
||||
},
|
||||
};
|
||||
return Object.freeze({ storage, operations, values });
|
||||
}
|
||||
Reference in New Issue
Block a user