Files
clean-architecture-frontend…/tests/integration/http-scenario-catalog.test.ts
T
DongHyeonka ceaf316f65
frontend-quality-gates / merge / static checks (pull_request) Canceled after 0s
frontend-quality-gates / merge / tests (pull_request) Canceled after 0s
frontend-quality-gates / merge / browser tests (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-014 / config-compatibility (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-026 / lab-performance (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-028 / manual-accessibility-review (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-015 / immutable-release-candidate (pull_request) Canceled after 0s
frontend-quality-gates / external-vulnerability-provider (pull_request) Canceled after 0s
frontend-quality-gates / external-provenance-provider (pull_request) Canceled after 0s
frontend-quality-gates / promote-verified-immutable-candidate (pull_request) Canceled after 0s
frontend-quality-gates / deploy-promoted-release (pull_request) Canceled after 0s
frontend-quality-gates / main / self-tests and removal drills (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-019 / hosting-header (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-021 / runbook-boot-config (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-022 / runbook-chunk-mismatch (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-023 / runbook-api-degradation (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-024 / runbook-telemetry (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-025 / runbook-release-rollback (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-018 / field-web-vitals (pull_request) Canceled after 0s
frontend-quality-gates / FE-GATE-017 / diagram-review (pull_request) Canceled after 0s
test(setup): 시험 timeout을 모두 desktop 실측값으로 주고 손 숫자를 거절한다
문제: 자식 프로세스 시험의 timeout은 `childProcessTimeout(<잰 ms>)`로
정한다(docs/work/reference/testing.md 2절). 그런데 기본값이 아니라서
`tests/setup.ts` 검사에 걸리지 않는 손 숫자가 남았다. 리뷰가 센 54곳 중
둘(`browser-soak.contract.test.ts`의 `evaluateSoak` 인자)은 timeout이 아니고,
AST로 다시 세면 80곳이다: 시험·describe·hook에 쓴 숫자, `VERIFY_TIMEOUT`·
`PROCESS_HEAVY_TIMEOUT_MS`(파일 넷에 따로)·`PROVIDER_BASE_FIXTURE_TIMEOUT_MS`
같은 상수, `PROCESS_WATCHDOG_MS * 3` 같은 식. taxonomy는 옛 상수를 예로
들었다. 실측으로 보면 손 숫자 하나가 모자랐다: system 시험의 provider 묶음
30초는 desktop 여섯 번 중 두 번 넘겼다.

고친 것:
- 80곳을 desktop에서 잰 가장 긴 실행으로 `childProcessTimeout(<ms>)`에
  넘긴다. coverage 묶음(maxWorkers=4)·test:unit 각 9회, integration 3회,
  scale(traffic·session·composition·global-local) 3회, 시스템 컨테이너의
  tests/system 6회, JSON reporter로 받았다. hook은 파일 시간에서 시험 합을
  뺀 값이다. 시간 초과로 끝난 실행은 그 시간을 하한으로 넣었다.
- 상수 사본은 지운다. 한 파일 안에서 여럿이 쓰는 값만
  `const X = childProcessTimeout(...)`로 둔다(`VERIFY_TIMEOUT`). guardian
  시험의 lease는 시험 timeout이 아니라 `childProcessTimeout(0)`(helper의 최소
  30초)로 둔다.
- `tests/unit/test-timeout-declarations.test.ts`가 tests/의 Vitest 파일을
  Babel로 읽어 시험·describe·hook의 timeout 인자가 helper 호출(또는 그것을
  담은 파일 상수)이 아니면 파일과 줄로 실패한다. 프로세스 안에서 도는 긴
  시험(scale tier)도 같은 helper를 쓴다.
- taxonomy 예시, testing.md 2절, `.claude/rules/tests.md` 9번, helper 머리
  주석을 같은 방법으로 맞춘다.

버린 대안:
- `tests/setup.ts` 검사를 「helper가 만든 값이 아닌 timeout」까지 넓히기.
  helper 값은 30초 단위 숫자라 손으로 쓴 60_000과 가릴 수 없고, hook과
  프로세스 안 시험은 보지 못한다.
- 프로세스 안 시험은 손 숫자 그대로 두기. 근거 없는 숫자가 남고 검사가
  예외를 가져야 한다.

위험·한계:
- 값이 커진 곳이 있다(15배 규칙): `manual-code-blocks` 300초 → 900초,
  provider 묶음 30초 → 480초 등. hang을 늦게 잡는다. 줄어든 곳도 있다
  (scale traffic 60초 → 30초 등, 잰 값의 15배 안).
- 측정은 desktop 부하 6~35에서 했다. runner가 더 느리면 다시 잰다.
- 검사 시험은 Vitest 파일 전부를 파싱해 coverage 묶음에서 최대 13.5초
  걸렸고, 그 값으로 자기 timeout을 준다. 이름 있는 함수를 콜백으로 넘긴
  시험(`it("x", run, 30_000)`)은 못 찾는다.
- 다운스트림: 손 숫자 timeout이 있는 제품 시험은 test:unit에서 이 검사에
  걸린다. 메시지가 helper를 알린다.

검증(desktop):
- RED: 검사 시험이 고치기 전 80곳을 파일·줄로 내고 실패. GREEN: 검사·
  child-process-timeout 6개 통과. check:types·eslint(바뀐 34파일) rc=0,
  verify:documentation PASS_SCOPED, 노트북 `scripts/verify.sh docs` OK.
- 측정 원자료: /tmp/claude-gates/w2fe2-t08m, -t08m2, -t08m3.
- 게이트 결과는 PR 본문에 적는다.

Refs: 1112/03-08
2026-11-13 06:47:02 +09:00

463 lines
15 KiB
TypeScript

import { mkdir, rm } from "node:fs/promises";
import path from "node:path";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { readBoundedBytes } from "../../src/adapters/http/bounded-body-reader.ts";
import {
createContractHttpExecutor,
type HttpExecutionObservation,
} from "../../src/adapters/http/http-execution-v3.ts";
import {
type InstalledHttpContract,
} from "../../src/contracts/external-contract-runtime.ts";
import type { CacheScopeSnapshot } from "../../src/contracts/server-state-scope.ts";
import {
computeHttpScenarioCatalogDigest,
httpScenarioReceiptSchema,
type AttemptStatus,
type BodyDisposition,
type HttpScenarioAssertionGroups,
type RetryReason,
} from "../../scripts/lib/http-scenario-evidence.ts";
import { writeValidatedJsonArtifact } from "../../scripts/lib/validated-json-artifact.ts";
import { TEST_LIST_HTTP_CONTRACT } from "../helpers/external-contract-fixture.ts";
import { createHttpScenarioHandler } from "../mocks/handlers/http-scenarios.ts";
import { createTestEntityScenarioHandlers } from "../mocks/handlers/test-entities.ts";
import { createStrictMockServer } from "../mocks/server.ts";
import {
assertOperationScenario,
HTTP_SCENARIO_CONTRACTS,
HTTP_SCENARIO_EXECUTION_IDS,
HTTP_SCENARIO_EXPECTATIONS,
HTTP_SCENARIO_RECEIPT_SCHEMA_VERSION,
OPERATION_SCENARIO_CATALOG,
type HttpScenarioExpectation,
} from "../mocks/scenarios/catalog.ts";
import { childProcessTimeout } from "../helpers/child-process-timeout.ts";
/** A low-cardinality route identity, as a feature binding passes it. */
const ROUTE_ID = "TEST_ENTITY_ROUTE";
const RECEIPT_PATH = path.resolve(
"artifacts/tests/http-scenario-executions.json",
);
const mockApi = createStrictMockServer();
beforeAll(mockApi.listen);
afterAll(mockApi.close);
type AttemptTrace = {
status: AttemptStatus;
media: string | null;
pulledBytes: number;
appliedCeiling: number | null;
completed: boolean;
cancelled: boolean;
};
type CatalogOperationId = HttpScenarioExpectation["operationId"];
function operationFor(operationId: CatalogOperationId) {
return HTTP_SCENARIO_CONTRACTS[operationId] as InstalledHttpContract<
unknown,
unknown,
unknown
>;
}
function inputFor(operationId: CatalogOperationId): unknown {
if (operationId === "TEST_LIST_ENTITIES") return { limit: 20 };
if (operationId === "TEST_GET_ENTITY") return { entityId: "entity-1" };
return { name: "Created" };
}
function normalizeMedia(response: Response): string | null {
const value = response.headers.get("content-type");
return value?.split(";", 1)[0]?.trim().toLowerCase() || null;
}
function tracingFetcher(attempts: AttemptTrace[]): typeof fetch {
return async (input, init) => {
const trace: AttemptTrace = {
status: "PENDING_ABORT",
media: null,
pulledBytes: 0,
appliedCeiling: null,
completed: false,
cancelled: false,
};
attempts.push(trace);
let response: Response;
try {
response = await fetch(input, init);
} catch (error) {
trace.status = init?.signal?.aborted
? "PENDING_ABORT"
: "NETWORK_REJECTION";
throw error;
}
trace.status = response.status;
trace.media = normalizeMedia(response);
if (!response.body) return response;
const reader = response.body.getReader();
const proxy = new ReadableStream<Uint8Array>(
{
async pull(controller) {
try {
const chunk = await reader.read();
if (chunk.done) {
trace.completed = true;
controller.close();
return;
}
trace.pulledBytes += chunk.value.byteLength;
controller.enqueue(chunk.value);
} catch (error) {
controller.error(error);
}
},
cancel(reason) {
trace.cancelled = true;
// MSW-transferred oversized bodies leave the original cancel promise
// permanently pending. Record and request cancellation without making
// the proxy response less responsive than the real executor contract.
void reader.cancel(reason).catch(() => {});
},
},
{ highWaterMark: 0 },
);
return new Response(proxy, {
status: response.status,
statusText: response.statusText,
headers: response.headers,
});
};
}
function outcomeDetail(outcome: Readonly<Record<string, unknown>>): string | null {
if (outcome.kind === "CONTRACT_VIOLATION") {
return String((outcome.violation as Readonly<{ kind: string }>).kind);
}
if (outcome.kind === "TRANSPORT_FAILURE") {
return String((outcome.failure as Readonly<{ kind: string }>).kind);
}
if (outcome.kind === "PROBLEM") {
return String((outcome.metadata as Readonly<{ status: number }>).status);
}
return null;
}
function retryReason(status: AttemptStatus): RetryReason {
if (status === "NETWORK_REJECTION") return "NETWORK_FAILURE";
if (status === 429) return "HTTP_429";
if (status === 503) return "HTTP_503";
throw new Error(`Sleep followed a non-retryable attempt: ${String(status)}`);
}
function bodyDisposition(
trace: AttemptTrace,
outcome: Readonly<Record<string, unknown>>,
): BodyDisposition {
if (trace.status === "NETWORK_REJECTION" || trace.status === "PENDING_ABORT") {
return "NO_RESPONSE";
}
if (
outcome.kind === "CONTRACT_VIOLATION" &&
outcomeDetail(outcome) === "RESPONSE_TOO_LARGE" &&
trace.cancelled
) {
return "REJECTED_LIMIT";
}
if (trace.completed) return "FULLY_READ_WITHIN_BOUND";
if (trace.cancelled) return "CANCELLED_WITHOUT_READ";
throw new Error(`Response body was neither consumed nor cancelled: ${trace.status}`);
}
async function waitForHandler(
ready: Promise<void>,
executionId: string,
): Promise<void> {
let watchdog: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
ready,
new Promise<never>((_resolve, reject) => {
watchdog = setTimeout(
() => reject(new Error(`Handler readiness timed out: ${executionId}`)),
2_000,
);
}),
]);
} finally {
if (watchdog !== undefined) clearTimeout(watchdog);
}
}
async function executeScenario(
entry: HttpScenarioExpectation,
): Promise<HttpScenarioAssertionGroups> {
const physicalAttempts: AttemptTrace[] = [];
const sleeps: RetryReason[] = [];
const observations: HttpExecutionObservation[] = [];
const caller = new AbortController();
const scopeLifetime = new AbortController();
let scopeCurrent = true;
let handlerReady!: () => void;
const ready = new Promise<void>((resolve) => {
handlerReady = resolve;
});
const scope: CacheScopeSnapshot = Object.freeze({
generation: 1,
fingerprint: "scenario-scope-1",
identities: Object.freeze({}) as CacheScopeSnapshot["identities"],
signal: scopeLifetime.signal,
isCurrent: () => scopeCurrent,
});
const baseOperation = operationFor(entry.operationId);
const operation =
entry.testDeadlineOverrideMs === null
? baseOperation
: Object.freeze({
...baseOperation,
frontend: Object.freeze({
...baseOperation.frontend,
totalDeadlineMs: entry.testDeadlineOverrideMs,
}),
});
const fetcher = tracingFetcher(physicalAttempts);
const executor = createContractHttpExecutor({
baseUrl: "https://api.test",
maxRetryAttempts: 2,
attachCredentials: () => ({ kind: "READY", headers: {} }),
fetcher,
random: () => 0,
readBoundedResponseBytes: async (response, maximumBytes) => {
const trace = physicalAttempts.at(-1);
if (!trace) throw new Error("Body reader ran before a physical attempt");
if (trace.appliedCeiling !== null) {
throw new Error("Body reader ran more than once for one attempt");
}
trace.appliedCeiling = maximumBytes;
return readBoundedBytes(response, maximumBytes);
},
sleep: async () => {
const prior = physicalAttempts.at(-1);
if (!prior) throw new Error("Retry sleep occurred before an attempt");
sleeps.push(retryReason(prior.status));
},
observe: (observation) => observations.push(observation),
});
mockApi.server.use(
...createTestEntityScenarioHandlers({
scenarios: { [entry.operationId]: entry.scenarioId },
onAttempt(attempt) {
if (
attempt.operationId === entry.operationId &&
attempt.scenarioId === entry.scenarioId
) {
handlerReady();
}
},
}),
);
try {
const execution = executor.execute(operation, inputFor(entry.operationId), {
routeId: ROUTE_ID,
scope,
signal: caller.signal,
...(entry.operationId === "TEST_CREATE_ENTITY"
? {
intent: Object.freeze({
intentId: `intent-${entry.scenarioId}`,
operationId: entry.operationId,
canonicalInputIdentity: "scenario-input",
idempotencyKey: `scenario-${entry.scenarioId}`,
createdAtMonotonicMs: 1,
}),
}
: {}),
});
if (entry.scenarioId === "timeout" || entry.scenarioId === "aborted") {
await waitForHandler(ready, entry.executionId);
if (entry.scenarioId === "aborted") {
if (entry.operationId === "TEST_LIST_ENTITIES") {
caller.abort();
} else {
scopeCurrent = false;
scopeLifetime.abort();
}
}
}
const outcome = (await execution) as unknown as Readonly<Record<string, unknown>>;
expect(observations, `${entry.executionId} observer count`).toHaveLength(1);
const observation = observations[0]!;
const observedSignal = scopeLifetime.signal.aborted ? "ABORTED" : "ACTIVE";
const cancellationOwner =
observation.terminalReason === "TIMEOUT"
? "DEADLINE"
: caller.signal.aborted
? "CALLER"
: scopeLifetime.signal.aborted
? "SCOPE_FENCE"
: "NONE";
const observed: HttpScenarioAssertionGroups = Object.freeze({
status: Object.freeze({
attempts: Object.freeze(physicalAttempts.map((attempt) => attempt.status)),
final: physicalAttempts.at(-1)!.status,
}),
outcome: Object.freeze({
kind: String(outcome.kind),
detail: outcomeDetail(outcome),
}),
effect: Object.freeze({
outcome: String(outcome.effect),
observer: observation.terminalReason,
}),
retry: Object.freeze({ count: sleeps.length, reasons: Object.freeze(sleeps) }),
fetch: Object.freeze({
count: physicalAttempts.length,
observerAttempts: observation.attemptCount,
agrees: physicalAttempts.length === observation.attemptCount,
}),
media: Object.freeze({
attempts: Object.freeze(physicalAttempts.map((attempt) => attempt.media)),
final: physicalAttempts.at(-1)?.media ?? null,
}),
body: Object.freeze({
attempts: Object.freeze(
physicalAttempts.map((attempt) =>
Object.freeze({
disposition: bodyDisposition(attempt, outcome),
pulledBytes: attempt.pulledBytes,
ceiling: attempt.appliedCeiling ?? 0,
}),
),
),
}),
scope: Object.freeze({
start: "CURRENT",
end: scopeCurrent ? "CURRENT" : "STALE",
signal: observedSignal,
cancellationOwner,
}),
});
return observed;
} finally {
caller.abort();
scopeLifetime.abort();
mockApi.reset();
}
}
describe("HTTP scenario catalog execution evidence", () => {
it("declares exactly the 57 behaviorally distinct operation scenarios", () => {
const executionIds = Object.entries(OPERATION_SCENARIO_CATALOG).flatMap(
([operationId, scenarioIds]) =>
scenarioIds.map((scenarioId) => `${operationId}::${scenarioId}`),
);
expect(OPERATION_SCENARIO_CATALOG.TEST_LIST_ENTITIES).toHaveLength(20);
expect(OPERATION_SCENARIO_CATALOG.TEST_GET_ENTITY).toHaveLength(19);
expect(OPERATION_SCENARIO_CATALOG.TEST_CREATE_ENTITY).toHaveLength(18);
expect(executionIds).toHaveLength(57);
expect(new Set(executionIds)).toHaveLength(57);
expect(executionIds).toEqual(HTTP_SCENARIO_EXECUTION_IDS);
});
it("executes every declared scenario and publishes one complete receipt", async () => {
await rm(RECEIPT_PATH, { force: true });
const rows: Array<Readonly<{
executionId: string;
expected: HttpScenarioAssertionGroups;
observed: HttpScenarioAssertionGroups;
testDeadlineOverrideMs: number | null;
}>> = [];
for (const entry of HTTP_SCENARIO_EXPECTATIONS) {
const observed = await executeScenario(entry);
expect(observed, entry.executionId).toEqual(entry.expected);
rows.push(
Object.freeze({
executionId: entry.executionId,
expected: entry.expected,
observed,
testDeadlineOverrideMs: entry.testDeadlineOverrideMs,
}),
);
}
const sortedRows = [...rows].sort((left, right) =>
left.executionId.localeCompare(right.executionId),
);
await mkdir(path.dirname(RECEIPT_PATH), { recursive: true });
await writeValidatedJsonArtifact({
path: RECEIPT_PATH,
schema: httpScenarioReceiptSchema,
value: {
schemaVersion: HTTP_SCENARIO_RECEIPT_SCHEMA_VERSION,
catalogDigest: computeHttpScenarioCatalogDigest(
HTTP_SCENARIO_RECEIPT_SCHEMA_VERSION,
HTTP_SCENARIO_EXPECTATIONS,
),
catalogTotal: 57,
executedIds: sortedRows.map((row) => row.executionId),
rows: sortedRows,
},
});
}, childProcessTimeout(2_334));
});
describe("HTTP scenario handler for an operation outside the catalog", () => {
// 16-02. A feature test passes its own installed contract; the catalog has
// no entry for it. The ceiling is larger than the catalog's, so the
// oversized body shows it was read from this descriptor.
const featureList = Object.freeze({
...TEST_LIST_HTTP_CONTRACT,
contract: Object.freeze({
...TEST_LIST_HTTP_CONTRACT.contract,
operationId: "FEATURE_LIST_ITEMS",
pathTemplate: "/api/feature-groups/{groupId}/items",
}),
frontend: Object.freeze({
...TEST_LIST_HTTP_CONTRACT.frontend,
responseByteLimit: 65_536,
}),
});
it("answers on the descriptor's route with the descriptor's ceiling", async () => {
const attempts: unknown[] = [];
mockApi.server.use(
createHttpScenarioHandler(featureList, {
scenario: "response-too-large",
payload: [],
onAttempt: (attempt) => attempts.push(attempt),
}),
);
try {
const response = await fetch("https://api.test/api/feature-groups/g-1/items");
expect(response.status).toBe(200);
expect((await response.arrayBuffer()).byteLength).toBe(65_537);
expect(attempts).toEqual([
{
operationId: "FEATURE_LIST_ITEMS",
scenarioId: "response-too-large",
attempt: 1,
},
]);
} finally {
mockApi.reset();
}
});
it("keeps the catalog's evidence to its own operations", () => {
expect(() =>
assertOperationScenario("FEATURE_LIST_ITEMS", "success"),
).toThrow("FEATURE_LIST_ITEMS is not a catalog operation");
});
});