Merge lane/J: test system (10-02, 10-04~10-07, 11-04~11-08, F-2, F-4, F-12, 00-03)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
donghyeon-ka
2026-09-27 13:28:55 +09:00
co-authored by Claude Opus 5.5
43 changed files with 1908 additions and 172 deletions
+22 -4
View File
@@ -41,10 +41,28 @@ Performance evidence is deliberately split by measurement context:
예산을 올릴 때는 이 표의 기준선과 새 측정값, 이유를 같은 변경에 적는다.
lab 측정(`scripts/test-performance.ts`, FE-GATE-026)은 local 프로필 build를
`vite preview`로 띄워 한 번만 잰다. named interaction 시간은 Node의
`performance.now()`로 Playwright 클릭부터 heading 대기까지 재므로 Playwright
왕복 시간이 들어간다. 한 번 측정한 값은 흔들린다. 경계 근처 값으로 판단할
때는 여러 번 돌려 중앙값을 본다.
`vite preview --strictPort`로 띄워 잰다(리뷰 11-07).
- 포트는 `LAB_PERFORMANCE_PORT`(기본 4173)다. 서버를 띄우기 전에 그 포트가
이미 응답하면 실패하고, 응답하는 `/release-manifest.json`의 build·release가
로컬 `dist/release-manifest.json`과 다르면 실패한다
(`scripts/lib/lab-performance.ts`, `tests/unit/lab-performance.test.ts`).
- 새 브라우저 context에서 `--runs=N`(기본 5)번 재고, 예산은 중앙값으로
판정한다. 모든 측정값과 최소·최대는 `lab.json`의 `context.sampling`에 남는다.
- named interaction은 페이지 안의 시계로 잰다. 클릭 이벤트의 `timeStamp`부터
대상 heading이 DOM에 들어온 시각까지다. Playwright 왕복 시간은 들어가지 않고,
lazy chunk 다운로드(200 KB/s 제한)는 들어간다. 클릭의 Event Timing entry
길이(INP의 입력값)는 `context.sampling.clickEventDurationMs`에 따로 남는다.
- CPU 모델, 논리 CPU 수, 측정 전후 loadavg를 `context.runner`에 남긴다.
- preview 서버는 자기 프로세스 그룹으로 띄우고 끝날 때 그룹 전체를 멈춘다.
`corepack`에만 신호를 보내던 때는 vite가 포트를 계속 잡고 있었다.
2026-09-27 한 빌드를 10회 잰 결과(i5-10400F 12 CPU, loadavg 11~17): 중앙값
LCP 2,386ms, interaction 184.5ms로 통과했다. 한 번씩 판정했다면 interaction
10회 중 4회(최대 274ms), LCP 10회 중 2회(최대 2,644ms)가 실패였다. 같은
빌드를 loadavg 7~9에서 다시 10회 재면 interaction 120~148ms, LCP
1,968~2,072ms로 모두 통과했다. 한 번 측정한 판정은 호스트 부하에 따라
뒤집히고, 중앙값 판정은 두 번 모두 통과였다.
The field minimum eligible-sample threshold is intentionally unresolved until
a privacy-approved telemetry baseline exists. Therefore the field command
@@ -8,7 +8,7 @@ Plan bundle: `docs/reviews/2026-09-23-extreme-scale-frontend-plan-bundle/`. One
| 2 | Composition does not contain known quadratic accumulation patterns. | 완료 | `src/platform/cardinality/compose-unique-record.ts`; `tests/platform/cardinality/compose-unique-record.test.ts`; `tests/platform/cardinality/extreme-scale-registry.test.ts`; growth gate `scripts/scale/run-composition-scale.ts` + `src/platform/cardinality/growth-regression.ts` (`tests/scale/composition-scale.test.ts`, `tests/scale/growth-regression.test.ts`, artifact `artifacts/scale/main/composition-s0-s2.json`); stored-baseline gate `pnpm check:scale:baseline` (`config/scale/benchmark-baseline.json`) |
| 3 | Membership checks on scale-sensitive paths use appropriate indexed structures. | 완료 | `src/contracts/product-features.ts` (`Set` membership); `tests/unit/product-features.test.ts`; `tests/features/product-features-scale.test.ts`; 100k selection/resolution in `tests/platform/cardinality/extreme-scale-registry.test.ts`; workloads `product-feature-selection`/`product-feature-resolution` in `scripts/scale/composition-workloads.ts` |
| 4 | Concrete imports across unrelated features fail architecture checks. | 완료 | `.dependency-cruiser.json` rules `features-do-not-import-other-features`, `contexts-do-not-import-other-contexts`; rejected fixtures `tests/fixtures/architecture/dependency-graph/cross-boundary/`; assertions in `scripts/check-architecture.ts`; `pnpm check:architecture` |
| 5 | Context/app cardinality ceilings are explicit and testable. | 완료 | `src/platform/cardinality/cardinality-budget.ts`; `src/contexts/bounded-context-module.ts`; `src/apps/deployable-app-module.ts`; `tests/architecture/topology-cardinality.test.ts`; `tests/architecture/module-hierarchy.test.ts`; `tests/platform/cardinality/cardinality-budget.test.ts` |
| 5 | Context/app cardinality ceilings are explicit and testable. | 완료 | `src/platform/cardinality/cardinality-budget.ts`; `src/contexts/bounded-context-module.ts`; `src/apps/deployable-app-module.ts`; `tests/architecture/topology-cardinality.test.ts`; `tests/architecture/module-hierarchy.test.ts`; `tests/platform/cardinality/cardinality-budget.test.ts`. The ceilings are declared limits, not a measured capacity: see "Growth inside one app" below (11-04) |
| 6 | An unchanged local app does not grow materially when global logical topology grows toward S3/S4 fixtures. | 완료 | (1) Ownership source: `src/apps/template-app/app-module.ts`, `src/contexts/reference-context/context-module.ts`, `FEATURE_CONTEXT_ID`. (2) Production projection: `scripts/generate-product-feature-index.ts` selects one app (`--app=`/`PRODUCT_APP_ID`) → `src/.generated/{product-feature-index,product-context-index,product-app}.ts` → `src/apps/template-app.ts`. (3) S0→S4 invariant: `tests/scale/global-local-independence.test.ts` — S0/S1/S2 real on-disk repos through the generator command (identical generated files, sha256 and reachable source graph); S0→S4 through the same `selectAppIndexes` (identical feature/context ids, route roots, generated files, hashes and compile-input paths; only App A's declarations read at 1M), pinned to the filesystem source by an equivalence test. (4) Build-input gate: `scripts/check-app-build-input.ts` (`pnpm check:app-build-input`, and `check:app-build-input:bundle` on the fresh production bundle), negative fixtures in `tests/architecture/app-build-input.test.ts` |
| 7 | Initial routing is hierarchical/lazy rather than a product-wide flat feature table. | 완료 | `src/platform/routing/context-route-module.ts`; `src/presentation/routes/app-router.tsx` (`ContextRouteGateway`); `tests/platform/routing/context-route-module.test.ts`; `tests/architecture/context-lazy-loading.test.ts`; `tests/features/reference-feature/installed-context-routing.test.ts` |
| 8 | User interactions carry bounded request, retry, concurrency, cancellation, and deadline semantics. | 완료 | `src/platform/execution/execution-context.ts`; `src/platform/traffic/{request-budget,concurrency-budget,traffic-budget}.ts`; `src/platform/resilience/{retry-budget,deadline}.ts`; enforcement in `src/adapters/http/http-execution-v3.ts`; operation policy `HttpExecutionPolicy.traffic` (`src/contracts/external-contract-runtime.ts`); `tests/platform/traffic/http-budget-integration.test.ts`; `tests/platform/traffic/traffic-amplification.test.ts`; `pnpm check:traffic` |
@@ -20,10 +20,34 @@ Plan bundle: `docs/reviews/2026-09-23-extreme-scale-frontend-plan-bundle/`. One
| 14 | Telemetry failure does not fail business behavior. | 완료 | `tests/platform/telemetry/telemetry-pipeline.test.ts` (transport throw/reject); `tests/platform/telemetry/telemetry-port.test.ts` (`failOpenTelemetryPort`); `tests/features/reference-feature/reference-telemetry.test.ts`; collector outage in `tests/scale/telemetry/telemetry-storm.test.ts` |
| 15 | Failure boundaries prevent one feature/context from unnecessarily collapsing unrelated UI. | 완료 | `src/platform/errors/{scoped-error-boundary,feature-error-boundary,context-error-boundary}.tsx`; wiring in `src/presentation/routes/app-router.tsx`; `tests/platform/errors/failure-boundary.test.tsx` |
| 16 | Spike and sustained-load tests detect request, retry, telemetry, and memory amplification. | 완료 | `tests/scale/traffic/*`, `tests/scale/telemetry/telemetry-storm.test.ts`, `tests/scale/session/*`; real Chromium soak `scripts/scale/browser-soak.ts` against the fresh build with telemetry on and a 503 API: 2,012 cycles, heap/listeners/timers/storage stable, 0 in-flight at end, 1.43 API requests per reference visit (≤ 3), telemetry ≤ 14 posts/min (≤ 40), checks in `artifacts/scale/nightly/browser-soak.json`; evaluator tests `tests/scale/browser-soak.contract.test.ts`; skipped ⇒ tier `incomplete` (non-zero). Platform telemetry adapter now send-rate limited (`src/adapters/telemetry/best-effort-telemetry.ts`, tests in `tests/unit/telemetry.test.ts`) |
| 17 | Multi-replica release tests verify fail-closed coherence and old-asset reachability. | 완료 | `src/platform/release/release-coherence.ts` (delegated to by `src/bootstrap/load-runtime-config.ts`, `src/bootstrap/load-release-manifest.ts`); `src/platform/release/release-mismatch-recovery.ts` + `src/bootstrap/boot-release-recovery.ts`; `tests/platform/release/*.test.ts`; `tests/release/{old-asset-reachability,mixed-replica,release-matrix}.test.ts`; `pnpm check:release-coherence` |
| 17 | Multi-replica release tests verify fail-closed coherence and old-asset reachability. | 완료 | `src/platform/release/release-coherence.ts` (delegated to by `src/bootstrap/load-runtime-config.ts`, `src/bootstrap/load-release-manifest.ts`); `src/platform/release/release-mismatch-recovery.ts` + `src/bootstrap/boot-release-recovery.ts`; `tests/platform/release/*.test.ts`; `tests/release/{mixed-replica,release-matrix}.test.ts`; `pnpm check:release-coherence`. Old-asset reachability is a hosting provider duty (`docs/operations/release-cache-rollback.md`); `tests/release/old-asset-reachability.test.ts` checks only the in-memory hosting fixture against `config/hosting/cache-policy.json`. An old tab of the default build (no Service Worker) whose chunk is gone after a deploy is checked in real browsers by `tests/e2e/old-tab-chunk-recovery.spec.ts`: one reload into the new build, no second reload (11-05) |
Overall: **완료** — every spec §54 criterion is `완료`.
## Growth inside one app (11-04)
The rows above show that one app does not change when the *global* topology grows. What one app costs as its *own* feature count grows is measured separately, with copies of the real reference feature: `scripts/scale/measure-feature-shape-growth.ts` copies `src/features/reference-feature` N times with every feature-owned identifier renamed (feature id, routes, operations, schemas, query namespace, invalidation topic, messages), into contexts of 200 features each, and proves that the copies compose (the registries throw at module evaluation on any duplicate). The baseline is `config/scale/feature-shape-growth.json`, re-created with `node scripts/scale/measure-feature-shape-growth.ts --sizes=0,25,100 --repeats=2 --write` and checked by `tests/unit/feature-shape-growth.test.ts`.
Measured 2026-09-27, two runs per size, Intel i5-10400F (12 logical CPUs), Node 24.14.0, TypeScript 7.0.2, Vite 8.1.5. Load average 25–30 during the runs (other agents were building in parallel), so wall times are upper bounds; counts and bytes are identical in both runs.
| Added features | Routes | tsc instantiations | tsc memory | tsc wall (run 1 / 2) | vite build wall | Initial JS files / gzip | Lazy JS files / gzip |
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 0 | 10 | 712,089 | 0.5 GB | 3.4 s / 3.9 s | 1.2 s / 1.5 s | 13 / 200,001 B | 14 / 24,938 B |
| 25 | 110 | 3,579,093 | 1.4 GB | 8.1 s / 8.7 s | 1.7 s / 2.1 s | 39 / 274,493 B | 165 / 171,638 B |
| 100 | 410 | 12,157,304 | 4.2 GB | 31.3 s / 43.2 s | 6.0 s / 3.7 s | 114 / 490,658 B | 615 / 704,938 B |
What this means for the declared ceilings (`CONTEXT_MAX_FEATURES = 200`, `src/contexts/reference-context/context-module.ts:10`; `APP_MAX_CONTEXTS = 20`, `src/apps/template-app/app-module.ts:7`):
| Growth axis | Verified | Condition | Not verified |
| --- | --- | --- | --- |
| Typecheck of one app | Linear, about 115k instantiations and 37 MB of tsc memory per feature, up to 100 features | Reference-shaped features, `tsconfig.app.json`, the host above | 200 features (one full context) and beyond; 4,000 (20 × 200) would need on the order of 150 GB by the same slope |
| Production build | Completes at 100 features in seconds | Same | Beyond 100 |
| Initial JavaScript | Grows by about 2.9 KB gzip and one file per feature: every installed feature's contracts and module load at boot | Same | — the initial budget (204,800 B, `config/performance/budgets.json`) (about 196.5 KB used by `check:bundle`) is exceeded after about three more reference-shaped features, far below the context ceiling |
| Lazy JavaScript | About 6.8 KB gzip and six files per feature, every file far below the 120 KiB lazy-chunk budget | Same | Download cost of many lazy files |
| Routes | 4 per feature, composed without conflict | Same | Router matching cost at runtime |
So the ceilings bound the *count*; they are not a verified capacity. The practical limit of one app today is the initial JavaScript budget, then typecheck memory. 400 features were not measured (D-18 makes it optional; at the observed slope a run needs about 16 GB of tsc memory).
## Executable evidence (supported runtime)
Re-run on Node **v24.11.0** / pnpm 11.17.0 inside the `workmachine` container (whose default Node v22.23.2 is below `engines.node`):
@@ -142,8 +142,12 @@ rasterization과 실제 device farm은 프로젝트가 요구할 때 연결한
operation별 success/empty/slow/network/timeout/content/envelope/schema/auth/
403/404/409/422/429/retry/terminal을 포함한 19개 scenario catalog와 strict
unhandled-request server를 공유한다. clock/random/scheduler/storage helper와
browser console/page/network failure 정책으로 비결정적 우회를 차단한다.
unhandled-request server(`tests/mocks/server.ts`의
`onUnhandledRequest: "error"`)를 공유한다. browser test는 공용 fixture
(`tests/support/browser/strict-browser-test.ts`)가 console error/warning,
page error, 실패한 request를 test 실패로 만든다. 시간은 각 test가
`vi.useFakeTimers()`로 통제하고, 전역 `afterEach`가 실제 timer로 되돌린다
(`tests/setup.ts`). 공용 clock/random/scheduler helper는 두지 않는다.
#### Built-dist와 compact E2E
@@ -153,11 +157,25 @@ Chromium, Firefox, WebKit과 compact project로 실행한다. 빠른 Vite 개발
#### 위험 기반 coverage
V8 text/JSON/LCOV를 생성하고 전체 기준과 retry/storage/telemetry/application
composition/compatibility/performance/promotion/chunk/diagnostics/reference HTTP
operation/query-mutation/registry compatibility 19개 high-risk module에 80개
scoped threshold를 적용한다. critical module 누락 또는 threshold 미달 fixture는
merge gate를 실패시킨다.
V8 text/JSON/LCOV를 생성하고 `config/testing/risk-coverage.json`의 하한을
적용한다. 하한은 전체 평균 4개(`summary`: lines 75, statements 73, functions 80,
branches 68)와 `criticalModules` 20개의 모듈별 하한 80개(모듈마다 4개)다.
`check-risk-coverage.ts`는 두 종류를 합쳐 "84 thresholds"로 보고한다. 모듈은
HTTP 실행(`http-execution-v3.ts`, live 재시도 규칙 `http-retry-lifecycle.ts`,
계약 경로 규칙 `http-contract-bridge.ts`, legacy V2 `retry-policy.ts`, 요청 조립,
bounded body/JSON), server-state scope, Service Worker lifecycle, 저장소
(`browser-storage-adapter.ts`, `indexeddb-runtime.ts`), telemetry, application
조립, compatibility, chunk recovery, release manifest, boot JSON, diagnostics,
reference HTTP gateway, application query다. 모듈별 하한은 목표치가 아니라
등록할 때 측정값 근처로 둔 ratchet이다(되돌아가지 못하게 막는 하한). 새로
넣은 세 모듈은 2026-09-26 측정값을 내림한 값이다.
critical module 누락 또는 threshold 미달 fixture는 merge gate를 실패시킨다.
realtime(`src/adapters/realtime/`)과 Web Push(`src/adapters/web-push/`)는
`test:realtime-removal`이 통째로 지우는 선택 runtime이라
(`scripts/test-realtime-runtime-removal.ts`의 `runtimePaths`) 모듈별 하한에 넣지
않는다. 넣으면 그 runtime을 지운 다운스트림의 coverage gate가 없는 파일 때문에
실패한다. 두 영역은 전체 평균 하한만 받는다.
Repository inventory의 정적 provenance는 runtime 실행 가능 여부가 아니라 현재
Vitest/V8 producer가 file counter를 생성하는 문장의 존재 여부를 나타낸다. 따라서
@@ -724,15 +742,19 @@ tests/factories/
| `malformed-json` | 파싱 불가능한 JSON |
| `envelope-mismatch` | top-level contract 위반 |
| `schema-mismatch` | data payload contract 위반 |
| `auth-recover-once` | 첫 401 후 복구 성공 |
| `auth-persistent-401` | 복구 뒤에도 401 |
| `response-too-large` | 응답 본문이 크기 상한을 넘어 읽기를 멈춤 |
| `unauthenticated-401` | 401. 명령은 적용되지 않음(`NOT_APPLIED`) |
| `forbidden-403` | 권한 없음 |
| `not-found-404` | 리소스 없음 |
| `conflict-409` | mutation conflict |
| `validation-422` | 승인된 field/global error |
| `rate-limited-429` | Retry-After 포함 |
| `server-retry-success` | bounded retry 뒤 성공 |
| `server-terminal-500` | retry 소진 |
| `server-terminal-503` | 503. 조회는 retry 2회 뒤 소진, 명령은 retry 없이 결과 불명(`MAYBE_APPLIED`) |
이 표는 `tests/mocks/scenarios/catalog.ts`의 `HTTP_SCENARIO_IDS`와 같은 순서다.
401 뒤 인증 복구 흐름은 catalog가 아니라 `tests/integration/auth-recovery.test.ts`가
다룬다.
### 11.3 Catalog 규칙
@@ -951,6 +973,13 @@ fixture로 확인한다.
### 14.2 위험별 목표
아래 표는 올려 갈 **목표**다. 지금 gate가 막는 값은 이 표가 아니라
`config/testing/risk-coverage.json`의 현재값 ratchet이다(2.3절 "위험 기반
coverage"). 예를 들어 `service-worker-lifecycle.ts`의 branch 하한은 43,
`http-execution-v3.ts`는 52로 표의 목표보다 낮다. 테스트를 보강해 측정값이
오르면 그 모듈의 하한을 새 측정값으로 올린다. 하한을 내리는 변경은 이유를
커밋 메시지에 적는다.
| 영역 | Line/statement 참고 | Branch 우선 목표 |
| --- | --- | --- |
| registry/codec/schema | 95~100% | 100%에 가깝게 |
+36 -8
View File
@@ -16,9 +16,11 @@ Each gate is blocking in its declared scope. Failures are not downgraded with
| end-to-end | `pnpm test:e2e` | pinned browser engines | `artifacts/tests/e2e/` |
| accessibility | `pnpm test:a11y` | pinned browser engines | `artifacts/tests/a11y.json` |
`test:all` is the normal product-development loop. It intentionally includes
runtime-schema, unit, capability-contract, component, integration, reference
feature and recipe suites, but does not include `test:system`. CI-runner and
`test:all` is the normal product-development loop. It runs, in order,
`test:runtime-schema`, `test:unit`, `test:contract`, `test:component`,
`test:integration`, `test:reference-feature`, `test:platform` (`tests/platform`
and `tests/architecture`), `check:release-coherence` and `test:recipes`
(`package.json` `test:all`). It does not include `test:system`. CI-runner and
supply-chain assurance has different host prerequisites and is invoked
explicitly in the assurance path.
@@ -37,6 +39,24 @@ That runner:
`vitest.config.ts` also fixes `NODE_ENV=test` so a direct Vitest invocation
cannot accidentally select React's production behavior.
To run one test file, use the same runner:
```sh
node scripts/run-vitest.ts run tests/component/router.test.tsx
```
The runner renders the gitignored `src/.generated/` product index first
(`scripts/run-vitest.ts`), so this works in a fresh clone. A direct
`pnpm exec vitest run <file>` skips that step and fails there with
`Failed to resolve import "../.generated/product-feature-index.ts"`.
`pnpm test:unit -- <file>` does not narrow the run, because the script already
passes the whole `tests/unit` directory.
`vitest.config.ts` collects only `tests/**/*.test.{ts,tsx}`; Playwright
`*.spec.ts` files are not Vitest tests. `tests/system/` is collected only when
the command names it, as `test:system` does, so a bare `vitest` run does not
start the CI-runner suite.
The system suite additionally runs
`scripts/check-system-test-prerequisites.ts` and fails with one prerequisite
report when the CI-runner host does not provide its required Linux facilities.
@@ -64,11 +84,19 @@ CI / release assurance:
A host-level process/sandbox test must not be placed in `tests/unit` merely
because it uses Vitest. The classification follows the system boundary and
prerequisites, not the test framework. Process-heavy tests that spawn child
processes but do not require privileged host facilities belong in
`tests/integration`; for example the provider guardian transaction protocol
lives at `tests/integration/provider-guardian-transaction.test.ts`. Its
handshake timeout is a test watchdog, not a production protocol deadline.
prerequisites, not the test framework:
- Tests that need systemd, bubblewrap, cgroup v2 or CI-provider process
controls belong in `tests/system`.
- Tests of a long-lived child-process protocol belong in `tests/integration`;
for example the provider guardian transaction protocol lives at
`tests/integration/provider-guardian-transaction.test.ts`. Its handshake
timeout is a test watchdog, not a production protocol deadline.
- A script's own tests may start that script, Node or Vitest as a short-lived
child process to check its command-line contract, and stay in `tests/unit`
next to the script's other tests. Such a suite declares its own longer
timeout instead of raising the global one; for example
`tests/unit/risk-coverage.test.ts` uses `PROCESS_HEAVY_TIMEOUT_MS`.
End-to-end and automated accessibility scenarios run on the pinned Chromium,
Firefox, and WebKit engines. The responsive contract explicitly exercises