docs: define platform-owned frontend assurance delivery
This commit is contained in:
DongHyeonka
+424
@@ -0,0 +1,424 @@
|
||||
# Platform-owned Frontend Assurance and Delivery Design
|
||||
|
||||
## Purpose
|
||||
|
||||
This design moves test assurance and CI/CD orchestration to the two platforms
|
||||
that own those responsibilities while keeping product behavior and product
|
||||
tests in the frontend template. It covers the complete vertical path from risk
|
||||
selection through immutable static-site promotion.
|
||||
|
||||
The three repositories are:
|
||||
|
||||
- `/home/donghyeon/workspace/desktop-server-git/test-assurance-platform`
|
||||
- `/home/donghyeon/workspace/desktop-server-git/cicd-platform`
|
||||
- `/home/donghyeon/workspace/desktop-server-git/clean-architecture-frontend-template`
|
||||
|
||||
The selected approach is platform-native integration. A CI-first translation
|
||||
bridge and a test-assurance-only migration were rejected because each would
|
||||
leave one platform temporarily owning semantics assigned to the other.
|
||||
|
||||
## Authority boundaries
|
||||
|
||||
### Frontend product repository
|
||||
|
||||
The frontend repository owns:
|
||||
|
||||
- production source and product contracts;
|
||||
- test implementations, fixtures, mocks, scenario catalogs, and assertions;
|
||||
- package scripts that run one declared suite;
|
||||
- raw product-specific reports and codecs for product artifacts;
|
||||
- repository declarations: risks, obligations, suites, change surfaces, build
|
||||
components, outputs, and selected platform capabilities.
|
||||
|
||||
It does not own remote scheduling, pipeline DAG generation, evidence
|
||||
assessment, provider orchestration, retention, signing, or promotion.
|
||||
|
||||
### Test assurance platform
|
||||
|
||||
The test assurance platform owns:
|
||||
|
||||
- risk-to-obligation selection;
|
||||
- scheduler-neutral execution plans;
|
||||
- suite execution contracts and bounded local execution;
|
||||
- report normalization and false-green prevention;
|
||||
- evidence digest chains and obligation assessment;
|
||||
- flaky, quarantine, waiver, and capability-readiness semantics.
|
||||
|
||||
It does not own runner allocation, workflow fan-out/fan-in, build publication,
|
||||
release approval, or promotion.
|
||||
|
||||
### CI/CD platform
|
||||
|
||||
The CI/CD platform owns:
|
||||
|
||||
- required workflows, source materialization, runner trust, and toolchain pins;
|
||||
- remote projection of test-assurance work items;
|
||||
- install, lint, typecheck, deterministic build, and artifact publication;
|
||||
- immutable candidate assembly and artifact identity;
|
||||
- vulnerability, SBOM, provenance, signature, and provider evidence;
|
||||
- release approval, promotion, retention, and terminal status publication.
|
||||
|
||||
It consumes test-assurance results without reinterpreting their test meaning.
|
||||
|
||||
## End-to-end authority flow
|
||||
|
||||
```text
|
||||
frontend declarations and tests
|
||||
-> testctl validate/lock/compile/select/plan
|
||||
-> CI projects source work items to untrusted runners
|
||||
-> testctl normalize/bundle/assess source results
|
||||
-> CI builds one immutable frontend candidate
|
||||
-> CI supplies the candidate digest to artifact-bound work items
|
||||
-> testctl normalize/bundle/assess artifact results
|
||||
-> CI obtains vulnerability/SBOM/provenance/signature evidence
|
||||
-> release-control promotes the exact assessed candidate
|
||||
```
|
||||
|
||||
Every transition carries canonical identities. No stage may replace a missing
|
||||
identity with a path, timestamp, branch name, or mutable tag.
|
||||
|
||||
## Test assurance platform changes
|
||||
|
||||
### Frontend capability family
|
||||
|
||||
Add the following independent capabilities:
|
||||
|
||||
- `unit-typescript-vitest`
|
||||
- `component-react-vitest`
|
||||
- `integration-http-msw`
|
||||
- `architecture-typescript`
|
||||
- `coverage-v8`
|
||||
- `e2e-playwright-firefox`
|
||||
- `e2e-playwright-webkit`
|
||||
- `accessibility-web`
|
||||
- `visual-regression-web`
|
||||
|
||||
The existing `e2e-playwright-chromium` capability is upgraded to the same
|
||||
contract revision. Browser capabilities retain separate readiness cards; no
|
||||
aggregate frontend-readiness card or score is introduced.
|
||||
|
||||
Each capability defines its own artifacts, timeouts, isolation, false-green
|
||||
rules, max evidence age, and minimum readiness. A repository obligation may
|
||||
require all three browser capabilities without merging their readiness.
|
||||
|
||||
### Toolchain catalog
|
||||
|
||||
Create a new catalog revision containing the frontend template's supported
|
||||
toolchain:
|
||||
|
||||
- Node.js `24.14.0`
|
||||
- pnpm `11.17.0`
|
||||
- TypeScript `7.0.2`
|
||||
- Vitest `4.1.10`
|
||||
- Playwright `1.62.0`
|
||||
|
||||
The exact installed versions in `package.json`, `.nvmrc`, and the lockfile are
|
||||
validated against the catalog. The lock generator records immutable artifact
|
||||
digests. A missing digest or version mismatch is `TA-LOCK-003`/exit 30 and does
|
||||
not fall back to the host toolchain.
|
||||
|
||||
### Adapters and normalizers
|
||||
|
||||
Add adapters and normalizers for:
|
||||
|
||||
- Vitest JUnit XML and Vitest-discovered test counts;
|
||||
- V8 coverage summary plus the repository production-module inventory;
|
||||
- typed HTTP scenario execution receipts;
|
||||
- TypeScript architecture graph and violation JSON;
|
||||
- Playwright JSON/JUnit, trace, screenshot, console, and network summaries;
|
||||
- accessibility findings and manual-review records;
|
||||
- visual baseline identity and image-diff results;
|
||||
- production-shaped read/write E2E mutation receipts.
|
||||
|
||||
Normalizers validate regular non-symlink files, bounded byte sizes, fatal UTF-8,
|
||||
strict schemas, canonical repository-relative paths, and report-specific
|
||||
cross-field invariants.
|
||||
|
||||
### False-green rules
|
||||
|
||||
The following outcomes can never normalize to PASS:
|
||||
|
||||
- zero discovered tests or an all-skipped required suite;
|
||||
- a pass created only by retry;
|
||||
- a missing, empty, malformed, oversized, or mismatched report;
|
||||
- a declared HTTP scenario without an exact executed receipt;
|
||||
- a missing required browser project;
|
||||
- a non-empty source tree with zero architecture modules or dependencies;
|
||||
- an unresolved import or dependency cycle;
|
||||
- a production module absent from the coverage universe;
|
||||
- a browser write test without observed response, mutation, and reload reads;
|
||||
- an accessibility or visual result whose baseline/provider identity is absent.
|
||||
|
||||
Negative product fixtures may be schema-valid FAIL evidence; command outcome and
|
||||
assessment, not artifact shape alone, determine satisfaction.
|
||||
|
||||
### Artifact-bound execution contract
|
||||
|
||||
The current v2 execution schemas cannot bind a work item to an immutable build
|
||||
input. The platform therefore adds v3 execution contracts rather than silently
|
||||
changing v2 semantics.
|
||||
|
||||
`SuiteDefinition` and `WorkItem` gain required fields:
|
||||
|
||||
```text
|
||||
executionPhase: SOURCE | ARTIFACT
|
||||
requiredInputArtifacts[]:
|
||||
artifactId
|
||||
mediaType
|
||||
sha256
|
||||
```
|
||||
|
||||
An artifact-bound `ExecutionRequest` carries the same artifact references. The
|
||||
plan, raw result set, normalized result, evidence bundle, and assessment all
|
||||
bind the input artifact tuple. Evidence from v2 and v3 cannot be merged.
|
||||
|
||||
The platform dual-reads existing v2 JVM manifests during migration. New
|
||||
frontend capabilities require v3, and all new platform outputs are v3. There is
|
||||
no implicit phase default.
|
||||
|
||||
## CI/CD platform changes
|
||||
|
||||
### Test-assurance integration
|
||||
|
||||
Add `ci-test-assurance`. It depends on `ci-standard-core` and invokes a
|
||||
digest-pinned `testctl` distribution using argument arrays only.
|
||||
|
||||
Its responsibilities are:
|
||||
|
||||
1. validate and lock the repository assurance manifest;
|
||||
2. compile policy and create source and artifact execution requests;
|
||||
3. obtain deterministic plans;
|
||||
4. project `execute-one` work items to the required trust partition;
|
||||
5. preserve plan digest, work-item ID, attempt, exit code, and raw artifacts;
|
||||
6. call testctl normalization, bundling, and assessment;
|
||||
7. expose only canonical assessment and evidence digests downstream.
|
||||
|
||||
CI may choose runner parallelism but may not change selection, retry, timeout,
|
||||
expected artifacts, status, or obligation satisfaction. Missing work-item
|
||||
results are platform defects, never successful no-ops.
|
||||
|
||||
### Node and frontend capabilities
|
||||
|
||||
`ci-node-typescript` owns reproducible pnpm install, declared lint, and declared
|
||||
typecheck. It no longer executes or assesses unit/coverage suites when
|
||||
`ci-test-assurance` is selected.
|
||||
|
||||
`ci-frontend` owns one deterministic build of the selected source revision and
|
||||
verifies:
|
||||
|
||||
- a non-empty static output;
|
||||
- byte-identical rebuild evidence in an isolated verification workspace;
|
||||
- absence of undeclared environment and build-host values;
|
||||
- declared size budgets;
|
||||
- a canonical tree digest and archive manifest.
|
||||
|
||||
The candidate consumed after this point is the first verified candidate. The
|
||||
verification build proves determinism but is never promoted.
|
||||
|
||||
### Release capability family
|
||||
|
||||
Add the following capabilities:
|
||||
|
||||
- `ci-dependency-vulnerability`
|
||||
- `ci-artifact-signing`
|
||||
- `ci-static-artifact-supply-chain`
|
||||
- `ci-static-site-publish`
|
||||
|
||||
Generalize existing `ci-sbom` and `ci-provenance` subject contracts so that a
|
||||
static archive is a supported immutable subject without weakening their
|
||||
container behavior. `ci-static-artifact-supply-chain` is a composite that
|
||||
references, rather than copies, child evidence.
|
||||
|
||||
`delivery-release-control` promotes the exact candidate whose digest appears in
|
||||
the signed release identity. It may not build, modify, or repackage the
|
||||
candidate. Static-site publication returns provider and served-content digests;
|
||||
both must equal the approved subject before promotion succeeds.
|
||||
|
||||
### Release identity
|
||||
|
||||
The release manifest binds:
|
||||
|
||||
```text
|
||||
source revision
|
||||
candidate archive SHA-256 and canonical member manifest
|
||||
test-assurance manifest and policy digests
|
||||
source plan, evidence, and assessment digests
|
||||
artifact plan, evidence, and assessment digests
|
||||
dependency vulnerability, SBOM, and provenance digests
|
||||
signature key ID and signature digest
|
||||
CI policy, capability-registry, toolchain, and platform-catalog digests
|
||||
target environment and approval identity
|
||||
```
|
||||
|
||||
A report for another candidate, a rebuilt candidate, a changed policy, an
|
||||
expired approval, or an unconfirmed provider mutation blocks promotion.
|
||||
|
||||
### Required workflow
|
||||
|
||||
The centrally installed required workflow remains a thin bootstrap. It contains
|
||||
no language, test, build, provider, or promotion logic. It materializes the
|
||||
exact source revision, verifies the signed platform catalog, invokes pinned
|
||||
`cicdctl`, and publishes one terminal sentinel.
|
||||
|
||||
Product repositories do not copy this workflow.
|
||||
|
||||
## Frontend consumer contract
|
||||
|
||||
The frontend repository adds:
|
||||
|
||||
- `delivery-platform.yaml`;
|
||||
- `test-assurance.yaml`;
|
||||
- a generated `test-assurance.lock.json`;
|
||||
- risk, obligation, suite, and change-surface documents under
|
||||
`config/test-assurance/`.
|
||||
|
||||
`delivery-platform.yaml` selects the core, Node, frontend, test-assurance,
|
||||
dependency vulnerability, SBOM, provenance, signing, static supply-chain,
|
||||
static publication, and release-control capabilities. It pins the signed
|
||||
platform version.
|
||||
|
||||
Suite definitions reference existing product-owned package scripts. One suite
|
||||
definition executes one bounded test purpose and declares its raw artifacts.
|
||||
The repository does not wrap several assurance decisions in one script.
|
||||
|
||||
The product retains:
|
||||
|
||||
- production runtime and adapter tests;
|
||||
- Vitest and Playwright configuration;
|
||||
- mocks, scenarios, fixtures, and browser assertions;
|
||||
- V8 instrumentation configuration;
|
||||
- product artifact codecs such as runtime/release manifest schemas.
|
||||
|
||||
It removes after cutover:
|
||||
|
||||
- the copied `.gitea/workflows/quality-gates.yml`;
|
||||
- `config/ci/gates.json` and its runner/generator/checker;
|
||||
- local risk selection, waiver, normalization, and assessment engines;
|
||||
- local provider, retention, promotion, and CI evidence orchestration;
|
||||
- package scripts whose only purpose is to reproduce platform policy.
|
||||
|
||||
## Treatment of completed and in-progress frontend work
|
||||
|
||||
Runtime production changes and their product tests remain unchanged.
|
||||
|
||||
Repository-wide V8 instrumentation, HTTP scenario execution, and browser
|
||||
read/write assertions remain as product test inputs. Their local selection,
|
||||
reconciliation, evidence assessment, and waiver logic moves to test assurance.
|
||||
|
||||
Product runtime/release artifact codecs remain local. Supply-chain provider
|
||||
policy, archive transfer, signing, retention, and promotion move to CI/CD.
|
||||
|
||||
The uncommitted Task 3 worktree is not reset or overwritten. Before migration,
|
||||
every changed path is classified as product-owned, test-assurance-owned,
|
||||
CI/CD-owned, or unrelated/user-owned. Reusable validators and adversarial tests
|
||||
move to their owning platform through explicit patches. Unrelated and
|
||||
origin-unknown changes are preserved.
|
||||
|
||||
## Migration sequence
|
||||
|
||||
### Phase 1: Test assurance capability readiness
|
||||
|
||||
Implement v3 contracts, frontend capabilities, adapters, normalizers, locks,
|
||||
positive fixtures, adversarial fixtures, and independent readiness cards. Each
|
||||
new capability reaches at least R1 before a consumer may select it in shadow.
|
||||
|
||||
### Phase 2: CI/CD integration and immutable release
|
||||
|
||||
Implement pinned testctl integration, source/artifact work-item projection,
|
||||
frontend candidate identity, provider evidence, signing, static publication,
|
||||
and release-control binding. Each CI capability receives independent P1
|
||||
evidence; activation remains shadow.
|
||||
|
||||
### Phase 3: Consumer declarations
|
||||
|
||||
Add both manifests and the assurance declarations to the frontend repository.
|
||||
Map existing product suites to capabilities without deleting the legacy path.
|
||||
Validate all manifests with the released platform binaries.
|
||||
|
||||
### Phase 4: Shadow parity
|
||||
|
||||
Run legacy and platform paths against the same source revision. Compare:
|
||||
|
||||
- selected suites and discovered/executed counts;
|
||||
- PASS, FAIL, FLAKY, INCOMPLETE, and platform-defect classification;
|
||||
- coverage production-module universe;
|
||||
- HTTP scenario declared/executed identities;
|
||||
- Chromium, Firefox, and WebKit results;
|
||||
- candidate archive and member digests;
|
||||
- provider evidence and promotion readiness.
|
||||
|
||||
The platform path is the only candidate producer in shadow. Legacy release and
|
||||
promotion commands become read-only comparison probes. No two writers may
|
||||
publish or promote.
|
||||
|
||||
### Phase 5: Cutover
|
||||
|
||||
After parity and fault tests pass, activate the platform capabilities, attach
|
||||
the central required workflow/status, and remove copied workflow and local
|
||||
policy engines. Product tests and declared suite commands remain.
|
||||
|
||||
## Failure and rollback semantics
|
||||
|
||||
- Unsupported capability or unavailable toolchain is explicit UNSUPPORTED or
|
||||
INCOMPLETE, not a skipped pass.
|
||||
- Schema-major mismatch stops before execution and never invokes a local
|
||||
fallback.
|
||||
- Missing or corrupt raw results stop normalization.
|
||||
- Provider timeout, response loss, or digest mismatch blocks promotion and is
|
||||
reconciled by operation ID where mutation may have occurred.
|
||||
- A candidate, source assessment, or artifact assessment digest change
|
||||
invalidates approval.
|
||||
- A missing terminal sentinel blocks the required status.
|
||||
|
||||
Rollback changes only the repository's signed `platformVersion`/catalog pin to
|
||||
the previous proven release and restores the previous capability activation.
|
||||
It never restores a copied product workflow. Release rollback promotes the
|
||||
previous stable immutable subject through release-control.
|
||||
|
||||
## Verification strategy
|
||||
|
||||
### Test assurance platform
|
||||
|
||||
- schema positive and adversarial corpus;
|
||||
- compiler/selector/plan determinism;
|
||||
- each normalizer's valid, malformed, missing, empty, oversized, symlink, and
|
||||
cross-field cases;
|
||||
- zero-discovery, all-skipped, retry, browser-matrix, coverage-omission,
|
||||
architecture-empty/cycle, scenario-omission, and mutation-receipt fixtures;
|
||||
- v2/v3 isolation and digest-chain tests;
|
||||
- full conformance chain and Gradle verification.
|
||||
|
||||
### CI/CD platform
|
||||
|
||||
- manifest compiler and capability dependency/activation tests;
|
||||
- pinned testctl invocation and exit-code preservation;
|
||||
- fan-out completeness and missing-result fault tests;
|
||||
- deterministic static candidate and exact-member archive tests;
|
||||
- provider timeout, invalid signature, changed digest, response-loss, and stale
|
||||
approval tests;
|
||||
- required workflow thinness and exact required-status tests;
|
||||
- registry, boundary, fixture, fault, and readiness verification.
|
||||
|
||||
### Frontend consumer
|
||||
|
||||
- manifest validation with released platform binaries;
|
||||
- existing focused product tests;
|
||||
- platform shadow run against the actual repository;
|
||||
- semantic parity and digest reports;
|
||||
- one-writer and rollback drills;
|
||||
- removal tests proving the template works without copied workflow or local
|
||||
policy engines.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Test selection and assessment have one authority: test assurance.
|
||||
- CI orchestration and promotion have one authority: CI/CD.
|
||||
- The frontend repository contains product tests and declarations, not copied
|
||||
platform engines.
|
||||
- Every required frontend capability has independent readiness evidence.
|
||||
- Source and artifact test evidence bind the exact revision and candidate.
|
||||
- The promoted static site is byte-identical to the assessed, signed candidate.
|
||||
- Missing evidence, unsupported capability, platform failure, and provider
|
||||
uncertainty cannot produce a passing required status.
|
||||
- Cutover and rollback require no copied workflow logic in the product
|
||||
repository.
|
||||
Reference in New Issue
Block a user