465 lines
18 KiB
Markdown
465 lines
18 KiB
Markdown
# Platform-owned Frontend Assurance and Delivery Design
|
|
|
|
## Purpose
|
|
|
|
This design moves test assurance and CI/CD orchestration to the two platforms
|
|
that own those responsibilities while keeping product behavior and product
|
|
tests in the frontend template. It covers the complete vertical path from risk
|
|
selection through immutable static-site promotion.
|
|
|
|
The three repositories are:
|
|
|
|
- `/home/donghyeon/workspace/desktop-server-git/test-assurance-platform`
|
|
- `/home/donghyeon/workspace/desktop-server-git/cicd-platform`
|
|
- `/home/donghyeon/workspace/desktop-server-git/clean-architecture-frontend-template`
|
|
|
|
The selected approach is platform-native integration. A CI-first translation
|
|
bridge and a test-assurance-only migration were rejected because each would
|
|
leave one platform temporarily owning semantics assigned to the other.
|
|
|
|
## Authority boundaries
|
|
|
|
### Frontend product repository
|
|
|
|
The frontend repository owns:
|
|
|
|
- production source and product contracts;
|
|
- test implementations, fixtures, mocks, scenario catalogs, and assertions;
|
|
- package scripts that run one declared suite;
|
|
- raw product-specific reports and codecs for product artifacts;
|
|
- repository declarations: risks, obligations, suites, change surfaces, build
|
|
components, outputs, and selected platform capabilities.
|
|
|
|
It does not own remote scheduling, pipeline DAG generation, evidence
|
|
assessment, provider orchestration, retention, signing, or promotion.
|
|
|
|
### Test assurance platform
|
|
|
|
The test assurance platform owns:
|
|
|
|
- risk-to-obligation selection;
|
|
- scheduler-neutral execution plans;
|
|
- suite execution contracts and bounded local execution;
|
|
- report normalization and false-green prevention;
|
|
- evidence digest chains and obligation assessment;
|
|
- flaky, quarantine, waiver, and capability-readiness semantics.
|
|
|
|
It does not own runner allocation, workflow fan-out/fan-in, build publication,
|
|
release approval, or promotion.
|
|
|
|
### CI/CD platform
|
|
|
|
The CI/CD platform owns:
|
|
|
|
- required workflows, source materialization, runner trust, and toolchain pins;
|
|
- remote projection of test-assurance work items;
|
|
- install, lint, typecheck, deterministic build, and artifact publication;
|
|
- immutable candidate assembly and artifact identity;
|
|
- vulnerability, SBOM, provenance, signature, and provider evidence;
|
|
- release approval, promotion, retention, and terminal status publication.
|
|
|
|
It consumes test-assurance results without reinterpreting their test meaning.
|
|
|
|
## End-to-end authority flow
|
|
|
|
```text
|
|
frontend declarations and tests
|
|
-> testctl validate/lock/compile/select/plan
|
|
-> CI projects source work items to untrusted runners
|
|
-> testctl normalize/bundle/assess source results
|
|
-> CI builds one immutable frontend candidate
|
|
-> CI supplies the candidate digest to artifact-bound work items
|
|
-> testctl normalize/bundle/assess artifact results
|
|
-> CI obtains vulnerability/SBOM/provenance/signature evidence
|
|
-> release-control promotes the exact assessed candidate
|
|
```
|
|
|
|
Every transition carries canonical identities. No stage may replace a missing
|
|
identity with a path, timestamp, branch name, or mutable tag.
|
|
|
|
## Test assurance platform changes
|
|
|
|
### Frontend capability family
|
|
|
|
Add the following independent capabilities:
|
|
|
|
- `unit-typescript-vitest`
|
|
- `component-react-vitest`
|
|
- `integration-http-msw`
|
|
- `architecture-typescript`
|
|
- `coverage-v8`
|
|
- `e2e-playwright-firefox`
|
|
- `e2e-playwright-webkit`
|
|
- `accessibility-web`
|
|
- `visual-regression-web`
|
|
|
|
The existing `e2e-playwright-chromium` capability is upgraded to the same
|
|
contract revision. Browser capabilities retain separate readiness cards; no
|
|
aggregate frontend-readiness card or score is introduced.
|
|
|
|
Each capability defines its own artifacts, timeouts, isolation, false-green
|
|
rules, max evidence age, and minimum readiness. A repository obligation may
|
|
require all three browser capabilities without merging their readiness.
|
|
|
|
### Toolchain catalog
|
|
|
|
Create a new catalog revision containing the frontend template's supported
|
|
toolchain:
|
|
|
|
- Node.js `24.14.0`
|
|
- pnpm `11.17.0`
|
|
- TypeScript `7.0.2`
|
|
- Vitest `4.1.10`
|
|
- Playwright `1.62.0`
|
|
|
|
The exact installed versions in `package.json`, `.nvmrc`, and the lockfile are
|
|
validated against the catalog. The lock generator records immutable artifact
|
|
digests. A missing digest or version mismatch is `TA-LOCK-003`/exit 30 and does
|
|
not fall back to the host toolchain.
|
|
|
|
### Adapters and normalizers
|
|
|
|
Add adapters and normalizers for:
|
|
|
|
- Vitest JUnit XML and Vitest-discovered test counts;
|
|
- V8 coverage summary plus the repository production-module inventory;
|
|
- typed HTTP scenario execution receipts;
|
|
- TypeScript architecture graph and violation JSON;
|
|
- Playwright JSON/JUnit, trace, screenshot, console, and network summaries;
|
|
- accessibility findings and manual-review records;
|
|
- visual baseline identity and image-diff results;
|
|
- production-shaped read/write E2E mutation receipts.
|
|
|
|
Normalizers validate regular non-symlink files, bounded byte sizes, fatal UTF-8,
|
|
strict schemas, canonical repository-relative paths, and report-specific
|
|
cross-field invariants.
|
|
|
|
### False-green rules
|
|
|
|
The following outcomes can never normalize to PASS:
|
|
|
|
- zero discovered tests or an all-skipped required suite;
|
|
- a pass created only by retry;
|
|
- a missing, empty, malformed, oversized, or mismatched report;
|
|
- a declared HTTP scenario without an exact executed receipt;
|
|
- a missing required browser project;
|
|
- a non-empty source tree with zero architecture modules or dependencies;
|
|
- an unresolved import or dependency cycle;
|
|
- a production module absent from the coverage universe;
|
|
- a browser write test without observed response, mutation, and reload reads;
|
|
- an accessibility or visual result whose baseline/provider identity is absent.
|
|
|
|
Negative product fixtures may be schema-valid FAIL evidence; command outcome and
|
|
assessment, not artifact shape alone, determine satisfaction.
|
|
|
|
### Artifact-bound execution contract
|
|
|
|
The current v2 execution schemas cannot bind a work item to an immutable build
|
|
input. The platform therefore adds v3 execution contracts rather than silently
|
|
changing v2 semantics.
|
|
|
|
`SuiteDefinition` and `WorkItem` gain required fields:
|
|
|
|
```text
|
|
executionPhase: SOURCE | ARTIFACT
|
|
requiredInputArtifacts[]:
|
|
artifactId
|
|
mediaType
|
|
sha256
|
|
```
|
|
|
|
An artifact-bound `ExecutionRequest` carries the same artifact references. The
|
|
plan, raw result set, normalized result, evidence bundle, and assessment all
|
|
bind the input artifact tuple. Evidence from v2 and v3 cannot be merged.
|
|
|
|
The platform dual-reads existing v2 JVM manifests during migration. New
|
|
frontend capabilities require v3, and all new platform outputs are v3. There is
|
|
no implicit phase default.
|
|
|
|
### Artifact suite declaration amendment
|
|
|
|
A repository cannot know the SHA-256 of a candidate that CI has not built yet.
|
|
The repository contract therefore separates a static declaration from an
|
|
executable suite:
|
|
|
|
- `ArtifactSuiteTemplate` is repository-owned and declares the suite command,
|
|
raw artifacts, `executionPhase: ARTIFACT`, and required input artifact IDs
|
|
and media types. Its schema forbids `sha256` and digest placeholders.
|
|
- `SuiteDefinition` remains the executable v3 contract required above. For an
|
|
artifact suite it always contains the concrete `requiredInputArtifacts`
|
|
tuple including SHA-256.
|
|
- after `ci-frontend` freezes the candidate, CI creates the artifact
|
|
`ExecutionRequest` with that candidate tuple; testctl matches it to the
|
|
template and materializes the executable `SuiteDefinition` and `WorkItem`.
|
|
|
|
CI supplies artifact identity but does not construct or reinterpret test suite
|
|
semantics. An unmatched artifact ID/media type, an unresolved template, or a
|
|
digest in a committed template stops planning. Templates never enter raw,
|
|
normalized, evidence, or assessment documents.
|
|
|
|
## CI/CD platform changes
|
|
|
|
### Test-assurance integration
|
|
|
|
Add `ci-test-assurance`. It depends on `ci-standard-core` and invokes a
|
|
digest-pinned `testctl` distribution using argument arrays only.
|
|
|
|
Its responsibilities are:
|
|
|
|
1. validate and lock the repository assurance manifest;
|
|
2. compile policy and create source and artifact execution requests;
|
|
3. obtain deterministic plans;
|
|
4. project `execute-one` work items to the required trust partition;
|
|
5. preserve plan digest, work-item ID, attempt, exit code, and raw artifacts;
|
|
6. call testctl normalization, bundling, and assessment;
|
|
7. expose only canonical assessment and evidence digests downstream.
|
|
|
|
CI may choose runner parallelism but may not change selection, retry, timeout,
|
|
expected artifacts, status, or obligation satisfaction. Missing work-item
|
|
results are platform defects, never successful no-ops.
|
|
|
|
### Node and frontend capabilities
|
|
|
|
`ci-node-typescript` owns reproducible pnpm install, declared lint, and declared
|
|
typecheck. It no longer executes or assesses unit/coverage suites when
|
|
`ci-test-assurance` is selected.
|
|
|
|
`ci-frontend` owns one deterministic build of the selected source revision and
|
|
verifies:
|
|
|
|
- a non-empty static output;
|
|
- byte-identical rebuild evidence in an isolated verification workspace;
|
|
- absence of undeclared environment and build-host values;
|
|
- declared size budgets;
|
|
- a canonical tree digest and archive manifest.
|
|
|
|
The candidate consumed after this point is the first verified candidate. The
|
|
verification build proves determinism but is never promoted.
|
|
|
|
### Release capability family
|
|
|
|
Add the following capabilities:
|
|
|
|
- `ci-dependency-vulnerability`
|
|
- `ci-artifact-signing`
|
|
- `ci-static-artifact-supply-chain`
|
|
- `ci-static-site-publish`
|
|
|
|
Generalize existing `ci-sbom` and `ci-provenance` subject contracts so that a
|
|
static archive is a supported immutable subject without weakening their
|
|
container behavior. `ci-static-artifact-supply-chain` is a composite that
|
|
references, rather than copies, child evidence.
|
|
|
|
`delivery-release-control` promotes the exact candidate whose digest appears in
|
|
the signed release identity. It may not build, modify, or repackage the
|
|
candidate. Static-site publication returns provider and served-content digests;
|
|
both must equal the approved subject before promotion succeeds.
|
|
|
|
### Release identity
|
|
|
|
The signed release identity is split into two immutable documents so that one
|
|
candidate can be promoted to more than one environment without rewriting its
|
|
candidate manifest:
|
|
|
|
- `ReleaseManifestV2` binds the candidate, test, supply-chain, policy,
|
|
toolchain, and platform identities;
|
|
- `PromotionSubject` binds the release-manifest digest to the target
|
|
environment, approval identity/policy, confirmed publication, and expected
|
|
Git CAS state.
|
|
|
|
Together they bind:
|
|
|
|
```text
|
|
source revision
|
|
candidate archive SHA-256 and canonical member manifest
|
|
test-assurance manifest and policy digests
|
|
source plan, evidence, and assessment digests
|
|
artifact plan, evidence, and assessment digests
|
|
dependency vulnerability, SBOM, and provenance digests
|
|
signature key ID and signature digest
|
|
CI policy, capability-registry, toolchain, and platform-catalog digests
|
|
target environment and approval identity
|
|
```
|
|
|
|
Promotion history is an append-only record and is not a mutable field inside
|
|
`ReleaseManifestV2`.
|
|
|
|
A report for another candidate, a rebuilt candidate, a changed policy, an
|
|
expired approval, or an unconfirmed provider mutation blocks promotion.
|
|
|
|
### Required workflow
|
|
|
|
The centrally installed required workflow remains a thin bootstrap. It contains
|
|
no language, test, build, provider, or promotion logic. It materializes the
|
|
exact source revision, verifies the signed platform catalog, invokes pinned
|
|
`cicdctl`, and publishes one terminal sentinel.
|
|
|
|
Product repositories do not copy this workflow.
|
|
|
|
## Frontend consumer contract
|
|
|
|
The frontend repository adds:
|
|
|
|
- `delivery-platform.yaml`;
|
|
- `test-assurance.yaml`;
|
|
- a generated `test-assurance.lock.json`;
|
|
- risk, obligation, suite, and change-surface documents under
|
|
`config/test-assurance/`.
|
|
|
|
`delivery-platform.yaml` selects the core, Node, frontend, test-assurance,
|
|
dependency vulnerability, SBOM, provenance, signing, static supply-chain,
|
|
static publication, and release-control capabilities. It pins the signed
|
|
platform version.
|
|
|
|
Suite definitions reference existing product-owned package scripts. One suite
|
|
definition executes one bounded test purpose and declares its raw artifacts.
|
|
The repository does not wrap several assurance decisions in one script.
|
|
|
|
The product retains:
|
|
|
|
- production runtime and adapter tests;
|
|
- Vitest and Playwright configuration;
|
|
- mocks, scenarios, fixtures, and browser assertions;
|
|
- V8 instrumentation configuration;
|
|
- product artifact codecs such as runtime/release manifest schemas.
|
|
|
|
It removes after cutover:
|
|
|
|
- the copied `.gitea/workflows/quality-gates.yml`;
|
|
- `config/ci/gates.json` and its runner/generator/checker;
|
|
- local risk selection, waiver, normalization, and assessment engines;
|
|
- local provider, retention, promotion, and CI evidence orchestration;
|
|
- package scripts whose only purpose is to reproduce platform policy.
|
|
|
|
## Treatment of completed and in-progress frontend work
|
|
|
|
Runtime production changes and their product tests remain unchanged.
|
|
|
|
Repository-wide V8 instrumentation, HTTP scenario execution, and browser
|
|
read/write assertions remain as product test inputs. Their local selection,
|
|
reconciliation, evidence assessment, and waiver logic moves to test assurance.
|
|
|
|
Product runtime/release artifact codecs remain local. Supply-chain provider
|
|
policy, archive transfer, signing, retention, and promotion move to CI/CD.
|
|
|
|
The uncommitted Task 3 worktree is not reset or overwritten. Before migration,
|
|
every changed path is classified as product-owned, test-assurance-owned,
|
|
CI/CD-owned, or unrelated/user-owned. Reusable validators and adversarial tests
|
|
move to their owning platform through explicit patches. Unrelated and
|
|
origin-unknown changes are preserved.
|
|
|
|
## Migration sequence
|
|
|
|
### Phase 1: Test assurance capability readiness
|
|
|
|
Implement v3 contracts, frontend capabilities, adapters, normalizers, locks,
|
|
positive fixtures, adversarial fixtures, and independent readiness cards. Each
|
|
new capability reaches at least R1 before a consumer may select it in shadow.
|
|
|
|
### Phase 2: CI/CD integration and immutable release
|
|
|
|
Implement pinned testctl integration, source/artifact work-item projection,
|
|
frontend candidate identity, provider evidence, signing, static publication,
|
|
and release-control binding. Each CI capability receives independent P1
|
|
evidence; activation remains shadow.
|
|
|
|
### Phase 3: Consumer declarations
|
|
|
|
Add both manifests and the assurance declarations to the frontend repository.
|
|
Map existing product suites to capabilities without deleting the legacy path.
|
|
Validate all manifests with the released platform binaries.
|
|
|
|
### Phase 4: Shadow parity
|
|
|
|
Run legacy and platform paths against the same source revision. Compare:
|
|
|
|
- selected suites and discovered/executed counts;
|
|
- PASS, FAIL, FLAKY, INCOMPLETE, and platform-defect classification;
|
|
- coverage production-module universe;
|
|
- HTTP scenario declared/executed identities;
|
|
- Chromium, Firefox, and WebKit results;
|
|
- candidate archive and member digests;
|
|
- provider evidence and promotion readiness.
|
|
|
|
The platform path is the only candidate producer in shadow. Legacy release and
|
|
promotion commands become read-only comparison probes. No two writers may
|
|
publish or promote.
|
|
|
|
Before shadow execution, the legacy workflow registration is disabled and its
|
|
required status is detached. The centrally installed workflow runs in shadow,
|
|
and its runner may invoke legacy test/evidence commands only as read-only
|
|
comparison probes. Observed state records zero legacy candidate/provider/
|
|
promotion invocations and exactly one platform candidate writer.
|
|
|
|
### Phase 5: Cutover
|
|
|
|
After parity and fault tests pass, activate the platform capabilities, attach
|
|
the central required workflow/status, and remove copied workflow and local
|
|
policy engines. Product tests and declared suite commands remain.
|
|
|
|
## Failure and rollback semantics
|
|
|
|
- Unsupported capability or unavailable toolchain is explicit UNSUPPORTED or
|
|
INCOMPLETE, not a skipped pass.
|
|
- Schema-major mismatch stops before execution and never invokes a local
|
|
fallback.
|
|
- Missing or corrupt raw results stop normalization.
|
|
- Provider timeout, response loss, or digest mismatch blocks promotion and is
|
|
reconciled by operation ID where mutation may have occurred.
|
|
- A candidate, source assessment, or artifact assessment digest change
|
|
invalidates approval.
|
|
- A missing terminal sentinel blocks the required status.
|
|
|
|
Rollback changes only the repository's signed `platformVersion`/catalog pin to
|
|
the previous proven release and restores the previous capability activation.
|
|
It never restores a copied product workflow. Release rollback promotes the
|
|
previous stable immutable subject through release-control.
|
|
|
|
## Verification strategy
|
|
|
|
### Test assurance platform
|
|
|
|
- schema positive and adversarial corpus;
|
|
- compiler/selector/plan determinism;
|
|
- each normalizer's valid, malformed, missing, empty, oversized, symlink, and
|
|
cross-field cases;
|
|
- zero-discovery, all-skipped, retry, browser-matrix, coverage-omission,
|
|
architecture-empty/cycle, scenario-omission, and mutation-receipt fixtures;
|
|
- v2/v3 isolation and digest-chain tests;
|
|
- full conformance chain and Gradle verification.
|
|
|
|
### CI/CD platform
|
|
|
|
- manifest compiler and capability dependency/activation tests;
|
|
- pinned testctl invocation and exit-code preservation;
|
|
- fan-out completeness and missing-result fault tests;
|
|
- deterministic static candidate and exact-member archive tests;
|
|
- provider timeout, invalid signature, changed digest, response-loss, and stale
|
|
approval tests;
|
|
- required workflow thinness and exact required-status tests;
|
|
- registry, boundary, fixture, fault, and readiness verification.
|
|
|
|
### Frontend consumer
|
|
|
|
- manifest validation with released platform binaries;
|
|
- existing focused product tests;
|
|
- platform shadow run against the actual repository;
|
|
- semantic parity and digest reports;
|
|
- one-writer and rollback drills;
|
|
- removal tests proving the template works without copied workflow or local
|
|
policy engines.
|
|
|
|
## Acceptance criteria
|
|
|
|
- Test selection and assessment have one authority: test assurance.
|
|
- CI orchestration and promotion have one authority: CI/CD.
|
|
- The frontend repository contains product tests and declarations, not copied
|
|
platform engines.
|
|
- Every required frontend capability has independent readiness evidence.
|
|
- Source and artifact test evidence bind the exact revision and candidate.
|
|
- The promoted static site is byte-identical to the assessed, signed candidate.
|
|
- Missing evidence, unsupported capability, platform failure, and provider
|
|
uncertainty cannot produce a passing required status.
|
|
- Cutover and rollback require no copied workflow logic in the product
|
|
repository.
|