문제: merge job 셋은 runner의 새 컨테이너에서 매번 Node(GitHub 다운로드), corepack의 pnpm, 패키지 404개를 내려받고, pnpm 11은 lockfile 542개 항목의 공급망 정책을 레지스트리에 물어 검증한다. run 169에서 job마다 setup-node 11~18초, 설치 12~29초였고 설치 시간의 대부분은 정책 검증의 메타데이터 요청이었다(10.1~27.9초). FE-GATE-013의 시스템 컨테이너는 같은 설치를 한 번 더 한다. merge 25분 목표에 0.8분 모자랐다. 고친 것(권하는 안으로 정함 2026-11-13, 총괄): - job 단계 `dependency-cache`를 더한다. setup-node 앞에서 Node toolcache와 `/tmp/fe-dependency-cache`(corepack, pnpm store, pnpm 메타데이터 캐시)를 `.nvmrc`·`pnpm-lock.yaml` 해시 키로 복원한다. main run은 actions/cache로 복원·저장하고, 그 밖의 ref는 actions/cache/restore로 복원만 한다. pin은 짝 백엔드의 JDK·Gradle·Trivy 캐시와 같다(v4.3.0). - 생성기가 그 job에 `COREPACK_HOME`·`PNPM_CONFIG_STORE_DIR`· `PNPM_CONFIG_CACHE_DIR`를 주어 job의 모든 pnpm 호출이 같은 곳을 본다. frozen install 단계는 그대로이고 캐시와 관계없이 돈다. - PR run은 main이 저장한 항목을 같은 키로 읽기만 한다. 저장할 수 있는 actions/cache 단계는 `refs/heads/main` 조건 아래에만 있고, 그 밖의 ref가 타는 actions/cache/restore에는 저장하는 post 단계가 없다(시험으로 고정). - 계약: merge job 셋만 이 단계를 가진다(release·promotion·provider·main job은 거절, 단계 순서 고정에 반영). 그 env를 job이 따로 묶으면 거절한다. 키 입력은 `config/repository-layout.json`의 새 집합 `dependency-cache-key`(`.nvmrc`, `pnpm-lock.yaml`)에서 고른다. - `scripts/run-in-system-container.sh`가 그 env가 있으면 store와 메타데이터 캐시를 같은 경로로 컨테이너에 넘긴다. 없으면(desktop) 지금과 같다. - 결정 색인, ci-quality-gates.md, O1 출처 문구를 총괄이 정한 말로 맞춘다. 버린 대안: - CI 이미지에 의존성 굽기: 레지스트리 쓰기 토큰이 든다. - 브랜치 run도 저장하기: 캐시 항목은 다음 run이 믿는 입력이다. pnpm은 같은 lockfile의 공급망 검증 기록이 캐시에 있으면 다시 검증하지 않으므로, 브랜치가 쓴 항목이 main의 검증을 건너뛰게 할 수 있다. - `cache-hit`일 때 설치를 건너뛰기: 결과가 캐시 상태에 달린다. - apt 글꼴 캐시: 캐시가 없을 때 새 패키지 버전이 깔려 시각 결과가 달라질 수 있다. 위험·한계: - PR run은 main이 같은 키(또는 restore-keys로 앞 항목)를 저장한 뒤에야 hit한다. 이 PR 자신의 run은 비어서 시작한다. - runner 캐시 저장소(5GB)는 짝 백엔드와 같이 쓴다. 항목 하나는 Node·store· 메타데이터로 압축 전 약 0.7GB다. lockfile이 바뀔 때마다 새 항목이 생긴다. - Node toolcache 경로(`/opt/hostedtoolcache`)가 job 이미지와 다르면 그 부분만 효과가 없고 setup-node가 내려받는다. 검증(desktop): - 설치 실험(같은 lockfile, 새 폴더): 빈 캐시 13.9·17.2초, 캐시 있음 1.6초, store 파일 40개·메타데이터 40개를 망가뜨린 캐시 3.6초(27개만 다시 받음). 다섯 번 모두 node_modules 트리 해시가 같다. pnpm 11은 대문자 `PNPM_CONFIG_STORE_DIR`·`PNPM_CONFIG_CACHE_DIR`를 읽는다. - FE-GATE-013 흐름을 캐시 env로 재현: 빈 캐시 설치 13.7초, 캐시 설치 2.6초, 시스템 컨테이너 안 설치 1.5초(`Recreating node_modules` 없음, 지금은 31.3초). 그 실행의 FE-GATE-013은 provider 묶음 시험 하나가 손 timeout 30초에 걸려 실패했다(캐시와 무관, PR #39가 고치는 문제). - RED: 계약·생성기를 둔 채 gates.json에서 캐시 단계를 빼면 새 시험 6개 실패. 키 입력을 코드에 직접 적은 첫 판은 repository-layout R2 시험이 잡았다. GREEN: repository-layout·ci-workflow-generation 456개 통과, check:ci·check:types·eslint rc=0, verify:documentation rc=0, 노트북 `scripts/verify.sh docs` OK.
786 lines
32 KiB
TypeScript
786 lines
32 KiB
TypeScript
import { randomUUID } from "node:crypto";
|
|
import { constants, type Stats } from "node:fs";
|
|
import {
|
|
mkdir,
|
|
open,
|
|
readFile,
|
|
rename,
|
|
rm,
|
|
} from "node:fs/promises";
|
|
import { fileURLToPath } from "node:url";
|
|
import path from "node:path";
|
|
|
|
import {
|
|
DEPENDENCY_CACHE,
|
|
indexCiGateContract,
|
|
loadCiGateContract,
|
|
resolveCiStepActionUses,
|
|
resolveCiSubActionUses,
|
|
type CiGateContract,
|
|
type CiWorkflowJob,
|
|
type CiWorkflowStep,
|
|
} from "./contracts/ci-gates.ts";
|
|
import { loadProductCiOverlay, type ProductGate } from "./contracts/product-ci-overlay.ts";
|
|
import {
|
|
assertSafeExistingPublishPath,
|
|
assertSafePublishLeaf,
|
|
ensureSafePublishDirectory,
|
|
} from "./lib/ci-gate-log.ts";
|
|
import { VISUAL_TEST_FONT_INSTALL_ARGUMENTS } from "./lib/visual-test-fonts.ts";
|
|
|
|
export type GenerateCiWorkflowOptions = Readonly<{
|
|
root: string;
|
|
contract?: CiGateContract;
|
|
/** Product merge gates; read from the root's config/ci/product-gates.json when omitted. */
|
|
productGates?: readonly ProductGate[];
|
|
check: boolean;
|
|
}>;
|
|
|
|
export type GenerateCiWorkflowResult = Readonly<{
|
|
target: string;
|
|
written: boolean;
|
|
matches: boolean;
|
|
firstDifferenceByte: number | null;
|
|
firstDifferenceLine: number | null;
|
|
}>;
|
|
|
|
export type CiWorkflowFileSystem = Readonly<{
|
|
mkdir(directory: string): Promise<unknown>;
|
|
readFile(target: string): Promise<Buffer>;
|
|
open(target: string, flags: number, mode: number): Promise<{
|
|
writeFile(content: string, encoding: "utf8"): Promise<unknown>;
|
|
chmod(mode: number): Promise<unknown>;
|
|
sync(): Promise<unknown>;
|
|
close(): Promise<unknown>;
|
|
}>;
|
|
openDirectory(target: string): Promise<{ sync(): Promise<unknown>; close(): Promise<unknown> }>;
|
|
rename(source: string, destination: string): Promise<unknown>;
|
|
rm(target: string): Promise<unknown>;
|
|
}>;
|
|
|
|
const defaultFileSystem: CiWorkflowFileSystem = Object.freeze({
|
|
mkdir: async (directory) => mkdir(directory, { recursive: true }),
|
|
readFile: async (target) => readFile(target),
|
|
open: async (target, flags, mode) => open(target, flags, mode),
|
|
openDirectory: async (target) => open(target, constants.O_RDONLY),
|
|
rename: async (source, destination) => rename(source, destination),
|
|
rm: async (target) => rm(target, { force: true }),
|
|
});
|
|
|
|
const PRODUCT_GATE_JOB = "product_merge_gate";
|
|
/**
|
|
* 1105/13-02. The job that waits for the product gates. A failing product
|
|
* gate stopped nothing but a PR merge: the same run went on to release, build,
|
|
* promote and deploy, and the fixed job graph left a product no way to link
|
|
* them. release_gate is the first release-stage job, and every later one
|
|
* depends on it as it depends on the merge jobs. The link is rendered from the
|
|
* overlay, so the template contract's job graph (and its check) stays the
|
|
* template's, and without product gates the workflow bytes do not change.
|
|
*/
|
|
const PRODUCT_GATES_HOLD = "release_gate";
|
|
|
|
/**
|
|
* A newer run in the same group cancels the older one. Gitea 1.27 cancels the
|
|
* older run's waiting and blocked jobs always and its running jobs only when
|
|
* cancel-in-progress is true. A pull request run has the ref
|
|
* `refs/pull/<number>/head`, so one group per pull request; a branch push is
|
|
* one group per branch; a release tag has its own ref. A manual run is one
|
|
* group per ref and stage and does not cancel a running run: a later release
|
|
* dispatch must not stop an earlier one between verification and promotion.
|
|
*/
|
|
const CONCURRENCY_LINES = Object.freeze([
|
|
"concurrency:",
|
|
" group: frontend-quality-gates-${{ gitea.event_name }}-${{ gitea.ref }}-${{ inputs.stage }}",
|
|
" cancel-in-progress: ${{ gitea.event_name != 'workflow_dispatch' }}",
|
|
]);
|
|
|
|
export function renderCiWorkflow(
|
|
contract: CiGateContract,
|
|
productGates: readonly ProductGate[] = [],
|
|
): string {
|
|
const index = indexCiGateContract(contract);
|
|
const transfers = new Map<string, { name: string }>();
|
|
for (const job of contract.jobs) {
|
|
for (const step of job.steps) {
|
|
if (step.kind === "upload") transfers.set(step.transferId, { name: step.name });
|
|
}
|
|
}
|
|
// 1105/13-02. With product gates, the release stage waits for them too.
|
|
const productNeeds = productGates.length > 0 ? [PRODUCT_GATE_JOB] : [];
|
|
const lines = [
|
|
"# GENERATED FILE — edit config/ci/gates.json and run `corepack pnpm generate:ci-workflow`.",
|
|
"name: frontend-quality-gates",
|
|
"",
|
|
"on:",
|
|
" push:",
|
|
" branches: [main, develop]",
|
|
' tags: ["v*"]',
|
|
" pull_request:",
|
|
" workflow_dispatch:",
|
|
" inputs:",
|
|
" stage:",
|
|
" description: Highest promotion tier to evaluate",
|
|
" required: true",
|
|
" default: merge",
|
|
" type: choice",
|
|
" options:",
|
|
" - merge",
|
|
" - main",
|
|
" - release",
|
|
" - production",
|
|
" - field",
|
|
" - documentation",
|
|
"",
|
|
"permissions:",
|
|
" contents: read",
|
|
"",
|
|
...CONCURRENCY_LINES,
|
|
"",
|
|
"env:",
|
|
' CI: "true"',
|
|
' VITE_BUILD_ID: "gitea-${{ gitea.run_id }}-${{ gitea.run_attempt }}"',
|
|
' VITE_COMMIT_SHA: "${{ gitea.sha }}"',
|
|
' RELEASE_ID: "${{ gitea.ref }}-${{ gitea.run_id }}-${{ gitea.run_attempt }}"',
|
|
' CI_RUNNER_IMAGE: "${{ vars.RUNNER_IMAGE_DIGEST }}"',
|
|
"",
|
|
"jobs:",
|
|
];
|
|
for (const [jobIndex, job] of contract.jobs.entries()) {
|
|
if (jobIndex > 0) lines.push("");
|
|
lines.push(...renderJob(job, index, transfers, job.id === PRODUCT_GATES_HOLD ? productNeeds : []));
|
|
}
|
|
if (productGates.length > 0) lines.push("", ...renderProductGateJob(productGates));
|
|
return `${lines.join("\n").replace(/\n+$/u, "")}\n`;
|
|
}
|
|
|
|
/**
|
|
* 1103/12-07. The product's merge gates (config/ci/product-gates.json) run in
|
|
* a job after every template job, on the same merge condition. A product gate
|
|
* is one package script; it has no log artifact or evidence of the template
|
|
* gate contract and is not part of its readiness formula. The release stage
|
|
* waits for this job (1105/13-02); the product makes it a required check where
|
|
* it wants it to block a merge.
|
|
*/
|
|
function renderProductGateJob(productGates: readonly ProductGate[]): string[] {
|
|
return [
|
|
` ${PRODUCT_GATE_JOB}:`,
|
|
` name: ${yamlScalar("${{ matrix.gate }} / ${{ matrix.name }}")}`,
|
|
` if: ${renderCondition("merge")}`,
|
|
" runs-on: ubuntu-latest",
|
|
" timeout-minutes: 45",
|
|
" strategy:",
|
|
" fail-fast: false",
|
|
" matrix:",
|
|
" include:",
|
|
...productGates.map(
|
|
(gate) => ` - { gate: ${gate.id}, name: ${gate.name}, script: ${yamlScalar(gate.script)} }`,
|
|
),
|
|
" steps:",
|
|
` - uses: ${requiredStepActionUses("checkout")}`,
|
|
" with:",
|
|
" persist-credentials: false",
|
|
` - uses: ${requiredStepActionUses("setup-node")}`,
|
|
" with:",
|
|
" node-version-file: .nvmrc",
|
|
" - name: Frozen install",
|
|
" run: |",
|
|
" corepack enable",
|
|
" corepack pnpm install --frozen-lockfile --ignore-scripts",
|
|
// The overlay limits the script to [A-Za-z0-9:_-], a plain shell word.
|
|
" - name: Run product gate",
|
|
" run: corepack pnpm ${{ matrix.script }}",
|
|
];
|
|
}
|
|
|
|
function renderJob(
|
|
job: CiWorkflowJob,
|
|
index: ReturnType<typeof indexCiGateContract>,
|
|
transfers: ReadonlyMap<string, Readonly<{ name: string }>>,
|
|
productNeeds: readonly string[],
|
|
): string[] {
|
|
const lines = [` ${yamlKey(job.id)}:`, ` name: ${yamlScalar(job.displayName)}`];
|
|
const needs = [...job.needs, ...productNeeds];
|
|
if (needs.length === 1) lines.push(` needs: ${yamlKey(needs[0]!)}`);
|
|
if (needs.length > 1) lines.push(` needs: [${needs.map(yamlKey).join(", ")}]`);
|
|
const condition = renderCondition(job.condition);
|
|
if (condition) lines.push(` if: ${condition}`);
|
|
lines.push(
|
|
...renderHostRequirements(job, index),
|
|
" runs-on: ubuntu-latest",
|
|
` timeout-minutes: ${job.timeoutMinutes}`,
|
|
);
|
|
const image = jobImage(job);
|
|
if (image) lines.push(" container:", ` image: ${yamlScalar(image)}`);
|
|
if (job.kind === "immutable") {
|
|
const archive = job.steps.find((step) => step.kind === "archive-candidate");
|
|
if (!archive || archive.kind !== "archive-candidate") throw new TypeError("immutable job lacks archive step");
|
|
lines.push(
|
|
" outputs:",
|
|
` ${archive.distOutputName}: \${{ steps.${archive.stepId}.outputs.${archive.distOutputName} }}`,
|
|
` ${archive.archiveOutputName}: \${{ steps.${archive.stepId}.outputs.${archive.archiveOutputName} }}`,
|
|
);
|
|
}
|
|
if (job.kind === "provider") {
|
|
const supervisor = job.steps.find((step) => step.kind === "run-provider");
|
|
if (!supervisor || supervisor.kind !== "run-provider") {
|
|
throw new TypeError("provider job lacks supervisor step");
|
|
}
|
|
lines.push(
|
|
" outputs:",
|
|
` invocation_nonce: \${{ steps.${supervisor.stepId}.outputs.invocation_nonce }}`,
|
|
);
|
|
}
|
|
if (job.kind === "deployment") {
|
|
lines.push(" outputs:", " deployed: ${{ steps.deploy.outputs.deployed }}");
|
|
}
|
|
const jobEnvironment = job.environment.filter((binding) => !isStepScopedSecret(job, binding));
|
|
// The build records the image it ran in (scripts/lib/build-environment.ts):
|
|
// a job in its own image names that one, not the runner label's.
|
|
// Every pnpm and corepack call of the job reads the cached directories, the
|
|
// install and the system container's own install alike.
|
|
const cacheEnvironment = job.steps.some(({ kind }) => kind === "dependency-cache")
|
|
? DEPENDENCY_CACHE.environment
|
|
: [];
|
|
const environmentLines = [
|
|
...(image ? [` CI_RUNNER_IMAGE: ${yamlScalar(image)}`] : []),
|
|
...[...cacheEnvironment, ...jobEnvironment].map(
|
|
(binding) => ` ${binding.name}: ${yamlScalar(binding.value)}`,
|
|
),
|
|
];
|
|
if (environmentLines.length > 0) lines.push(" env:", ...environmentLines);
|
|
if (job.kind === "gate-matrix") {
|
|
lines.push(" strategy:", " fail-fast: false", " matrix:", " include:");
|
|
const includesBrowser = job.steps.some(({ kind }) => kind === "browser-install");
|
|
for (const gateId of job.gateIds) {
|
|
const gate = index.gates.get(gateId);
|
|
if (!gate) throw new TypeError(`unknown gate while rendering: ${gateId}`);
|
|
const browser = job.browserGateIds.includes(gateId);
|
|
lines.push(
|
|
` - { gate: ${gateId}, name: ${gate.name}${includesBrowser ? `, browser: ${browser ? "true" : "false"}` : ""} }`,
|
|
);
|
|
}
|
|
}
|
|
lines.push(" steps:");
|
|
for (const step of job.steps) lines.push(...renderStep(job, step, transfers, index));
|
|
return lines;
|
|
}
|
|
|
|
/**
|
|
* 1105/12-01. A gate's host requirements (`hostRequirements` in
|
|
* config/ci/gates.json) sit as comments beside the runner label of the job
|
|
* that runs the gate, where whoever picks the runner reads them. FE-GATE-013
|
|
* needs unprivileged user namespaces, and that was written nowhere a runner
|
|
* is chosen. The contract keeps line breaks out of each requirement.
|
|
*/
|
|
function renderHostRequirements(
|
|
job: CiWorkflowJob,
|
|
index: ReturnType<typeof indexCiGateContract>,
|
|
): string[] {
|
|
return [
|
|
...job.gateIds.flatMap((gateId) =>
|
|
(index.gates.get(gateId)?.hostRequirements ?? []).map(
|
|
(requirement) => ` # ${gateId} runner requirement: ${requirement}`,
|
|
),
|
|
),
|
|
// 1101/13-03. A provider job runs the provider sandbox and has no gate.
|
|
...(job.hostRequirements ?? []).map(
|
|
(requirement) => ` # provider sandbox runner requirement: ${requirement}`,
|
|
),
|
|
];
|
|
}
|
|
|
|
/**
|
|
* A gate whose host requirements the job container cannot meet runs in the
|
|
* environment its contract names (`hostEnvironment`). FE-GATE-013's provider
|
|
* sandbox needs systemd and bubblewrap, which the runner's job image lacks.
|
|
*/
|
|
function gateHostPrefix(gate: Readonly<{ hostEnvironment?: "system-container" | undefined }> | undefined): string {
|
|
return gate?.hostEnvironment === "system-container" ? "scripts/run-in-system-container.sh " : "";
|
|
}
|
|
|
|
/** The Playwright image a browser job runs in (`browser-install.image`), if any. */
|
|
function jobImage(job: CiWorkflowJob): string | undefined {
|
|
for (const step of job.steps) {
|
|
if (step.kind === "browser-install" && step.image) return step.image;
|
|
}
|
|
return undefined;
|
|
}
|
|
|
|
function renderStep(
|
|
job: CiWorkflowJob,
|
|
step: CiWorkflowStep,
|
|
transfers: ReadonlyMap<string, Readonly<{ name: string }>>,
|
|
index: ReturnType<typeof indexCiGateContract>,
|
|
): string[] {
|
|
switch (step.kind) {
|
|
case "checkout":
|
|
return [
|
|
` - uses: ${requiredStepActionUses(step.kind)}`,
|
|
" with:",
|
|
" persist-credentials: false",
|
|
];
|
|
case "setup-node":
|
|
return [
|
|
` - uses: ${requiredStepActionUses(step.kind)}`,
|
|
" with:",
|
|
" node-version-file: .nvmrc",
|
|
];
|
|
case "dependency-cache": {
|
|
// DEPENDENCY_CACHE: main restores and saves, every other ref restores.
|
|
// The frozen install after it runs either way.
|
|
const inputs = DEPENDENCY_CACHE.keyInputs.map((file) => `'${file}'`).join(", ");
|
|
const cache = [
|
|
" with:",
|
|
" path: |",
|
|
...DEPENDENCY_CACHE.paths.map((cachePath) => ` ${cachePath}`),
|
|
` key: ${DEPENDENCY_CACHE.keyPrefix}\${{ hashFiles(${inputs}) }}`,
|
|
" restore-keys: |",
|
|
` ${DEPENDENCY_CACHE.keyPrefix}`,
|
|
];
|
|
return [
|
|
" - name: Restore and save the dependency cache",
|
|
" if: ${{ gitea.ref == 'refs/heads/main' }}",
|
|
` uses: ${requiredStepActionUses(step.kind)}`,
|
|
...cache,
|
|
" - name: Restore the dependency cache",
|
|
" if: ${{ gitea.ref != 'refs/heads/main' }}",
|
|
` uses: ${resolveCiSubActionUses("cache", "restore")}`,
|
|
...cache,
|
|
];
|
|
}
|
|
case "frozen-install":
|
|
return [
|
|
" - name: Frozen install",
|
|
" run: |",
|
|
" corepack enable",
|
|
" corepack pnpm install --frozen-lockfile --ignore-scripts",
|
|
];
|
|
case "browser-install":
|
|
if (step.image) {
|
|
// The job image holds the browsers and their OS libraries; the goldens'
|
|
// faces are still OS packages it lacks, and it ships no package lists.
|
|
return [
|
|
" - name: Install visual-test fonts",
|
|
" run: |",
|
|
" $(command -v sudo || true) apt-get update",
|
|
` $(command -v sudo || true) ${VISUAL_TEST_FONT_INSTALL_ARGUMENTS}`,
|
|
];
|
|
}
|
|
return [
|
|
" - name: Install Playwright browsers",
|
|
...(job.kind === "gate-matrix" ? [" if: ${{ matrix.browser }}"] : []),
|
|
" run: |",
|
|
" corepack pnpm exec playwright install --with-deps chromium firefox webkit",
|
|
// 1103/11-02. The visual goldens' faces (tests/visual/fonts.conf) are
|
|
// not among Playwright's dependencies. `--with-deps` has refreshed the
|
|
// package lists; sudo only where the runner is not root.
|
|
` $(command -v sudo || true) ${VISUAL_TEST_FONT_INSTALL_ARGUMENTS}`,
|
|
];
|
|
case "run-gate": {
|
|
if (job.kind === "gate-matrix") {
|
|
return [" - name: Run blocking gate", " run: corepack pnpm ci:gate -- ${{ matrix.gate }}"];
|
|
}
|
|
if (job.gateIds.length === 0) throw new TypeError(`run-gate step lacks ownership: ${job.id}`);
|
|
// F2·F1 (design-docs repo-structure/03-ci.md). A job of several gates
|
|
// shares one install; each gate is a step named by its id, and a failed
|
|
// gate does not skip the ones after it, as the matrix entries did not.
|
|
if (job.kind === "gate-sequence") {
|
|
return job.gateIds.flatMap((gateId) => [
|
|
` - name: ${yamlScalar(`${gateId} / ${index.gates.get(gateId)?.name ?? ""}`)}`,
|
|
" if: ${{ !cancelled() }}",
|
|
` run: ${gateHostPrefix(index.gates.get(gateId))}corepack pnpm ci:gate -- ${gateId}`,
|
|
]);
|
|
}
|
|
// 12-02: every gate the job owns runs, in order. Rendering only the first
|
|
// left FE-GATE-027 (release admission) out of the workflow.
|
|
return job.gateIds.flatMap((gateId) => [
|
|
` - name: ${runGateStepName(job.id, gateId)}`,
|
|
` run: corepack pnpm ci:gate -- ${gateId}`,
|
|
]);
|
|
}
|
|
case "archive-candidate": {
|
|
const archive = shellDoubleQuoted(step.archivePath);
|
|
const lines = [
|
|
" - name: Archive and validate the exact candidate file set",
|
|
` id: ${yamlKey(step.stepId)}`,
|
|
" run: |",
|
|
" mkdir -p .release",
|
|
` tar --sort=name --mtime="@0" --owner=0 --group=0 --numeric-owner -czf ${archive} \\`,
|
|
];
|
|
step.members.forEach((member, memberIndex) => {
|
|
lines.push(` ${shellWord(member)}${memberIndex === step.members.length - 1 ? "" : " \\"}`);
|
|
});
|
|
lines.push(
|
|
` node scripts/verify-ci-candidate-archive.ts --archive ${archive} --github-output "$GITHUB_OUTPUT"`,
|
|
);
|
|
return lines;
|
|
}
|
|
case "download":
|
|
return [
|
|
` - name: Download ${humanize(step.transferId)}`,
|
|
` uses: ${requiredStepActionUses(step.kind)}`,
|
|
" with:",
|
|
` name: ${yamlScalar(requiredTransferName(transfers, step.transferId))}`,
|
|
` path: ${yamlScalar(step.path)}`,
|
|
];
|
|
case "extract":
|
|
return [
|
|
" - name: Verify and extract the candidate through one inode-bound operation",
|
|
` run: node scripts/verify-ci-candidate-archive.ts --archive ${shellDoubleQuoted(step.archivePath)} --extract-to ${shellDoubleQuoted(step.targetRoot)}`,
|
|
];
|
|
case "run-provider": {
|
|
return [
|
|
` - name: Run and validate external ${step.provider} provider in one trusted supervisor`,
|
|
` id: ${yamlKey(step.stepId)}`,
|
|
` run: node scripts/run-and-validate-provider.ts --kind ${step.provider}`,
|
|
];
|
|
}
|
|
case "validate-provider-evidence":
|
|
return [
|
|
` - name: Confirm sealed ${step.provider} provider evidence`,
|
|
' run: test -s "$VALIDATED_PROVIDER_REPORT_PATH"',
|
|
];
|
|
case "verify-promotion":
|
|
return [
|
|
" - name: Finalize verified promotion from inode-bound captured inputs",
|
|
` id: ${yamlKey(step.stepId)}`,
|
|
" run: node scripts/stage-verified-promotion.ts",
|
|
];
|
|
case "cleanup-promotion":
|
|
return [
|
|
" - name: Always remove private promotion staging",
|
|
" if: always()",
|
|
" env:",
|
|
` PROMOTION_STAGING_ROOT: \${{ steps.${step.finalizerStepId}.outputs.staging_root }}`,
|
|
` PROMOTION_CLEANUP_TOKEN: \${{ steps.${step.finalizerStepId}.outputs.cleanup_token }}`,
|
|
` PROMOTION_RUNNER_TEMP_DEV: \${{ steps.${step.finalizerStepId}.outputs.runner_temp_dev }}`,
|
|
` PROMOTION_RUNNER_TEMP_INO: \${{ steps.${step.finalizerStepId}.outputs.runner_temp_ino }}`,
|
|
` PROMOTION_STAGING_DEV: \${{ steps.${step.finalizerStepId}.outputs.staging_dev }}`,
|
|
` PROMOTION_STAGING_INO: \${{ steps.${step.finalizerStepId}.outputs.staging_ino }}`,
|
|
" run: |",
|
|
' if [ -n "$PROMOTION_STAGING_ROOT" ] && [ -n "$PROMOTION_CLEANUP_TOKEN" ] && [ -n "$PROMOTION_RUNNER_TEMP_DEV" ] && [ -n "$PROMOTION_RUNNER_TEMP_INO" ] && [ -n "$PROMOTION_STAGING_DEV" ] && [ -n "$PROMOTION_STAGING_INO" ]; then',
|
|
" node scripts/cleanup-verified-promotion.ts",
|
|
" fi",
|
|
];
|
|
// 1101/13-01. `run` fails on the first failing command, so deployed=true
|
|
// is written only after the project's script succeeded.
|
|
case "deploy": {
|
|
const secrets = job.environment.filter((binding) => isStepScopedSecret(job, binding));
|
|
return step.script
|
|
? [
|
|
" - name: Deploy the promoted release",
|
|
" id: deploy",
|
|
...(secrets.length > 0
|
|
? [" env:", ...secrets.map((binding) => ` ${binding.name}: ${yamlScalar(binding.value)}`)]
|
|
: []),
|
|
" run: |",
|
|
// The contract limits the name to [A-Za-z0-9:_-], a plain shell word.
|
|
` corepack pnpm ${step.script}`,
|
|
' echo "deployed=true" >> "$GITHUB_OUTPUT"',
|
|
]
|
|
: [
|
|
" - name: Deployment slot is empty",
|
|
" id: deploy",
|
|
' run: echo "deployed=false" >> "$GITHUB_OUTPUT"',
|
|
];
|
|
}
|
|
case "upload": {
|
|
const lines = [
|
|
` - name: Upload ${humanize(step.transferId)}`,
|
|
...(step.always ? [" if: always()"] : []),
|
|
` uses: ${requiredStepActionUses(step.kind)}`,
|
|
" with:",
|
|
` name: ${yamlScalar(step.name)}`,
|
|
];
|
|
if (step.paths.length === 1) lines.push(` path: ${yamlScalar(step.paths[0]!)}`);
|
|
else {
|
|
lines.push(" path: |");
|
|
for (const target of step.paths) lines.push(` ${target}`);
|
|
}
|
|
lines.push(" if-no-files-found: error");
|
|
return lines;
|
|
}
|
|
}
|
|
}
|
|
|
|
/**
|
|
* 1103/13-05. Job env reaches every step of the job: checkout, setup-node and
|
|
* the third-party download action as well. In the deployment slot a
|
|
* `secrets.*` binding is rendered on the deploy step instead, the one step
|
|
* that uses it.
|
|
*/
|
|
function isStepScopedSecret(job: CiWorkflowJob, binding: CiWorkflowJob["environment"][number]): boolean {
|
|
return job.kind === "deployment" && /^\$\{\{ secrets\./u.test(binding.value);
|
|
}
|
|
|
|
function requiredStepActionUses(stepKind: string): string {
|
|
const uses = resolveCiStepActionUses(stepKind);
|
|
if (!uses) throw new TypeError(`workflow step has no registered CI action: ${stepKind}`);
|
|
return uses;
|
|
}
|
|
|
|
function renderCondition(condition: CiWorkflowJob["condition"]): string | null {
|
|
const expressions: Record<CiWorkflowJob["condition"], string | null> = {
|
|
always: null,
|
|
"needs-success": null,
|
|
merge: "${{ gitea.event_name != 'workflow_dispatch' || (inputs.stage != 'documentation' && inputs.stage != 'main') }}",
|
|
// A push to a branch (main, develop) or a dispatch that asks for it; never
|
|
// a pull request or a tag.
|
|
main: "${{ (gitea.event_name == 'push' && startsWith(gitea.ref, 'refs/heads/')) || (gitea.event_name == 'workflow_dispatch' && inputs.stage == 'main') }}",
|
|
release: "${{ startsWith(gitea.ref, 'refs/tags/v') || (gitea.event_name == 'workflow_dispatch' && (inputs.stage == 'release' || inputs.stage == 'production' || inputs.stage == 'field')) }}",
|
|
production: "${{ gitea.event_name == 'workflow_dispatch' && (inputs.stage == 'production' || inputs.stage == 'field') }}",
|
|
field: "${{ gitea.event_name == 'workflow_dispatch' && inputs.stage == 'field' }}",
|
|
documentation: "${{ gitea.event_name == 'workflow_dispatch' && inputs.stage == 'documentation' }}",
|
|
};
|
|
return expressions[condition];
|
|
}
|
|
|
|
function requiredTransferName(
|
|
transfers: ReadonlyMap<string, Readonly<{ name: string }>>,
|
|
transferId: string,
|
|
): string {
|
|
const transfer = transfers.get(transferId);
|
|
if (!transfer) throw new TypeError(`download transfer has no typed producer: ${transferId}`);
|
|
return transfer.name;
|
|
}
|
|
|
|
function yamlKey(value: string): string {
|
|
if (!/^[A-Za-z0-9_-]+$/u.test(value)) throw new TypeError(`unsafe YAML key: ${value}`);
|
|
return value;
|
|
}
|
|
|
|
function yamlScalar(value: string): string {
|
|
if (/^[A-Za-z0-9._/-]+$/u.test(value)) return value;
|
|
return `"${value.replaceAll("\\", "\\\\").replaceAll('"', '\\"').replaceAll("\n", "\\n")}"`;
|
|
}
|
|
|
|
function shellWord(value: string): string {
|
|
if (!/^[A-Za-z0-9._/-]+$/u.test(value)) throw new TypeError(`unsafe shell word: ${value}`);
|
|
return value;
|
|
}
|
|
|
|
function shellDoubleQuoted(value: string): string {
|
|
const expressions: string[] = [];
|
|
const withoutExpressions = value.replace(/\$\{\{ [A-Za-z0-9_.-]+ \}\}/gu, (expression) => {
|
|
expressions.push(expression);
|
|
return `__CI_EXPRESSION_${expressions.length - 1}__`;
|
|
});
|
|
if (withoutExpressions.includes("$")) {
|
|
throw new TypeError(`unapproved shell interpolation in workflow value: ${value}`);
|
|
}
|
|
let escaped = withoutExpressions
|
|
.replaceAll("\\", "\\\\")
|
|
.replaceAll('"', '\\"')
|
|
.replaceAll("`", "\\`");
|
|
expressions.forEach((expression, index) => {
|
|
escaped = escaped.replace(`__CI_EXPRESSION_${index}__`, expression);
|
|
});
|
|
return `"${escaped}"`;
|
|
}
|
|
|
|
function runGateStepName(jobId: string, gateId: string): string {
|
|
if (gateId === "FE-GATE-015") return "Build candidate once and verify local evidence";
|
|
if (gateId === "FE-GATE-027") return "Admit the candidate to the declared release target";
|
|
return jobId === "documentation_gate" ? "Run documentation gate" : "Run blocking gate";
|
|
}
|
|
|
|
/**
|
|
* 12-02 / D-12. Every contract gate must be executed by the workflow text: a
|
|
* literal `ci:gate -- <id>` run line, or a matrix entry of a job whose run step
|
|
* is `ci:gate -- ${{ matrix.gate }}`. Owning a gate in the contract is not
|
|
* enough; the generator once dropped all but the first.
|
|
*/
|
|
export function workflowGateExecutionFailures(
|
|
contract: CiGateContract,
|
|
workflow: string,
|
|
productGates: readonly ProductGate[] = [],
|
|
): string[] {
|
|
const executed = new Set<string>();
|
|
const jobs = workflow.split(/^(?= {2}[a-z_]+:$)/mu);
|
|
const productJob = jobs.find((job) => job.startsWith(` ${PRODUCT_GATE_JOB}:\n`)) ?? "";
|
|
const productRuns = /^ {8}run: corepack pnpm \$\{\{ matrix\.script \}\}$/mu.test(productJob);
|
|
const productFailures = productGates
|
|
.filter(
|
|
(gate) =>
|
|
!productRuns ||
|
|
!productJob.includes(
|
|
` - { gate: ${gate.id}, name: ${gate.name}, script: ${yamlScalar(gate.script)} }\n`,
|
|
),
|
|
)
|
|
.map(({ id }) => `product gate is not executed by the ${PRODUCT_GATE_JOB} job: ${id}`);
|
|
// 1105/13-02. Run is not enough: a failing product gate must also hold the
|
|
// release stage.
|
|
const holdingJob = jobs.find((job) => job.startsWith(` ${PRODUCT_GATES_HOLD}:\n`)) ?? "";
|
|
const heldBy = /^ {4}needs: (?:\[([^\]\n]*)\]|(\S+))$/mu.exec(holdingJob);
|
|
if (
|
|
productGates.length > 0 &&
|
|
!(heldBy?.[1] ?? heldBy?.[2] ?? "").split(", ").includes(PRODUCT_GATE_JOB)
|
|
) {
|
|
productFailures.push(`product gates do not hold the release stage: ${PRODUCT_GATES_HOLD} does not need ${PRODUCT_GATE_JOB}`);
|
|
}
|
|
const hostPrefixes = new Map(contract.gates.map((gate) => [gate.id, gateHostPrefix(gate)]));
|
|
for (const job of jobs) {
|
|
for (const match of job.matchAll(/^ +run: (\S+ )?corepack pnpm ci:gate -- (FE-GATE-\d{3})$/gmu)) {
|
|
// A gate runs through the prefix its contract names, and only that one.
|
|
if ((match[1] ?? "") === hostPrefixes.get(match[2]!)) executed.add(match[2]!);
|
|
}
|
|
if (/^ +run: corepack pnpm ci:gate -- \$\{\{ matrix\.gate \}\}$/mu.test(job)) {
|
|
for (const match of job.matchAll(/^ +- \{ gate: (FE-GATE-\d{3}),/gmu)) {
|
|
executed.add(match[1]!);
|
|
}
|
|
}
|
|
}
|
|
return [
|
|
...contract.gates
|
|
.filter(({ id }) => !executed.has(id))
|
|
.map(({ id }) => `gate is not executed by any workflow run step: ${id}`),
|
|
...productFailures,
|
|
];
|
|
}
|
|
|
|
function humanize(value: string): string {
|
|
return value.replaceAll("-", " ");
|
|
}
|
|
|
|
export function createCiWorkflowGenerator(
|
|
dependencies: Readonly<{
|
|
fileSystem?: CiWorkflowFileSystem;
|
|
createNonce?: () => string;
|
|
}> = {},
|
|
) {
|
|
const fileSystem = dependencies.fileSystem ?? defaultFileSystem;
|
|
const createNonce = dependencies.createNonce ?? randomUUID;
|
|
return async function generate(options: GenerateCiWorkflowOptions): Promise<GenerateCiWorkflowResult> {
|
|
const root = path.resolve(options.root);
|
|
const contract = options.contract ?? (await loadCiGateContract(root));
|
|
const target = path.resolve(root, contract.providerAdapter);
|
|
if (path.relative(root, target).startsWith("..") || path.relative(root, target) === "") {
|
|
throw new TypeError(`workflow target escapes repository root: ${contract.providerAdapter}`);
|
|
}
|
|
const productGates = options.productGates ?? (await loadProductCiOverlay(root)).mergeGates;
|
|
const expected = Buffer.from(renderCiWorkflow(contract, productGates), "utf8");
|
|
let actual: Buffer | null = null;
|
|
const existingPathIsSafe =
|
|
fileSystem === defaultFileSystem
|
|
? await assertSafeExistingPublishPath(root, target)
|
|
: true;
|
|
if (existingPathIsSafe) {
|
|
try {
|
|
actual = await fileSystem.readFile(target);
|
|
} catch (error) {
|
|
if (!hasErrorCode(error, "ENOENT")) throw error;
|
|
}
|
|
}
|
|
const difference = firstDifference(expected, actual);
|
|
if (options.check || difference === null) {
|
|
return Object.freeze({
|
|
target,
|
|
written: false,
|
|
matches: difference === null,
|
|
firstDifferenceByte: difference?.byte ?? null,
|
|
firstDifferenceLine: difference?.line ?? null,
|
|
});
|
|
}
|
|
|
|
let parentIdentity: Stats | undefined;
|
|
if (fileSystem === defaultFileSystem) {
|
|
parentIdentity = await ensureSafePublishDirectory(root, path.dirname(target));
|
|
await assertSafePublishLeaf(target, contract.providerAdapter);
|
|
} else {
|
|
await fileSystem.mkdir(path.dirname(target));
|
|
}
|
|
const temporary = path.join(path.dirname(target), `.${path.basename(target)}.${createNonce()}.tmp`);
|
|
let ownsTemporary = false;
|
|
try {
|
|
const handle = await fileSystem.open(
|
|
temporary,
|
|
constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW,
|
|
0o644,
|
|
);
|
|
ownsTemporary = true;
|
|
let failure: unknown;
|
|
try {
|
|
await handle.writeFile(expected.toString("utf8"), "utf8");
|
|
await handle.chmod(0o644);
|
|
await handle.sync();
|
|
} catch (error) {
|
|
failure = error;
|
|
}
|
|
try {
|
|
await handle.close();
|
|
} catch (error) {
|
|
failure ??= error;
|
|
}
|
|
if (failure) throw failure;
|
|
if (fileSystem === defaultFileSystem && parentIdentity) {
|
|
const current = await ensureSafePublishDirectory(root, path.dirname(target));
|
|
if (
|
|
parentIdentity.dev <= 0 ||
|
|
parentIdentity.ino <= 0 ||
|
|
current.dev !== parentIdentity.dev ||
|
|
current.ino !== parentIdentity.ino
|
|
) {
|
|
throw new TypeError("CI workflow parent directory identity changed");
|
|
}
|
|
await assertSafePublishLeaf(target, contract.providerAdapter);
|
|
}
|
|
await fileSystem.rename(temporary, target);
|
|
ownsTemporary = false;
|
|
const directory = await fileSystem.openDirectory(path.dirname(target));
|
|
try {
|
|
try {
|
|
await directory.sync();
|
|
} catch (error) {
|
|
if (!hasErrorCode(error, "EINVAL") && !hasErrorCode(error, "ENOTSUP")) throw error;
|
|
}
|
|
} finally {
|
|
await directory.close();
|
|
}
|
|
} catch (error) {
|
|
if (ownsTemporary) {
|
|
try {
|
|
await fileSystem.rm(temporary);
|
|
} catch {
|
|
// The owned sibling temp is the only cleanup target; preserve the publish failure.
|
|
}
|
|
}
|
|
throw error;
|
|
}
|
|
return Object.freeze({ target, written: true, matches: true, firstDifferenceByte: null, firstDifferenceLine: null });
|
|
};
|
|
}
|
|
|
|
export const generateCiWorkflow = createCiWorkflowGenerator();
|
|
|
|
function firstDifference(expected: Buffer, actual: Buffer | null): { byte: number; line: number } | null {
|
|
if (actual?.equals(expected)) return null;
|
|
const limit = Math.min(expected.byteLength, actual?.byteLength ?? 0);
|
|
let byte = 0;
|
|
while (byte < limit && expected[byte] === actual?.[byte]) byte += 1;
|
|
const line = expected.subarray(0, byte).toString("utf8").split("\n").length;
|
|
return { byte, line };
|
|
}
|
|
|
|
function hasErrorCode(error: unknown, code: string): boolean {
|
|
return Boolean(error && typeof error === "object" && "code" in error && error.code === code);
|
|
}
|
|
|
|
const isCli = process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
|
|
if (isCli) {
|
|
const check = process.argv.includes("--check");
|
|
try {
|
|
const result = await generateCiWorkflow({ root: process.cwd(), check });
|
|
if (!result.matches) {
|
|
process.stderr.write(
|
|
`CI workflow drift: ${result.target} differs at byte ${result.firstDifferenceByte ?? 0}, line ${result.firstDifferenceLine ?? 1}\n`,
|
|
);
|
|
process.exitCode = 1;
|
|
} else {
|
|
process.stdout.write(check ? "CI workflow bytes: PASS\n" : "CI workflow generated atomically\n");
|
|
}
|
|
} catch (error) {
|
|
process.stderr.write(`CI workflow generation failed: ${error instanceof Error ? error.message : String(error)}\n`);
|
|
process.exitCode = 1;
|
|
}
|
|
}
|