From 9655958bdae9199f4ea84684eddb803da3262762 Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Thu, 5 Nov 2026 22:06:59 +0900 Subject: [PATCH 1/2] =?UTF-8?q?fix(ci):=20test:system=20=EC=82=AC=EC=A0=84?= =?UTF-8?q?=20=EC=A0=90=EA=B2=80=EC=9D=B4=20bwrap=20namespace=EB=A5=BC=20?= =?UTF-8?q?=EC=8B=A4=EC=A0=9C=EB=A1=9C=20=EB=A7=8C=EB=93=A4=EC=96=B4=20?= =?UTF-8?q?=EB=B3=B8=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 문제: FE-GATE-013의 provider sandbox는 bwrap을 --unshare-pid --unshare-ipc --unshare-uts --unshare-net으로 띄운다. bwrap이 setuid가 아니므로 비특권 user namespace가 있어야 한다. 사전 점검 (scripts/check-system-test-prerequisites.ts)은 실행 파일·cgroup·systemd bus만 보고 bwrap을 한 번도 실행하지 않는다. 그래서 이 제한이 있는 호스트 (Ubuntu 24.04, kernel.apparmor_restrict_unprivileged_userns=1)에서 PASS를 낸 뒤 provider 테스트 114건 중 18건이 "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted" 또는 그 provider를 기다린 시간 초과로 실패한다. 이 조건은 runner 요구 목록, gate 정의, workflow 어디에도 없고 운영 원장의 일회성 메모에만 있다. 고른 방법: - 사전 점검이 provider와 같은 namespace 옵션으로 bwrap에서 /bin/true를 한 번 띄운다. 실패하면 "this host does not allow unprivileged user and network namespaces", bwrap stderr(최대 4줄), user namespace sysctl 값, 고치는 방법 셋(AppArmor profile, sysctl, 다른 runner)을 적고 exit 1. 기존 검사(실행 파일 3개, cgroup v2, systemd bus)는 그대로 둔다. - 옵션 목록을 scripts/lib/provider-sandbox-namespaces.ts로 옮기고 run-and-validate-provider.ts와 사전 점검이 같은 상수를 쓴다. 둘이 갈라지면 점검이 다른 것을 시험하게 된다. - 점검을 findSystemTestPrerequisiteFailures(host)로 내보내고 access· readFile·spawn을 주입받는다. CLI 동작(출력·exit 코드)은 같다. - config/ci/gates.json의 FE-GATE-013에 hostRequirements 두 줄을 둔다. gate schema에 선택 필드를 더하고, generator가 그 gate를 도는 job의 runs-on 바로 위에 "# FE-GATE-013 runner requirement: ..." 주석으로 그린다. runner를 고르는 사람이 보는 자리다. 줄바꿈은 기존 nonEmpty 규칙이 막으므로 주석 밖으로 나갈 수 없다. 이 필드는 gate의 명령·증거가 아니라 runner 설명이라 기준선 gate shape 해시에는 넣지 않았다(기준선 재기록 불필요). - 요구 목록 세 곳(docs/operations/ci-quality-gates.md, docs/security/supply-chain.md, docs/testing/taxonomy.md)과 README merge 게이트 절에 조건을 적었다. ci-quality-gates.md에 AppArmor profile 예시를 두었다. 버린 대안: - namespace가 없으면 provider 테스트를 skip: 거짓 초록이다. - FE-GATE-013을 release 단계로 옮김: PR마다 sandbox 검사를 잃는다. - probe를 systemd-run --user --scope 안에서 돌림: 리뷰 11-R10에서 scope 안과 밖의 결과가 같았다. 직접 실행이 더 단순하다. - workflow에 요구 사항을 runs-on label로 표현: runner label 체계가 아직 없다. 주석이 지금 가능한 가장 작은 표시다. 위험·한계: - probe는 bwrap에 --ro-bind / /만 준다. --size 같은 provider의 다른 옵션 지원은 여기서 보지 않는다(기존과 같다). probe가 다른 이유로 실패해도 namespace 문구가 나오지만 bwrap stderr를 같이 적는다. - 호스트를 고칠지는 호스트 소유자가 정한다. 이 커밋은 조건을 적고 알리기만 한다. 이 desktop에서 FE-GATE-013은 여전히 실패하고, 이제 첫 단계에서 이유와 함께 실패한다. - gate schema에 선택 필드 hostRequirements가 생겼다. 기존 gates.json과 제품 overlay는 그대로 통과한다. 검증: desktop batch에서 확인 - node scripts/run-vitest.ts run tests/unit/system-test-prerequisites.test.ts tests/unit/ci-workflow-generation.test.ts - corepack pnpm check:ci, check:types, lint, verify:documentation - desktop에서 node scripts/check-system-test-prerequisites.ts가 exit 1과 namespace 문구를 내는지, FE-GATE-013이 그 단계에서 멈추는지 Refs: 1105/12-01, 1105/11-02 --- .gitea/workflows/quality-gates.yml | 2 + README.md | 5 +- config/ci/gates.json | 6 +- docs/operations/ci-quality-gates.md | 40 ++++ docs/security/supply-chain.md | 10 +- docs/testing/taxonomy.md | 7 +- scripts/check-system-test-prerequisites.ts | 179 ++++++++++++++---- scripts/contracts/ci-gates.ts | 8 + scripts/generate-ci-workflow.ts | 24 ++- scripts/lib/provider-sandbox-namespaces.ts | 18 ++ scripts/run-and-validate-provider.ts | 3 +- .../ci-workflow-generation.test.ts.snap | 2 + tests/unit/ci-workflow-generation.test.ts | 30 +++ tests/unit/system-test-prerequisites.test.ts | 125 ++++++++++++ 14 files changed, 413 insertions(+), 46 deletions(-) create mode 100644 scripts/lib/provider-sandbox-namespaces.ts create mode 100644 tests/unit/system-test-prerequisites.test.ts diff --git a/.gitea/workflows/quality-gates.yml b/.gitea/workflows/quality-gates.yml index 3841d543..796ff2cc 100644 --- a/.gitea/workflows/quality-gates.yml +++ b/.gitea/workflows/quality-gates.yml @@ -34,6 +34,8 @@ jobs: merge_gate: name: "${{ matrix.gate }} / ${{ matrix.name }}" if: ${{ gitea.event_name != 'workflow_dispatch' || inputs.stage != 'documentation' }} + # FE-GATE-013 runner requirement: Linux with executable /usr/bin/bwrap (with --size), /usr/bin/prlimit, /usr/bin/systemd-run, /usr/bin/systemctl and /usr/bin/tar, an active systemd user manager 254 or newer, and unified cgroup v2 with delegated memory, pids and cpu controllers + # FE-GATE-013 runner requirement: unprivileged user and network namespaces for bwrap (the provider sandbox runs bwrap --unshare-pid --unshare-ipc --unshare-uts --unshare-net); Ubuntu 23.10 and later refuse them while kernel.apparmor_restrict_unprivileged_userns=1 unless /usr/bin/bwrap has an AppArmor profile that allows userns; scripts/check-system-test-prerequisites.ts probes this before test:system runs runs-on: ubuntu-latest timeout-minutes: 45 strategy: diff --git a/README.md b/README.md index be85eada..6a0f27e3 100644 --- a/README.md +++ b/README.md @@ -328,7 +328,10 @@ tags. The Git host of this repository may not have Actions enabled, so a pull request can show no gate status at all; no status is not a pass. Until the host runs the workflow and marks the merge gates as required checks, run the 14 merge gates locally before merging (FE-GATE-008 and 009 need Playwright -browsers; FE-GATE-012 builds the app first). Product checks listed in +browsers; FE-GATE-012 builds the app first; FE-GATE-013 needs a Linux host where +bubblewrap may create unprivileged user and network namespaces, which Ubuntu +23.10 and later refuse by default — its first step says so, and +`docs/operations/ci-quality-gates.md` lists the fixes). Product checks listed in `config/ci/product-gates.json` (`PRODUCT-GATE-NNN`) run in the workflow's own `product_merge_gate` job: diff --git a/config/ci/gates.json b/config/ci/gates.json index 46a4d27f..f32e169e 100644 --- a/config/ci/gates.json +++ b/config/ci/gates.json @@ -2036,7 +2036,11 @@ "artifact-artifacts-security-supply-chain-fixtures-json", "artifact-artifacts-security-supply-chain-provider-fixtures-json" ], - "retentionClassId": "release-coherence" + "retentionClassId": "release-coherence", + "hostRequirements": [ + "Linux with executable /usr/bin/bwrap (with --size), /usr/bin/prlimit, /usr/bin/systemd-run, /usr/bin/systemctl and /usr/bin/tar, an active systemd user manager 254 or newer, and unified cgroup v2 with delegated memory, pids and cpu controllers", + "unprivileged user and network namespaces for bwrap (the provider sandbox runs bwrap --unshare-pid --unshare-ipc --unshare-uts --unshare-net); Ubuntu 23.10 and later refuse them while kernel.apparmor_restrict_unprivileged_userns=1 unless /usr/bin/bwrap has an AppArmor profile that allows userns; scripts/check-system-test-prerequisites.ts probes this before test:system runs" + ] }, { "id": "FE-GATE-014", diff --git a/docs/operations/ci-quality-gates.md b/docs/operations/ci-quality-gates.md index 2fbbbfc7..1171b76e 100644 --- a/docs/operations/ci-quality-gates.md +++ b/docs/operations/ci-quality-gates.md @@ -188,6 +188,46 @@ trust/archive/path 같은 host-side preflight 실패는 raw report 생성 전에 만든 뒤 scope 안에서만 확정할 수 있으며, 이 단계의 실패는 해당 inode를 identity-bound cleanup하고 fail closed한다. +**비특권 user namespace (1105/12-01).** runner는 비특권 process가 bubblewrap으로 +user namespace와 network namespace를 만들 수 있게 허용해야 한다. provider +sandbox는 `bwrap --unshare-pid --unshare-ipc --unshare-uts --unshare-net`으로 +뜬다(`scripts/lib/provider-sandbox-namespaces.ts`). bubblewrap은 setuid가 아니므로 +이 옵션마다 비특권 user namespace가 필요하다. 이 조건은 provider job과 merge +게이트 FE-GATE-013(`test:system`의 provider 테스트) 둘 다에 걸린다. +`config/ci/gates.json`의 FE-GATE-013 `hostRequirements`에 같은 조건이 있고, +생성 workflow는 그것을 merge job의 `runs-on` 옆 주석으로 보여 준다. + +- Ubuntu 23.10부터는 `kernel.apparmor_restrict_unprivileged_userns=1`이 + 기본이다(24.04도 같다). `/usr/bin/bwrap`에 `userns`를 허용하는 AppArmor + profile이 없으면 bwrap이 + `bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted`로 끝난다. +- `test:system`은 먼저 `scripts/check-system-test-prerequisites.ts`를 돈다. 이 + 검사는 provider와 같은 namespace 옵션으로 bwrap에서 `/bin/true`를 한 번 + 띄운다. 실패하면 "this host does not allow unprivileged user and network + namespaces"와 bwrap의 stderr, 위 sysctl 값, 고치는 방법을 적고 exit 1로 + 멈춘다. 전에는 이 검사가 PASS를 낸 뒤 provider 테스트가 bwrap 원문 오류로 + 실패했다(2026-11-05 desktop에서 114건 중 18건). +- 고치는 방법은 셋 중 하나다. 호스트의 보안 설정을 바꾸는 일이므로 호스트 + 소유자가 고른다. + 1. `/usr/bin/bwrap`에 `userns`를 허용하는 AppArmor profile을 둔다. Ubuntu + 문서가 안내하는 형태는 아래와 같다. `/etc/apparmor.d/bwrap`에 두고 + `sudo apparmor_parser -r /etc/apparmor.d/bwrap`으로 읽게 한다. + 2. 직접 관리하는 호스트라면 sysctl + `kernel.apparmor_restrict_unprivileged_userns=0`으로 제한을 끈다. 이 설정이 + 없는 커널에서는 `kernel.unprivileged_userns_clone=1`이 같은 역할이다. + 3. 비특권 user namespace를 허용하는 다른 runner에서 FE-GATE-013과 provider + job을 돌린다. + +```text +abi , +include + +profile bwrap /usr/bin/bwrap flags=(unconfined) { + userns, + include if exists +} +``` + 각 provider는 고유한 collected user scope에서 실행된다. Supervisor는 실행 전에 실제 cgroup membership과 `memory.max=1073741824`, `memory.swap.max=0`, `pids.max=64`, `cpu.max="100000 100000"`을 확인한다. 내부 process에는 core 0, diff --git a/docs/security/supply-chain.md b/docs/security/supply-chain.md index 1c9c505f..b41673df 100644 --- a/docs/security/supply-chain.md +++ b/docs/security/supply-chain.md @@ -183,7 +183,15 @@ External provider supervision is fail-closed and requires a Linux runner with executable `/usr/bin/bwrap`, `/usr/bin/prlimit`, `/usr/bin/systemd-run`, and `/usr/bin/systemctl`, bubblewrap support for `--size`, an active user bus, a systemd user manager version 254 or newer, unified cgroup v2, and delegated -memory, pids, and CPU controllers. Executable access, user-manager/version, +memory, pids, and CPU controllers. The runner must also let an unprivileged +process create user and network namespaces with bubblewrap: the sandbox runs +`bwrap --unshare-pid --unshare-ipc --unshare-uts --unshare-net`, and Ubuntu +23.10 and later refuse that while +`kernel.apparmor_restrict_unprivileged_userns=1` unless `/usr/bin/bwrap` has +an AppArmor profile that allows `userns` (1105/12-01). The same holds for +FE-GATE-013, whose `test:system` probes it first +(`scripts/check-system-test-prerequisites.ts`); the fixes are in +`docs/operations/ci-quality-gates.md`. Executable access, user-manager/version, trust, archive, and path failures are rejected before raw creation. Bubblewrap `--size` acceptance and effective delegated controller values can only be verified after the owned raw inode exists inside a new scope; failures there diff --git a/docs/testing/taxonomy.md b/docs/testing/taxonomy.md index f3a199e9..1bb5b9da 100644 --- a/docs/testing/taxonomy.md +++ b/docs/testing/taxonomy.md @@ -12,7 +12,7 @@ Each gate is blocking in its declared scope. Failures are not downgraded with | capability contract | `pnpm test:contract` | reusable capability consumer contracts | `artifacts/tests/contract.xml` | | component | `pnpm test:component` | React/hook/UI behavior | `artifacts/tests/component.xml` | | integration | `pnpm test:integration` | HTTP/MSW, IndexedDB, composed browser-runtime boundaries, child-process/filesystem/IPC integration | `artifacts/tests/integration.xml` | -| system / CI runner | `pnpm test:system` | compatible Linux host with systemd, bubblewrap, cgroup v2 and CI-provider process controls | `artifacts/tests/system.xml` | +| system / CI runner | `pnpm test:system` | compatible Linux host with systemd, bubblewrap, cgroup v2 and CI-provider process controls, where bubblewrap may create unprivileged user and network namespaces | `artifacts/tests/system.xml` | | end-to-end | `pnpm test:e2e` | pinned browser engines | `artifacts/tests/e2e/` | | accessibility | `pnpm test:a11y` | pinned browser engines | `artifacts/tests/a11y.json` | @@ -60,6 +60,11 @@ start the CI-runner suite. The system suite additionally runs `scripts/check-system-test-prerequisites.ts` and fails with one prerequisite report when the CI-runner host does not provide its required Linux facilities. +Besides the executables, cgroup v2 and the systemd bus, it runs bubblewrap once +with the provider sandbox's namespace options on `/bin/true`, so a host that +refuses unprivileged user namespaces (Ubuntu 23.10 and later with +`kernel.apparmor_restrict_unprivileged_userns=1`) fails there by name instead of +in the provider tests (1105/12-01, `docs/operations/ci-quality-gates.md`). ## Development paths diff --git a/scripts/check-system-test-prerequisites.ts b/scripts/check-system-test-prerequisites.ts index c43e32fe..50462fc7 100644 --- a/scripts/check-system-test-prerequisites.ts +++ b/scripts/check-system-test-prerequisites.ts @@ -1,55 +1,154 @@ import { constants } from "node:fs"; import { access, readFile } from "node:fs/promises"; import { spawnSync } from "node:child_process"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; -const failures: string[] = []; +import { PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS } from "./lib/provider-sandbox-namespaces.ts"; -async function requireExecutable(path: string, label: string): Promise { - try { - await access(path, constants.X_OK); - } catch { - failures.push(`${label} is required at ${path}`); - } -} +/** What the check reads from the host; a test passes a fake. */ +export type SystemTestHost = Readonly<{ + /** Resolves when `target` is executable, rejects otherwise. */ + access(target: string): Promise; + readFile(target: string): Promise; + spawn( + command: string, + args: readonly string[], + timeoutMs: number, + ): Readonly<{ status: number | null; stderr: string; error?: Error | undefined }>; +}>; -await Promise.all([ - requireExecutable("/usr/bin/bwrap", "bubblewrap"), - requireExecutable("/usr/bin/systemctl", "systemctl"), - requireExecutable("/usr/bin/tar", "tar"), +export const nodeSystemTestHost: SystemTestHost = Object.freeze({ + access: async (target: string) => access(target, constants.X_OK), + readFile: async (target: string) => readFile(target, "utf8"), + spawn: (command: string, args: readonly string[], timeoutMs: number) => { + const result = spawnSync(command, [...args], { encoding: "utf8", timeout: timeoutMs }); + return { status: result.status, stderr: result.stderr ?? "", error: result.error }; + }, +}); + +const BWRAP = "/usr/bin/bwrap"; +const SYSTEMCTL = "/usr/bin/systemctl"; + +/** + * 1105/12-01. The provider sandbox's own namespace options on /bin/true. The + * check used to see only that /usr/bin/bwrap was executable, so a host that + * refuses unprivileged user namespaces passed it and then failed 18 provider + * tests with `bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted`. + */ +export const NAMESPACE_PROBE_ARGUMENTS: readonly string[] = Object.freeze([ + ...PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS, + "--ro-bind", "/", "/", + "--", "/bin/true", ]); -try { - const controllers = await readFile("/sys/fs/cgroup/cgroup.controllers", "utf8"); - if (controllers.trim().length === 0) { - failures.push("cgroup v2 controllers are unavailable"); +/** Host settings that decide whether an unprivileged process may create user namespaces. */ +const USER_NAMESPACE_SETTINGS = Object.freeze([ + ["kernel.apparmor_restrict_unprivileged_userns", "/proc/sys/kernel/apparmor_restrict_unprivileged_userns"], + ["kernel.unprivileged_userns_clone", "/proc/sys/kernel/unprivileged_userns_clone"], + ["user.max_user_namespaces", "/proc/sys/user/max_user_namespaces"], +] as const); + +const PROBE_TIMEOUT_MS = 5_000; +const MAX_REPORTED_STDERR_LINES = 4; + +export async function findSystemTestPrerequisiteFailures( + host: SystemTestHost = nodeSystemTestHost, +): Promise { + const failures: string[] = []; + const executable = new Map(); + for (const [target, label] of [ + [BWRAP, "bubblewrap"], + [SYSTEMCTL, "systemctl"], + ["/usr/bin/tar", "tar"], + ] as const) { + try { + await host.access(target); + executable.set(target, true); + } catch { + executable.set(target, false); + failures.push(`${label} is required at ${target}`); + } } -} catch { - failures.push("cgroup v2 is required at /sys/fs/cgroup/cgroup.controllers"); -} -if (!failures.some((failure) => failure.includes("systemctl"))) { - const probe = spawnSync("/usr/bin/systemctl", ["show-environment"], { - encoding: "utf8", - timeout: 3_000, - }); - if (probe.error || probe.status !== 0) { - failures.push("a reachable systemd manager bus is required"); + try { + const controllers = await host.readFile("/sys/fs/cgroup/cgroup.controllers"); + if (controllers.trim().length === 0) { + failures.push("cgroup v2 controllers are unavailable"); + } + } catch { + failures.push("cgroup v2 is required at /sys/fs/cgroup/cgroup.controllers"); } + + if (executable.get(SYSTEMCTL)) { + const probe = host.spawn(SYSTEMCTL, ["show-environment"], 3_000); + if (probe.error || probe.status !== 0) { + failures.push("a reachable systemd manager bus is required"); + } + } + + if (executable.get(BWRAP)) { + const probe = host.spawn(BWRAP, NAMESPACE_PROBE_ARGUMENTS, PROBE_TIMEOUT_MS); + if (probe.error || probe.status !== 0) { + failures.push(await namespaceFailure(host, probe)); + } + } + return failures; } -if (failures.length > 0) { - process.stderr.write( - [ - "System/CI-runner test prerequisites are unavailable:", - ...failures.map((failure) => ` - ${failure}`), - "", - "Run test:unit/test:contract/test:component/test:integration for the", - "developer loop. test:system is intentionally reserved for a compatible", - "Linux CI-runner host.", - "", - ].join("\n"), - ); - process.exit(1); +async function namespaceFailure( + host: SystemTestHost, + probe: ReturnType, +): Promise { + const settings: string[] = []; + for (const [name, file] of USER_NAMESPACE_SETTINGS) { + try { + settings.push(`${name} = ${(await host.readFile(file)).trim()}`); + } catch { + // Absent on this kernel; the others still say what the host allows. + } + } + const stderr = probe.stderr + .split("\n") + .map((line) => line.trim()) + .filter((line) => line.length > 0) + .slice(0, MAX_REPORTED_STDERR_LINES); + const outcome = probe.error + ? `did not finish: ${probe.error.message}` + : `exited ${String(probe.status)}`; + return [ + "this host does not allow unprivileged user and network namespaces, which the provider " + + `sandbox needs (bwrap ${PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS.join(" ")}).`, + ` probe: ${BWRAP} ${NAMESPACE_PROBE_ARGUMENTS.join(" ")} ${outcome}`, + ...(stderr.length > 0 ? stderr.map((line) => ` bwrap stderr: ${line}`) : [" bwrap stderr: (empty)"]), + ...(settings.length > 0 ? [` host settings: ${settings.join(", ")}`] : []), + " To fix, do one of these:", + " - give /usr/bin/bwrap an AppArmor profile that allows `userns` (Ubuntu 23.10 and later),", + " - on a host you own, set sysctl kernel.apparmor_restrict_unprivileged_userns=0", + " (or kernel.unprivileged_userns_clone=1 where that setting exists),", + " - run FE-GATE-013 on another runner that allows them.", + " Runner requirements: docs/operations/ci-quality-gates.md.", + ].join("\n"); } -process.stdout.write("System test prerequisites: PASS\n"); +export function formatSystemTestPrerequisiteReport(failures: readonly string[]): string { + return [ + "System/CI-runner test prerequisites are unavailable:", + ...failures.map((failure) => ` - ${failure}`), + "", + "Run test:unit/test:contract/test:component/test:integration for the", + "developer loop. test:system is intentionally reserved for a compatible", + "Linux CI-runner host.", + "", + ].join("\n"); +} + +const isCli = process.argv[1] !== undefined && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url); +if (isCli) { + const failures = await findSystemTestPrerequisiteFailures(); + if (failures.length > 0) { + process.stderr.write(formatSystemTestPrerequisiteReport(failures)); + process.exit(1); + } + process.stdout.write("System test prerequisites: PASS\n"); +} diff --git a/scripts/contracts/ci-gates.ts b/scripts/contracts/ci-gates.ts index 73d05af5..5111a329 100644 --- a/scripts/contracts/ci-gates.ts +++ b/scripts/contracts/ci-gates.ts @@ -290,6 +290,14 @@ const gateSchema = z evidenceArtifactIds: z.array(id).min(1).max(128), retentionClassId: id, requiresEnvironment: z.array(z.string().max(128).regex(/^[A-Z][A-Z0-9_]*$/u)).max(32).optional(), + /** + * 1105/12-01. What the runner host must provide for the gate to pass, one + * line each. The workflow shows them as comments beside the job's + * `runs-on`; a line cannot hold a line break, so it cannot leave the + * comment. They describe the runner, not the gate's commands or evidence, + * so the reviewed gate shape (canonicalGateShapeSha256) leaves them out. + */ + hostRequirements: z.array(nonEmpty.max(512)).min(1).max(8).optional(), }) .strict(); diff --git a/scripts/generate-ci-workflow.ts b/scripts/generate-ci-workflow.ts index 0f8fdaa5..00a38cc3 100644 --- a/scripts/generate-ci-workflow.ts +++ b/scripts/generate-ci-workflow.ts @@ -167,7 +167,11 @@ function renderJob( if (job.needs.length > 1) lines.push(` needs: [${job.needs.map(yamlKey).join(", ")}]`); const condition = renderCondition(job.condition); if (condition) lines.push(` if: ${condition}`); - lines.push(" runs-on: ubuntu-latest", ` timeout-minutes: ${job.timeoutMinutes}`); + lines.push( + ...renderHostRequirements(job, index), + " runs-on: ubuntu-latest", + ` timeout-minutes: ${job.timeoutMinutes}`, + ); if (job.kind === "immutable") { const archive = job.steps.find((step) => step.kind === "archive-candidate"); if (!archive || archive.kind !== "archive-candidate") throw new TypeError("immutable job lacks archive step"); @@ -214,6 +218,24 @@ function renderJob( return lines; } +/** + * 1105/12-01. A gate's host requirements (`hostRequirements` in + * config/ci/gates.json) sit as comments beside the runner label of the job + * that runs the gate, where whoever picks the runner reads them. FE-GATE-013 + * needs unprivileged user namespaces, and that was written nowhere a runner + * is chosen. The contract keeps line breaks out of each requirement. + */ +function renderHostRequirements( + job: CiWorkflowJob, + index: ReturnType, +): string[] { + return job.gateIds.flatMap((gateId) => + (index.gates.get(gateId)?.hostRequirements ?? []).map( + (requirement) => ` # ${gateId} runner requirement: ${requirement}`, + ), + ); +} + function renderStep( job: CiWorkflowJob, step: CiWorkflowStep, diff --git a/scripts/lib/provider-sandbox-namespaces.ts b/scripts/lib/provider-sandbox-namespaces.ts new file mode 100644 index 00000000..618474fa --- /dev/null +++ b/scripts/lib/provider-sandbox-namespaces.ts @@ -0,0 +1,18 @@ +/** + * 1105/12-01. The namespaces the provider sandbox asks bubblewrap for + * (scripts/run-and-validate-provider.ts). bubblewrap is not setuid, so each of + * them is an unprivileged user namespace plus the namespace named, and + * `--unshare-net` also brings up loopback inside it. + * + * scripts/check-system-test-prerequisites.ts probes the host with this same + * list. A host that refuses them (Ubuntu 24.04 with + * kernel.apparmor_restrict_unprivileged_userns=1 and no AppArmor profile for + * bwrap) then fails the prerequisite check by name, instead of passing it and + * failing the provider tests with a raw netlink error. + */ +export const PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS: readonly string[] = Object.freeze([ + "--unshare-pid", + "--unshare-ipc", + "--unshare-uts", + "--unshare-net", +]); diff --git a/scripts/run-and-validate-provider.ts b/scripts/run-and-validate-provider.ts index 09511f4a..268318ab 100644 --- a/scripts/run-and-validate-provider.ts +++ b/scripts/run-and-validate-provider.ts @@ -30,6 +30,7 @@ import { type ProviderGuardianLease, } from "./lib/provider-guardian-client.ts"; import { createProviderOutputLimiter } from "./lib/provider-output-limiter.ts"; +import { PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS } from "./lib/provider-sandbox-namespaces.ts"; import { providerProcessTraceEnvironment, traceOwnProviderProcess, @@ -253,7 +254,7 @@ async function runProviderInSandbox( const cpuSeconds = providerCpuSeconds(); const bwrapArguments = [ "--die-with-parent", "--new-session", "--as-pid-1", - "--unshare-pid", "--unshare-ipc", "--unshare-uts", "--unshare-net", + ...PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS, "--dev", "/dev", "--remount-ro", "/dev", "--proc", "/proc", "--remount-ro", "/proc", "--size", String(PROVIDER_TMP_BYTES), "--tmpfs", "/tmp", diff --git a/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap b/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap index a7f51531..6b84e76f 100644 --- a/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap +++ b/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap @@ -37,6 +37,8 @@ jobs: merge_gate: name: "\${{ matrix.gate }} / \${{ matrix.name }}" if: \${{ gitea.event_name != 'workflow_dispatch' || inputs.stage != 'documentation' }} + # FE-GATE-013 runner requirement: Linux with executable /usr/bin/bwrap (with --size), /usr/bin/prlimit, /usr/bin/systemd-run, /usr/bin/systemctl and /usr/bin/tar, an active systemd user manager 254 or newer, and unified cgroup v2 with delegated memory, pids and cpu controllers + # FE-GATE-013 runner requirement: unprivileged user and network namespaces for bwrap (the provider sandbox runs bwrap --unshare-pid --unshare-ipc --unshare-uts --unshare-net); Ubuntu 23.10 and later refuse them while kernel.apparmor_restrict_unprivileged_userns=1 unless /usr/bin/bwrap has an AppArmor profile that allows userns; scripts/check-system-test-prerequisites.ts probes this before test:system runs runs-on: ubuntu-latest timeout-minutes: 45 strategy: diff --git a/tests/unit/ci-workflow-generation.test.ts b/tests/unit/ci-workflow-generation.test.ts index be94a2c0..03d8ad2d 100644 --- a/tests/unit/ci-workflow-generation.test.ts +++ b/tests/unit/ci-workflow-generation.test.ts @@ -1698,6 +1698,36 @@ describe("CI workflow generation", () => { expect(workflow).toMatch(/^on:\n {2}push:\n {4}branches: \[main, develop\]$/mu); }); + // 1105/12-01. The provider sandbox in FE-GATE-013 needs unprivileged user + // and network namespaces. No runner requirement said so, and a host that + // refuses them failed 18 provider tests after the prerequisite check passed. + it("states FE-GATE-013's runner requirements beside the runner label of the job that runs it", async () => { + const contract = await loadCiGateContract(process.cwd()); + const requirements = indexCiGateContract(contract).gates.get("FE-GATE-013")?.hostRequirements ?? []; + expect(requirements.join("\n")).toMatch(/unprivileged user and network namespaces/u); + expect(requirements.join("\n")).toContain("--unshare-net"); + expect(requirements.join("\n")).toContain("kernel.apparmor_restrict_unprivileged_userns"); + expect(requirements.join("\n")).toContain("scripts/check-system-test-prerequisites.ts"); + + const workflow = renderCiWorkflow(contract); + const merge = workflow.slice(workflow.indexOf(" merge_gate:\n"), workflow.indexOf(" release_gate:\n")); + const header = merge.slice(0, merge.indexOf(" runs-on: ubuntu-latest\n")); + expect(header.split("\n").filter((line) => line.startsWith(" # "))).toEqual( + requirements.map((requirement) => ` # FE-GATE-013 runner requirement: ${requirement}`), + ); + expect(workflow.match(/ runner requirement: /gu)).toHaveLength(requirements.length); + }); + + it("rejects a runner requirement that would leave its workflow comment (1105/12-01)", async () => { + const contract = JSON.parse( + JSON.stringify(await loadCiGateContract(process.cwd())), + ) as Record; + contract.gates.find((gate: Record) => gate.id === "FE-GATE-013").hostRequirements = [ + "namespaces\n runs-on: self-hosted", + ]; + expect(() => parseCiGateContract(contract)).toThrow(/line-break characters are forbidden/u); + }); + // 12-02 / 13-01: immutable_build owns FE-GATE-015 and FE-GATE-027, but only // the first gate of a non-matrix job was ever rendered, so release admission // never ran in CI. diff --git a/tests/unit/system-test-prerequisites.test.ts b/tests/unit/system-test-prerequisites.test.ts new file mode 100644 index 00000000..3ff28dc4 --- /dev/null +++ b/tests/unit/system-test-prerequisites.test.ts @@ -0,0 +1,125 @@ +import { readFile } from "node:fs/promises"; + +import { describe, expect, it } from "vitest"; + +import { + findSystemTestPrerequisiteFailures, + formatSystemTestPrerequisiteReport, + NAMESPACE_PROBE_ARGUMENTS, + type SystemTestHost, +} from "../../scripts/check-system-test-prerequisites.ts"; +import { PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS } from "../../scripts/lib/provider-sandbox-namespaces.ts"; + +type SpawnCall = Readonly<{ command: string; args: readonly string[] }>; +type SpawnResult = ReturnType; + +const RTM_NEWADDR = "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted"; + +/** A host that has every prerequisite, except what `bwrap` and `files` say. */ +function fakeHost( + bwrap: SpawnResult, + files: Readonly> = {}, +): { host: SystemTestHost; calls: SpawnCall[] } { + const calls: SpawnCall[] = []; + const host: SystemTestHost = { + access: async () => undefined, + readFile: async (target) => { + if (target === "/sys/fs/cgroup/cgroup.controllers") return "cpuset cpu io memory pids\n"; + const content = files[target]; + if (content === undefined) throw new Error(`ENOENT: ${target}`); + return content; + }, + spawn: (command, args) => { + calls.push({ command, args }); + if (command === "/usr/bin/systemctl") return { status: 0, stderr: "" }; + if (command === "/usr/bin/bwrap") return bwrap; + throw new Error(`unexpected spawn: ${command}`); + }, + }; + return { host, calls }; +} + +// 1105/12-01. The check looked only for the bwrap executable, so a host that +// refuses unprivileged user namespaces printed PASS and then failed 18 +// provider tests with bwrap's raw netlink error. +describe("system test prerequisites (1105/12-01)", () => { + it("passes when bubblewrap creates the provider sandbox's namespaces", async () => { + const { host, calls } = fakeHost({ status: 0, stderr: "" }); + + expect(await findSystemTestPrerequisiteFailures(host)).toEqual([]); + + const probes = calls.filter(({ command }) => command === "/usr/bin/bwrap"); + expect(probes).toHaveLength(1); + expect(probes[0]?.args).toEqual(NAMESPACE_PROBE_ARGUMENTS); + expect(probes[0]?.args.at(-1)).toBe("/bin/true"); + }); + + it("probes with exactly the namespace options the provider sandbox uses", async () => { + expect(PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS).toEqual([ + "--unshare-pid", + "--unshare-ipc", + "--unshare-uts", + "--unshare-net", + ]); + expect(NAMESPACE_PROBE_ARGUMENTS.slice(0, PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS.length)).toEqual( + PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS, + ); + // The provider takes the list from the same module instead of its own copy. + const provider = await readFile("scripts/run-and-validate-provider.ts", "utf8"); + expect(provider).toContain("...PROVIDER_SANDBOX_NAMESPACE_ARGUMENTS,"); + expect(provider).not.toMatch(/"--unshare-/u); + }); + + it("fails with the namespace restriction, bwrap's stderr and what to do when bwrap cannot set up loopback", async () => { + const { host } = fakeHost( + { status: 1, stderr: `${RTM_NEWADDR}\n` }, + { "/proc/sys/kernel/apparmor_restrict_unprivileged_userns": "1\n" }, + ); + + const failures = await findSystemTestPrerequisiteFailures(host); + + expect(failures).toHaveLength(1); + const failure = failures[0] ?? ""; + expect(failure).toMatch(/^this host does not allow unprivileged user and network namespaces/u); + expect(failure).toContain("--unshare-net"); + expect(failure).toContain(`bwrap stderr: ${RTM_NEWADDR}`); + expect(failure).toContain("kernel.apparmor_restrict_unprivileged_userns = 1"); + expect(failure).toMatch(/AppArmor profile/u); + expect(failure).toMatch(/sysctl kernel\.apparmor_restrict_unprivileged_userns=0/u); + expect(failure).toMatch(/another runner/u); + expect(formatSystemTestPrerequisiteReport(failures)).toContain( + " - this host does not allow unprivileged user and network namespaces", + ); + }); + + it("fails the same way when the probe cannot run to the end", async () => { + const { host } = fakeHost({ status: null, stderr: "", error: new Error("spawnSync /usr/bin/bwrap ETIMEDOUT") }); + + const failures = await findSystemTestPrerequisiteFailures(host); + + expect(failures).toHaveLength(1); + expect(failures[0]).toMatch(/^this host does not allow unprivileged user and network namespaces/u); + expect(failures[0]).toContain("did not finish: spawnSync /usr/bin/bwrap ETIMEDOUT"); + }); + + it("keeps the executable, cgroup v2 and systemd bus checks, and probes bwrap only when it is there", async () => { + const calls: SpawnCall[] = []; + const host: SystemTestHost = { + access: async (target) => { + if (target === "/usr/bin/bwrap") throw new Error(`EACCES: ${target}`); + }, + readFile: async () => "\n", + spawn: (command, args) => { + calls.push({ command, args }); + return { status: 1, stderr: "Failed to connect to bus" }; + }, + }; + + expect(await findSystemTestPrerequisiteFailures(host)).toEqual([ + "bubblewrap is required at /usr/bin/bwrap", + "cgroup v2 controllers are unavailable", + "a reachable systemd manager bus is required", + ]); + expect(calls.map(({ command }) => command)).toEqual(["/usr/bin/systemctl"]); + }); +}); From 629c713ce4fefa4ce8a7a9c73c55e79b8031905d Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Thu, 5 Nov 2026 22:12:30 +0900 Subject: [PATCH 2/2] =?UTF-8?q?fix(ci):=20immutable=5Fbuild=EA=B0=80=20gat?= =?UTF-8?q?e=20=EB=A1=9C=EA=B7=B8=EC=99=80=20advisory=20=EA=B8=B0=EB=A1=9D?= =?UTF-8?q?=EC=9D=84=20=ED=95=AD=EC=83=81=20=EC=98=AC=EB=A6=B0=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 문제: ci:gate는 명령 출력을 artifacts/quality/gates/FE-GATE-.txt에만 쓰고 콘솔에는 PASS/FAIL 한 줄만 남긴다(scripts/run-ci-gate.ts). release 단계 job immutable_build는 후보 tar만, 그것도 성공했을 때만 올린다. 그래서 실제 run에서 FE-GATE-015·027이 왜 실패했는지와 FE-GATE-027이 쓰는 artifacts/security/dependency-advisories.json(1103/13-03의 신호)이 runner 작업 디렉터리와 함께 사라진다. 다른 gate job은 artifacts/를 if: always()로 올린다. 고른 방법: - config/ci/gates.json의 immutable_build 마지막에 upload 단계를 더한다. transfer ID는 immutable-gate-evidence, 이름은 immutable-gate-evidence--, always: true. 대상은 FE-GATE-015·027의 log와 evidence 산출물(gate 순서)과 advisory 기록 7개다. FE-GATE-015의 verification.json 등은 후보가 만들어지지 않으면 후보 안에도 없으므로 같이 올린다. - 계약(scripts/contracts/ci-gates.ts)이 이 단계를 고정한다. immutable_build의 단계 목록에 upload 하나가 늘고, 이 upload가 always이고 마지막 단계이며 경로가 "job이 도는 gate의 log·evidence 경로 + advisory 기록"과 같아야 한다. 경로는 계약의 artifact 등록에서 계산하므로 log 경로가 바뀌면 함께 따라온다. advisory 기록은 gate evidence가 아니라(막지 않는 신호) 경로를 scripts/lib/dependency-advisories.ts의 상수로 옮기고 scan 스크립트도 그 상수를 쓴다. - 후보 archive의 member set, 후보 upload(always 없음), 하위 job의 download는 그대로다. 생성 workflow를 generator 출력과 같게 손으로 고쳤다. 버린 대안: - immutable_build도 artifacts/ 전체를 올림: 다른 job과 같지만 후보 증거가 두 이름으로 나가 어느 것이 승격 대상인지 흐려진다. 필요한 파일만 고정했다. - ci:gate가 실패한 명령의 마지막 N줄을 콘솔에 씀(리뷰 대안 c, d의 콘솔 한 줄 포함): 모든 job에 걸리고 provider 출력의 비밀 노출과 로그 상한을 다시 따져야 한다. 담당 지시 범위(업로드) 밖이라 이번에 하지 않았다. 위험·한계: - checkout·install 단계에서 실패하면 올릴 파일이 없어 이 단계도 if-no-files-found: error로 실패한다. job은 이미 실패한 뒤라 결과는 같다. merge·release job의 upload도 같은 동작이다. - 실제 Gitea runner에서 stage=release를 돌려 산출물이 남는지는 확인하지 못했다(runner 없음). 확인한 것은 계약·생성 workflow까지다. - 콘솔에는 여전히 한 줄만 나온다. 사람이 산출물을 열어야 이유를 본다. 검증: desktop batch에서 확인 - node scripts/run-vitest.ts run tests/unit/ci-workflow-generation.test.ts - corepack pnpm check:ci, check:types, lint, verify:documentation Refs: 1105/13-01, 1103/13-03 --- .gitea/workflows/quality-gates.yml | 14 +++ config/ci/gates.json | 15 +++ docs/operations/ci-quality-gates.md | 12 ++- scripts/contracts/ci-gates.ts | 46 +++++++- scripts/lib/dependency-advisories.ts | 6 ++ scripts/scan-dependency-advisories.ts | 7 +- .../ci-workflow-generation.test.ts.snap | 14 +++ tests/unit/ci-workflow-generation.test.ts | 101 +++++++++++++++++- 8 files changed, 210 insertions(+), 5 deletions(-) diff --git a/.gitea/workflows/quality-gates.yml b/.gitea/workflows/quality-gates.yml index 796ff2cc..35af29d9 100644 --- a/.gitea/workflows/quality-gates.yml +++ b/.gitea/workflows/quality-gates.yml @@ -207,6 +207,20 @@ jobs: name: "release-candidate-${{ gitea.run_id }}-${{ gitea.run_attempt }}" path: ".release/release-candidate-${{ gitea.run_id }}-${{ gitea.run_attempt }}.tar.gz" if-no-files-found: error + - name: Upload immutable gate evidence + if: always() + uses: https://github.com/ChristopherHX/gitea-upload-artifact@81f940d004763f986ba3582c007fd842dd5cb0d7 + with: + name: "immutable-gate-evidence-${{ gitea.run_id }}-${{ gitea.run_attempt }}" + path: | + artifacts/quality/gates/FE-GATE-015.txt + artifacts/release/release-candidate.json + artifacts/release/verification.json + artifacts/security/supply-chain-coherence.json + artifacts/quality/gates/FE-GATE-027.txt + artifacts/release/deployment-admission.json + artifacts/security/dependency-advisories.json + if-no-files-found: error vulnerability_provider: name: external-vulnerability-provider diff --git a/config/ci/gates.json b/config/ci/gates.json index f32e169e..31175c46 100644 --- a/config/ci/gates.json +++ b/config/ci/gates.json @@ -2508,6 +2508,21 @@ "paths": [ ".release/release-candidate-${{ gitea.run_id }}-${{ gitea.run_attempt }}.tar.gz" ] + }, + { + "kind": "upload", + "transferId": "immutable-gate-evidence", + "name": "immutable-gate-evidence-${{ gitea.run_id }}-${{ gitea.run_attempt }}", + "paths": [ + "artifacts/quality/gates/FE-GATE-015.txt", + "artifacts/release/release-candidate.json", + "artifacts/release/verification.json", + "artifacts/security/supply-chain-coherence.json", + "artifacts/quality/gates/FE-GATE-027.txt", + "artifacts/release/deployment-admission.json", + "artifacts/security/dependency-advisories.json" + ], + "always": true } ] }, diff --git a/docs/operations/ci-quality-gates.md b/docs/operations/ci-quality-gates.md index 1171b76e..511689b1 100644 --- a/docs/operations/ci-quality-gates.md +++ b/docs/operations/ci-quality-gates.md @@ -163,7 +163,12 @@ Release tier의 `immutable_build`는 `dist`의 hidden `.vite` entries, raw 번 생성해 run ID/attempt로 구분된 archive로 게시한다. Raw lock bytes의 SHA-256은 dependency inventory와 candidate manifest 양쪽의 `lockfileSha256`과 같아야 한다. Producer는 게시 전에 manifest가 선언한 exact member set과 각 byte digest를 -검증하고 archive SHA-256을 job output으로 고정한다. Vulnerability/provenance +검증하고 archive SHA-256을 job output으로 고정한다. 같은 job은 마지막 단계에서 +FE-GATE-015·027의 gate log와 evidence, `scan:dependency-advisories` 기록을 후보와 +다른 이름(`immutable-gate-evidence--`)으로 성공·실패와 상관없이 +올린다(1105/13-01). `ci:gate`는 명령 출력을 gate log에만 쓰고 콘솔에는 한 줄만 +남기므로, release 단계의 실패 이유는 이 산출물에서 읽는다. 후보 archive의 member +set은 그대로다. Vulnerability/provenance provider job은 같은 archive를 각각 내려받아 output SHA-256과 exact member set을 확인한 뒤 checkout과 분리된 root에만 추출하고, `CANDIDATE_LOCKFILE_PATH`와 `CANDIDATE_DIST_SHA256`를 외부 command에 전달한다. Provider command 실행 후에는 @@ -525,6 +530,11 @@ Repository variables required by higher tiers: findings) or `UNVERIFIED` (no registry). It never fails the gate, because an advisory database that changes under unchanged code would break releases; the signed verdict of the vulnerability provider still decides promotion. + The job console shows only one PASS/FAIL line per gate, so the + immutable_build job uploads both gates' logs and evidence and this record as + the run artifact `immutable-gate-evidence--`, whether the job + passed or failed (1105/13-01). Read why FE-GATE-015 or FE-GATE-027 failed, + and the advisory status, there. `renovate.json` opens security update PRs once a Renovate runner is attached; a dependency update re-approves the dependency baseline in the same change (`update:dependency-baseline`). diff --git a/scripts/contracts/ci-gates.ts b/scripts/contracts/ci-gates.ts index 5111a329..ae02aedb 100644 --- a/scripts/contracts/ci-gates.ts +++ b/scripts/contracts/ci-gates.ts @@ -4,6 +4,7 @@ import path from "node:path"; import { z } from "zod"; +import { DEPENDENCY_ADVISORIES_RECORD_PATH } from "../lib/dependency-advisories.ts"; import { PROMOTION_FORMULA } from "../lib/promotion-readiness.ts"; import { RELEASE_CANDIDATE_EVIDENCE_PATHS, @@ -952,7 +953,7 @@ function validateContractSemantics( const expectedStepKinds: Readonly> = { merge_gate: ["checkout", "setup-node", "frozen-install", "browser-install", "run-gate", "upload"], release_gate: ["checkout", "setup-node", "frozen-install", "browser-install", "run-gate", "upload"], - immutable_build: ["checkout", "setup-node", "frozen-install", "run-gate", "archive-candidate", "upload"], + immutable_build: ["checkout", "setup-node", "frozen-install", "run-gate", "archive-candidate", "upload", "upload"], vulnerability_provider: ["checkout", "setup-node", "frozen-install", "download", "run-provider", "validate-provider-evidence", "upload"], provenance_provider: ["checkout", "setup-node", "frozen-install", "download", "run-provider", "validate-provider-evidence", "upload"], promotion: ["checkout", "setup-node", "frozen-install", "download", "download", "download", "verify-promotion", "upload", "cleanup-promotion"], @@ -1150,6 +1151,29 @@ function validateContractSemantics( validateCanonicalStepFields(contract, issue); } +const IMMUTABLE_GATE_EVIDENCE_TRANSFER_ID = "immutable-gate-evidence"; + +/** + * 1105/13-01. What immutable_build uploads besides the candidate: the log and + * evidence artifacts of each gate it runs, in gate order, then the advisory + * record of scan:dependency-advisories (FE-GATE-027), which is not a gate + * evidence artifact because it never blocks. + */ +function immutableGateEvidencePaths( + contract: Pick, "artifacts" | "gates">, + immutable: Pick, "gateIds">, +): string[] { + const paths = immutable.gateIds.flatMap((gateId) => { + const gate = contract.gates.find(({ id }) => id === gateId); + return gate + ? [gate.logArtifactId, ...gate.evidenceArtifactIds].flatMap((artifactId) => + contract.artifacts.filter(({ id }) => id === artifactId).map(({ path: artifactPath }) => artifactPath), + ) + : []; + }); + return [...new Set([...paths, DEPENDENCY_ADVISORIES_RECORD_PATH])]; +} + function validateCanonicalStepFields( contract: z.infer, issue: (message: string, path?: PropertyKey[]) => void, @@ -1168,6 +1192,26 @@ function validateCanonicalStepFields( ) { issue("immutable archive and upload fields must remain linked"); } + // 1105/13-01. ci:gate writes command output only to the gate log and prints + // one PASS/FAIL line, and this job uploaded only the candidate, on success. + // Why FE-GATE-015 or FE-GATE-027 failed, and the advisory record FE-GATE-027 + // writes, went with the runner's workspace. A separate transfer, always and + // last, carries them; the candidate archive's member set stays as it is. + const gateEvidence = immutable?.steps.find( + (step) => step.kind === "upload" && step.transferId === IMMUTABLE_GATE_EVIDENCE_TRANSFER_ID, + ); + if ( + !immutable || + gateEvidence?.kind !== "upload" || + gateEvidence.always !== true || + immutable.steps.at(-1) !== gateEvidence || + JSON.stringify(gateEvidence.paths) !== JSON.stringify(immutableGateEvidencePaths(contract, immutable)) + ) { + issue( + "immutable_build must upload its gates' logs and evidence and the dependency advisory record " + + `as ${IMMUTABLE_GATE_EVIDENCE_TRANSFER_ID}, with always, as its last step`, + ); + } const providerExpectations = { vulnerability_provider: { diff --git a/scripts/lib/dependency-advisories.ts b/scripts/lib/dependency-advisories.ts index 083540be..ffcd91ee 100644 --- a/scripts/lib/dependency-advisories.ts +++ b/scripts/lib/dependency-advisories.ts @@ -1,5 +1,11 @@ import { validateVulnerabilityReport } from "./supply-chain.ts"; +/** + * Where scan:dependency-advisories writes its record. 1105/13-01: the + * immutable_build job uploads it on every outcome (scripts/contracts/ci-gates.ts). + */ +export const DEPENDENCY_ADVISORIES_RECORD_PATH = "artifacts/security/dependency-advisories.json"; + /** * 1103/13-03. The vulnerability verdict belongs to the external provider of * the production promotion (signed, independent of the build). Without a diff --git a/scripts/scan-dependency-advisories.ts b/scripts/scan-dependency-advisories.ts index dc5a8fdf..14621e2e 100644 --- a/scripts/scan-dependency-advisories.ts +++ b/scripts/scan-dependency-advisories.ts @@ -2,7 +2,10 @@ import { spawnSync } from "node:child_process"; import { createHash } from "node:crypto"; import { mkdir, readFile, writeFile } from "node:fs/promises"; -import { summarizeDependencyAdvisories } from "./lib/dependency-advisories.ts"; +import { + DEPENDENCY_ADVISORIES_RECORD_PATH, + summarizeDependencyAdvisories, +} from "./lib/dependency-advisories.ts"; /** * 1103/13-03. Release-stage signal for known dependency advisories @@ -17,7 +20,7 @@ import { summarizeDependencyAdvisories } from "./lib/dependency-advisories.ts"; * result is UNVERIFIED, not PASS. The signed verdict of the vulnerability * provider stays the only one that decides a promotion. */ -const RECORD_PATH = "artifacts/security/dependency-advisories.json"; +const RECORD_PATH = DEPENDENCY_ADVISORIES_RECORD_PATH; const [policy, exceptions, lockfile] = await Promise.all([ readFile("config/security/vulnerability-policy.json", "utf8").then((text) => JSON.parse(text) as Record), diff --git a/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap b/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap index 6b84e76f..a64963d1 100644 --- a/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap +++ b/tests/unit/__snapshots__/ci-workflow-generation.test.ts.snap @@ -210,6 +210,20 @@ jobs: name: "release-candidate-\${{ gitea.run_id }}-\${{ gitea.run_attempt }}" path: ".release/release-candidate-\${{ gitea.run_id }}-\${{ gitea.run_attempt }}.tar.gz" if-no-files-found: error + - name: Upload immutable gate evidence + if: always() + uses: https://github.com/ChristopherHX/gitea-upload-artifact@81f940d004763f986ba3582c007fd842dd5cb0d7 + with: + name: "immutable-gate-evidence-\${{ gitea.run_id }}-\${{ gitea.run_attempt }}" + path: | + artifacts/quality/gates/FE-GATE-015.txt + artifacts/release/release-candidate.json + artifacts/release/verification.json + artifacts/security/supply-chain-coherence.json + artifacts/quality/gates/FE-GATE-027.txt + artifacts/release/deployment-admission.json + artifacts/security/dependency-advisories.json + if-no-files-found: error vulnerability_provider: name: external-vulnerability-provider diff --git a/tests/unit/ci-workflow-generation.test.ts b/tests/unit/ci-workflow-generation.test.ts index 03d8ad2d..f5ad6f8a 100644 --- a/tests/unit/ci-workflow-generation.test.ts +++ b/tests/unit/ci-workflow-generation.test.ts @@ -1654,7 +1654,7 @@ describe("CI workflow generation", () => { 'if [ -n "$PROMOTION_STAGING_ROOT" ] && [ -n "$PROMOTION_CLEANUP_TOKEN" ] && [ -n "$PROMOTION_RUNNER_TEMP_DEV" ] && [ -n "$PROMOTION_RUNNER_TEMP_INO" ] && [ -n "$PROMOTION_STAGING_DEV" ] && [ -n "$PROMOTION_STAGING_INO" ]; then', ); const actionUses = [...first.matchAll(/^\s+-?\s*uses: (.+)$/gmu)].map((match) => match[1]); - expect(actionUses).toHaveLength(33); + expect(actionUses).toHaveLength(34); expect(new Set(actionUses)).toEqual( new Set([ "https://github.com/actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5", @@ -1743,6 +1743,105 @@ describe("CI workflow generation", () => { expect(workflowGateExecutionFailures(contract, workflow)).toEqual([]); }); + // 1105/13-01. ci:gate writes each command's output only to the gate log and + // prints one PASS/FAIL line. immutable_build uploaded only the candidate, on + // success, so why FE-GATE-015 or FE-GATE-027 failed, and the advisory record + // FE-GATE-027 writes (1103/13-03), went with the runner's workspace. + const IMMUTABLE_GATE_EVIDENCE_PATHS = [ + "artifacts/quality/gates/FE-GATE-015.txt", + "artifacts/release/release-candidate.json", + "artifacts/release/verification.json", + "artifacts/security/supply-chain-coherence.json", + "artifacts/quality/gates/FE-GATE-027.txt", + "artifacts/release/deployment-admission.json", + "artifacts/security/dependency-advisories.json", + ]; + + it("uploads immutable_build's gate logs, evidence and advisory record on every outcome (1105/13-01)", async () => { + const contract = await loadCiGateContract(process.cwd()); + const index = indexCiGateContract(contract); + const gatePaths = ["FE-GATE-015", "FE-GATE-027"].flatMap((gateId) => { + const gate = index.gates.get(gateId)!; + return [gate.logArtifactId, ...gate.evidenceArtifactIds].map((artifactId) => index.artifacts.get(artifactId)!.path); + }); + expect(IMMUTABLE_GATE_EVIDENCE_PATHS).toEqual([...gatePaths, "artifacts/security/dependency-advisories.json"]); + + const workflow = renderCiWorkflow(contract); + const immutable = workflow.slice( + workflow.indexOf(" immutable_build:\n"), + workflow.indexOf(" vulnerability_provider:\n"), + ); + const candidateUpload = immutable.indexOf(" - name: Upload release candidate\n"); + const evidenceUpload = immutable.indexOf(" - name: Upload immutable gate evidence\n"); + expect(candidateUpload).toBeGreaterThan(immutable.indexOf("ci:gate -- FE-GATE-027")); + expect(evidenceUpload).toBeGreaterThan(candidateUpload); + expect(immutable.slice(evidenceUpload)).toBe( + [ + " - name: Upload immutable gate evidence", + " if: always()", + ` uses: ${resolveCiActionUses("uploadArtifact")}`, + " with:", + ' name: "immutable-gate-evidence-${{ gitea.run_id }}-${{ gitea.run_attempt }}"', + " path: |", + ...IMMUTABLE_GATE_EVIDENCE_PATHS.map((target) => ` ${target}`), + " if-no-files-found: error", + "", + "", + ].join("\n"), + ); + // The candidate still goes up only after its archive was validated, and + // the archive's member set is not widened by the evidence. + expect(immutable.slice(candidateUpload, evidenceUpload)).not.toContain("if: always()"); + const archive = contract.jobs + .find(({ id }) => id === "immutable_build") + ?.steps.find((step) => step.kind === "archive-candidate"); + expect(archive?.kind === "archive-candidate" ? archive.members : []).not.toContain( + "artifacts/quality/gates/FE-GATE-027.txt", + ); + }); + + it.each([ + [ + "is missing", + (steps: Record[]) => steps.splice(steps.findIndex((step) => step.transferId === "immutable-gate-evidence"), 1), + ], + [ + "runs only on success", + (steps: Record[]) => delete steps.find((step) => step.transferId === "immutable-gate-evidence")!.always, + ], + [ + "leaves out the advisory record", + (steps: Record[]) => { + const upload = steps.find((step) => step.transferId === "immutable-gate-evidence")!; + upload.paths = upload.paths.filter((target: string) => target !== "artifacts/security/dependency-advisories.json"); + }, + ], + [ + "leaves out the FE-GATE-027 log", + (steps: Record[]) => { + const upload = steps.find((step) => step.transferId === "immutable-gate-evidence")!; + upload.paths = upload.paths.filter((target: string) => target !== "artifacts/quality/gates/FE-GATE-027.txt"); + }, + ], + [ + "comes before the candidate upload", + (steps: Record[]) => { + const evidence = steps.splice(steps.findIndex((step) => step.transferId === "immutable-gate-evidence"), 1)[0]!; + steps.splice(steps.findIndex((step) => step.transferId === "release-candidate"), 0, evidence); + }, + ], + ])("rejects an immutable_build whose gate evidence upload %s (1105/13-01)", async (_name, mutate) => { + const contract = JSON.parse( + JSON.stringify(await loadCiGateContract(process.cwd())), + ) as Record; + const steps = contract.jobs.find((job: Record) => job.id === "immutable_build").steps; + expect(steps.filter((step: Record) => step.transferId === "immutable-gate-evidence")).toHaveLength(1); + mutate(steps); + expect(() => parseCiGateContract(contract)).toThrow( + /immutable_build must upload its gates' logs and evidence and the dependency advisory record/u, + ); + }); + it.each([ ["a non-matrix gate step", " run: corepack pnpm ci:gate -- FE-GATE-027\n", "FE-GATE-027"], ["a matrix entry", " - { gate: FE-GATE-020, name: removability, browser: false }\n", "FE-GATE-020"],