test(system): provider 테스트 묶음 전체에 provider 시간 예산을 쓴다

문제
tests/system/ci-runner/ci-artifact-contract.test.ts는 provider 묶음에 30초
예산(PROCESS_HEAVY_TIMEOUT_MS)을 두고 "10s default is sized for pure-JS
unit tests"라고 적는다. 그런데 같은 describe의 테스트 선언 8개(케이스
10개)는 여전히 10초다. 그래서 바쁜 호스트에서 "kills descendants … timeout
termination"이 10,116 ms로 시간 초과해 FE-GATE-013이 가끔 빨개진다. 또 기반
fixture(build:release-candidate)를 처음 createProviderFixture()를 부른 테스트
안에서 만들어서, -t로 하나만 골라 돌리면 그 테스트가 빌드를 떠안아 항상
실패한다.

고른 방법
- describe 옵션으로 묶음 전체에 PROCESS_HEAVY_TIMEOUT_MS를 주고, 테스트마다
  적은 10_000·15_000·20_000·30_000을 지운다. 40_000을 준 tar 폭탄 테스트
  하나는 그대로 둔다.
- 기반 fixture는 describe의 beforeAll에서 만든다. 상한은 빌드의 spawn
  상한(120초)에 복사·압축 여유를 더한 180초다.
- 버린 대안: 전역 testTimeout 올리기. 파일 주석이 이미 버렸다(진짜 멈춘
  테스트를 숨긴다). 테스트마다 상수로 바꾸기. 상수를 안 쓰는 테스트가 다시
  생길 수 있다.

위험·한계
- 실제로 멈춘 provider 테스트는 실패까지 최대 30초 걸린다.
- runProviderSupervisor의 spawn 상한(기본 8초)은 바꾸지 않았다. 부하에서
  systemd user manager가 느리면 "provider unit remained loaded"로 따로
  실패할 수 있다(이번 실행에서 load 24일 때 한 번 봄, 운영 코드의 상한).
- loadavg 20 이상에서 test:system 전체 5회 반복은 하지 않았다.

검증
- -t "kills descendants|hides repository metadata"로 골라 돌렸을 때 두
  kills descendants 테스트가 통과(3.0 s, 2.9 s). 기반 빌드는 beforeAll이
  떠안았다.
- eslint 통과.

Refs: docs/reviews/2026-11-03-maturity-review/11-tests-browser-release-scale.md 11-03
This commit is contained in:
DongHyeonka
2026-11-04 21:03:59 +09:00
parent c4d587052e
commit 90440082f9
@@ -6,7 +6,7 @@ import { tmpdir } from "node:os";
import path from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { afterAll, afterEach, describe, expect, it } from "vitest";
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
import type {
CiGateArtifact,
@@ -62,6 +62,11 @@ import {
* default instead would hide a genuinely hung test.
*/
const PROCESS_HEAVY_TIMEOUT_MS = 30_000;
/**
* The provider base fixture runs `build:release-candidate`, bounded at 120s
* (ensureProviderBaseFixture), and then copies and archives the result.
*/
const PROVIDER_BASE_FIXTURE_TIMEOUT_MS = 180_000;
const temporaryRoots: string[] = [];
let providerBaseRoot: string | undefined;
@@ -448,7 +453,16 @@ describe("CI artifact validator", () => {
});
});
describe("candidate archive and provider upload boundaries", () => {
// 1103/11-03. Every test here spawns a provider launch chain or copies the
// release-candidate fixture, so the whole block runs on the provider budget;
// eight tests were still on the 10s unit default and timed out under load.
// The base fixture is built once in beforeAll, on its own ceiling, so a test
// selected alone with -t no longer pays for the build inside its own budget.
describe("candidate archive and provider upload boundaries", { timeout: PROCESS_HEAVY_TIMEOUT_MS }, () => {
beforeAll(async () => {
await ensureProviderBaseFixture();
}, PROVIDER_BASE_FIXTURE_TIMEOUT_MS);
it("accepts only the manifest-bound candidate member set and bytes", async () => {
const fixture = await createCandidateArchiveFixture();
await expect(
@@ -601,7 +615,7 @@ describe("candidate archive and provider upload boundaries", () => {
providerValidationInput(fixture, Buffer.from(`${JSON.stringify(report)}\n`)),
),
).rejects.toThrow(/candidate identity mismatch/i);
}, 30_000);
});
it("uses the reverified archive manifest when extracted candidate files are mutated", async () => {
const fixture = await createProviderFixture();
@@ -686,7 +700,6 @@ describe("candidate archive and provider upload boundaries", () => {
});
expect(retried.status, retried.stderr).toBe(0);
},
20_000,
);
it("drains and kills provider background processes before sealing evidence", async () => {
@@ -728,7 +741,7 @@ describe("candidate archive and provider upload boundaries", () => {
expect(JSON.parse(await readFile(sealedPath, "utf8"))).toEqual(
expect.objectContaining({ provider: "fixture" }),
);
}, 10_000);
});
it("does not expose or mutate a host path outside the sandboxed workspace", async () => {
const fixture = await createProviderFixture();
@@ -749,7 +762,7 @@ describe("candidate archive and provider upload boundaries", () => {
});
expect(result.status, result.stderr).toBe(0);
await expect(readFile(canary, "utf8")).resolves.toBe("host-secret\n");
}, 10_000);
});
it("applies effective aggregate cgroup limits without exposing command or credentials", async () => {
const fixture = await createProviderFixture();
@@ -851,7 +864,7 @@ describe("candidate archive and provider upload boundaries", () => {
expect(trace.every((entry) => !processExists(entry.pid))).toBe(true);
await expect(lstat(fixture.reportPath)).rejects.toMatchObject({ code: "ENOENT" });
await expect(findProviderRawReferences(fixture.reportPath, reportIdentity)).resolves.toEqual([]);
}, PROCESS_HEAVY_TIMEOUT_MS);
});
it("kills and collects an active provider when its guardian dies", async () => {
const fixture = await createProviderFixture();
@@ -894,7 +907,7 @@ describe("candidate archive and provider upload boundaries", () => {
sealedPath,
});
expect(retried.status, retried.stderr).toBe(0);
}, 20_000);
});
it("collects the whole provider scope when its supervisor dies", async () => {
const fixture = await createProviderFixture();
@@ -934,7 +947,7 @@ describe("candidate archive and provider upload boundaries", () => {
await expect(lstat(cgroupRoot)).rejects.toMatchObject({ code: "ENOENT" });
expect(cgroupPids.every((pid) => !processExists(pid))).toBe(true);
await expect(findProviderRawReferences(fixture.reportPath, reportIdentity)).resolves.toEqual([]);
}, 10_000);
});
it("fails closed when the guardian dies after scope collection and before commit", async () => {
const fixture = await createProviderFixture();
@@ -979,7 +992,7 @@ describe("candidate archive and provider upload boundaries", () => {
await execution.completion;
}
}
}, 10_000);
});
it("cleans published evidence when the supervisor dies and permits same-workspace retry", async () => {
const fixture = await createProviderFixture();
@@ -1025,7 +1038,7 @@ describe("candidate archive and provider upload boundaries", () => {
await execution.completion;
}
}
}, 15_000);
});
it("observes EMFILE at the provider FD limit and still emits valid evidence", async () => {
const fixture = await createProviderFixture();
@@ -1046,7 +1059,7 @@ describe("candidate archive and provider upload boundaries", () => {
});
expect(result.status, result.stderr).toBe(0);
await expect(lstat(fixture.reportPath)).rejects.toMatchObject({ code: "ENOENT" });
}, 10_000);
});
it("enforces CPU RLIMIT before the independent wall-clock timeout", async () => {
const fixture = await createProviderFixture();
@@ -1063,7 +1076,7 @@ describe("candidate archive and provider upload boundaries", () => {
expect(result.stderr).toMatch(/exit=(?:137|152)|signal=SIG(?:XCPU|KILL)/iu);
expect(result.stderr).not.toMatch(/timed out/iu);
await expect(lstat(fixture.reportPath)).rejects.toMatchObject({ code: "ENOENT" });
}, 10_000);
});
it.each(["output", "timeout"] as const)(
"kills descendants and collects the scope on provider %s termination",
@@ -1107,7 +1120,6 @@ describe("candidate archive and provider upload boundaries", () => {
expect(cgroupPids.every((pid) => !processExists(pid))).toBe(true);
await expect(findProviderRawReferences(fixture.reportPath, reportIdentity)).resolves.toEqual([]);
},
10_000,
);
it.each(["vulnerability", "provenance"] as const)(
@@ -1150,7 +1162,6 @@ describe("candidate archive and provider upload boundaries", () => {
await waitForChildClose(canary.child);
}
},
10_000,
);
});