From 2c180b90595e8d312da974997a49fe27ca25eeeb Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Thu, 12 Nov 2026 01:37:37 +0900 Subject: [PATCH] =?UTF-8?q?ci(main):=20main=EA=B3=BC=20integration=20?= =?UTF-8?q?=EB=8B=A8=EA=B3=84=EB=A5=BC=20job=20=ED=95=98=EB=82=98,=20Gradl?= =?UTF-8?q?e=20=EC=8B=A4=ED=96=89=20=ED=95=9C=20=EB=B2=88=EC=9C=BC?= =?UTF-8?q?=EB=A1=9C=20=EB=8F=88=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 문제: main push는 `main`(17.1분)과 `integration`(17.4분) 두 job을 차례로 돈다 (run 79, runner capacity 1). 두 job이 같은 모듈을 각자 설정·컴파일하고, integration의 컨테이너 lane은 실서버 슬롯(ExclusiveRealServerLaneService) 하나를 차례로 쓰는 동안 나머지 worker가 놀고, main의 분석(testQualityCheck, 생성 명령 왕복의 안쪽 빌드)은 다른 job이라 그 시간에 겹치지 못한다. 고른 방법: `./gradlew :mainCheck :integrationCheck --continue` 한 줄을 `main` job이 돈다. strict lane 단계 선언에 `line`을 더해, 같은 job을 쓰는 단계가 선언한 한 줄을 나눠 쓸 수 있게 한다. 「단계마다 정해진 한 줄」 규칙은 그대로다: 두 단계 모두 그 정확한 줄을 요구하고, line에는 태스크 경로만 쓸 수 있고 자기 단계를 빼면 설정 단계에서 거절한다(TestKit 시험 셋). scripts/verify.sh의 main 단계도 같은 줄을 쓴다. 버린 대안: 한 job에 두 줄을 차례로 두기. mainCheck가 실패하면 integration이 돌지 않는다. 실서버 슬롯을 둘로 늘리기. 시간에 민감한 lane이 서로 겹쳐 불안정해진다 (그 서비스가 생긴 이유). 위험·한계: 컨테이너 lane이 main 단계의 CPU 작업과 겹친다. 시간에 민감한 system lane은 지금도 main 단계 안에서 같은 겹침을 견디고, integration lane은 대부분 컨테이너 계약이다. 한 job이 길어져(예상 22~25분) timeout을 240분으로 둔다. 검증: build-logic 시험(desktop), verifyStrictLaneStages --dry-run, runner main 수동 실행의 main job 시간(이 PR 본문에 적는다). --- .gitea/workflows/main.yml | 34 ++++------ build-logic/AGENTS.md | 2 +- .../strictlane/StrictLaneStagesExtension.java | 19 ++++-- .../StrictLaneStagesFunctionalTest.java | 66 +++++++++++++++++++ build.gradle | 10 ++- docs/ci-cd/template-vs-consumer.md | 2 +- docs/work/reference/gates.md | 6 +- scripts/verify.sh | 3 +- 8 files changed, 108 insertions(+), 34 deletions(-) diff --git a/.gitea/workflows/main.yml b/.gitea/workflows/main.yml index bb370d42..4c48b774 100644 --- a/.gitea/workflows/main.yml +++ b/.gitea/workflows/main.yml @@ -4,12 +4,13 @@ # On every push to main: the main stage, the container lanes, the build-logic tests, configuration # cache, links and the dependency scan. Nightly (and on a manual run) also the jobs that need a matrix # or an environment the stage tasks do not have, and last the performance lanes, which need a -# machine nothing else is using: the runner takes one job at a time, and `needs` puts them after -# every other heavy job (owner decision 2026-11-07). +# machine nothing else is using: the runner takes one job at a time, and the performance job is the +# last one in this file (owner decision 2026-11-07; the job says why not `needs`). # -# Which lanes a stage means is decided by its Gradle task (build.gradle). The stage jobs -# (`main`, `integration`, `performance`) run it as one line; `verifyStrictLaneStages` fails a pull -# request that changes that line, its condition, or adds a way for it to pass quietly. +# Which lanes a stage means is decided by its Gradle task (build.gradle). The stage jobs (`main` +# for the main and integration stages, `performance`) run them as one line; +# `verifyStrictLaneStages` fails a pull request that changes that line, its condition, or adds a +# way for it to pass quietly. name: main on: @@ -37,27 +38,20 @@ env: TESTCONTAINERS_REUSE_ENABLE: "false" jobs: - # Environment contract, full-boot qualification, public-path baseline, transport and messaging - # qualification, document-contract lanes, test-code analysis, the feature generator round trip. - # `--continue`: one failed lane must not hide the others. + # Both main-branch stages in one Gradle run (build.gradle `strictLaneStages` says why). + # mainCheck: environment contract, full-boot qualification, public-path baseline, transport and + # messaging qualification, document-contract lanes, test-code analysis, the feature generator + # round trip. integrationCheck: every integration-category strict lane, against real PostgreSQL, + # MongoDB, Kafka, MinIO, nginx and Toxiproxy; a lane joins by declaring `integration()`, no line + # here names it, and lanes that need an input this job does not have are explicitOnly and named + # in the output. `--continue`: one failed lane must not hide the others. main: - runs-on: ubuntu-latest - timeout-minutes: 120 - steps: - - uses: https://github.com/actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2 - - uses: ./.gitea/actions/setup-java-gradle - - run: ./gradlew :mainCheck --continue - - # Every integration-category strict lane: real PostgreSQL, MongoDB, Kafka, MinIO, nginx, Toxiproxy. - # A lane joins by declaring `integration()`; no line here names it. Lanes that need an input this - # job does not have are explicitOnly and named in the output. - integration: runs-on: ubuntu-latest timeout-minutes: 240 steps: - uses: https://github.com/actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2 - uses: ./.gitea/actions/setup-java-gradle - - run: ./gradlew :integrationCheck --continue + - run: ./gradlew :mainCheck :integrationCheck --continue # The included build's own tests (TestKit for the convention plugins). The main task graph does # not reach them; `ci` already runs every plugin they test (owner decision 2026-11-07: main stage). diff --git a/build-logic/AGENTS.md b/build-logic/AGENTS.md index 10d81125..85a4cb4c 100644 --- a/build-logic/AGENTS.md +++ b/build-logic/AGENTS.md @@ -26,7 +26,7 @@ registry, settings, build-logic, 루트 라이프사이클이 정하는 것을 ## 단계 job의 모양 - `verifyStrictLaneStages --dry-run`(PR)은 `explicitOnly`가 아닌 strict lane마다 `build.gradle`의 `strictLaneStages`가 선언한 단계 태스크 하나의 계획에 들어 있는지 본다(그래프 검사). -- 단계마다 선언한 workflow job은 step의 `run: ./gradlew <단계> <옵션>` 줄을 가지고, 다른 step `run:`도 셸 문법 없는 `./gradlew` 한 줄이나 `bash scripts/<이름>.sh` 한 줄이며, 선언 밖의 `if:`, 선언한 조건이 `!cancelled()`·`always()`가 아닌데 `needs:`, `continue-on-error`·`shell:`·`defaults:`, 따옴표 키·flow 형식, 이 빌드 밖의 `working-directory`가 없다. workflow는 선언한 이벤트로, 선언한 필터 한 줄만 달고 시작하고, PR 단계 job 하나가 이 검사를 돌린다(형식 검사, 셸을 해석하지 않는다). +- 단계마다 선언한 workflow job은 step의 `run: ./gradlew <단계> <옵션>` 줄을 가지고(같은 job을 쓰는 단계는 선언한 `line`대로 한 줄을 나눠 쓸 수 있다: `./gradlew :mainCheck :integrationCheck --continue`), 다른 step `run:`도 셸 문법 없는 `./gradlew` 한 줄이나 `bash scripts/<이름>.sh` 한 줄이며, 선언 밖의 `if:`, 선언한 조건이 `!cancelled()`·`always()`가 아닌데 `needs:`, `continue-on-error`·`shell:`·`defaults:`, 따옴표 키·flow 형식, 이 빌드 밖의 `working-directory`가 없다. workflow는 선언한 이벤트로, 선언한 필터 한 줄만 달고 시작하고, PR 단계 job 하나가 이 검사를 돌린다(형식 검사, 셸을 해석하지 않는다). - 단계를 더하거나 job을 바꾸면 `strictLaneStages`의 `stage` 줄과 workflow를 같은 커밋에서 고친다. ## 테스트 diff --git a/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java b/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java index 3cd5c32c..1aa3c22e 100644 --- a/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java +++ b/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java @@ -21,7 +21,10 @@ import org.gradle.api.file.DirectoryProperty; * } * * - *

The job runs the stage as {@code ./gradlew } on a line of its own. {@code + *

The job runs the stage as {@code ./gradlew } on a line of its own. Stages + * that share a job may share that line: each then declares the same {@code line}, the stage tasks + * in the order the line names them, and the line is {@code ./gradlew }. One Gradle + * run compiles once and lets the stages' work overlap. {@code * filter} is the one line under the event (none if absent; a schedule takes its cron lines), and * {@code condition} is the job's {@code if:}: both are written out here so that changing when a * stage runs is a reviewed change of the build rather than an edit to a workflow. Single quotes, @@ -45,7 +48,14 @@ public abstract class StrictLaneStagesExtension { job.get("filter"), required(job, "job", taskPath), job.getOrDefault("options", ""), - job.get("condition")); + job.get("condition"), + job.getOrDefault("line", taskPath)); + List lineTasks = List.of(stage.line().split(" ", -1)); + if (!lineTasks.contains(taskPath) + || !lineTasks.stream().allMatch(task -> task.startsWith(":") && task.length() > 1)) { + throw new GradleException( + "stage " + taskPath + ": line '" + stage.line() + "' must name task paths, this one too"); + } for (String option : stage.options().split(" ", -1)) { if (!option.isEmpty() && !STAGE_OPTIONS.contains(option)) { throw new GradleException( @@ -75,9 +85,10 @@ public abstract class StrictLaneStagesExtension { String filter, String job, String options, - String condition) { + String condition, + String line) { String command() { - return options.isEmpty() ? "./gradlew " + taskPath : "./gradlew " + taskPath + " " + options; + return options.isEmpty() ? "./gradlew " + line : "./gradlew " + line + " " + options; } } } diff --git a/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java b/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java index 373b7e95..3c140e24 100644 --- a/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java +++ b/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java @@ -358,6 +358,54 @@ class StrictLaneStagesFunctionalTest { assertThat(result.getOutput()).contains("run it with --dry-run"); } + @Test + @DisplayName("같은 job을 쓰는 두 단계는 선언한 line대로 한 줄을 나눠 쓸 수 있다") + void twoStagesOfOneJobMayShareTheLineTheyDeclare() throws Exception { + // given + shareTheIntegrationLineWithAMainStage(":mainCheck :integrationCheck"); + workflow( + "main.yml", + MAIN_WORKFLOW.replace( + "run: ./gradlew :integrationCheck --continue", + "run: ./gradlew :mainCheck :integrationCheck --continue")); + + // when + BuildResult result = runner(":verifyStrictLaneStages", "--dry-run").build(); + + // then + assertThat(result.getOutput()).contains("every strict lane is in a stage its workflow job runs"); + } + + @Test + @DisplayName("선언한 한 줄 대신 단계마다 따로 된 줄이 있으면 두 단계 모두 실패한다") + void aSharedLineDeclaredButNotRunFailsBothStages() throws Exception { + // given + shareTheIntegrationLineWithAMainStage(":mainCheck :integrationCheck"); + + // when + BuildResult result = runner(":verifyStrictLaneStages", "--dry-run").buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains( + "main.yml job integration has no line `run: ./gradlew :mainCheck :integrationCheck" + + " --continue`"); + } + + @Test + @DisplayName("단계 자신을 부르지 않는 line은 설정 단계에서 거절한다") + void refusesALineThatLeavesItsOwnStageOut() throws Exception { + // given + shareTheIntegrationLineWithAMainStage(":mainCheck"); + + // when + BuildResult result = runner(":verifyStrictLaneStages", "--dry-run").buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains("stage :integrationCheck: line ':mainCheck' must name task paths, this one too"); + } + @Test @DisplayName("단계 줄의 옵션이 정해진 것 밖이면 설정 단계에서 거절한다") void refusesAStageOptionThatChangesWhatRuns() throws Exception { @@ -435,6 +483,24 @@ class StrictLaneStagesFunctionalTest { .formatted(lanes)); } + /** Declares a `:mainCheck` stage on the integration job, both stages naming {@code line}. */ + private void shareTheIntegrationLineWithAMainStage(String line) throws Exception { + Files.writeString( + gradleRoot.resolve("build.gradle"), + Files.readString(gradleRoot.resolve("build.gradle")) + .replace( + "tasks.register('integrationCheck')", + "tasks.register('mainCheck')\ntasks.register('integrationCheck')") + .replace( + "job: 'integration', options: '--continue', condition: '${{ !cancelled() }}'", + "job: 'integration', options: '--continue', condition: '${{ !cancelled() }}'," + + " line: ':mainCheck :integrationCheck'\n" + + " stage ':mainCheck', workflow: 'main.yml', event: 'push'," + + " filter: 'branches: [main]', job: 'integration', options: '--continue'," + + " condition: '${{ !cancelled() }}', line: ':mainCheck :integrationCheck'") + .replace("line: ':mainCheck :integrationCheck'\n", "line: '" + line + "'\n")); + } + private void workflow(String name, String content) throws Exception { Files.writeString(repository.resolve(".gitea/workflows").resolve(name), content); } diff --git a/build.gradle b/build.gradle index fd8da614..8df4fdb1 100644 --- a/build.gradle +++ b/build.gradle @@ -243,10 +243,14 @@ tasks.register('prCheck') { strictLaneStages { workflowDirectory = layout.projectDirectory.dir('.gitea/workflows') stage ':prCheck', workflow: 'pr.yml', event: 'pull_request', job: 'done' + // One job and one Gradle run for both main-branch stages: it compiles once, and the + // main stage's analysis and the feature generator's nested build run while the integration + // lanes take turns on the one real-server slot. Two jobs spent about 34 runner minutes on a push + // (run 79), two of them compiling the same modules twice. stage ':mainCheck', workflow: 'main.yml', event: 'push', filter: 'branches: [main]', job: 'main', - options: '--continue' - stage ':integrationCheck', workflow: 'main.yml', event: 'push', filter: 'branches: [main]', job: 'integration', - options: '--continue' + options: '--continue', line: ':mainCheck :integrationCheck' + stage ':integrationCheck', workflow: 'main.yml', event: 'push', filter: 'branches: [main]', job: 'main', + options: '--continue', line: ':mainCheck :integrationCheck' stage ':performanceCheck', workflow: 'main.yml', event: 'schedule', job: 'performance', options: '--continue', condition: '${{ github.event_name != \'push\' }}' stage ':releaseCheck', workflow: 'release.yml', event: 'push', filter: 'tags: ["v*"]', job: 'release' diff --git a/docs/ci-cd/template-vs-consumer.md b/docs/ci-cd/template-vs-consumer.md index cafd9cde..3cb6295a 100644 --- a/docs/ci-cd/template-vs-consumer.md +++ b/docs/ci-cd/template-vs-consumer.md @@ -29,7 +29,7 @@ | `configContractCheck` | 환경변수 계약 | | `verifyDependencyLocks` | 재현 가능한 의존성 해석 | | `pr.yml`(`./gradlew :prCheck`, 필수 검사 `pr / done`) | PR 단계. 무엇을 돌리는지는 Gradle 태스크 `prCheck`가 정한다 | -| `main.yml`의 `main`·`integration`·`build-logic`·`configuration-cache` job | main 단계(`mainCheck`, `integrationCheck`) | +| `main.yml`의 `main`(두 단계를 한 줄로)·`build-logic`·`configuration-cache` job | main 단계(`mainCheck`, `integrationCheck`) | | `main.yml`의 `performance` job | nightly 성능 단계(`performanceCheck`) | | `pr.yml` `done`의 Trivy step과 `main.yml`의 `vulnerabilities` job(`scripts/ci/trivy-scan.sh`) | Trivy와 CISA KEV | | `release.yml` | `releaseCheck`, 이미지 · SBOM · 서명 | diff --git a/docs/work/reference/gates.md b/docs/work/reference/gates.md index 68a9885c..1e76a01a 100644 --- a/docs/work/reference/gates.md +++ b/docs/work/reference/gates.md @@ -17,7 +17,7 @@ Gradle 명령과 `scripts/verify.sh`는 모두 저장소 루트(Gradle 루트) | `docs` | `check-work-docs.py`, `check-docs.py`, `scripts/tests` 자기 시험, `check-repo-layout.py`(Gradle 없음, 노트북 가능) | `pr.yml`의 `prCheck` 안 | | `leaf ` | 그 leaf의 `check` | — | | `pr` | `./gradlew :prCheck`, `./gradlew :verifyStrictLaneStages --dry-run` | `pr.yml` `done` | -| `main` | `:mainCheck --continue`, `:integrationCheck --continue`, `-p build-logic test`, `scripts/ci/verify-configuration-cache.sh` | `main.yml`(push) | +| `main` | `:mainCheck :integrationCheck --continue`, `-p build-logic test`, `scripts/ci/verify-configuration-cache.sh` | `main.yml`(push) | | `nightly` | compose smoke 전부, 마지막에 `:performanceCheck --continue` | `main.yml`(schedule·수동) | | `release` | `:releaseCheck`(지금 커밋을 provenance로) | `release.yml` `release` | | `app` | `scripts/bootstrap-local.sh`(앱을 띄워 health와 기본 endpoint 확인) | — | @@ -39,8 +39,8 @@ workflow에만 있는 job: `pr.yml` `done`의 마지막 step과 `main.yml`의 `v | `verifyStrictLaneStages --dry-run` | `explicitOnly`가 아닌 strict lane이 단계 태스크 하나의 계획에 들어 있는가(그래프 검사), 단계마다 선언한 workflow job이 `./gradlew <단계>` 한 줄을 정해진 조건으로 돌리는가(형식 검사) | PR | `pr.yml` `done` 둘째 줄. 형식은 `build-logic/AGENTS.md` | | `python3 scripts/check-repo-layout.py` | 저장소 루트와 `src/` 최상위에 허용 목록 밖의 항목이 없는가, registry 경로가 허용 폴더 안인가. 아무도 가리키지 않는 compose·스크립트 파일은 경고 | PR(`prCheck` 안) | `verifyRepositoryLayout` | | `python3 scripts/check-docs.py` | 살아 있는 문서의 언어, 기록·ADR 머리 블록, 문서의 인라인 코드 경로·Gradle 경로가 지금 있는가, 복사한 숫자, 상대 링크(`docs/README.md` 「문서 검사」) | PR(`prCheck` 안) | `documentCheck` | -| `mainCheck` | `configContractCheck` + `qualificationCheck`(`testQualityCheck` 포함) + 공개 경로 스냅숏 + 전송·메시징 qualification lane + 생성 명령 왕복(`verifyFeatureScaffold`) + 모든 leaf의 `mainCheck`(document-contract lane 등) | main | `main.yml` `main`: `./gradlew :mainCheck --continue` | -| `integrationCheck` | 선언된 strict integration lane 전체(실 컨테이너). 입력이 없는 lane은 빼고 이름을 댄다 | main·nightly | `main.yml` `integration` | +| `mainCheck` | `configContractCheck` + `qualificationCheck`(`testQualityCheck` 포함) + 공개 경로 스냅숏 + 전송·메시징 qualification lane + 생성 명령 왕복(`verifyFeatureScaffold`) + 모든 leaf의 `mainCheck`(document-contract lane 등) | main | `main.yml` `main`: `./gradlew :mainCheck :integrationCheck --continue` | +| `integrationCheck` | 선언된 strict integration lane 전체(실 컨테이너). 입력이 없는 lane은 빼고 이름을 댄다 | main·nightly | `main.yml` `main`: `./gradlew :mainCheck :integrationCheck --continue`(두 단계를 Gradle 한 번에. 한 번 컴파일하고, 직렬 lane이 도는 동안 나머지가 겹친다) | | `performanceCheck` | 선언된 performance lane 전체(부하·종료·풀·자원 한계, 실 서버와 컨테이너). 상한을 매번 단언한다. 손으로 돌리든 workflow든 같은 상한이다 | nightly(다른 무거운 job 뒤: 파일 맨 끝 job이고 runner 하나가 job을 만든 순서대로 돈다. `needs`는 쓰지 않는다 — Gitea 1.27이 `needs`와 `if`를 함께 가진 job을 Blocked에 둔다) | `main.yml` `performance`. 5절 | | `releaseCheck` | 릴리스 provenance + 공개 경로 스냅숏. 커밋을 `GITHUB_SHA`·`GIT_SHA`·`-PgitRevision=`로 받고, 없으면 설계대로 실패한다 | 태그 | `release.yml` `release` | | `:app-bootstrap:verifyEnvKeys` | env 키 레지스트리와 profile YAML·`.env.example` | `mainCheck` 안(`configContractCheck`) | build-logic `ca.config-contract` | diff --git a/scripts/verify.sh b/scripts/verify.sh index bb816b03..6d5d386d 100755 --- a/scripts/verify.sh +++ b/scripts/verify.sh @@ -71,8 +71,7 @@ run_pr() { } run_main() { - gradle :mainCheck --continue - gradle :integrationCheck --continue + gradle :mainCheck :integrationCheck --continue gradle -p build-logic test bash scripts/ci/verify-configuration-cache.sh }