From e6b6e18eae73aab8b19546a2f4458ed3d7a2198b Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Fri, 6 Nov 2026 20:33:47 +0900 Subject: [PATCH] =?UTF-8?q?fix(strict-lane)!:=20=EC=84=B1=EB=8A=A5=20lane?= =?UTF-8?q?=EC=9D=B4=20=EC=83=81=ED=95=9C=EC=9D=84=20=EC=9E=AC=EC=A7=80=20?= =?UTF-8?q?=EC=95=8A=EA=B3=A0=20=ED=86=B5=EA=B3=BC=ED=95=98=EC=A7=80=20?= =?UTF-8?q?=EB=AA=BB=ED=95=98=EA=B2=8C=20=ED=95=9C=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 문제: 성능 lane의 기계 의존 상한(httpclient 스트리밍 힙 증가, Mongo 시간 gate)이 performance.assertions.enabled 뒤에 있었고 기본값이 자리마다 달랐다(httpclient build false, Mongo build true, Mongo 테스트 코드 false). workflow만 GRADLE_OPTS로 켰다. 그래서 desktop에서 손으로 돌린 ./gradlew :performanceCheck가 httpclient 힙 상한을 재지 않고 통과했다 (2026-11-05 r10 게이트). JPA pool lane도 전에 같은 속성 뒤에서 아무 상한도 비교하지 않았다(PoolLaneClaimTest). 방법: 규칙을 build-logic strict-lane convention 한 곳에 둔다. - PERFORMANCE lane 테스트 JVM에 performance.assertions.enabled=true를 넣는다. 빌드(-P, GRADLE_OPTS)나 lane(systemProperty, jvmArgs, task 설정)이 true 아닌 값을 주면 테스트 전에 실패하고, 지우거나 true로 바꾸라고 알려 준다. true는 받아 준다(파생 서비스 workflow가 넘긴다). - PERFORMANCE lane은 skip된 테스트가 있으면 실패한다. - httpclient·Mongo 테스트는 조건 없이 단언하고 잰 값을 한 줄 남긴다. 두 leaf의 속성 선언과 integration-main·release workflow의 GRADLE_OPTS 줄을 지웠다. 사람과 workflow가 같은 명령을 쓴다. verifyStrictLaneStages가 읽는 run: 줄은 그대로다. 버린 대안: 두 leaf 기본값만 true로(정의가 leaf마다 남아 다음 lane이 다시 복사한다), 속성 삭제만(옛 속성을 기본값 false로 읽는 파생 서비스 테스트가 조용히 건너뛴다), 측정 전용 태스크(쓸 곳이 없다: 숫자만 볼 때는 JMH, 실패한 단언과 매 실행 출력이 값을 남긴다). 같은 원인으로 함께 고친 문서: httpclient support-matrix·release-checklist의 없는 lane 이름 spring62CompatibilityTest와 빠진 main push 주기. 위험·한계: 손으로 돌린 performanceCheck가 이제 상한을 실제로 재므로 느린 기계에서는 실패할 수 있다. JVM 인자 공급자(jvmArgumentProviders)가 넣는 -D는 보지 않는다. Docker가 없을 때 skip되는 통합·시스템 lane은 지금 모두 실패한다(전부 skip이면 executed no test, 아니면 skip 안 되는 컨테이너 테스트가 실패) — 거짓 통과가 아니어서 고치지 않았다. 검증(desktop ~/b4-logs, flock 아래, 노트북과 같은 트리 — git diff sha256 일치): - RED(고치기 전 트리 + 새 테스트): build-logic 새 테스트 4건이 UnexpectedBuildSuccess로 실패 — aPerformanceLaneFailsOnAnExceededBound WhenTheBuildSetsNoAssertionsProperty, …FailsBeforeTestingWhenTheBuild TurnsItsAssertionsOff, …FailsWhenItsOwnDeclarationTurnsTheAssertionsOff, …FailsWhenOneOfItsTestsIsSkipped. jvmArgs 경로(…FailsWhenItsJvmArguments TurnTheAssertionsOff)도 같은 실패. 실제 lane: 속성 없이 httpClientPerformanceTest rc=0인데 "skipped machine-dependent bound" 출력. - GREEN: -p build-logic check javadoc rc=0. :performanceCheck rc=0(lane 5개 테스트 26, skip 0; httpclient 힙 증가 5,243,880 B / 상한 16,777,216 B, Mongo p50·p95·p99 1ms, examined/returned 1.0, pool 대기 0). -p optional-platforms performanceCheck rc=0(grpc 1). 두 명령에 -Pperformance.assertions.enabled=false → rc=1, 6개 lane 모두 테스트 전 실패. =true → rc=0. verifyStrictLaneStages --dry-run 두 빌드 rc=0, architectureCheck rc=0, ci·mainCheck·integrationCheck·releaseCheck· performanceCheck --dry-run --configuration-cache rc=0, :mainCheck --continue --warning-mode=fail rc=0, -p optional-platforms ci rc=0, persistence-mongo:check rc=0, verify-httpclient-docs.py rc=0. - 이 변경 밖 실패: :ci :verifyDependencyLocks --continue와 httpclient:check가 CircuitBreakerStatusAccountingTest JDK 2건으로 rc=1. 기준 커밋 3d73b6f1에서도 10/10 실패(이 트리도 10/10). ci의 다른 task는 모두 통과했다. Refs: docs/upgrade/2026-11-maturity-review.md B-43, docs/testing/TESTING_STRATEGY.md 6절 BREAKING CHANGE: PERFORMANCE lane은 performance.assertions.enabled가 true가 아니거나 skip된 테스트가 있으면 실패한다. 자기 성능 lane의 그 속성 선언(false)과 skip을 없앤다. --- .github/workflows/integration-main.yml | 5 +- .github/workflows/release.yml | 4 - AGENTS.md | 6 +- CLAUDE.md | 7 +- docs/httpclient/performance-baseline.md | 17 +- docs/httpclient/release-checklist.md | 5 +- docs/httpclient/support-matrix.md | 22 +-- docs/testing/TESTING_STRATEGY.md | 16 ++ docs/upgrade/2026-11-maturity-review.md | 5 +- src/README.md | 2 +- src/adapter/outbound/httpclient/CLAUDE.md | 5 +- src/adapter/outbound/httpclient/README.md | 5 +- src/adapter/outbound/httpclient/build.gradle | 7 +- .../performance/LargeBodyResourceTest.java | 13 +- .../performance/PerformanceAssertions.java | 27 +-- .../outbound/persistence-mongo/build.gradle | 5 +- .../MongoResourceBudgetLaneTest.java | 18 +- src/build-logic/README.md | 2 +- .../strictlane/StrictTestLanePlugin.java | 84 ++++++++- .../strictlane/StrictTestLaneSpec.java | 6 + .../StrictTestLaneConventionTest.java | 167 ++++++++++++++++++ 21 files changed, 346 insertions(+), 82 deletions(-) diff --git a/.github/workflows/integration-main.yml b/.github/workflows/integration-main.yml index 7bd70230..2fdf8b10 100644 --- a/.github/workflows/integration-main.yml +++ b/.github/workflows/integration-main.yml @@ -189,13 +189,12 @@ jobs: # never catches. This job replaced `web-load-abuse-and-shutdown`, which despite its name ran # `:adapter:inbound:web:test` and `webJettyCompatTest` — two lanes the pull-request and main stages # already run — and never the load lanes (review 2026-11-01 08-01), and `httpclient-performance`. + # No assertions switch: the strict-lane convention asserts every performance lane's bounds, so this + # job and a person on a workstation run the same command and get the same bounds. performance-check: runs-on: ubuntu-latest timeout-minutes: 120 env: - # A project property rather than a command-line flag, so the run command stays a plain, - # verifiable task invocation while the machine-dependent bounds are still asserted. - GRADLE_OPTS: -Dorg.gradle.project.performance.assertions.enabled=true TESTCONTAINERS_REUSE_ENABLE: "false" steps: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8cdecab9..654739fe 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -160,10 +160,6 @@ jobs: defaults: run: working-directory: src - env: - # A project property rather than a command-line flag, so each run command stays a plain, - # verifiable task invocation while the machine-dependent bounds are still asserted. - GRADLE_OPTS: -Dorg.gradle.project.performance.assertions.enabled=true steps: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2 - uses: ./.github/actions/setup-gradle-java diff --git a/AGENTS.md b/AGENTS.md index b37bbaf3..063bf11f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,6 +73,10 @@ root `CLAUDE.md`는 이 목록의 동기화된 요약이다. 두 문서가 어 `INTEGRATION`·`SYSTEM`·`PERFORMANCE` lane과 `mainStage()` lane은 이미지·네트워크·시간·문서처럼 입력으로 잡히지 않는 것에 기대므로 매번 다시 돈다. hermetic lane이 저장소 파일을 읽으면 그 파일을 입력으로 선언해야 캐시가 거짓 통과를 내지 않는다. +- `PERFORMANCE` lane은 상한을 매번 단언한다. 끄는 스위치가 없다: convention이 테스트 JVM에 + `performance.assertions.enabled=true`를 넣고, 빌드(`-P`·`GRADLE_OPTS`)나 lane이 다른 값을 주면 + 테스트 전에 실패한다. skip된 테스트가 하나라도 있어도 실패한다(`rejectSkipped`를 따로 쓰지 않아도). + 사람이 손으로 돌리는 `./gradlew performanceCheck`와 workflow가 같은 상한을 단언한다. - `verifyEnvKeys`는 app-bootstrap 태스크로 유지한다(소유자 결정 4). 구현은 build-logic의 `ca.config-contract`다. `src/build-tools/` included build는 비워서 삭제했다. - `src/build.gradle`: 루트 라이프사이클(`ci`, `mainCheck`, `integrationCheck`, `performanceCheck`, @@ -222,7 +226,7 @@ cd src ./gradlew ci # PR 게이트 = leaf check + architecture + quality ./gradlew mainCheck # main 단계 = env 계약·전체 기동(샘플 포함)·public path·전송/메시징 레인·leaf 문서 대조 레인·기능 생성 왕복(verifyFeatureScaffold)·테스트 코드 정적 분석(testQualityCheck) ./gradlew integrationCheck # 실제 컨테이너 integration 레인 전체(main·nightly, Docker 필요) -./gradlew performanceCheck # 부하·종료·자원 한계 레인(main·nightly) +./gradlew performanceCheck # 부하·종료·자원 한계 레인(main·nightly). 상한은 항상 단언, 끄면 실패 ./gradlew verifyStrictLaneStages --dry-run # 이 빌드의 strict lane이 workflow가 실행하는 단계에 속하는지(optional 빌드는 -p optional-platforms로 따로) ./gradlew verifyPublicPathSnapshot ./gradlew :app-bootstrap:verifyEnvKeys diff --git a/CLAUDE.md b/CLAUDE.md index 4921a632..396f0dab 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -33,7 +33,9 @@ If this summary drifts from `AGENTS.md`, `AGENTS.md` wins and this summary must in that leaf's `mainCheck`; a merge-blocking contract over non-document repository files is a pull-request lane whose inputs are exactly those files (app-bootstrap `repository-contract`). `TEST`/`ARCHITECTURE` lanes are cacheable; `INTEGRATION`/`SYSTEM`/`PERFORMANCE` and main-stage - lanes rerun every time. + lanes rerun every time. A `PERFORMANCE` lane asserts its bounds in every run and rejects skipped + tests: the convention sets `performance.assertions.enabled=true`, and a build or lane that sets + anything else fails before a test runs. - `verifyEnvKeys` stays an app-bootstrap task (owner decision 4); its implementation is build-logic's `ca.config-contract`. The `src/build-tools/` included build was emptied and removed. - `src/build.gradle`: the root lifecycle (`ci`, `mainCheck`, `integrationCheck`, `performanceCheck`, @@ -145,7 +147,8 @@ of domain-core/shared-contract/application-*, Git-carryable sources, ArchUnit), (SpotBugs, FindSecBugs over production code), `testQualityCheck` (the same over every test source set, against `config/spotbugs/exclude-test.xml`), `configContractCheck` (the environment contract), `integrationCheck` (the declared strict integration lanes; a lane that needs an input the build was not given is left out -and named), `performanceCheck` (the declared performance lanes; main and nightly). `ci` (pull +and named), `performanceCheck` (the declared performance lanes, bounds always asserted — the same +command by hand or in a workflow; main and nightly). `ci` (pull request) is leaf checks (API surfaces included, owner decision 20) + architectureCheck + qualityCheck, and the workflow also runs `verifyStrictLaneStages --dry-run`; `mainCheck` (main branch) is configContractCheck + qualificationCheck (which includes testQualityCheck) + the sample web contract + the sample full boot (`:sample-portfolio:systemTest`) diff --git a/docs/httpclient/performance-baseline.md b/docs/httpclient/performance-baseline.md index 7335d020..0a1ee87e 100644 --- a/docs/httpclient/performance-baseline.md +++ b/docs/httpclient/performance-baseline.md @@ -8,19 +8,20 @@ from an explicit profile setting. ## How to run ```bash -# structural bounds only (default; still executes every test) +# every bound, structural and machine-dependent ./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest --console=plain -# full certification, including machine-dependent bounds -./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \ - -Pperformance.assertions.enabled=true --console=plain - -# JMH benchmarks +# JMH benchmarks: numbers only, never a gate ./gradlew :adapter:outbound:httpclient:jmh --console=plain ``` -Machine-dependent assertions are reported as explicitly skipped when the flag is absent — the lane -never silently degrades into a pass. +The lane asserts every bound in every run. There is no switch that turns the machine-dependent one +off: the strict-lane convention fails a performance lane before it runs when the build sets +`performance.assertions.enabled` to anything but `true`, and fails it when one of its tests is +skipped. The same command is what `integration-main.yml` (`:performanceCheck`) and `release.yml` +run. Every run prints the measured heap increase (`[httpclient-performance] streaming download heap +increase … bytes, bound … bytes`) to the test output and report, and a bound that fails repeats the +value in its assertion message; for numbers without a verdict, use JMH. ## Certified bounds diff --git a/docs/httpclient/release-checklist.md b/docs/httpclient/release-checklist.md index 6b21cf41..a11e0f7f 100644 --- a/docs/httpclient/release-checklist.md +++ b/docs/httpclient/release-checklist.md @@ -9,11 +9,10 @@ cd src ./gradlew :adapter:outbound:httpclient:test --console=plain ./gradlew :adapter:outbound:httpclient:httpClientStableContractTest --console=plain ./gradlew :adapter:outbound:httpclient:httpClientSecurityTest --console=plain -./gradlew :adapter:outbound:httpclient:spring62CompatibilityTest --console=plain +./gradlew :adapter:outbound:httpclient:spring62ApiSurfaceScan --console=plain ./gradlew :adapter:outbound:httpclient:spring70CompatibilityTest --console=plain ./gradlew :adapter:outbound:httpclient:httpClientFailureInjectionTest --console=plain # needs Docker -./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \ - -Pperformance.assertions.enabled=true --console=plain +./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest --console=plain # every bound asserted ./gradlew verifyCleanArchitectureDependencies --console=plain ./gradlew :app-bootstrap:architectureTest --console=plain python3 ../scripts/verify-httpclient-docs.py diff --git a/docs/httpclient/support-matrix.md b/docs/httpclient/support-matrix.md index 7d3966c1..a253cfd9 100644 --- a/docs/httpclient/support-matrix.md +++ b/docs/httpclient/support-matrix.md @@ -57,19 +57,19 @@ gives route-scoped pooling and Dynamic Target pinning, JDK gives HTTP/2. | Profile | Frequency | Release gate | Task | |---|---|---|---| -| Spring Framework 7.0 (repository baseline) | every PR | required | `spring70CompatibilityTest` | -| Spring Framework 6.2 API surface | every PR | required | `spring62CompatibilityTest` | -| Apache HC5 + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=apache` | -| JDK HttpClient + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=jdk` | -| SSRF / cardinality suite | every PR | required | `httpClientSecurityTest` | -| Toxiproxy fault suite | nightly, release | required | `httpClientFailureInjectionTest` | -| Performance certification | nightly, release | required | `httpClientPerformanceTest -Pperformance.assertions.enabled=true` | -| Jetty HTTP/3 | nightly | Experimental, non-blocking | `test -Phttp3.tests.enabled=true` | +| Spring Framework 7.0 (repository baseline) | PR touching this leaf, main push, release | required | `spring70CompatibilityTest` | +| Spring Framework 6.2 API surface | every PR (`check`), release | required | `spring62ApiSurfaceScan` | +| Apache HC5 + RestClient | PR touching this leaf (both transports together on every PR through `check`), release | required | `httpClientStableContractTest -Phttpclient.contract.transports=apache` | +| JDK HttpClient + RestClient | PR touching this leaf (both transports together on every PR through `check`), release | required | `httpClientStableContractTest -Phttpclient.contract.transports=jdk` | +| SSRF / cardinality suite | every PR (`check`), release | required | `httpClientSecurityTest` | +| Toxiproxy fault suite | main push, nightly (`integrationCheck`), release | required | `httpClientFailureInjectionTest` | +| Performance certification | main push, nightly (`performanceCheck`), release | required | `httpClientPerformanceTest` (every bound asserted; no switch turns them off) | +| Jetty HTTP/3 | main push, nightly | Experimental, non-blocking | `test -Phttp3.tests.enabled=true` | ### Known limitation of the Spring 6.2 lane This repository's Spring Boot 4.0 baseline pins Spring Framework 7, so a real 6.2 runtime cannot be -resolved here. `spring62CompatibilityTest` therefore verifies the **API surface**: the common +resolved here. `spring62ApiSurfaceScan` therefore verifies the **API surface**: the common packages must not reference any Spring 7-only type, and no class uses `org.springframework.web.service.registry`. Executing the suite against an actual 6.2 distribution requires a host project on that line. This limitation is stated rather than hidden behind a passing @@ -83,5 +83,5 @@ Stated so the matrix is read as a measurement rather than an aspiration. | Gap | Why | What is proven instead | |---|---|---| | HTTP/2 frame injection (`REFUSED_STREAM`, arbitrary `GOAWAY`) | The fixture server exposes no frame-level control, and a purpose-built h2 server is a larger dependency than the guarantee is worth here | `Http2EvidenceMapperTest` proves the frame → evidence mapping, and `NegotiatedProtocolContractTest` proves h2 is really negotiated | -| Spring 6.2 runtime | This repository's Boot 4.0 baseline pins Spring 7 | `spring62CompatibilityTest` confines the common packages to the 6.2 API surface | -| Performance latency baseline | Numbers measured on a build agent are not a certification | `httpClientPerformanceTest` asserts structural bounds unconditionally; latency and heap bounds run under `-Pperformance.assertions.enabled=true` | +| Spring 6.2 runtime | This repository's Boot 4.0 baseline pins Spring 7 | `spring62ApiSurfaceScan` confines the common packages to the 6.2 API surface | +| Performance latency baseline | Numbers measured on a build agent are not a certification | `httpClientPerformanceTest` asserts its structural bounds and the streaming heap bound in every run; it asserts no latency figure | diff --git a/docs/testing/TESTING_STRATEGY.md b/docs/testing/TESTING_STRATEGY.md index 19e705ea..8f870afa 100644 --- a/docs/testing/TESTING_STRATEGY.md +++ b/docs/testing/TESTING_STRATEGY.md @@ -154,6 +154,22 @@ fixture는 **지원 코드**다. TEST/INTEGRATION/SYSTEM/ARCHITECTURE/PERFORMANC PERFORMANCE는 기본 로컬 `check`에 붙이지 않는다. 명시적 benchmark/performance task나 scheduled workflow가 실행한다. +PERFORMANCE lane은 상한을 매번 단언한다. 측정만 하고 통과하는 모드는 없다. + +- convention(`ca.strict-test-lane`)이 테스트 JVM에 `performance.assertions.enabled=true`를 넣는다. + leaf나 lane이 이 값을 따로 선언하지 않는다. +- 빌드가 `-Pperformance.assertions.enabled`(또는 `GRADLE_OPTS`의 + `-Dorg.gradle.project.performance.assertions.enabled`)에 `true` 아닌 값을 주거나, lane이 테스트 + JVM 값을 덮어쓰면 테스트 전에 실패한다. 메시지가 지우거나 `true`로 바꿀 곳을 알려 준다. +- skip된 테스트가 하나라도 있으면 실패한다. skip은 아무도 재지 않은 상한이다. +- 그래서 사람이 desktop에서 돌리는 `./gradlew performanceCheck`와 workflow가 같은 상한을 단언한다. +- 판정 없이 숫자만 보려면 JMH가 있는 leaf의 `./gradlew ::jmh`를 쓴다. 게이트가 아니다. + 실패한 상한은 잰 값을 단언 메시지에 남긴다. + +2026-11-05 desktop 게이트에서 httpclient lane이 이 속성 기본값 `false`로 돌아, 힙 상한을 재지 않고 +통과했다. 그 전에는 JPA pool lane이 같은 속성 뒤에 숨어 아무 상한도 비교하지 않았다 +(`PoolLaneClaimTest`). 이 규칙은 그 두 번을 되풀이하지 않기 위한 것이다. + ## 7. 새 테스트를 추가할 때 판단 순서 1. 실제 외부 인프라가 필요한가? diff --git a/docs/upgrade/2026-11-maturity-review.md b/docs/upgrade/2026-11-maturity-review.md index aaa5e736..ef50f035 100644 --- a/docs/upgrade/2026-11-maturity-review.md +++ b/docs/upgrade/2026-11-maturity-review.md @@ -4,7 +4,7 @@ - 커밋: `34565130` High·P1 19건 · `1400dc15` 남은 발견·매뉴얼·생성 명령 · `5786126c` 남은 항목과 합친 뒤 결함 · `7ef8d0b4` 재리뷰 기록 · `bbe944bd` 재리뷰 결함 · `ecc74b7d` 2차 재리뷰 기록·이 안내 · `1885bf8d` 2차 재리뷰 결함 · `339cf8fd` 3차 확인 리뷰 기록 · `f40c03c6` 3차 확인 리뷰 Minor · `c2e55510` 마지막 확인 리뷰 기록 · - `2678955c` 테스트 코드 정적 분석(B-40) · `70fb80bf` 소유자가 다시 정한 기본값 3가지(늦은 멱등 완료 기록, `failure-retention` 기동 검사, workflow `cd`) — `docs(...)` 커밋은 문서만 바꾼다 + `2678955c` 테스트 코드 정적 분석(B-40) · `70fb80bf` 소유자가 다시 정한 기본값 3가지(늦은 멱등 완료 기록, `failure-retention` 기동 검사, workflow `cd`) · B4 성능 lane 단언 상시화(2026-11-06, B-43) — `docs(...)` 커밋은 문서만 바꾼다 - 근거 문서: `docs/reviews/2026-11-01-maturity-review/README.md` 13~16절, `docs/reviews/2026-11-03-rereview/README.md` 4·6·8~13절, 각 커밋의 `BREAKING CHANGE:` 꼬리말 - 아래 표의 각 줄은 위 범위의 diff와 코드로 확인했다. 확인하지 못한 것은 그 자리에 "확인 못 함"으로 적었고 9절에 @@ -36,6 +36,7 @@ | 13 | `.env`에 옛 `.env.example` 값을 복사해 두었다면 5.6절 키를 다시 맞춘다 | 새 기본값이 안 먹거나(JDBC batch), 기동 실패(알림 콜백 상한) | `.env`를 복사해 쓰는 서비스 | | 14 | 자기 테스트 코드에 `./gradlew testQualityCheck`를 돌려 보고, 발견을 고치거나 이유를 단 Match로 뺀다(B-40) | `mainCheck`가 실패 | 테스트 코드가 있는 모든 서비스 | | 15 | staging·prod 배포 마이그레이션 도구에 `flyway.postgresql.transactional.lock=false`(환경 변수 `FLYWAY_POSTGRESQL_TRANSACTIONAL_LOCK=false`)를 넣는다. 자기 Flyway stream은 `Flyway.configure()` 대신 `FlywaySchemaPolicy.configure()`로 만든다(5절 「Flyway 잠금」) | `CREATE INDEX CONCURRENTLY`·`REINDEX … CONCURRENTLY` 마이그레이션에서 배포가 오류 없이 멈춘다. 자기 stream이 `Flyway.configure()`를 부르면 `:app-bootstrap:architectureTest` 실패 | persistence-jpa 선택 + 배포 도구로 마이그레이션하거나 자기 stream을 둔 서비스 | +| 16 | 자기 성능 lane(`performance()`)에서 `performance.assertions.enabled`를 `false`로 두는 `systemProperty` 줄을 지우고, 그 속성을 읽어 상한 단언을 건너뛰던 테스트는 항상 단언하게 고친다. 성능 lane의 `@Disabled`·assumption skip을 없앤다(B-43) | `performanceCheck`가 테스트 전에 실패(`its test JVM would get performance.assertions.enabled=false`)하거나 skip 때문에 실패(`forbids skipped tests`) | 자기 성능 lane이 있는 서비스 | ### 되돌릴 수 없는 것 @@ -109,6 +110,7 @@ | B-37 | app-bootstrap 테스트 lane 분리: 저장소 파일을 읽는 테스트는 태그 `document-contract`(main 단계) 또는 `repository-contract`(`check`)여야 하고, 일반 `test` lane에는 `ca.repository.root`가 없다 | app-bootstrap에 둔 자기 테스트가 저장소 파일을 읽으면 `test`에서 실패 | 태그를 단다 | `1400dc15` · `src/app-bootstrap/build.gradle` | | B-40 | 테스트 코드 정적 분석 `testQualityCheck`: SpotBugs·FindSecBugs를 main 밖 모든 source set(test, testFixtures, integrationTest, systemTest 등)에 돌린다. 테스트용 제외 목록은 `config/spotbugs/exclude-test.xml`(보안 범주·내부 상태 노출·생성자 예외·반환값 null 역참조·아키텍처 위반 fixture). 분석 classpath가 그 source set의 runtime classpath로 넓어졌고, 분석이 불완전하면(빠진 클래스) 실패한다. main 빌드는 `mainCheck`(`qualificationCheck`), optional 빌드는 `ci` | 자기 테스트 코드의 SpotBugs 발견(예: `CountDownLatch.await` 반환값 무시, 공유 변수 비원자 쓰기, 닫지 않은 `Statement`)이 main 단계를 막는다. 테스트 classpath에 없는 타입을 참조하면 `SpotBugs analysis incomplete` | 고치거나, 클래스·메서드로 좁힌 Match를 이유와 함께 `exclude-test.xml`에 더한다. 빠진 타입은 그 테스트 suite 의존성에 넣는다 | `2678955c` · `src/build-logic/.../quality/QualityConventionsPlugin.java`, `src/config/spotbugs/exclude-test.xml`, `src/build.gradle` | | B-42 | shared-contract 패키지 규칙 `SHARED_CONTRACT_CONTAINS_ONLY_OPERATIONAL_CONTRACT_PACKAGES`가 실제로 검사한다. 대상은 `dev.caskeleton.shared..`뿐이고, 허용은 목록의 패키지(`concurrency digest error metrics observability ratelimit request response retry runtime security tracing`)와 그 아래, 뿌리 패키지에는 `package-info`만이다. 전에는 뿌리를 허용 목록에 넣고 접두어로 비교해서 모듈 안은 아무것도 잡지 못했고, 다른 leaf의 `..shared..` 패키지를 잡았다 | `dev.caskeleton.shared`에 목록에 없는 패키지나 뿌리 클래스를 둔 파생 서비스는 architectureTest 실패. 다른 leaf의 `shared` 패키지는 더는 걸리지 않는다 | 업무 개념이면 그것을 소유하는 모듈로 옮긴다. 운영 계약이면 `ForbiddenApiArchitectureTest.SHARED_CONTRACT_PACKAGES`와 shared-contract CLAUDE.md 「Responsibility」에 함께 더한다 | r10 · `src/app-bootstrap/src/architectureTest/.../ForbiddenApiArchitectureTest.java`, `SharedContractPackageRuleTest.java` | +| B-43 | 성능 lane 단언 상시화: `performance()` lane은 convention이 테스트 JVM에 `performance.assertions.enabled=true`를 넣는다. 빌드가 `-P`·`GRADLE_OPTS`(`-Dorg.gradle.project.…`)로 `true` 아닌 값을 주거나, lane `systemProperty`·`jvmArgs`(`-D…`)·task 설정이 테스트 JVM 값을 `true` 아닌 값으로 덮으면 테스트 전에 실패한다. 성능 lane은 `rejectSkipped` 없이도 skip된 테스트가 있으면 실패한다. httpclient·Mongo leaf의 속성 선언과 `integration-main.yml` `performance-check`·`release.yml` `httpclient-release-gate`의 `GRADLE_OPTS` 줄을 지웠다. 측정만 하는 모드는 만들지 않았다(JMH가 그 용도) | `performanceCheck`·`::performanceTestLaneCheck`가 `is a performance lane, and this build sets performance.assertions.enabled=false`, `its test JVM would get performance.assertions.enabled=false`, `forbids skipped tests … a skipped test is a bound nobody measured` 중 하나로 실패 | `-Pperformance.assertions.enabled=false`를 지운다. 자기 lane의 그 `systemProperty`·`jvmArgs` 줄을 지우고, 그 속성을 읽던 테스트는 항상 단언하게 고친다. 성능 lane의 skip은 없애거나 그 테스트를 다른 lane으로 옮긴다. `GRADLE_OPTS=-Dorg.gradle.project.performance.assertions.enabled=true`·`-P…=true`는 받아 주므로 그대로 둬도 된다 | B4(2026-11-06) · `src/build-logic/.../strictlane/StrictTestLanePlugin.java`, `.../strictlane/StrictTestLaneConventionTest.java`(test), `docs/testing/TESTING_STRATEGY.md` 6절 | ### 3.5 모듈 registry·새 leaf @@ -240,6 +242,7 @@ V21 실패의 분류는 `docs/runbooks/migration-failed.md` "Classify the failur | 서버 로그(GraphQL·gRPC·STOMP) | carrier로 답한 실패를 web과 같은 줄(shared-contract `ServerFailureLog`)로 ERROR에 남긴다: persistence 실패는 분류만, 5xx carrier는 예외와 함께, 4xx carrier는 남기지 않는다. gRPC는 ` (method=…)`, GraphQL은 ` (executionId=…)`를 붙인다. gRPC는 인식 못 한 실패(`INTERNAL_ERROR`)도 `unhandled failure on …`로 남긴다. 클라이언트가 취소한 호출은 남기지 않고, 서비스가 `CANCELLED` status로 끝낸 살아 있는 호출은 남긴다. raw status는 넘겼든 던졌든 설명을 남기지 않는다(원인 예외가 있으면 그것만 붙인다). `IllegalArgumentException`(`BAD_PARAMETER`)은 web과 같은 WARN 한 줄 `request refused as BAD_PARAMETER by <예외 타입>`(메시지 없음)으로 남긴다. STOMP `@MessageMapping` 실패는 Spring의 `Unhandled exception from message handler method`(예외 전체) 대신 같은 줄로 남는다 | ERROR 로그가 늘어난다(전에는 gRPC가 아무것도 남기지 않았다). GraphQL·gRPC·STOMP는 잘못된 입력마다 WARN 한 줄이 생긴다. STOMP에서 Spring 문구로 경보를 걸었다면 고친다. `IDEMPOTENCY_RECOVERY_REQUIRED`의 scope digest를 모든 전송에서 찾을 수 있다(`docs/runbooks/idempotency-unavailable.md`) | r9 · `src/shared-contract/.../error/ServerFailureLog.java`, `GrpcExceptionHandlingInterceptor.java`, `GraphQlWireErrorMapper.java`, `StompHandlerFailureLog.java` | | sample-portfolio | 익명 쓰기 401, 포스터 제목 대소문자 무시 유일(409 `POSTER_ALREADY_EXISTS`), 관리자도 남의 work log 수정 불가, 샘플 이벤트 tenant `global` | sample을 남긴 서비스만 | `34565130`, `1400dc15` | | Flyway 잠금 | Flyway의 PostgreSQL 마이그레이션 잠금을 트랜잭션 대신 세션에 둔다. Spring Boot의 Flyway는 module-defaults `spring.flyway.postgresql.transactional-lock: false`, 템플릿이 직접 만드는 Flyway(멱등성·알림·sample stream, schema-per-tenant, testkit `MigrationContractRunner`)는 `FlywaySchemaPolicy.configure()`, messaging inbox stream은 같은 값을 직접 넣는다. 운영 코드의 `Flyway.configure()` 직접 호출은 `JpaProductionArchitectureTest`가 거부한다 | Flyway 기본값에서는 동시 인덱스 마이그레이션이 Flyway 자신의 열린 트랜잭션을 기다려 끝나지 않았다. 이제 끝난다. 마이그레이션은 잠금을 잡은 연결 하나로 돈다. transaction 모드 PgBouncer 뒤에서는 잠금 해제가 다른 서버 연결로 가 다음 배포가 기다릴 수 있으니 마이그레이션은 직접 연결이나 session 모드 pool로 돌린다. 배포 도구는 이 repo 밖이라 2절 15번으로 맞춘다 | 재리뷰 round6 RR6-03-01 · `persistence-jpa.yml`, `.../migration/FlywaySchemaPolicy.java`, `docs/jpa/migration-guide.md` "Who may change the schema"·"Concurrent index builds" | +| 성능 lane(HTTP client·Mongo) | httpclient `LargeBodyResourceTest`의 힙 상한(32 MiB 스트리밍 중 힙 증가 16 MiB 미만)을 매번 단언한다. 전에는 `-Pperformance.assertions.enabled=true`일 때만 단언하고 아니면 건너뛴다는 줄만 찍었다. Mongo `MongoResourceBudgetLaneTest`는 표준 gate 전체(p99 500ms, pool 대기 250ms, 힙 512 MiB, examined/returned 10, spill 금지)를 Gradle 밖 실행에서도 단언한다(B-43) | 손으로 돌린 `performanceCheck`가 상한을 실제로 재므로, 전에는 통과하던 느린 기계에서 실패할 수 있다 | B4(2026-11-06) · `LargeBodyResourceTest`, `MongoResourceBudgetLaneTest` | ## 6. 따라오는 순서 diff --git a/src/README.md b/src/README.md index c6ba7a83..4e8b0fa4 100644 --- a/src/README.md +++ b/src/README.md @@ -27,7 +27,7 @@ Checkstyle은 **포매터가 표현할 수 없는 것만** — 네이밍과 몇 | `configContractCheck` | `:app-bootstrap:verifyEnvKeys`로 환경 설정 계약을 검사 | | `qualificationCheck` | 전체 기동(system) 레인과 보조 source set Checkstyle을 실행 | | `integrationCheck` | 선언된 strict integration lane(sample 포함)을 실행. 대부분 container runtime이 필요. 입력이 필요한 lane(Redis topology: `-Predis.topology.mode`, AWS: 보호 자격 증명 env)은 입력이 없으면 빠지고 집계가 그 이유를 출력 | -| `performanceCheck` | 선언된 strict performance lane(부하·종료·pool·자원 상한)을 실행. 실서버·container가 필요. `integration-main.yml` `performance-check`가 main·nightly에 돌림 | +| `performanceCheck` | 선언된 strict performance lane(부하·종료·pool·자원 상한)을 실행. 실서버·container가 필요. 상한은 매번 단언하고 끄는 속성이 없다(`-Pperformance.assertions.enabled`가 `true`가 아니면 실패, skip된 테스트가 있어도 실패). `integration-main.yml` `performance-check`가 main·nightly에 같은 명령으로 돌림 | | `verifyStrictLaneStages` | `--dry-run` 전용. `explicitOnly`가 아닌 strict lane이 workflow가 실제 실행하는 단계(`ci`·`:sample-portfolio:check`·`mainCheck`·`integrationCheck`·`performanceCheck`·`releaseCheck`) 중 하나의 계획에 있는지, 각 단계를 workflow가 `--dry-run` 없이 실행하는지 검사. Gradle이 작업을 돌리지 않는 명령(`--dry-run`·`-v`·`--help` 등)과, 실패해도 단계가 실패하지 않을 수 있거나(`\|\| true`, pipefail 없는 파이프, `if` 조건 등) 돌지 않을 수 있거나(조건부 `exit 0` 뒤, 요청으로만 시작하는 workflow 등) 검사가 읽지 않는(`\|\| exit 1`, `{ }` 안 등) 명령은 실행으로 세지 않는다. 자세한 목록은 `build-logic/README.md`. PR 게이트가 돌림 | | `ci` | PR 게이트: 모든 leaf `check`(API surface 검사 포함, 소유자 결정 20) + `architectureCheck` + `qualityCheck` | | `mainCheck` | main 단계: `configContractCheck` + `qualificationCheck` + sample web 계약 + sample 전체 기동(`:sample-portfolio:systemTest`) + `verifyPublicPathSnapshot` + 전송·메시징 qualification 레인 + 각 leaf의 `mainCheck`(문서 대조 레인 `*DocumentContractTest`, objectstorage MinIO 레인) | diff --git a/src/adapter/outbound/httpclient/CLAUDE.md b/src/adapter/outbound/httpclient/CLAUDE.md index 9b819b9a..6b7e27d2 100644 --- a/src/adapter/outbound/httpclient/CLAUDE.md +++ b/src/adapter/outbound/httpclient/CLAUDE.md @@ -81,8 +81,9 @@ Default lane: `./gradlew :adapter:outbound:httpclient:test`. Additional lanes, a | `jmh` | per-call overhead benchmarks | A selected lane never skips silently: the fault lane throws without Docker, the contract lane throws -on an empty or unknown transport selection, and the performance lane prints which machine-dependent -bounds were not asserted. +on an empty or unknown transport selection, and the performance lane asserts every bound in every +run — the strict-lane convention fails it when the build sets `performance.assertions.enabled` to +anything but `true` or when one of its tests is skipped. The transport port (`BlockingTransportProvider`) has one abstract contract in the test fixtures, `testkit/BlockingTransportProviderContract`. The engines pass it over a real socket diff --git a/src/adapter/outbound/httpclient/README.md b/src/adapter/outbound/httpclient/README.md index 9ae38727..4130afdc 100644 --- a/src/adapter/outbound/httpclient/README.md +++ b/src/adapter/outbound/httpclient/README.md @@ -111,8 +111,9 @@ cd src ``` No lane skips silently. The fault lane fails without Docker, the contract lane fails on an empty or -unknown transport selection, and the performance lane reports which machine-dependent bounds it did -not assert. +unknown transport selection, and the performance lane asserts every bound in every run (it fails +when the build sets `performance.assertions.enabled` to anything but `true`, or when a test is +skipped). ## Further reading diff --git a/src/adapter/outbound/httpclient/build.gradle b/src/adapter/outbound/httpclient/build.gradle index 40108ed1..2e4041d9 100644 --- a/src/adapter/outbound/httpclient/build.gradle +++ b/src/adapter/outbound/httpclient/build.gradle @@ -138,8 +138,9 @@ dependencies { } // Which transports the contract suites cover and whether the HTTP/3 opt-in runs: the same two -// properties for every Test task of this leaf, declared once (review 2026-11-01 O-10a). Values a -// single lane needs (the performance assertions, the httpbin image) stay in that lane. +// properties for every Test task of this leaf, declared once (review 2026-11-01 O-10a). A value a +// single lane needs (the httpbin image) stays in that lane. The performance lane's bounds have no +// switch here: the strict-lane convention asserts them in every run. tasks.withType(Test).configureEach { systemProperty 'httpclient.contract.transports', providers.gradleProperty('httpclient.contract.transports').orElse('apache,jdk').get() @@ -193,8 +194,6 @@ strictTestLanes { performance() sourceSet = 'httpClientPerformanceTest' description = 'Certifies pool, streaming, retry, and rotation resource bounds (design §28.8).' - systemProperty 'performance.assertions.enabled', - providers.gradleProperty('performance.assertions.enabled').orElse('false').get() } lane('httpClientFailureInjectionTest') { integration() diff --git a/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/LargeBodyResourceTest.java b/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/LargeBodyResourceTest.java index 2c2837da..68f54585 100644 --- a/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/LargeBodyResourceTest.java +++ b/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/LargeBodyResourceTest.java @@ -52,11 +52,18 @@ class LargeBodyResourceTest { } } long peakIncrease = usedHeap() - before; + // The measured value, so a passing run is a recorded number and not only a green line. + System.out.println( + "[httpclient-performance] streaming download heap increase " + + peakIncrease + + " bytes, bound " + + payloadBytes / 2 + + " bytes"); assertThat(consumed).isEqualTo(payloadBytes); - PerformanceAssertions.machineDependent( - "streaming download heap increase stays well below the payload size", - () -> assertThat(peakIncrease).isLessThan((long) payloadBytes / 2)); + assertThat(peakIncrease) + .as("streaming download heap increase stays well below the payload size") + .isLessThan((long) payloadBytes / 2); } } } diff --git a/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/PerformanceAssertions.java b/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/PerformanceAssertions.java index ca812905..924e9b10 100644 --- a/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/PerformanceAssertions.java +++ b/src/adapter/outbound/httpclient/src/httpClientPerformanceTest/java/dev/caskeleton/adapter/outbound/httpclient/performance/PerformanceAssertions.java @@ -3,32 +3,19 @@ package dev.caskeleton.adapter.outbound.httpclient.performance; import static org.assertj.core.api.Assertions.assertThat; /** - * Separates deterministic bounds from machine-dependent ones (design §28.8, §37). + * Structural bounds — retry budget arithmetic, a refresh collapsing to one request, drain + * completing — that hold on any machine (design §28.8, §37). * - *

Structural bounds — retry budget arithmetic, streaming not buffering the whole payload, drain - * completing — always run. Latency and absolute memory numbers depend on the host, so they are - * asserted only when the certification flag is set; when it is off the lane still executes and - * still reports, rather than becoming a silent pass. + *

The machine-dependent bound (the streaming heap increase) is asserted in place, in every run. + * It used to sit behind {@code performance.assertions.enabled}, which defaulted to {@code false} in + * this leaf, so a lane run by hand reported success with the bound compared to nothing. A + * performance lane now asserts all of its bounds; the strict-lane convention rejects a build that + * tries to turn them off. */ final class PerformanceAssertions { private PerformanceAssertions() {} - static boolean certificationEnabled() { - return Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false")); - } - - static void machineDependent(String description, Runnable assertion) { - if (certificationEnabled()) { - assertion.run(); - return; - } - System.out.println( - "[httpclient-performance] skipped machine-dependent bound (enable with " - + "-Pperformance.assertions.enabled=true): " - + description); - } - static void structural(String description, boolean condition) { assertThat(condition).describedAs(description).isTrue(); } diff --git a/src/adapter/outbound/persistence-mongo/build.gradle b/src/adapter/outbound/persistence-mongo/build.gradle index e32912e3..c33c1752 100644 --- a/src/adapter/outbound/persistence-mongo/build.gradle +++ b/src/adapter/outbound/persistence-mongo/build.gradle @@ -239,9 +239,8 @@ strictTestLanes { providers.gradleProperty('mongoCompatibilityImage').orElse(testContainerImages['mongo-7']).get() systemProperty 'mongodb.toxiproxy.image', providers.gradleProperty('mongoToxiproxyImage').orElse(testContainerImages['toxiproxy']).get() - // Assertions on by default: a performance gate without bounds is only a report. - systemProperty 'performance.assertions.enabled', - providers.gradleProperty('performance.assertions.enabled').orElse('true').get() + // No assertions switch: the strict-lane convention asserts a performance lane's bounds in + // every run, because a performance gate without bounds is only a report. } } diff --git a/src/adapter/outbound/persistence-mongo/src/mongoPerformanceTest/java/dev/caskeleton/adapter/outbound/mongo/performance/MongoResourceBudgetLaneTest.java b/src/adapter/outbound/persistence-mongo/src/mongoPerformanceTest/java/dev/caskeleton/adapter/outbound/mongo/performance/MongoResourceBudgetLaneTest.java index 969d1690..15441d34 100644 --- a/src/adapter/outbound/persistence-mongo/src/mongoPerformanceTest/java/dev/caskeleton/adapter/outbound/mongo/performance/MongoResourceBudgetLaneTest.java +++ b/src/adapter/outbound/persistence-mongo/src/mongoPerformanceTest/java/dev/caskeleton/adapter/outbound/mongo/performance/MongoResourceBudgetLaneTest.java @@ -27,16 +27,16 @@ import org.junit.jupiter.api.Test; * it is the same on an idle machine and a busy one, and it is what actually predicts the query * falling over as the collection grows. * - *

Timing bounds are asserted only when {@code -Pperformance.assertions.enabled=true}, for the - * same reason. The ratio bound is always asserted. + *

The timing bounds are asserted too, in every run, but the gate's bounds are generous for the + * same reason: they catch a query that falls over, not one that was slower than last time. They + * used to sit behind {@code performance.assertions.enabled}, which the build defaulted to on and + * this class to off, so a run outside Gradle skipped them; the strict-lane convention now asserts a + * performance lane's bounds in every run. */ class MongoResourceBudgetLaneTest { private static final int DOCUMENT_COUNT = 5_000; - private static final boolean TIMING_ASSERTIONS = - Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false")); - private static MongoSingleReplicaSetContainer container; private static MongoClient client; @@ -75,18 +75,16 @@ class MongoResourceBudgetLaneTest { @Test void anIndexedQueryStaysInsideTheResourceBudget() { MongoResourceBudgetReport report = measure(new Document("status", "PENDING"), "ix_status"); + // The measured values, so a passing run is a recorded number and not only a green line. + System.out.println("[mongo-performance] indexed query " + report.asArtifact()); assertThat(report.documentsReturned()).isEqualTo(DOCUMENT_COUNT / 50); assertThat(report.examinedToReturnedRatio()) .as("an index seek examines about what it returns") .isLessThanOrEqualTo(MongoPerformanceGate.standard().maximumExaminedToReturnedRatio()); assertThat(MongoPerformanceGate.standard().violations(report)) - .filteredOn(violation -> violation.startsWith("examined/returned")) + .as("an indexed query stays inside every bound of the standard gate") .isEmpty(); - - if (TIMING_ASSERTIONS) { - assertThat(MongoPerformanceGate.standard().passes(report)).isTrue(); - } } @Test diff --git a/src/build-logic/README.md b/src/build-logic/README.md index 22a7340b..52ae07a2 100644 --- a/src/build-logic/README.md +++ b/src/build-logic/README.md @@ -15,7 +15,7 @@ capability, and the files it reads are declared as that test task's inputs. | `ca.quality-conventions` | Spotless, Checkstyle, Error Prone, SpotBugs/FindSecBugs wiring shared by Java leaves | | `ca.java-library` / `ca.spring-library` / `ca.spring-config` | leaf-level composition of the Java/quality conventions for plain and Spring libraries | | `ca.platform-module` / `ca.grpc-platform-module` | what a vendored platform leaf (`messaging:*`, `grpc:*`, `grpc-advanced:*`) is: `java-library`, plus the gRPC BOM for the leaves that had it | -| `ca.strict-test-lane` | the one "named tests ran, none skipped" mechanism: tagged / named / own-source-set `Test` lanes that fail when they execute nothing, miss a `requires` class, or (with `rejectSkipped`) skip anything. `mainStage()` moves a tagged lane out of `test` into the leaf's `mainCheck`; `explicitOnly(reason)` keeps a lane that needs an outside input out of every aggregate, and the aggregate prints the reason. Extra test source sets are Gradle `jvm-test-suite` suites declared in the leaf (`testing { suites { … } }`); a lane named after its suite makes the suite's own `Test` task strict, and `ca.java-conventions` gives every suite the BOM's JUnit | +| `ca.strict-test-lane` | the one "named tests ran, none skipped" mechanism: tagged / named / own-source-set `Test` lanes that fail when they execute nothing, miss a `requires` class, or (with `rejectSkipped`) skip anything. A `performance()` lane always rejects skips and runs with `performance.assertions.enabled=true`; a build (`-P`, `GRADLE_OPTS`) or lane that sets another value fails it before a test runs. `mainStage()` moves a tagged lane out of `test` into the leaf's `mainCheck`; `explicitOnly(reason)` keeps a lane that needs an outside input out of every aggregate, and the aggregate prints the reason. Extra test source sets are Gradle `jvm-test-suite` suites declared in the leaf (`testing { suites { … } }`); a lane named after its suite makes the suite's own `Test` task strict, and `ca.java-conventions` gives every suite the BOM's JUnit | | `ca.strict-lane-stages` (root of each build with strict lanes: the main build and optional-platforms) | the other half of the lane mechanism: `verifyStrictLaneStages --dry-run` fails when a strict lane of that build that is not `explicitOnly` is in the task plan of no declared stage that a workflow executes, or when a declared stage is executed by no workflow. The workflows are read together (`WorkflowGradleInvocations.readAll`, so a reusable workflow is judged by its callers) and their `run:` scripts parsed as bash (`ShellScript`: quotes, comments, heredoc bodies, substitutions and compound commands are read as the shell reads them, so a `cd` in a heredoc or a string is data). A command does not count when Gradle runs no task with it — `--dry-run`/`-m` (also in a bundle such as `-qm`), `-v`/`--version`, `-h`/`-?`/`--help`, `--status`, `--stop`, `--foreground` (also in a bundle such as `-qv`) — or rejects it: a short-option bundle whose later letter takes a value (`-mx`: only the first letter of a bundle takes one), or a value given to an option that takes none (`-m=yes`). The reader follows Gradle 9.0's parser. `GradleCommandLineTest` holds its option tables to the `--help` of the Gradle that runs build-logic — an upgrade that adds an option, as 9.2 adds `--task-graph`, fails until the option is placed — and runs the launcher with each no-task option but `--stop` and `--foreground` (both return a daemon action, not a build, in Gradle's `BuildActionsFactory`); which options take a value was probed against 9.0.0 in round 9 and is not re-run. Nor does a command for another build count (the build a command starts is its step's `working-directory`, or the job's or workflow's `defaults.run.working-directory`, moved by each `cd ` the same `run:` shell surely ran before it — alone in its list, or before the command in the same `&&` list — then joined with `-p`; the plugin reads this build's own directory from where it sits under the repository that holds `.github/workflows`). Every other command that does not count gives one of three reasons. **Its failure may not fail its step**: `\|\| true`, `\|\| :` or any `\|\|` but `\|\| exit `, a pipe's left side without `pipefail` (on with `shell: bash` or a top-level `set -o pipefail`), `!`, `&`, an `if`/`while` condition, the part before `&&` of a list that is not the script's last command, after `set +e`, after a `trap` whose `EXIT` or `ERR` action may `exit` with status 0 (a bare `exit` keeps the status in an `EXIT` trap, not in an `ERR` one), and `continue-on-error`. **It may not run**: after `\|\|`, after `&&` in a list that is not the script's last command, inside `if`/`while`/`for`/`case` or a function body, after a top-level `exit`, `exec ` or `set -n`, after one elsewhere — in a compound command, an and-or list or a function the script calls — that may end the script with status 0 (`exit 0`, a bare `exit` other than right after `\|\|`, `exit "$code"`), replace it or stop it running commands (`set -o noexec`), under an `if:` other than `always()`/`success()`/`!cancelled()`, in a workflow that starts only on events someone or something asks for (`workflow_dispatch`, `repository_dispatch`, `issue_comment`, a pull request with `types: [labeled]`, `workflow_run`, …; `push`, `pull_request`, `pull_request_target`, `merge_group`, `schedule` and `release` count), in a reusable (`workflow_call`-only) workflow that no counted job calls with `uses: ./.github/workflows/`, and in a job that `needs`, directly or through other jobs, a job with such an `if:` or with `continue-on-error`, unless its own `if:` is `always()` or `!cancelled()`. **This check does not read it**, although its failure would fail the step (a false failure on purpose): `\|\| exit 1`, `\|\| { …; exit 1; }`, inside `{ }` or `( )`, `$( )`, `bash -c`, `gradlew` as another command's argument (`timeout 60 ./gradlew`), after a directory change the reader does not follow (`cd "$VAR"`, `cd -`, `cd ~`, an absolute path, `pushd`/`popd`, `builtin cd`, `eval`/`source`, a `cd` anywhere inside a compound command or a function body, one joined with `\|\|`, piped or in the background, and the lines after a `cd` that shares an `&&` list with other commands — not guessed), a step whose `shell:` is not `bash` or `sh`, and a step with no `shell:` in a job whose `runs-on` may be a Windows runner, whose default shell is `pwsh` (a Windows label, an expression such as `${{ matrix.os }}`, a self-hosted label set without `linux` or `macos`; Linux and macOS runners default to `bash -e` and a `container:` job to `sh -e`, which are read alike). An `exit` that fails the step — `exit 1`, or `cmd \|\| exit` — and one in an uncalled function, a `( )` subshell or `$( )` stop nothing after them from counting. The failure names every command that names the stage but does not execute it, with its reason or the other build it starts, and says to run the stage as `./gradlew` on a line of its own. A lane the command removes with `-x` is not covered — matched the way Gradle matches task and project names, abbreviations included (`TaskSelectors`, a copy of Gradle's internal `NameMatcher`). A `run:` input of a `uses:` step is not a command. The plan is Gradle's own; the workflows are read for their `run:` steps with YAML folding applied (review 2026-11-01 08-01, 2026-11-03 RR-06-03/04, 2026-11-04 RR2-04-04, owner decision 2026-11-04 on `cd`, RR5-02-01/02/03 in round 9, 2026-11-05 RR6-04-01..06) | | `ca.redis-topology-lane` | the Redis topology lane (mode selection, `-P` property validation, per-mode required classes) the redis-sdk-topology workflow drives | | `ca.api-surface` | read-only public API surface verification with a separate update task, wired into the leaf's `check` (owner decision 20); optionally holds other leaves' imports to a reviewed adopter subset (`adopterBaseline`) | diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLanePlugin.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLanePlugin.java index 104671df..628cea88 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLanePlugin.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLanePlugin.java @@ -10,12 +10,26 @@ import org.gradle.api.NamedDomainObjectContainer; import org.gradle.api.Plugin; import org.gradle.api.Project; import org.gradle.api.plugins.JavaPluginExtension; +import org.gradle.api.provider.Provider; import org.gradle.api.tasks.SourceSet; import org.gradle.api.tasks.TaskProvider; import org.gradle.api.tasks.testing.Test; import org.gradle.testing.base.TestingExtension; public final class StrictTestLanePlugin implements Plugin { + /** + * The switch a performance lane's machine-dependent bounds used to sit behind, now always on. + * + *

Each leaf declared it with its own default — {@code false} for the httpclient lane, {@code + * true} for the Mongo lane — and only the workflows set it. {@code ./gradlew :performanceCheck} + * typed by hand therefore passed with the httpclient heap bound compared to nothing (the + * 2026-11-05 desktop gate). The convention owns it now: every performance lane's test JVM gets + * {@code true}, so a test that still reads the switch asserts, and a build that sets it to + * anything else fails before the lane runs. Nothing in this repository reads it any more; the + * value is for tests written against the old switch, such as a derived service's. + */ + static final String PERFORMANCE_ASSERTIONS = "performance.assertions.enabled"; + @Override public void apply(Project project) { project.getPluginManager().withPlugin("java", ignored -> configure(project)); @@ -229,7 +243,8 @@ public final class StrictTestLanePlugin implements Plugin { // judge the lane again rather than reuse a result produced under the looser policy. Without // this a lane that skipped a test stayed green, up to date and from the build cache, after // rejectSkipped was switched on (review 2026-11-03 RR-06-02). - test.getInputs().property("strictLane.rejectSkipped", project.provider(lane::isRejectSkipped)); + test.getInputs() + .property("strictLane.rejectSkipped", project.provider(() -> rejectsSkipped(lane))); test.getInputs() .property("strictLane.requiredTests", project.provider(lane::getRequiredTests)); test.getInputs() @@ -243,6 +258,17 @@ public final class StrictTestLanePlugin implements Plugin { test.getOutputs().upToDateWhen(ignored -> false); test.getOutputs().doNotCacheIf(rerunReason, ignored -> true); } + boolean performance = lane.getCategory() == TestLaneCategory.PERFORMANCE; + if (performance) { + // Set before the lane's own JVM arguments and system properties — Gradle turns a -D in + // jvmArgs into a system property — so a lane that still declares the switch replaces this + // value and the check below sees the override, instead of the override disappearing silently. + test.systemProperty(PERFORMANCE_ASSERTIONS, "true"); + Provider requested = project.getProviders().gradleProperty(PERFORMANCE_ASSERTIONS); + String laneName = lane.getName(); + test.doFirst( + running -> requireAssertionsOn(laneName, ownerPath, requested.getOrNull(), (Test) running)); + } if (lane.getMaxHeapSize() != null && !lane.getMaxHeapSize().isBlank()) { test.setMaxHeapSize(lane.getMaxHeapSize()); } @@ -288,7 +314,7 @@ public final class StrictTestLanePlugin implements Plugin { + detail + " — a lane that runs nothing reports success for whatever it was meant to prove."); } - if (lane.isRejectSkipped() && tracker.skippedCount() > 0L) { + if (rejectsSkipped(lane) && tracker.skippedCount() > 0L) { throw new GradleException( "strict test lane '" + lane.getName() @@ -296,7 +322,10 @@ public final class StrictTestLanePlugin implements Plugin { + ownerPath + " forbids skipped tests but observed " + tracker.skippedCount() - + " skipped test(s)."); + + " skipped test(s)." + + (performance + ? " A performance lane always does: a skipped test is a bound nobody measured." + : "")); } if (!required.isEmpty()) { List absent = @@ -318,6 +347,55 @@ public final class StrictTestLanePlugin implements Plugin { } } + /** + * A performance lane rejects skipped tests whatever it declares: its tests are its bounds, and a + * skip is a bound the run reports as passed without comparing it to anything. + */ + private static boolean rejectsSkipped(StrictTestLaneSpec lane) { + return lane.isRejectSkipped() || lane.getCategory() == TestLaneCategory.PERFORMANCE; + } + + /** + * Fails a performance lane, before any test runs, when the build or the lane turns its bounds off. + * {@code true} from the build is accepted, so a command that still passes it keeps working. + */ + private static void requireAssertionsOn( + String laneName, String ownerPath, String requestedByBuild, Test test) { + if (requestedByBuild != null && !Boolean.parseBoolean(requestedByBuild.trim())) { + throw new GradleException( + "strict test lane '" + + laneName + + "' in " + + ownerPath + + " is a performance lane, and this build sets " + + PERFORMANCE_ASSERTIONS + + "=" + + requestedByBuild + + " (-P" + + PERFORMANCE_ASSERTIONS + + ", or -Dorg.gradle.project." + + PERFORMANCE_ASSERTIONS + + " in GRADLE_OPTS). A performance lane asserts its bounds in every run: one without" + + " them measures nothing it can fail on and still reports success. Remove the" + + " property or set it to true."); + } + Object effective = test.getSystemProperties().get(PERFORMANCE_ASSERTIONS); + if (effective == null || !Boolean.parseBoolean(String.valueOf(effective).trim())) { + throw new GradleException( + "strict test lane '" + + laneName + + "' in " + + ownerPath + + " is a performance lane, but its test JVM would get " + + PERFORMANCE_ASSERTIONS + + "=" + + effective + + " from this project's build (a lane systemProperty or jvmArgs, or a block that" + + " configures the task). The strict-lane convention sets it to true for every" + + " performance lane; delete the override."); + } + } + private static boolean startsARealServer(StrictTestLaneSpec lane) { return lane.getCategory() != TestLaneCategory.TEST && lane.getCategory() != TestLaneCategory.ARCHITECTURE; diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneSpec.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneSpec.java index f34537e8..197d1020 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneSpec.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneSpec.java @@ -137,6 +137,12 @@ public class StrictTestLaneSpec implements Named { category = TestLaneCategory.ARCHITECTURE; } + /** + * A lane whose tests are resource or latency bounds. Besides rerunning every time and taking the + * build-wide real-server slot, it rejects skipped tests and always runs with its bounds asserted: + * the convention sets {@code performance.assertions.enabled=true}, and a build or lane that sets + * it to anything else fails before a test runs. + */ public void performance() { category = TestLaneCategory.PERFORMANCE; } diff --git a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneConventionTest.java b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneConventionTest.java index 5cda0950..208f2cde 100644 --- a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneConventionTest.java +++ b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictTestLaneConventionTest.java @@ -611,6 +611,147 @@ class StrictTestLaneConventionTest { assertThat(result.getOutput()).contains("rulesLane=false").contains("databaseLane=true"); } + @Test + void aPerformanceLaneFailsOnAnExceededBoundWhenTheBuildSetsNoAssertionsProperty() + throws Exception { + // given + buildFile(performanceLane("")); + writePerformanceTest( + "BoundTest.java", + """ + import org.junit.jupiter.api.Test; + class BoundTest { + @Test void staysInsideTheBound() { + long p99Millis = 900; + // The shape the httpclient lane had: the machine-dependent bound behind the switch. + if (Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false")) + && p99Millis > 500) { + throw new AssertionError("p99 900ms above the 500ms bound"); + } + } + } + """); + + // when + BuildResult result = runner("performanceTestLaneCheck").buildAndFail(); + + // then + assertThat(result.getOutput()).contains("BoundTest > staysInsideTheBound() FAILED"); + } + + @Test + void aPerformanceLaneFailsBeforeTestingWhenTheBuildTurnsItsAssertionsOff() throws Exception { + // given + buildFile(performanceLane("")); + writePerformanceTest( + "MeasuredTest.java", + """ + import org.junit.jupiter.api.Test; + class MeasuredTest { + @Test void measured() { throw new AssertionError("the lane ran a test"); } + } + """); + + // when + BuildResult result = + runner("performanceTestLaneCheck", "-Pperformance.assertions.enabled=false") + .buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains("this build sets performance.assertions.enabled=false") + .contains("set it to true") + .doesNotContain("MeasuredTest > measured() FAILED"); + } + + @Test + void aPerformanceLaneFailsWhenItsOwnDeclarationTurnsTheAssertionsOff() throws Exception { + // given + buildFile( + performanceLane( + """ + systemProperty 'performance.assertions.enabled', + providers.gradleProperty('performance.assertions.enabled').orElse('false').get() + """)); + writePerformanceTest( + "MeasuredTest.java", + """ + import org.junit.jupiter.api.Test; + class MeasuredTest { @Test void measured() {} } + """); + + // when + BuildResult result = runner("performanceTestLaneCheck").buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains("its test JVM would get performance.assertions.enabled=false") + .contains("delete the override"); + } + + @Test + void aPerformanceLaneFailsWhenItsJvmArgumentsTurnTheAssertionsOff() throws Exception { + // given + buildFile(performanceLane("jvmArgs '-Dperformance.assertions.enabled=false'")); + writePerformanceTest( + "MeasuredTest.java", + """ + import org.junit.jupiter.api.Test; + class MeasuredTest { @Test void measured() {} } + """); + + // when + BuildResult result = runner("performanceTestLaneCheck").buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains("its test JVM would get performance.assertions.enabled=false") + .contains("delete the override"); + } + + @Test + void aPerformanceLaneFailsWhenOneOfItsTestsIsSkipped() throws Exception { + // given + buildFile(performanceLane("")); + writePerformanceTest( + "PartlyMeasuredTest.java", + """ + import org.junit.jupiter.api.Disabled; + import org.junit.jupiter.api.Test; + class PartlyMeasuredTest { + @Test void measured() {} + @Test @Disabled("too slow on this machine") void quarantined() {} + } + """); + + // when + BuildResult result = runner("performanceTestLaneCheck").buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains("forbids skipped tests") + .contains("a skipped test is a bound nobody measured"); + } + + @Test + void aPerformanceLaneAcceptsABuildThatTurnsItsAssertionsOnExplicitly() throws Exception { + // given + buildFile(performanceLane("")); + writePerformanceTest( + "MeasuredTest.java", + """ + import org.junit.jupiter.api.Test; + class MeasuredTest { @Test void measured() {} } + """); + + // when + BuildResult result = + runner("performanceTestLaneCheck", "-Pperformance.assertions.enabled=true").build(); + + // then + assertThat(result.task(":loadLane").getOutcome()).isEqualTo(TaskOutcome.SUCCESS); + } + private void buildFile(String laneBlock) throws Exception { Files.writeString( projectDir.resolve("build.gradle"), @@ -631,6 +772,32 @@ class StrictTestLaneConventionTest { Files.writeString(testSource.resolve(fileName), source); } + private void writePerformanceTest(String fileName, String source) throws Exception { + Path performanceSource = projectDir.resolve("src/performance/java"); + Files.createDirectories(performanceSource); + Files.writeString(performanceSource.resolve(fileName), source); + } + + private static String performanceLane(String extraDeclarations) { + return """ + sourceSets { performance } + dependencies { + performanceImplementation platform('org.junit:junit-bom:5.11.3') + performanceImplementation 'org.junit.jupiter:junit-jupiter' + performanceRuntimeOnly 'org.junit.platform:junit-platform-launcher' + } + strictTestLanes { + lane('loadLane') { + performance() + sourceSet = 'performance' + description = 'Measures a resource bound.' + %s + } + } + """ + .formatted(extraDeclarations); + } + private static String junitDependencies() { return """ dependencies {