fix(strict-lane)!: 성능 lane이 상한을 재지 않고 통과하지 못하게 한다

문제: 성능 lane의 기계 의존 상한(httpclient 스트리밍 힙 증가, Mongo 시간
gate)이 performance.assertions.enabled 뒤에 있었고 기본값이 자리마다
달랐다(httpclient build false, Mongo build true, Mongo 테스트 코드 false).
workflow만 GRADLE_OPTS로 켰다. 그래서 desktop에서 손으로 돌린
./gradlew :performanceCheck가 httpclient 힙 상한을 재지 않고 통과했다
(2026-11-05 r10 게이트). JPA pool lane도 전에 같은 속성 뒤에서 아무 상한도
비교하지 않았다(PoolLaneClaimTest).

방법: 규칙을 build-logic strict-lane convention 한 곳에 둔다.
- PERFORMANCE lane 테스트 JVM에 performance.assertions.enabled=true를
  넣는다. 빌드(-P, GRADLE_OPTS)나 lane(systemProperty, jvmArgs, task
  설정)이 true 아닌 값을 주면 테스트 전에 실패하고, 지우거나 true로
  바꾸라고 알려 준다. true는 받아 준다(파생 서비스 workflow가 넘긴다).
- PERFORMANCE lane은 skip된 테스트가 있으면 실패한다.
- httpclient·Mongo 테스트는 조건 없이 단언하고 잰 값을 한 줄 남긴다.
  두 leaf의 속성 선언과 integration-main·release workflow의 GRADLE_OPTS
  줄을 지웠다. 사람과 workflow가 같은 명령을 쓴다. verifyStrictLaneStages가
  읽는 run: 줄은 그대로다.
버린 대안: 두 leaf 기본값만 true로(정의가 leaf마다 남아 다음 lane이 다시
복사한다), 속성 삭제만(옛 속성을 기본값 false로 읽는 파생 서비스 테스트가
조용히 건너뛴다), 측정 전용 태스크(쓸 곳이 없다: 숫자만 볼 때는 JMH, 실패한
단언과 매 실행 출력이 값을 남긴다).
같은 원인으로 함께 고친 문서: httpclient support-matrix·release-checklist의
없는 lane 이름 spring62CompatibilityTest와 빠진 main push 주기.

위험·한계: 손으로 돌린 performanceCheck가 이제 상한을 실제로 재므로 느린
기계에서는 실패할 수 있다. JVM 인자 공급자(jvmArgumentProviders)가 넣는
-D는 보지 않는다. Docker가 없을 때 skip되는 통합·시스템 lane은 지금
모두 실패한다(전부 skip이면 executed no test, 아니면 skip 안 되는 컨테이너
테스트가 실패) — 거짓 통과가 아니어서 고치지 않았다.

검증(desktop ~/b4-logs, flock 아래, 노트북과 같은 트리 — git diff sha256 일치):
- RED(고치기 전 트리 + 새 테스트): build-logic 새 테스트 4건이
  UnexpectedBuildSuccess로 실패 — aPerformanceLaneFailsOnAnExceededBound
  WhenTheBuildSetsNoAssertionsProperty, …FailsBeforeTestingWhenTheBuild
  TurnsItsAssertionsOff, …FailsWhenItsOwnDeclarationTurnsTheAssertionsOff,
  …FailsWhenOneOfItsTestsIsSkipped. jvmArgs 경로(…FailsWhenItsJvmArguments
  TurnTheAssertionsOff)도 같은 실패. 실제 lane: 속성 없이
  httpClientPerformanceTest rc=0인데 "skipped machine-dependent bound" 출력.
- GREEN: -p build-logic check javadoc rc=0. :performanceCheck rc=0(lane 5개
  테스트 26, skip 0; httpclient 힙 증가 5,243,880 B / 상한 16,777,216 B,
  Mongo p50·p95·p99 1ms, examined/returned 1.0, pool 대기 0).
  -p optional-platforms performanceCheck rc=0(grpc 1). 두 명령에
  -Pperformance.assertions.enabled=false → rc=1, 6개 lane 모두 테스트 전
  실패. =true → rc=0. verifyStrictLaneStages --dry-run 두 빌드 rc=0,
  architectureCheck rc=0, ci·mainCheck·integrationCheck·releaseCheck·
  performanceCheck --dry-run --configuration-cache rc=0,
  :mainCheck --continue --warning-mode=fail rc=0, -p optional-platforms ci
  rc=0, persistence-mongo:check rc=0, verify-httpclient-docs.py rc=0.
- 이 변경 밖 실패: :ci :verifyDependencyLocks --continue와
  httpclient:check가 CircuitBreakerStatusAccountingTest JDK 2건으로 rc=1.
  기준 커밋 3d73b6f1에서도 10/10 실패(이 트리도 10/10). ci의 다른
  task는 모두 통과했다.

Refs: docs/upgrade/2026-11-maturity-review.md B-43, docs/testing/TESTING_STRATEGY.md 6절
BREAKING CHANGE: PERFORMANCE lane은 performance.assertions.enabled가
true가 아니거나 skip된 테스트가 있으면 실패한다. 자기 성능 lane의 그 속성
선언(false)과 skip을 없앤다.
This commit is contained in:
DongHyeonka
2026-11-06 20:33:47 +09:00
parent 3d73b6f1ba
commit e6b6e18eae
21 changed files with 346 additions and 82 deletions
+2 -3
View File
@@ -189,13 +189,12 @@ jobs:
# never catches. This job replaced `web-load-abuse-and-shutdown`, which despite its name ran
# `:adapter:inbound:web:test` and `webJettyCompatTest` — two lanes the pull-request and main stages
# already run — and never the load lanes (review 2026-11-01 08-01), and `httpclient-performance`.
# No assertions switch: the strict-lane convention asserts every performance lane's bounds, so this
# job and a person on a workstation run the same command and get the same bounds.
performance-check:
runs-on: ubuntu-latest
timeout-minutes: 120
env:
# A project property rather than a command-line flag, so the run command stays a plain,
# verifiable task invocation while the machine-dependent bounds are still asserted.
GRADLE_OPTS: -Dorg.gradle.project.performance.assertions.enabled=true
TESTCONTAINERS_REUSE_ENABLE: "false"
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2
-4
View File
@@ -160,10 +160,6 @@ jobs:
defaults:
run:
working-directory: src
env:
# A project property rather than a command-line flag, so each run command stays a plain,
# verifiable task invocation while the machine-dependent bounds are still asserted.
GRADLE_OPTS: -Dorg.gradle.project.performance.assertions.enabled=true
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2
- uses: ./.github/actions/setup-gradle-java
+5 -1
View File
@@ -73,6 +73,10 @@ root `CLAUDE.md`는 이 목록의 동기화된 요약이다. 두 문서가 어
`INTEGRATION`·`SYSTEM`·`PERFORMANCE` lane과 `mainStage()` lane은 이미지·네트워크·시간·문서처럼
입력으로 잡히지 않는 것에 기대므로 매번 다시 돈다. hermetic lane이 저장소 파일을 읽으면 그 파일을
입력으로 선언해야 캐시가 거짓 통과를 내지 않는다.
- `PERFORMANCE` lane은 상한을 매번 단언한다. 끄는 스위치가 없다: convention이 테스트 JVM에
`performance.assertions.enabled=true`를 넣고, 빌드(`-P`·`GRADLE_OPTS`)나 lane이 다른 값을 주면
테스트 전에 실패한다. skip된 테스트가 하나라도 있어도 실패한다(`rejectSkipped`를 따로 쓰지 않아도).
사람이 손으로 돌리는 `./gradlew performanceCheck`와 workflow가 같은 상한을 단언한다.
- `verifyEnvKeys`는 app-bootstrap 태스크로 유지한다(소유자 결정 4). 구현은 build-logic의
`ca.config-contract`다. `src/build-tools/` included build는 비워서 삭제했다.
- `src/build.gradle`: 루트 라이프사이클(`ci`, `mainCheck`, `integrationCheck`, `performanceCheck`,
@@ -222,7 +226,7 @@ cd src
./gradlew ci # PR 게이트 = leaf check + architecture + quality
./gradlew mainCheck # main 단계 = env 계약·전체 기동(샘플 포함)·public path·전송/메시징 레인·leaf 문서 대조 레인·기능 생성 왕복(verifyFeatureScaffold)·테스트 코드 정적 분석(testQualityCheck)
./gradlew integrationCheck # 실제 컨테이너 integration 레인 전체(main·nightly, Docker 필요)
./gradlew performanceCheck # 부하·종료·자원 한계 레인(main·nightly)
./gradlew performanceCheck # 부하·종료·자원 한계 레인(main·nightly). 상한은 항상 단언, 끄면 실패
./gradlew verifyStrictLaneStages --dry-run # 이 빌드의 strict lane이 workflow가 실행하는 단계에 속하는지(optional 빌드는 -p optional-platforms로 따로)
./gradlew verifyPublicPathSnapshot
./gradlew :app-bootstrap:verifyEnvKeys
+5 -2
View File
@@ -33,7 +33,9 @@ If this summary drifts from `AGENTS.md`, `AGENTS.md` wins and this summary must
in that leaf's `mainCheck`; a merge-blocking contract over non-document repository files is a
pull-request lane whose inputs are exactly those files (app-bootstrap `repository-contract`).
`TEST`/`ARCHITECTURE` lanes are cacheable; `INTEGRATION`/`SYSTEM`/`PERFORMANCE` and main-stage
lanes rerun every time.
lanes rerun every time. A `PERFORMANCE` lane asserts its bounds in every run and rejects skipped
tests: the convention sets `performance.assertions.enabled=true`, and a build or lane that sets
anything else fails before a test runs.
- `verifyEnvKeys` stays an app-bootstrap task (owner decision 4); its implementation is build-logic's
`ca.config-contract`. The `src/build-tools/` included build was emptied and removed.
- `src/build.gradle`: the root lifecycle (`ci`, `mainCheck`, `integrationCheck`, `performanceCheck`,
@@ -145,7 +147,8 @@ of domain-core/shared-contract/application-*, Git-carryable sources, ArchUnit),
(SpotBugs, FindSecBugs over production code), `testQualityCheck` (the same over every test source
set, against `config/spotbugs/exclude-test.xml`), `configContractCheck` (the environment contract), `integrationCheck` (the
declared strict integration lanes; a lane that needs an input the build was not given is left out
and named), `performanceCheck` (the declared performance lanes; main and nightly). `ci` (pull
and named), `performanceCheck` (the declared performance lanes, bounds always asserted — the same
command by hand or in a workflow; main and nightly). `ci` (pull
request) is leaf checks (API surfaces included, owner decision 20) + architectureCheck +
qualityCheck, and the workflow also runs `verifyStrictLaneStages --dry-run`; `mainCheck` (main branch) is configContractCheck +
qualificationCheck (which includes testQualityCheck) + the sample web contract + the sample full boot (`:sample-portfolio:systemTest`)
+9 -8
View File
@@ -8,19 +8,20 @@ from an explicit profile setting.
## How to run
```bash
# structural bounds only (default; still executes every test)
# every bound, structural and machine-dependent
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest --console=plain
# full certification, including machine-dependent bounds
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \
-Pperformance.assertions.enabled=true --console=plain
# JMH benchmarks
# JMH benchmarks: numbers only, never a gate
./gradlew :adapter:outbound:httpclient:jmh --console=plain
```
Machine-dependent assertions are reported as explicitly skipped when the flag is absent — the lane
never silently degrades into a pass.
The lane asserts every bound in every run. There is no switch that turns the machine-dependent one
off: the strict-lane convention fails a performance lane before it runs when the build sets
`performance.assertions.enabled` to anything but `true`, and fails it when one of its tests is
skipped. The same command is what `integration-main.yml` (`:performanceCheck`) and `release.yml`
run. Every run prints the measured heap increase (`[httpclient-performance] streaming download heap
increase … bytes, bound … bytes`) to the test output and report, and a bound that fails repeats the
value in its assertion message; for numbers without a verdict, use JMH.
## Certified bounds
+2 -3
View File
@@ -9,11 +9,10 @@ cd src
./gradlew :adapter:outbound:httpclient:test --console=plain
./gradlew :adapter:outbound:httpclient:httpClientStableContractTest --console=plain
./gradlew :adapter:outbound:httpclient:httpClientSecurityTest --console=plain
./gradlew :adapter:outbound:httpclient:spring62CompatibilityTest --console=plain
./gradlew :adapter:outbound:httpclient:spring62ApiSurfaceScan --console=plain
./gradlew :adapter:outbound:httpclient:spring70CompatibilityTest --console=plain
./gradlew :adapter:outbound:httpclient:httpClientFailureInjectionTest --console=plain # needs Docker
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \
-Pperformance.assertions.enabled=true --console=plain
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest --console=plain # every bound asserted
./gradlew verifyCleanArchitectureDependencies --console=plain
./gradlew :app-bootstrap:architectureTest --console=plain
python3 ../scripts/verify-httpclient-docs.py
+11 -11
View File
@@ -57,19 +57,19 @@ gives route-scoped pooling and Dynamic Target pinning, JDK gives HTTP/2.
| Profile | Frequency | Release gate | Task |
|---|---|---|---|
| Spring Framework 7.0 (repository baseline) | every PR | required | `spring70CompatibilityTest` |
| Spring Framework 6.2 API surface | every PR | required | `spring62CompatibilityTest` |
| Apache HC5 + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=apache` |
| JDK HttpClient + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=jdk` |
| SSRF / cardinality suite | every PR | required | `httpClientSecurityTest` |
| Toxiproxy fault suite | nightly, release | required | `httpClientFailureInjectionTest` |
| Performance certification | nightly, release | required | `httpClientPerformanceTest -Pperformance.assertions.enabled=true` |
| Jetty HTTP/3 | nightly | Experimental, non-blocking | `test -Phttp3.tests.enabled=true` |
| Spring Framework 7.0 (repository baseline) | PR touching this leaf, main push, release | required | `spring70CompatibilityTest` |
| Spring Framework 6.2 API surface | every PR (`check`), release | required | `spring62ApiSurfaceScan` |
| Apache HC5 + RestClient | PR touching this leaf (both transports together on every PR through `check`), release | required | `httpClientStableContractTest -Phttpclient.contract.transports=apache` |
| JDK HttpClient + RestClient | PR touching this leaf (both transports together on every PR through `check`), release | required | `httpClientStableContractTest -Phttpclient.contract.transports=jdk` |
| SSRF / cardinality suite | every PR (`check`), release | required | `httpClientSecurityTest` |
| Toxiproxy fault suite | main push, nightly (`integrationCheck`), release | required | `httpClientFailureInjectionTest` |
| Performance certification | main push, nightly (`performanceCheck`), release | required | `httpClientPerformanceTest` (every bound asserted; no switch turns them off) |
| Jetty HTTP/3 | main push, nightly | Experimental, non-blocking | `test -Phttp3.tests.enabled=true` |
### Known limitation of the Spring 6.2 lane
This repository's Spring Boot 4.0 baseline pins Spring Framework 7, so a real 6.2 runtime cannot be
resolved here. `spring62CompatibilityTest` therefore verifies the **API surface**: the common
resolved here. `spring62ApiSurfaceScan` therefore verifies the **API surface**: the common
packages must not reference any Spring 7-only type, and no class uses
`org.springframework.web.service.registry`. Executing the suite against an actual 6.2 distribution
requires a host project on that line. This limitation is stated rather than hidden behind a passing
@@ -83,5 +83,5 @@ Stated so the matrix is read as a measurement rather than an aspiration.
| Gap | Why | What is proven instead |
|---|---|---|
| HTTP/2 frame injection (`REFUSED_STREAM`, arbitrary `GOAWAY`) | The fixture server exposes no frame-level control, and a purpose-built h2 server is a larger dependency than the guarantee is worth here | `Http2EvidenceMapperTest` proves the frame → evidence mapping, and `NegotiatedProtocolContractTest` proves h2 is really negotiated |
| Spring 6.2 runtime | This repository's Boot 4.0 baseline pins Spring 7 | `spring62CompatibilityTest` confines the common packages to the 6.2 API surface |
| Performance latency baseline | Numbers measured on a build agent are not a certification | `httpClientPerformanceTest` asserts structural bounds unconditionally; latency and heap bounds run under `-Pperformance.assertions.enabled=true` |
| Spring 6.2 runtime | This repository's Boot 4.0 baseline pins Spring 7 | `spring62ApiSurfaceScan` confines the common packages to the 6.2 API surface |
| Performance latency baseline | Numbers measured on a build agent are not a certification | `httpClientPerformanceTest` asserts its structural bounds and the streaming heap bound in every run; it asserts no latency figure |
+16
View File
@@ -154,6 +154,22 @@ fixture는 **지원 코드**다. TEST/INTEGRATION/SYSTEM/ARCHITECTURE/PERFORMANC
PERFORMANCE는 기본 로컬 `check`에 붙이지 않는다. 명시적 benchmark/performance task나
scheduled workflow가 실행한다.
PERFORMANCE lane은 상한을 매번 단언한다. 측정만 하고 통과하는 모드는 없다.
- convention(`ca.strict-test-lane`)이 테스트 JVM에 `performance.assertions.enabled=true`를 넣는다.
leaf나 lane이 이 값을 따로 선언하지 않는다.
- 빌드가 `-Pperformance.assertions.enabled`(또는 `GRADLE_OPTS`의
`-Dorg.gradle.project.performance.assertions.enabled`)에 `true` 아닌 값을 주거나, lane이 테스트
JVM 값을 덮어쓰면 테스트 전에 실패한다. 메시지가 지우거나 `true`로 바꿀 곳을 알려 준다.
- skip된 테스트가 하나라도 있으면 실패한다. skip은 아무도 재지 않은 상한이다.
- 그래서 사람이 desktop에서 돌리는 `./gradlew performanceCheck`와 workflow가 같은 상한을 단언한다.
- 판정 없이 숫자만 보려면 JMH가 있는 leaf의 `./gradlew :<leaf>:jmh`를 쓴다. 게이트가 아니다.
실패한 상한은 잰 값을 단언 메시지에 남긴다.
2026-11-05 desktop 게이트에서 httpclient lane이 이 속성 기본값 `false`로 돌아, 힙 상한을 재지 않고
통과했다. 그 전에는 JPA pool lane이 같은 속성 뒤에 숨어 아무 상한도 비교하지 않았다
(`PoolLaneClaimTest`). 이 규칙은 그 두 번을 되풀이하지 않기 위한 것이다.
## 7. 새 테스트를 추가할 때 판단 순서
1. 실제 외부 인프라가 필요한가?
+4 -1
View File
@@ -4,7 +4,7 @@
- 커밋: `34565130` High·P1 19건 · `1400dc15` 남은 발견·매뉴얼·생성 명령 · `5786126c` 남은 항목과 합친 뒤 결함 ·
`7ef8d0b4` 재리뷰 기록 · `bbe944bd` 재리뷰 결함 · `ecc74b7d` 2차 재리뷰 기록·이 안내 · `1885bf8d` 2차 재리뷰 결함 ·
`339cf8fd` 3차 확인 리뷰 기록 · `f40c03c6` 3차 확인 리뷰 Minor · `c2e55510` 마지막 확인 리뷰 기록 ·
`2678955c` 테스트 코드 정적 분석(B-40) · `70fb80bf` 소유자가 다시 정한 기본값 3가지(늦은 멱등 완료 기록, `failure-retention` 기동 검사, workflow `cd`) — `docs(...)` 커밋은 문서만 바꾼다
`2678955c` 테스트 코드 정적 분석(B-40) · `70fb80bf` 소유자가 다시 정한 기본값 3가지(늦은 멱등 완료 기록, `failure-retention` 기동 검사, workflow `cd`) · B4 성능 lane 단언 상시화(2026-11-06, B-43) — `docs(...)` 커밋은 문서만 바꾼다
- 근거 문서: `docs/reviews/2026-11-01-maturity-review/README.md` 13~16절, `docs/reviews/2026-11-03-rereview/README.md`
4·6·8~13절, 각 커밋의 `BREAKING CHANGE:` 꼬리말
- 아래 표의 각 줄은 위 범위의 diff와 코드로 확인했다. 확인하지 못한 것은 그 자리에 "확인 못 함"으로 적었고 9절에
@@ -36,6 +36,7 @@
| 13 | `.env`에 옛 `.env.example` 값을 복사해 두었다면 5.6절 키를 다시 맞춘다 | 새 기본값이 안 먹거나(JDBC batch), 기동 실패(알림 콜백 상한) | `.env`를 복사해 쓰는 서비스 |
| 14 | 자기 테스트 코드에 `./gradlew testQualityCheck`를 돌려 보고, 발견을 고치거나 이유를 단 Match로 뺀다(B-40) | `mainCheck`가 실패 | 테스트 코드가 있는 모든 서비스 |
| 15 | staging·prod 배포 마이그레이션 도구에 `flyway.postgresql.transactional.lock=false`(환경 변수 `FLYWAY_POSTGRESQL_TRANSACTIONAL_LOCK=false`)를 넣는다. 자기 Flyway stream은 `Flyway.configure()` 대신 `FlywaySchemaPolicy.configure()`로 만든다(5절 「Flyway 잠금」) | `CREATE INDEX CONCURRENTLY`·`REINDEX … CONCURRENTLY` 마이그레이션에서 배포가 오류 없이 멈춘다. 자기 stream이 `Flyway.configure()`를 부르면 `:app-bootstrap:architectureTest` 실패 | persistence-jpa 선택 + 배포 도구로 마이그레이션하거나 자기 stream을 둔 서비스 |
| 16 | 자기 성능 lane(`performance()`)에서 `performance.assertions.enabled`를 `false`로 두는 `systemProperty` 줄을 지우고, 그 속성을 읽어 상한 단언을 건너뛰던 테스트는 항상 단언하게 고친다. 성능 lane의 `@Disabled`·assumption skip을 없앤다(B-43) | `performanceCheck`가 테스트 전에 실패(`its test JVM would get performance.assertions.enabled=false`)하거나 skip 때문에 실패(`forbids skipped tests`) | 자기 성능 lane이 있는 서비스 |
### 되돌릴 수 없는 것
@@ -109,6 +110,7 @@
| B-37 | app-bootstrap 테스트 lane 분리: 저장소 파일을 읽는 테스트는 태그 `document-contract`(main 단계) 또는 `repository-contract`(`check`)여야 하고, 일반 `test` lane에는 `ca.repository.root`가 없다 | app-bootstrap에 둔 자기 테스트가 저장소 파일을 읽으면 `test`에서 실패 | 태그를 단다 | `1400dc15` · `src/app-bootstrap/build.gradle` |
| B-40 | 테스트 코드 정적 분석 `testQualityCheck`: SpotBugs·FindSecBugs를 main 밖 모든 source set(test, testFixtures, integrationTest, systemTest 등)에 돌린다. 테스트용 제외 목록은 `config/spotbugs/exclude-test.xml`(보안 범주·내부 상태 노출·생성자 예외·반환값 null 역참조·아키텍처 위반 fixture). 분석 classpath가 그 source set의 runtime classpath로 넓어졌고, 분석이 불완전하면(빠진 클래스) 실패한다. main 빌드는 `mainCheck`(`qualificationCheck`), optional 빌드는 `ci` | 자기 테스트 코드의 SpotBugs 발견(예: `CountDownLatch.await` 반환값 무시, 공유 변수 비원자 쓰기, 닫지 않은 `Statement`)이 main 단계를 막는다. 테스트 classpath에 없는 타입을 참조하면 `SpotBugs analysis incomplete` | 고치거나, 클래스·메서드로 좁힌 Match를 이유와 함께 `exclude-test.xml`에 더한다. 빠진 타입은 그 테스트 suite 의존성에 넣는다 | `2678955c` · `src/build-logic/.../quality/QualityConventionsPlugin.java`, `src/config/spotbugs/exclude-test.xml`, `src/build.gradle` |
| B-42 | shared-contract 패키지 규칙 `SHARED_CONTRACT_CONTAINS_ONLY_OPERATIONAL_CONTRACT_PACKAGES`가 실제로 검사한다. 대상은 `dev.caskeleton.shared..`뿐이고, 허용은 목록의 패키지(`concurrency digest error metrics observability ratelimit request response retry runtime security tracing`)와 그 아래, 뿌리 패키지에는 `package-info`만이다. 전에는 뿌리를 허용 목록에 넣고 접두어로 비교해서 모듈 안은 아무것도 잡지 못했고, 다른 leaf의 `..shared..` 패키지를 잡았다 | `dev.caskeleton.shared`에 목록에 없는 패키지나 뿌리 클래스를 둔 파생 서비스는 architectureTest 실패. 다른 leaf의 `shared` 패키지는 더는 걸리지 않는다 | 업무 개념이면 그것을 소유하는 모듈로 옮긴다. 운영 계약이면 `ForbiddenApiArchitectureTest.SHARED_CONTRACT_PACKAGES`와 shared-contract CLAUDE.md 「Responsibility」에 함께 더한다 | r10 · `src/app-bootstrap/src/architectureTest/.../ForbiddenApiArchitectureTest.java`, `SharedContractPackageRuleTest.java` |
| B-43 | 성능 lane 단언 상시화: `performance()` lane은 convention이 테스트 JVM에 `performance.assertions.enabled=true`를 넣는다. 빌드가 `-P`·`GRADLE_OPTS`(`-Dorg.gradle.project.…`)로 `true` 아닌 값을 주거나, lane `systemProperty`·`jvmArgs`(`-D…`)·task 설정이 테스트 JVM 값을 `true` 아닌 값으로 덮으면 테스트 전에 실패한다. 성능 lane은 `rejectSkipped` 없이도 skip된 테스트가 있으면 실패한다. httpclient·Mongo leaf의 속성 선언과 `integration-main.yml` `performance-check`·`release.yml` `httpclient-release-gate`의 `GRADLE_OPTS` 줄을 지웠다. 측정만 하는 모드는 만들지 않았다(JMH가 그 용도) | `performanceCheck`·`:<leaf>:performanceTestLaneCheck`가 `is a performance lane, and this build sets performance.assertions.enabled=false`, `its test JVM would get performance.assertions.enabled=false`, `forbids skipped tests … a skipped test is a bound nobody measured` 중 하나로 실패 | `-Pperformance.assertions.enabled=false`를 지운다. 자기 lane의 그 `systemProperty`·`jvmArgs` 줄을 지우고, 그 속성을 읽던 테스트는 항상 단언하게 고친다. 성능 lane의 skip은 없애거나 그 테스트를 다른 lane으로 옮긴다. `GRADLE_OPTS=-Dorg.gradle.project.performance.assertions.enabled=true`·`-P…=true`는 받아 주므로 그대로 둬도 된다 | B4(2026-11-06) · `src/build-logic/.../strictlane/StrictTestLanePlugin.java`, `.../strictlane/StrictTestLaneConventionTest.java`(test), `docs/testing/TESTING_STRATEGY.md` 6절 |
### 3.5 모듈 registry·새 leaf
@@ -240,6 +242,7 @@ V21 실패의 분류는 `docs/runbooks/migration-failed.md` "Classify the failur
| 서버 로그(GraphQL·gRPC·STOMP) | carrier로 답한 실패를 web과 같은 줄(shared-contract `ServerFailureLog`)로 ERROR에 남긴다: persistence 실패는 분류만, 5xx carrier는 예외와 함께, 4xx carrier는 남기지 않는다. gRPC는 ` (method=…)`, GraphQL은 ` (executionId=…)`를 붙인다. gRPC는 인식 못 한 실패(`INTERNAL_ERROR`)도 `unhandled failure on <method> …`로 남긴다. 클라이언트가 취소한 호출은 남기지 않고, 서비스가 `CANCELLED` status로 끝낸 살아 있는 호출은 남긴다. raw status는 넘겼든 던졌든 설명을 남기지 않는다(원인 예외가 있으면 그것만 붙인다). `IllegalArgumentException`(`BAD_PARAMETER`)은 web과 같은 WARN 한 줄 `request refused as BAD_PARAMETER by <예외 타입>`(메시지 없음)으로 남긴다. STOMP `@MessageMapping` 실패는 Spring의 `Unhandled exception from message handler method`(예외 전체) 대신 같은 줄로 남는다 | ERROR 로그가 늘어난다(전에는 gRPC가 아무것도 남기지 않았다). GraphQL·gRPC·STOMP는 잘못된 입력마다 WARN 한 줄이 생긴다. STOMP에서 Spring 문구로 경보를 걸었다면 고친다. `IDEMPOTENCY_RECOVERY_REQUIRED`의 scope digest를 모든 전송에서 찾을 수 있다(`docs/runbooks/idempotency-unavailable.md`) | r9 · `src/shared-contract/.../error/ServerFailureLog.java`, `GrpcExceptionHandlingInterceptor.java`, `GraphQlWireErrorMapper.java`, `StompHandlerFailureLog.java` |
| sample-portfolio | 익명 쓰기 401, 포스터 제목 대소문자 무시 유일(409 `POSTER_ALREADY_EXISTS`), 관리자도 남의 work log 수정 불가, 샘플 이벤트 tenant `global` | sample을 남긴 서비스만 | `34565130`, `1400dc15` |
| Flyway 잠금 | Flyway의 PostgreSQL 마이그레이션 잠금을 트랜잭션 대신 세션에 둔다. Spring Boot의 Flyway는 module-defaults `spring.flyway.postgresql.transactional-lock: false`, 템플릿이 직접 만드는 Flyway(멱등성·알림·sample stream, schema-per-tenant, testkit `MigrationContractRunner`)는 `FlywaySchemaPolicy.configure()`, messaging inbox stream은 같은 값을 직접 넣는다. 운영 코드의 `Flyway.configure()` 직접 호출은 `JpaProductionArchitectureTest`가 거부한다 | Flyway 기본값에서는 동시 인덱스 마이그레이션이 Flyway 자신의 열린 트랜잭션을 기다려 끝나지 않았다. 이제 끝난다. 마이그레이션은 잠금을 잡은 연결 하나로 돈다. transaction 모드 PgBouncer 뒤에서는 잠금 해제가 다른 서버 연결로 가 다음 배포가 기다릴 수 있으니 마이그레이션은 직접 연결이나 session 모드 pool로 돌린다. 배포 도구는 이 repo 밖이라 2절 15번으로 맞춘다 | 재리뷰 round6 RR6-03-01 · `persistence-jpa.yml`, `.../migration/FlywaySchemaPolicy.java`, `docs/jpa/migration-guide.md` "Who may change the schema"·"Concurrent index builds" |
| 성능 lane(HTTP client·Mongo) | httpclient `LargeBodyResourceTest`의 힙 상한(32 MiB 스트리밍 중 힙 증가 16 MiB 미만)을 매번 단언한다. 전에는 `-Pperformance.assertions.enabled=true`일 때만 단언하고 아니면 건너뛴다는 줄만 찍었다. Mongo `MongoResourceBudgetLaneTest`는 표준 gate 전체(p99 500ms, pool 대기 250ms, 힙 512 MiB, examined/returned 10, spill 금지)를 Gradle 밖 실행에서도 단언한다(B-43) | 손으로 돌린 `performanceCheck`가 상한을 실제로 재므로, 전에는 통과하던 느린 기계에서 실패할 수 있다 | B4(2026-11-06) · `LargeBodyResourceTest`, `MongoResourceBudgetLaneTest` |
## 6. 따라오는 순서
+1 -1
View File
@@ -27,7 +27,7 @@ Checkstyle은 **포매터가 표현할 수 없는 것만** — 네이밍과 몇
| `configContractCheck` | `:app-bootstrap:verifyEnvKeys`로 환경 설정 계약을 검사 |
| `qualificationCheck` | 전체 기동(system) 레인과 보조 source set Checkstyle을 실행 |
| `integrationCheck` | 선언된 strict integration lane(sample 포함)을 실행. 대부분 container runtime이 필요. 입력이 필요한 lane(Redis topology: `-Predis.topology.mode`, AWS: 보호 자격 증명 env)은 입력이 없으면 빠지고 집계가 그 이유를 출력 |
| `performanceCheck` | 선언된 strict performance lane(부하·종료·pool·자원 상한)을 실행. 실서버·container가 필요. `integration-main.yml` `performance-check`가 main·nightly에 돌림 |
| `performanceCheck` | 선언된 strict performance lane(부하·종료·pool·자원 상한)을 실행. 실서버·container가 필요. 상한은 매번 단언하고 끄는 속성이 없다(`-Pperformance.assertions.enabled`가 `true`가 아니면 실패, skip된 테스트가 있어도 실패). `integration-main.yml` `performance-check`가 main·nightly에 같은 명령으로 돌림 |
| `verifyStrictLaneStages` | `--dry-run` 전용. `explicitOnly`가 아닌 strict lane이 workflow가 실제 실행하는 단계(`ci`·`:sample-portfolio:check`·`mainCheck`·`integrationCheck`·`performanceCheck`·`releaseCheck`) 중 하나의 계획에 있는지, 각 단계를 workflow가 `--dry-run` 없이 실행하는지 검사. Gradle이 작업을 돌리지 않는 명령(`--dry-run`·`-v`·`--help` 등)과, 실패해도 단계가 실패하지 않을 수 있거나(`\|\| true`, pipefail 없는 파이프, `if` 조건 등) 돌지 않을 수 있거나(조건부 `exit 0` 뒤, 요청으로만 시작하는 workflow 등) 검사가 읽지 않는(`\|\| exit 1`, `{ }` 안 등) 명령은 실행으로 세지 않는다. 자세한 목록은 `build-logic/README.md`. PR 게이트가 돌림 |
| `ci` | PR 게이트: 모든 leaf `check`(API surface 검사 포함, 소유자 결정 20) + `architectureCheck` + `qualityCheck` |
| `mainCheck` | main 단계: `configContractCheck` + `qualificationCheck` + sample web 계약 + sample 전체 기동(`:sample-portfolio:systemTest`) + `verifyPublicPathSnapshot` + 전송·메시징 qualification 레인 + 각 leaf의 `mainCheck`(문서 대조 레인 `*DocumentContractTest`, objectstorage MinIO 레인) |
+3 -2
View File
@@ -81,8 +81,9 @@ Default lane: `./gradlew :adapter:outbound:httpclient:test`. Additional lanes, a
| `jmh` | per-call overhead benchmarks |
A selected lane never skips silently: the fault lane throws without Docker, the contract lane throws
on an empty or unknown transport selection, and the performance lane prints which machine-dependent
bounds were not asserted.
on an empty or unknown transport selection, and the performance lane asserts every bound in every
run — the strict-lane convention fails it when the build sets `performance.assertions.enabled` to
anything but `true` or when one of its tests is skipped.
The transport port (`BlockingTransportProvider`) has one abstract contract in the test fixtures,
`testkit/BlockingTransportProviderContract`. The engines pass it over a real socket
+3 -2
View File
@@ -111,8 +111,9 @@ cd src
```
No lane skips silently. The fault lane fails without Docker, the contract lane fails on an empty or
unknown transport selection, and the performance lane reports which machine-dependent bounds it did
not assert.
unknown transport selection, and the performance lane asserts every bound in every run (it fails
when the build sets `performance.assertions.enabled` to anything but `true`, or when a test is
skipped).
## Further reading
+3 -4
View File
@@ -138,8 +138,9 @@ dependencies {
}
// Which transports the contract suites cover and whether the HTTP/3 opt-in runs: the same two
// properties for every Test task of this leaf, declared once (review 2026-11-01 O-10a). Values a
// single lane needs (the performance assertions, the httpbin image) stay in that lane.
// properties for every Test task of this leaf, declared once (review 2026-11-01 O-10a). A value a
// single lane needs (the httpbin image) stays in that lane. The performance lane's bounds have no
// switch here: the strict-lane convention asserts them in every run.
tasks.withType(Test).configureEach {
systemProperty 'httpclient.contract.transports',
providers.gradleProperty('httpclient.contract.transports').orElse('apache,jdk').get()
@@ -193,8 +194,6 @@ strictTestLanes {
performance()
sourceSet = 'httpClientPerformanceTest'
description = 'Certifies pool, streaming, retry, and rotation resource bounds (design §28.8).'
systemProperty 'performance.assertions.enabled',
providers.gradleProperty('performance.assertions.enabled').orElse('false').get()
}
lane('httpClientFailureInjectionTest') {
integration()
@@ -52,11 +52,18 @@ class LargeBodyResourceTest {
}
}
long peakIncrease = usedHeap() - before;
// The measured value, so a passing run is a recorded number and not only a green line.
System.out.println(
"[httpclient-performance] streaming download heap increase "
+ peakIncrease
+ " bytes, bound "
+ payloadBytes / 2
+ " bytes");
assertThat(consumed).isEqualTo(payloadBytes);
PerformanceAssertions.machineDependent(
"streaming download heap increase stays well below the payload size",
() -> assertThat(peakIncrease).isLessThan((long) payloadBytes / 2));
assertThat(peakIncrease)
.as("streaming download heap increase stays well below the payload size")
.isLessThan((long) payloadBytes / 2);
}
}
}
@@ -3,32 +3,19 @@ package dev.caskeleton.adapter.outbound.httpclient.performance;
import static org.assertj.core.api.Assertions.assertThat;
/**
* Separates deterministic bounds from machine-dependent ones (design §28.8, §37).
* Structural bounds — retry budget arithmetic, a refresh collapsing to one request, drain
* completing — that hold on any machine (design §28.8, §37).
*
* <p>Structural bounds — retry budget arithmetic, streaming not buffering the whole payload, drain
* completing — always run. Latency and absolute memory numbers depend on the host, so they are
* asserted only when the certification flag is set; when it is off the lane still executes and
* still reports, rather than becoming a silent pass.
* <p>The machine-dependent bound (the streaming heap increase) is asserted in place, in every run.
* It used to sit behind {@code performance.assertions.enabled}, which defaulted to {@code false} in
* this leaf, so a lane run by hand reported success with the bound compared to nothing. A
* performance lane now asserts all of its bounds; the strict-lane convention rejects a build that
* tries to turn them off.
*/
final class PerformanceAssertions {
private PerformanceAssertions() {}
static boolean certificationEnabled() {
return Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false"));
}
static void machineDependent(String description, Runnable assertion) {
if (certificationEnabled()) {
assertion.run();
return;
}
System.out.println(
"[httpclient-performance] skipped machine-dependent bound (enable with "
+ "-Pperformance.assertions.enabled=true): "
+ description);
}
static void structural(String description, boolean condition) {
assertThat(condition).describedAs(description).isTrue();
}
@@ -239,9 +239,8 @@ strictTestLanes {
providers.gradleProperty('mongoCompatibilityImage').orElse(testContainerImages['mongo-7']).get()
systemProperty 'mongodb.toxiproxy.image',
providers.gradleProperty('mongoToxiproxyImage').orElse(testContainerImages['toxiproxy']).get()
// Assertions on by default: a performance gate without bounds is only a report.
systemProperty 'performance.assertions.enabled',
providers.gradleProperty('performance.assertions.enabled').orElse('true').get()
// No assertions switch: the strict-lane convention asserts a performance lane's bounds in
// every run, because a performance gate without bounds is only a report.
}
}
@@ -27,16 +27,16 @@ import org.junit.jupiter.api.Test;
* it is the same on an idle machine and a busy one, and it is what actually predicts the query
* falling over as the collection grows.
*
* <p>Timing bounds are asserted only when {@code -Pperformance.assertions.enabled=true}, for the
* same reason. The ratio bound is always asserted.
* <p>The timing bounds are asserted too, in every run, but the gate's bounds are generous for the
* same reason: they catch a query that falls over, not one that was slower than last time. They
* used to sit behind {@code performance.assertions.enabled}, which the build defaulted to on and
* this class to off, so a run outside Gradle skipped them; the strict-lane convention now asserts a
* performance lane's bounds in every run.
*/
class MongoResourceBudgetLaneTest {
private static final int DOCUMENT_COUNT = 5_000;
private static final boolean TIMING_ASSERTIONS =
Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false"));
private static MongoSingleReplicaSetContainer container;
private static MongoClient client;
@@ -75,18 +75,16 @@ class MongoResourceBudgetLaneTest {
@Test
void anIndexedQueryStaysInsideTheResourceBudget() {
MongoResourceBudgetReport report = measure(new Document("status", "PENDING"), "ix_status");
// The measured values, so a passing run is a recorded number and not only a green line.
System.out.println("[mongo-performance] indexed query " + report.asArtifact());
assertThat(report.documentsReturned()).isEqualTo(DOCUMENT_COUNT / 50);
assertThat(report.examinedToReturnedRatio())
.as("an index seek examines about what it returns")
.isLessThanOrEqualTo(MongoPerformanceGate.standard().maximumExaminedToReturnedRatio());
assertThat(MongoPerformanceGate.standard().violations(report))
.filteredOn(violation -> violation.startsWith("examined/returned"))
.as("an indexed query stays inside every bound of the standard gate")
.isEmpty();
if (TIMING_ASSERTIONS) {
assertThat(MongoPerformanceGate.standard().passes(report)).isTrue();
}
}
@Test
File diff suppressed because one or more lines are too long
@@ -10,12 +10,26 @@ import org.gradle.api.NamedDomainObjectContainer;
import org.gradle.api.Plugin;
import org.gradle.api.Project;
import org.gradle.api.plugins.JavaPluginExtension;
import org.gradle.api.provider.Provider;
import org.gradle.api.tasks.SourceSet;
import org.gradle.api.tasks.TaskProvider;
import org.gradle.api.tasks.testing.Test;
import org.gradle.testing.base.TestingExtension;
public final class StrictTestLanePlugin implements Plugin<Project> {
/**
* The switch a performance lane's machine-dependent bounds used to sit behind, now always on.
*
* <p>Each leaf declared it with its own default — {@code false} for the httpclient lane, {@code
* true} for the Mongo lane — and only the workflows set it. {@code ./gradlew :performanceCheck}
* typed by hand therefore passed with the httpclient heap bound compared to nothing (the
* 2026-11-05 desktop gate). The convention owns it now: every performance lane's test JVM gets
* {@code true}, so a test that still reads the switch asserts, and a build that sets it to
* anything else fails before the lane runs. Nothing in this repository reads it any more; the
* value is for tests written against the old switch, such as a derived service's.
*/
static final String PERFORMANCE_ASSERTIONS = "performance.assertions.enabled";
@Override
public void apply(Project project) {
project.getPluginManager().withPlugin("java", ignored -> configure(project));
@@ -229,7 +243,8 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
// judge the lane again rather than reuse a result produced under the looser policy. Without
// this a lane that skipped a test stayed green, up to date and from the build cache, after
// rejectSkipped was switched on (review 2026-11-03 RR-06-02).
test.getInputs().property("strictLane.rejectSkipped", project.provider(lane::isRejectSkipped));
test.getInputs()
.property("strictLane.rejectSkipped", project.provider(() -> rejectsSkipped(lane)));
test.getInputs()
.property("strictLane.requiredTests", project.provider(lane::getRequiredTests));
test.getInputs()
@@ -243,6 +258,17 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
test.getOutputs().upToDateWhen(ignored -> false);
test.getOutputs().doNotCacheIf(rerunReason, ignored -> true);
}
boolean performance = lane.getCategory() == TestLaneCategory.PERFORMANCE;
if (performance) {
// Set before the lane's own JVM arguments and system properties — Gradle turns a -D in
// jvmArgs into a system property — so a lane that still declares the switch replaces this
// value and the check below sees the override, instead of the override disappearing silently.
test.systemProperty(PERFORMANCE_ASSERTIONS, "true");
Provider<String> requested = project.getProviders().gradleProperty(PERFORMANCE_ASSERTIONS);
String laneName = lane.getName();
test.doFirst(
running -> requireAssertionsOn(laneName, ownerPath, requested.getOrNull(), (Test) running));
}
if (lane.getMaxHeapSize() != null && !lane.getMaxHeapSize().isBlank()) {
test.setMaxHeapSize(lane.getMaxHeapSize());
}
@@ -288,7 +314,7 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
+ detail
+ " — a lane that runs nothing reports success for whatever it was meant to prove.");
}
if (lane.isRejectSkipped() && tracker.skippedCount() > 0L) {
if (rejectsSkipped(lane) && tracker.skippedCount() > 0L) {
throw new GradleException(
"strict test lane '"
+ lane.getName()
@@ -296,7 +322,10 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
+ ownerPath
+ " forbids skipped tests but observed "
+ tracker.skippedCount()
+ " skipped test(s).");
+ " skipped test(s)."
+ (performance
? " A performance lane always does: a skipped test is a bound nobody measured."
: ""));
}
if (!required.isEmpty()) {
List<String> absent =
@@ -318,6 +347,55 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
}
}
/**
* A performance lane rejects skipped tests whatever it declares: its tests are its bounds, and a
* skip is a bound the run reports as passed without comparing it to anything.
*/
private static boolean rejectsSkipped(StrictTestLaneSpec lane) {
return lane.isRejectSkipped() || lane.getCategory() == TestLaneCategory.PERFORMANCE;
}
/**
* Fails a performance lane, before any test runs, when the build or the lane turns its bounds off.
* {@code true} from the build is accepted, so a command that still passes it keeps working.
*/
private static void requireAssertionsOn(
String laneName, String ownerPath, String requestedByBuild, Test test) {
if (requestedByBuild != null && !Boolean.parseBoolean(requestedByBuild.trim())) {
throw new GradleException(
"strict test lane '"
+ laneName
+ "' in "
+ ownerPath
+ " is a performance lane, and this build sets "
+ PERFORMANCE_ASSERTIONS
+ "="
+ requestedByBuild
+ " (-P"
+ PERFORMANCE_ASSERTIONS
+ ", or -Dorg.gradle.project."
+ PERFORMANCE_ASSERTIONS
+ " in GRADLE_OPTS). A performance lane asserts its bounds in every run: one without"
+ " them measures nothing it can fail on and still reports success. Remove the"
+ " property or set it to true.");
}
Object effective = test.getSystemProperties().get(PERFORMANCE_ASSERTIONS);
if (effective == null || !Boolean.parseBoolean(String.valueOf(effective).trim())) {
throw new GradleException(
"strict test lane '"
+ laneName
+ "' in "
+ ownerPath
+ " is a performance lane, but its test JVM would get "
+ PERFORMANCE_ASSERTIONS
+ "="
+ effective
+ " from this project's build (a lane systemProperty or jvmArgs, or a block that"
+ " configures the task). The strict-lane convention sets it to true for every"
+ " performance lane; delete the override.");
}
}
private static boolean startsARealServer(StrictTestLaneSpec lane) {
return lane.getCategory() != TestLaneCategory.TEST
&& lane.getCategory() != TestLaneCategory.ARCHITECTURE;
@@ -137,6 +137,12 @@ public class StrictTestLaneSpec implements Named {
category = TestLaneCategory.ARCHITECTURE;
}
/**
* A lane whose tests are resource or latency bounds. Besides rerunning every time and taking the
* build-wide real-server slot, it rejects skipped tests and always runs with its bounds asserted:
* the convention sets {@code performance.assertions.enabled=true}, and a build or lane that sets
* it to anything else fails before a test runs.
*/
public void performance() {
category = TestLaneCategory.PERFORMANCE;
}
@@ -611,6 +611,147 @@ class StrictTestLaneConventionTest {
assertThat(result.getOutput()).contains("rulesLane=false").contains("databaseLane=true");
}
@Test
void aPerformanceLaneFailsOnAnExceededBoundWhenTheBuildSetsNoAssertionsProperty()
throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"BoundTest.java",
"""
import org.junit.jupiter.api.Test;
class BoundTest {
@Test void staysInsideTheBound() {
long p99Millis = 900;
// The shape the httpclient lane had: the machine-dependent bound behind the switch.
if (Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false"))
&& p99Millis > 500) {
throw new AssertionError("p99 900ms above the 500ms bound");
}
}
}
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput()).contains("BoundTest > staysInsideTheBound() FAILED");
}
@Test
void aPerformanceLaneFailsBeforeTestingWhenTheBuildTurnsItsAssertionsOff() throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest {
@Test void measured() { throw new AssertionError("the lane ran a test"); }
}
""");
// when
BuildResult result =
runner("performanceTestLaneCheck", "-Pperformance.assertions.enabled=false")
.buildAndFail();
// then
assertThat(result.getOutput())
.contains("this build sets performance.assertions.enabled=false")
.contains("set it to true")
.doesNotContain("MeasuredTest > measured() FAILED");
}
@Test
void aPerformanceLaneFailsWhenItsOwnDeclarationTurnsTheAssertionsOff() throws Exception {
// given
buildFile(
performanceLane(
"""
systemProperty 'performance.assertions.enabled',
providers.gradleProperty('performance.assertions.enabled').orElse('false').get()
"""));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest { @Test void measured() {} }
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput())
.contains("its test JVM would get performance.assertions.enabled=false")
.contains("delete the override");
}
@Test
void aPerformanceLaneFailsWhenItsJvmArgumentsTurnTheAssertionsOff() throws Exception {
// given
buildFile(performanceLane("jvmArgs '-Dperformance.assertions.enabled=false'"));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest { @Test void measured() {} }
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput())
.contains("its test JVM would get performance.assertions.enabled=false")
.contains("delete the override");
}
@Test
void aPerformanceLaneFailsWhenOneOfItsTestsIsSkipped() throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"PartlyMeasuredTest.java",
"""
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
class PartlyMeasuredTest {
@Test void measured() {}
@Test @Disabled("too slow on this machine") void quarantined() {}
}
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput())
.contains("forbids skipped tests")
.contains("a skipped test is a bound nobody measured");
}
@Test
void aPerformanceLaneAcceptsABuildThatTurnsItsAssertionsOnExplicitly() throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest { @Test void measured() {} }
""");
// when
BuildResult result =
runner("performanceTestLaneCheck", "-Pperformance.assertions.enabled=true").build();
// then
assertThat(result.task(":loadLane").getOutcome()).isEqualTo(TaskOutcome.SUCCESS);
}
private void buildFile(String laneBlock) throws Exception {
Files.writeString(
projectDir.resolve("build.gradle"),
@@ -631,6 +772,32 @@ class StrictTestLaneConventionTest {
Files.writeString(testSource.resolve(fileName), source);
}
private void writePerformanceTest(String fileName, String source) throws Exception {
Path performanceSource = projectDir.resolve("src/performance/java");
Files.createDirectories(performanceSource);
Files.writeString(performanceSource.resolve(fileName), source);
}
private static String performanceLane(String extraDeclarations) {
return """
sourceSets { performance }
dependencies {
performanceImplementation platform('org.junit:junit-bom:5.11.3')
performanceImplementation 'org.junit.jupiter:junit-jupiter'
performanceRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
strictTestLanes {
lane('loadLane') {
performance()
sourceSet = 'performance'
description = 'Measures a resource bound.'
%s
}
}
"""
.formatted(extraDeclarations);
}
private static String junitDependencies() {
return """
dependencies {