fix(strict-lane)!: 성능 lane이 상한을 재지 않고 통과하지 못하게 한다

문제: 성능 lane의 기계 의존 상한(httpclient 스트리밍 힙 증가, Mongo 시간
gate)이 performance.assertions.enabled 뒤에 있었고 기본값이 자리마다
달랐다(httpclient build false, Mongo build true, Mongo 테스트 코드 false).
workflow만 GRADLE_OPTS로 켰다. 그래서 desktop에서 손으로 돌린
./gradlew :performanceCheck가 httpclient 힙 상한을 재지 않고 통과했다
(2026-11-05 r10 게이트). JPA pool lane도 전에 같은 속성 뒤에서 아무 상한도
비교하지 않았다(PoolLaneClaimTest).

방법: 규칙을 build-logic strict-lane convention 한 곳에 둔다.
- PERFORMANCE lane 테스트 JVM에 performance.assertions.enabled=true를
  넣는다. 빌드(-P, GRADLE_OPTS)나 lane(systemProperty, jvmArgs, task
  설정)이 true 아닌 값을 주면 테스트 전에 실패하고, 지우거나 true로
  바꾸라고 알려 준다. true는 받아 준다(파생 서비스 workflow가 넘긴다).
- PERFORMANCE lane은 skip된 테스트가 있으면 실패한다.
- httpclient·Mongo 테스트는 조건 없이 단언하고 잰 값을 한 줄 남긴다.
  두 leaf의 속성 선언과 integration-main·release workflow의 GRADLE_OPTS
  줄을 지웠다. 사람과 workflow가 같은 명령을 쓴다. verifyStrictLaneStages가
  읽는 run: 줄은 그대로다.
버린 대안: 두 leaf 기본값만 true로(정의가 leaf마다 남아 다음 lane이 다시
복사한다), 속성 삭제만(옛 속성을 기본값 false로 읽는 파생 서비스 테스트가
조용히 건너뛴다), 측정 전용 태스크(쓸 곳이 없다: 숫자만 볼 때는 JMH, 실패한
단언과 매 실행 출력이 값을 남긴다).
같은 원인으로 함께 고친 문서: httpclient support-matrix·release-checklist의
없는 lane 이름 spring62CompatibilityTest와 빠진 main push 주기.

위험·한계: 손으로 돌린 performanceCheck가 이제 상한을 실제로 재므로 느린
기계에서는 실패할 수 있다. JVM 인자 공급자(jvmArgumentProviders)가 넣는
-D는 보지 않는다. Docker가 없을 때 skip되는 통합·시스템 lane은 지금
모두 실패한다(전부 skip이면 executed no test, 아니면 skip 안 되는 컨테이너
테스트가 실패) — 거짓 통과가 아니어서 고치지 않았다.

검증(desktop ~/b4-logs, flock 아래, 노트북과 같은 트리 — git diff sha256 일치):
- RED(고치기 전 트리 + 새 테스트): build-logic 새 테스트 4건이
  UnexpectedBuildSuccess로 실패 — aPerformanceLaneFailsOnAnExceededBound
  WhenTheBuildSetsNoAssertionsProperty, …FailsBeforeTestingWhenTheBuild
  TurnsItsAssertionsOff, …FailsWhenItsOwnDeclarationTurnsTheAssertionsOff,
  …FailsWhenOneOfItsTestsIsSkipped. jvmArgs 경로(…FailsWhenItsJvmArguments
  TurnTheAssertionsOff)도 같은 실패. 실제 lane: 속성 없이
  httpClientPerformanceTest rc=0인데 "skipped machine-dependent bound" 출력.
- GREEN: -p build-logic check javadoc rc=0. :performanceCheck rc=0(lane 5개
  테스트 26, skip 0; httpclient 힙 증가 5,243,880 B / 상한 16,777,216 B,
  Mongo p50·p95·p99 1ms, examined/returned 1.0, pool 대기 0).
  -p optional-platforms performanceCheck rc=0(grpc 1). 두 명령에
  -Pperformance.assertions.enabled=false → rc=1, 6개 lane 모두 테스트 전
  실패. =true → rc=0. verifyStrictLaneStages --dry-run 두 빌드 rc=0,
  architectureCheck rc=0, ci·mainCheck·integrationCheck·releaseCheck·
  performanceCheck --dry-run --configuration-cache rc=0,
  :mainCheck --continue --warning-mode=fail rc=0, -p optional-platforms ci
  rc=0, persistence-mongo:check rc=0, verify-httpclient-docs.py rc=0.
- 이 변경 밖 실패: :ci :verifyDependencyLocks --continue와
  httpclient:check가 CircuitBreakerStatusAccountingTest JDK 2건으로 rc=1.
  기준 커밋 3d73b6f1에서도 10/10 실패(이 트리도 10/10). ci의 다른
  task는 모두 통과했다.

Refs: docs/upgrade/2026-11-maturity-review.md B-43, docs/testing/TESTING_STRATEGY.md 6절
BREAKING CHANGE: PERFORMANCE lane은 performance.assertions.enabled가
true가 아니거나 skip된 테스트가 있으면 실패한다. 자기 성능 lane의 그 속성
선언(false)과 skip을 없앤다.
This commit is contained in:
DongHyeonka
2026-11-06 20:33:47 +09:00
parent 3d73b6f1ba
commit e6b6e18eae
21 changed files with 346 additions and 82 deletions
+9 -8
View File
@@ -8,19 +8,20 @@ from an explicit profile setting.
## How to run
```bash
# structural bounds only (default; still executes every test)
# every bound, structural and machine-dependent
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest --console=plain
# full certification, including machine-dependent bounds
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \
-Pperformance.assertions.enabled=true --console=plain
# JMH benchmarks
# JMH benchmarks: numbers only, never a gate
./gradlew :adapter:outbound:httpclient:jmh --console=plain
```
Machine-dependent assertions are reported as explicitly skipped when the flag is absent — the lane
never silently degrades into a pass.
The lane asserts every bound in every run. There is no switch that turns the machine-dependent one
off: the strict-lane convention fails a performance lane before it runs when the build sets
`performance.assertions.enabled` to anything but `true`, and fails it when one of its tests is
skipped. The same command is what `integration-main.yml` (`:performanceCheck`) and `release.yml`
run. Every run prints the measured heap increase (`[httpclient-performance] streaming download heap
increase … bytes, bound … bytes`) to the test output and report, and a bound that fails repeats the
value in its assertion message; for numbers without a verdict, use JMH.
## Certified bounds
+2 -3
View File
@@ -9,11 +9,10 @@ cd src
./gradlew :adapter:outbound:httpclient:test --console=plain
./gradlew :adapter:outbound:httpclient:httpClientStableContractTest --console=plain
./gradlew :adapter:outbound:httpclient:httpClientSecurityTest --console=plain
./gradlew :adapter:outbound:httpclient:spring62CompatibilityTest --console=plain
./gradlew :adapter:outbound:httpclient:spring62ApiSurfaceScan --console=plain
./gradlew :adapter:outbound:httpclient:spring70CompatibilityTest --console=plain
./gradlew :adapter:outbound:httpclient:httpClientFailureInjectionTest --console=plain # needs Docker
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \
-Pperformance.assertions.enabled=true --console=plain
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest --console=plain # every bound asserted
./gradlew verifyCleanArchitectureDependencies --console=plain
./gradlew :app-bootstrap:architectureTest --console=plain
python3 ../scripts/verify-httpclient-docs.py
+11 -11
View File
@@ -57,19 +57,19 @@ gives route-scoped pooling and Dynamic Target pinning, JDK gives HTTP/2.
| Profile | Frequency | Release gate | Task |
|---|---|---|---|
| Spring Framework 7.0 (repository baseline) | every PR | required | `spring70CompatibilityTest` |
| Spring Framework 6.2 API surface | every PR | required | `spring62CompatibilityTest` |
| Apache HC5 + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=apache` |
| JDK HttpClient + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=jdk` |
| SSRF / cardinality suite | every PR | required | `httpClientSecurityTest` |
| Toxiproxy fault suite | nightly, release | required | `httpClientFailureInjectionTest` |
| Performance certification | nightly, release | required | `httpClientPerformanceTest -Pperformance.assertions.enabled=true` |
| Jetty HTTP/3 | nightly | Experimental, non-blocking | `test -Phttp3.tests.enabled=true` |
| Spring Framework 7.0 (repository baseline) | PR touching this leaf, main push, release | required | `spring70CompatibilityTest` |
| Spring Framework 6.2 API surface | every PR (`check`), release | required | `spring62ApiSurfaceScan` |
| Apache HC5 + RestClient | PR touching this leaf (both transports together on every PR through `check`), release | required | `httpClientStableContractTest -Phttpclient.contract.transports=apache` |
| JDK HttpClient + RestClient | PR touching this leaf (both transports together on every PR through `check`), release | required | `httpClientStableContractTest -Phttpclient.contract.transports=jdk` |
| SSRF / cardinality suite | every PR (`check`), release | required | `httpClientSecurityTest` |
| Toxiproxy fault suite | main push, nightly (`integrationCheck`), release | required | `httpClientFailureInjectionTest` |
| Performance certification | main push, nightly (`performanceCheck`), release | required | `httpClientPerformanceTest` (every bound asserted; no switch turns them off) |
| Jetty HTTP/3 | main push, nightly | Experimental, non-blocking | `test -Phttp3.tests.enabled=true` |
### Known limitation of the Spring 6.2 lane
This repository's Spring Boot 4.0 baseline pins Spring Framework 7, so a real 6.2 runtime cannot be
resolved here. `spring62CompatibilityTest` therefore verifies the **API surface**: the common
resolved here. `spring62ApiSurfaceScan` therefore verifies the **API surface**: the common
packages must not reference any Spring 7-only type, and no class uses
`org.springframework.web.service.registry`. Executing the suite against an actual 6.2 distribution
requires a host project on that line. This limitation is stated rather than hidden behind a passing
@@ -83,5 +83,5 @@ Stated so the matrix is read as a measurement rather than an aspiration.
| Gap | Why | What is proven instead |
|---|---|---|
| HTTP/2 frame injection (`REFUSED_STREAM`, arbitrary `GOAWAY`) | The fixture server exposes no frame-level control, and a purpose-built h2 server is a larger dependency than the guarantee is worth here | `Http2EvidenceMapperTest` proves the frame → evidence mapping, and `NegotiatedProtocolContractTest` proves h2 is really negotiated |
| Spring 6.2 runtime | This repository's Boot 4.0 baseline pins Spring 7 | `spring62CompatibilityTest` confines the common packages to the 6.2 API surface |
| Performance latency baseline | Numbers measured on a build agent are not a certification | `httpClientPerformanceTest` asserts structural bounds unconditionally; latency and heap bounds run under `-Pperformance.assertions.enabled=true` |
| Spring 6.2 runtime | This repository's Boot 4.0 baseline pins Spring 7 | `spring62ApiSurfaceScan` confines the common packages to the 6.2 API surface |
| Performance latency baseline | Numbers measured on a build agent are not a certification | `httpClientPerformanceTest` asserts its structural bounds and the streaming heap bound in every run; it asserts no latency figure |