fix(strict-lane)!: 성능 lane이 상한을 재지 않고 통과하지 못하게 한다

문제: 성능 lane의 기계 의존 상한(httpclient 스트리밍 힙 증가, Mongo 시간
gate)이 performance.assertions.enabled 뒤에 있었고 기본값이 자리마다
달랐다(httpclient build false, Mongo build true, Mongo 테스트 코드 false).
workflow만 GRADLE_OPTS로 켰다. 그래서 desktop에서 손으로 돌린
./gradlew :performanceCheck가 httpclient 힙 상한을 재지 않고 통과했다
(2026-11-05 r10 게이트). JPA pool lane도 전에 같은 속성 뒤에서 아무 상한도
비교하지 않았다(PoolLaneClaimTest).

방법: 규칙을 build-logic strict-lane convention 한 곳에 둔다.
- PERFORMANCE lane 테스트 JVM에 performance.assertions.enabled=true를
  넣는다. 빌드(-P, GRADLE_OPTS)나 lane(systemProperty, jvmArgs, task
  설정)이 true 아닌 값을 주면 테스트 전에 실패하고, 지우거나 true로
  바꾸라고 알려 준다. true는 받아 준다(파생 서비스 workflow가 넘긴다).
- PERFORMANCE lane은 skip된 테스트가 있으면 실패한다.
- httpclient·Mongo 테스트는 조건 없이 단언하고 잰 값을 한 줄 남긴다.
  두 leaf의 속성 선언과 integration-main·release workflow의 GRADLE_OPTS
  줄을 지웠다. 사람과 workflow가 같은 명령을 쓴다. verifyStrictLaneStages가
  읽는 run: 줄은 그대로다.
버린 대안: 두 leaf 기본값만 true로(정의가 leaf마다 남아 다음 lane이 다시
복사한다), 속성 삭제만(옛 속성을 기본값 false로 읽는 파생 서비스 테스트가
조용히 건너뛴다), 측정 전용 태스크(쓸 곳이 없다: 숫자만 볼 때는 JMH, 실패한
단언과 매 실행 출력이 값을 남긴다).
같은 원인으로 함께 고친 문서: httpclient support-matrix·release-checklist의
없는 lane 이름 spring62CompatibilityTest와 빠진 main push 주기.

위험·한계: 손으로 돌린 performanceCheck가 이제 상한을 실제로 재므로 느린
기계에서는 실패할 수 있다. JVM 인자 공급자(jvmArgumentProviders)가 넣는
-D는 보지 않는다. Docker가 없을 때 skip되는 통합·시스템 lane은 지금
모두 실패한다(전부 skip이면 executed no test, 아니면 skip 안 되는 컨테이너
테스트가 실패) — 거짓 통과가 아니어서 고치지 않았다.

검증(desktop ~/b4-logs, flock 아래, 노트북과 같은 트리 — git diff sha256 일치):
- RED(고치기 전 트리 + 새 테스트): build-logic 새 테스트 4건이
  UnexpectedBuildSuccess로 실패 — aPerformanceLaneFailsOnAnExceededBound
  WhenTheBuildSetsNoAssertionsProperty, …FailsBeforeTestingWhenTheBuild
  TurnsItsAssertionsOff, …FailsWhenItsOwnDeclarationTurnsTheAssertionsOff,
  …FailsWhenOneOfItsTestsIsSkipped. jvmArgs 경로(…FailsWhenItsJvmArguments
  TurnTheAssertionsOff)도 같은 실패. 실제 lane: 속성 없이
  httpClientPerformanceTest rc=0인데 "skipped machine-dependent bound" 출력.
- GREEN: -p build-logic check javadoc rc=0. :performanceCheck rc=0(lane 5개
  테스트 26, skip 0; httpclient 힙 증가 5,243,880 B / 상한 16,777,216 B,
  Mongo p50·p95·p99 1ms, examined/returned 1.0, pool 대기 0).
  -p optional-platforms performanceCheck rc=0(grpc 1). 두 명령에
  -Pperformance.assertions.enabled=false → rc=1, 6개 lane 모두 테스트 전
  실패. =true → rc=0. verifyStrictLaneStages --dry-run 두 빌드 rc=0,
  architectureCheck rc=0, ci·mainCheck·integrationCheck·releaseCheck·
  performanceCheck --dry-run --configuration-cache rc=0,
  :mainCheck --continue --warning-mode=fail rc=0, -p optional-platforms ci
  rc=0, persistence-mongo:check rc=0, verify-httpclient-docs.py rc=0.
- 이 변경 밖 실패: :ci :verifyDependencyLocks --continue와
  httpclient:check가 CircuitBreakerStatusAccountingTest JDK 2건으로 rc=1.
  기준 커밋 3d73b6f1에서도 10/10 실패(이 트리도 10/10). ci의 다른
  task는 모두 통과했다.

Refs: docs/upgrade/2026-11-maturity-review.md B-43, docs/testing/TESTING_STRATEGY.md 6절
BREAKING CHANGE: PERFORMANCE lane은 performance.assertions.enabled가
true가 아니거나 skip된 테스트가 있으면 실패한다. 자기 성능 lane의 그 속성
선언(false)과 skip을 없앤다.
This commit is contained in:
DongHyeonka
2026-11-06 20:33:47 +09:00
parent 3d73b6f1ba
commit e6b6e18eae
21 changed files with 346 additions and 82 deletions
+1 -1
View File
@@ -27,7 +27,7 @@ Checkstyle은 **포매터가 표현할 수 없는 것만** — 네이밍과 몇
| `configContractCheck` | `:app-bootstrap:verifyEnvKeys`로 환경 설정 계약을 검사 |
| `qualificationCheck` | 전체 기동(system) 레인과 보조 source set Checkstyle을 실행 |
| `integrationCheck` | 선언된 strict integration lane(sample 포함)을 실행. 대부분 container runtime이 필요. 입력이 필요한 lane(Redis topology: `-Predis.topology.mode`, AWS: 보호 자격 증명 env)은 입력이 없으면 빠지고 집계가 그 이유를 출력 |
| `performanceCheck` | 선언된 strict performance lane(부하·종료·pool·자원 상한)을 실행. 실서버·container가 필요. `integration-main.yml` `performance-check`가 main·nightly에 돌림 |
| `performanceCheck` | 선언된 strict performance lane(부하·종료·pool·자원 상한)을 실행. 실서버·container가 필요. 상한은 매번 단언하고 끄는 속성이 없다(`-Pperformance.assertions.enabled`가 `true`가 아니면 실패, skip된 테스트가 있어도 실패). `integration-main.yml` `performance-check`가 main·nightly에 같은 명령으로 돌림 |
| `verifyStrictLaneStages` | `--dry-run` 전용. `explicitOnly`가 아닌 strict lane이 workflow가 실제 실행하는 단계(`ci`·`:sample-portfolio:check`·`mainCheck`·`integrationCheck`·`performanceCheck`·`releaseCheck`) 중 하나의 계획에 있는지, 각 단계를 workflow가 `--dry-run` 없이 실행하는지 검사. Gradle이 작업을 돌리지 않는 명령(`--dry-run`·`-v`·`--help` 등)과, 실패해도 단계가 실패하지 않을 수 있거나(`\|\| true`, pipefail 없는 파이프, `if` 조건 등) 돌지 않을 수 있거나(조건부 `exit 0` 뒤, 요청으로만 시작하는 workflow 등) 검사가 읽지 않는(`\|\| exit 1`, `{ }` 안 등) 명령은 실행으로 세지 않는다. 자세한 목록은 `build-logic/README.md`. PR 게이트가 돌림 |
| `ci` | PR 게이트: 모든 leaf `check`(API surface 검사 포함, 소유자 결정 20) + `architectureCheck` + `qualityCheck` |
| `mainCheck` | main 단계: `configContractCheck` + `qualificationCheck` + sample web 계약 + sample 전체 기동(`:sample-portfolio:systemTest`) + `verifyPublicPathSnapshot` + 전송·메시징 qualification 레인 + 각 leaf의 `mainCheck`(문서 대조 레인 `*DocumentContractTest`, objectstorage MinIO 레인) |
+3 -2
View File
@@ -81,8 +81,9 @@ Default lane: `./gradlew :adapter:outbound:httpclient:test`. Additional lanes, a
| `jmh` | per-call overhead benchmarks |
A selected lane never skips silently: the fault lane throws without Docker, the contract lane throws
on an empty or unknown transport selection, and the performance lane prints which machine-dependent
bounds were not asserted.
on an empty or unknown transport selection, and the performance lane asserts every bound in every
run — the strict-lane convention fails it when the build sets `performance.assertions.enabled` to
anything but `true` or when one of its tests is skipped.
The transport port (`BlockingTransportProvider`) has one abstract contract in the test fixtures,
`testkit/BlockingTransportProviderContract`. The engines pass it over a real socket
+3 -2
View File
@@ -111,8 +111,9 @@ cd src
```
No lane skips silently. The fault lane fails without Docker, the contract lane fails on an empty or
unknown transport selection, and the performance lane reports which machine-dependent bounds it did
not assert.
unknown transport selection, and the performance lane asserts every bound in every run (it fails
when the build sets `performance.assertions.enabled` to anything but `true`, or when a test is
skipped).
## Further reading
+3 -4
View File
@@ -138,8 +138,9 @@ dependencies {
}
// Which transports the contract suites cover and whether the HTTP/3 opt-in runs: the same two
// properties for every Test task of this leaf, declared once (review 2026-11-01 O-10a). Values a
// single lane needs (the performance assertions, the httpbin image) stay in that lane.
// properties for every Test task of this leaf, declared once (review 2026-11-01 O-10a). A value a
// single lane needs (the httpbin image) stays in that lane. The performance lane's bounds have no
// switch here: the strict-lane convention asserts them in every run.
tasks.withType(Test).configureEach {
systemProperty 'httpclient.contract.transports',
providers.gradleProperty('httpclient.contract.transports').orElse('apache,jdk').get()
@@ -193,8 +194,6 @@ strictTestLanes {
performance()
sourceSet = 'httpClientPerformanceTest'
description = 'Certifies pool, streaming, retry, and rotation resource bounds (design §28.8).'
systemProperty 'performance.assertions.enabled',
providers.gradleProperty('performance.assertions.enabled').orElse('false').get()
}
lane('httpClientFailureInjectionTest') {
integration()
@@ -52,11 +52,18 @@ class LargeBodyResourceTest {
}
}
long peakIncrease = usedHeap() - before;
// The measured value, so a passing run is a recorded number and not only a green line.
System.out.println(
"[httpclient-performance] streaming download heap increase "
+ peakIncrease
+ " bytes, bound "
+ payloadBytes / 2
+ " bytes");
assertThat(consumed).isEqualTo(payloadBytes);
PerformanceAssertions.machineDependent(
"streaming download heap increase stays well below the payload size",
() -> assertThat(peakIncrease).isLessThan((long) payloadBytes / 2));
assertThat(peakIncrease)
.as("streaming download heap increase stays well below the payload size")
.isLessThan((long) payloadBytes / 2);
}
}
}
@@ -3,32 +3,19 @@ package dev.caskeleton.adapter.outbound.httpclient.performance;
import static org.assertj.core.api.Assertions.assertThat;
/**
* Separates deterministic bounds from machine-dependent ones (design §28.8, §37).
* Structural bounds — retry budget arithmetic, a refresh collapsing to one request, drain
* completing — that hold on any machine (design §28.8, §37).
*
* <p>Structural bounds — retry budget arithmetic, streaming not buffering the whole payload, drain
* completing — always run. Latency and absolute memory numbers depend on the host, so they are
* asserted only when the certification flag is set; when it is off the lane still executes and
* still reports, rather than becoming a silent pass.
* <p>The machine-dependent bound (the streaming heap increase) is asserted in place, in every run.
* It used to sit behind {@code performance.assertions.enabled}, which defaulted to {@code false} in
* this leaf, so a lane run by hand reported success with the bound compared to nothing. A
* performance lane now asserts all of its bounds; the strict-lane convention rejects a build that
* tries to turn them off.
*/
final class PerformanceAssertions {
private PerformanceAssertions() {}
static boolean certificationEnabled() {
return Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false"));
}
static void machineDependent(String description, Runnable assertion) {
if (certificationEnabled()) {
assertion.run();
return;
}
System.out.println(
"[httpclient-performance] skipped machine-dependent bound (enable with "
+ "-Pperformance.assertions.enabled=true): "
+ description);
}
static void structural(String description, boolean condition) {
assertThat(condition).describedAs(description).isTrue();
}
@@ -239,9 +239,8 @@ strictTestLanes {
providers.gradleProperty('mongoCompatibilityImage').orElse(testContainerImages['mongo-7']).get()
systemProperty 'mongodb.toxiproxy.image',
providers.gradleProperty('mongoToxiproxyImage').orElse(testContainerImages['toxiproxy']).get()
// Assertions on by default: a performance gate without bounds is only a report.
systemProperty 'performance.assertions.enabled',
providers.gradleProperty('performance.assertions.enabled').orElse('true').get()
// No assertions switch: the strict-lane convention asserts a performance lane's bounds in
// every run, because a performance gate without bounds is only a report.
}
}
@@ -27,16 +27,16 @@ import org.junit.jupiter.api.Test;
* it is the same on an idle machine and a busy one, and it is what actually predicts the query
* falling over as the collection grows.
*
* <p>Timing bounds are asserted only when {@code -Pperformance.assertions.enabled=true}, for the
* same reason. The ratio bound is always asserted.
* <p>The timing bounds are asserted too, in every run, but the gate's bounds are generous for the
* same reason: they catch a query that falls over, not one that was slower than last time. They
* used to sit behind {@code performance.assertions.enabled}, which the build defaulted to on and
* this class to off, so a run outside Gradle skipped them; the strict-lane convention now asserts a
* performance lane's bounds in every run.
*/
class MongoResourceBudgetLaneTest {
private static final int DOCUMENT_COUNT = 5_000;
private static final boolean TIMING_ASSERTIONS =
Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false"));
private static MongoSingleReplicaSetContainer container;
private static MongoClient client;
@@ -75,18 +75,16 @@ class MongoResourceBudgetLaneTest {
@Test
void anIndexedQueryStaysInsideTheResourceBudget() {
MongoResourceBudgetReport report = measure(new Document("status", "PENDING"), "ix_status");
// The measured values, so a passing run is a recorded number and not only a green line.
System.out.println("[mongo-performance] indexed query " + report.asArtifact());
assertThat(report.documentsReturned()).isEqualTo(DOCUMENT_COUNT / 50);
assertThat(report.examinedToReturnedRatio())
.as("an index seek examines about what it returns")
.isLessThanOrEqualTo(MongoPerformanceGate.standard().maximumExaminedToReturnedRatio());
assertThat(MongoPerformanceGate.standard().violations(report))
.filteredOn(violation -> violation.startsWith("examined/returned"))
.as("an indexed query stays inside every bound of the standard gate")
.isEmpty();
if (TIMING_ASSERTIONS) {
assertThat(MongoPerformanceGate.standard().passes(report)).isTrue();
}
}
@Test
File diff suppressed because one or more lines are too long
@@ -10,12 +10,26 @@ import org.gradle.api.NamedDomainObjectContainer;
import org.gradle.api.Plugin;
import org.gradle.api.Project;
import org.gradle.api.plugins.JavaPluginExtension;
import org.gradle.api.provider.Provider;
import org.gradle.api.tasks.SourceSet;
import org.gradle.api.tasks.TaskProvider;
import org.gradle.api.tasks.testing.Test;
import org.gradle.testing.base.TestingExtension;
public final class StrictTestLanePlugin implements Plugin<Project> {
/**
* The switch a performance lane's machine-dependent bounds used to sit behind, now always on.
*
* <p>Each leaf declared it with its own default — {@code false} for the httpclient lane, {@code
* true} for the Mongo lane — and only the workflows set it. {@code ./gradlew :performanceCheck}
* typed by hand therefore passed with the httpclient heap bound compared to nothing (the
* 2026-11-05 desktop gate). The convention owns it now: every performance lane's test JVM gets
* {@code true}, so a test that still reads the switch asserts, and a build that sets it to
* anything else fails before the lane runs. Nothing in this repository reads it any more; the
* value is for tests written against the old switch, such as a derived service's.
*/
static final String PERFORMANCE_ASSERTIONS = "performance.assertions.enabled";
@Override
public void apply(Project project) {
project.getPluginManager().withPlugin("java", ignored -> configure(project));
@@ -229,7 +243,8 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
// judge the lane again rather than reuse a result produced under the looser policy. Without
// this a lane that skipped a test stayed green, up to date and from the build cache, after
// rejectSkipped was switched on (review 2026-11-03 RR-06-02).
test.getInputs().property("strictLane.rejectSkipped", project.provider(lane::isRejectSkipped));
test.getInputs()
.property("strictLane.rejectSkipped", project.provider(() -> rejectsSkipped(lane)));
test.getInputs()
.property("strictLane.requiredTests", project.provider(lane::getRequiredTests));
test.getInputs()
@@ -243,6 +258,17 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
test.getOutputs().upToDateWhen(ignored -> false);
test.getOutputs().doNotCacheIf(rerunReason, ignored -> true);
}
boolean performance = lane.getCategory() == TestLaneCategory.PERFORMANCE;
if (performance) {
// Set before the lane's own JVM arguments and system properties — Gradle turns a -D in
// jvmArgs into a system property — so a lane that still declares the switch replaces this
// value and the check below sees the override, instead of the override disappearing silently.
test.systemProperty(PERFORMANCE_ASSERTIONS, "true");
Provider<String> requested = project.getProviders().gradleProperty(PERFORMANCE_ASSERTIONS);
String laneName = lane.getName();
test.doFirst(
running -> requireAssertionsOn(laneName, ownerPath, requested.getOrNull(), (Test) running));
}
if (lane.getMaxHeapSize() != null && !lane.getMaxHeapSize().isBlank()) {
test.setMaxHeapSize(lane.getMaxHeapSize());
}
@@ -288,7 +314,7 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
+ detail
+ " — a lane that runs nothing reports success for whatever it was meant to prove.");
}
if (lane.isRejectSkipped() && tracker.skippedCount() > 0L) {
if (rejectsSkipped(lane) && tracker.skippedCount() > 0L) {
throw new GradleException(
"strict test lane '"
+ lane.getName()
@@ -296,7 +322,10 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
+ ownerPath
+ " forbids skipped tests but observed "
+ tracker.skippedCount()
+ " skipped test(s).");
+ " skipped test(s)."
+ (performance
? " A performance lane always does: a skipped test is a bound nobody measured."
: ""));
}
if (!required.isEmpty()) {
List<String> absent =
@@ -318,6 +347,55 @@ public final class StrictTestLanePlugin implements Plugin<Project> {
}
}
/**
* A performance lane rejects skipped tests whatever it declares: its tests are its bounds, and a
* skip is a bound the run reports as passed without comparing it to anything.
*/
private static boolean rejectsSkipped(StrictTestLaneSpec lane) {
return lane.isRejectSkipped() || lane.getCategory() == TestLaneCategory.PERFORMANCE;
}
/**
* Fails a performance lane, before any test runs, when the build or the lane turns its bounds off.
* {@code true} from the build is accepted, so a command that still passes it keeps working.
*/
private static void requireAssertionsOn(
String laneName, String ownerPath, String requestedByBuild, Test test) {
if (requestedByBuild != null && !Boolean.parseBoolean(requestedByBuild.trim())) {
throw new GradleException(
"strict test lane '"
+ laneName
+ "' in "
+ ownerPath
+ " is a performance lane, and this build sets "
+ PERFORMANCE_ASSERTIONS
+ "="
+ requestedByBuild
+ " (-P"
+ PERFORMANCE_ASSERTIONS
+ ", or -Dorg.gradle.project."
+ PERFORMANCE_ASSERTIONS
+ " in GRADLE_OPTS). A performance lane asserts its bounds in every run: one without"
+ " them measures nothing it can fail on and still reports success. Remove the"
+ " property or set it to true.");
}
Object effective = test.getSystemProperties().get(PERFORMANCE_ASSERTIONS);
if (effective == null || !Boolean.parseBoolean(String.valueOf(effective).trim())) {
throw new GradleException(
"strict test lane '"
+ laneName
+ "' in "
+ ownerPath
+ " is a performance lane, but its test JVM would get "
+ PERFORMANCE_ASSERTIONS
+ "="
+ effective
+ " from this project's build (a lane systemProperty or jvmArgs, or a block that"
+ " configures the task). The strict-lane convention sets it to true for every"
+ " performance lane; delete the override.");
}
}
private static boolean startsARealServer(StrictTestLaneSpec lane) {
return lane.getCategory() != TestLaneCategory.TEST
&& lane.getCategory() != TestLaneCategory.ARCHITECTURE;
@@ -137,6 +137,12 @@ public class StrictTestLaneSpec implements Named {
category = TestLaneCategory.ARCHITECTURE;
}
/**
* A lane whose tests are resource or latency bounds. Besides rerunning every time and taking the
* build-wide real-server slot, it rejects skipped tests and always runs with its bounds asserted:
* the convention sets {@code performance.assertions.enabled=true}, and a build or lane that sets
* it to anything else fails before a test runs.
*/
public void performance() {
category = TestLaneCategory.PERFORMANCE;
}
@@ -611,6 +611,147 @@ class StrictTestLaneConventionTest {
assertThat(result.getOutput()).contains("rulesLane=false").contains("databaseLane=true");
}
@Test
void aPerformanceLaneFailsOnAnExceededBoundWhenTheBuildSetsNoAssertionsProperty()
throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"BoundTest.java",
"""
import org.junit.jupiter.api.Test;
class BoundTest {
@Test void staysInsideTheBound() {
long p99Millis = 900;
// The shape the httpclient lane had: the machine-dependent bound behind the switch.
if (Boolean.parseBoolean(System.getProperty("performance.assertions.enabled", "false"))
&& p99Millis > 500) {
throw new AssertionError("p99 900ms above the 500ms bound");
}
}
}
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput()).contains("BoundTest > staysInsideTheBound() FAILED");
}
@Test
void aPerformanceLaneFailsBeforeTestingWhenTheBuildTurnsItsAssertionsOff() throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest {
@Test void measured() { throw new AssertionError("the lane ran a test"); }
}
""");
// when
BuildResult result =
runner("performanceTestLaneCheck", "-Pperformance.assertions.enabled=false")
.buildAndFail();
// then
assertThat(result.getOutput())
.contains("this build sets performance.assertions.enabled=false")
.contains("set it to true")
.doesNotContain("MeasuredTest > measured() FAILED");
}
@Test
void aPerformanceLaneFailsWhenItsOwnDeclarationTurnsTheAssertionsOff() throws Exception {
// given
buildFile(
performanceLane(
"""
systemProperty 'performance.assertions.enabled',
providers.gradleProperty('performance.assertions.enabled').orElse('false').get()
"""));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest { @Test void measured() {} }
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput())
.contains("its test JVM would get performance.assertions.enabled=false")
.contains("delete the override");
}
@Test
void aPerformanceLaneFailsWhenItsJvmArgumentsTurnTheAssertionsOff() throws Exception {
// given
buildFile(performanceLane("jvmArgs '-Dperformance.assertions.enabled=false'"));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest { @Test void measured() {} }
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput())
.contains("its test JVM would get performance.assertions.enabled=false")
.contains("delete the override");
}
@Test
void aPerformanceLaneFailsWhenOneOfItsTestsIsSkipped() throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"PartlyMeasuredTest.java",
"""
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
class PartlyMeasuredTest {
@Test void measured() {}
@Test @Disabled("too slow on this machine") void quarantined() {}
}
""");
// when
BuildResult result = runner("performanceTestLaneCheck").buildAndFail();
// then
assertThat(result.getOutput())
.contains("forbids skipped tests")
.contains("a skipped test is a bound nobody measured");
}
@Test
void aPerformanceLaneAcceptsABuildThatTurnsItsAssertionsOnExplicitly() throws Exception {
// given
buildFile(performanceLane(""));
writePerformanceTest(
"MeasuredTest.java",
"""
import org.junit.jupiter.api.Test;
class MeasuredTest { @Test void measured() {} }
""");
// when
BuildResult result =
runner("performanceTestLaneCheck", "-Pperformance.assertions.enabled=true").build();
// then
assertThat(result.task(":loadLane").getOutcome()).isEqualTo(TaskOutcome.SUCCESS);
}
private void buildFile(String laneBlock) throws Exception {
Files.writeString(
projectDir.resolve("build.gradle"),
@@ -631,6 +772,32 @@ class StrictTestLaneConventionTest {
Files.writeString(testSource.resolve(fileName), source);
}
private void writePerformanceTest(String fileName, String source) throws Exception {
Path performanceSource = projectDir.resolve("src/performance/java");
Files.createDirectories(performanceSource);
Files.writeString(performanceSource.resolve(fileName), source);
}
private static String performanceLane(String extraDeclarations) {
return """
sourceSets { performance }
dependencies {
performanceImplementation platform('org.junit:junit-bom:5.11.3')
performanceImplementation 'org.junit.jupiter:junit-jupiter'
performanceRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
strictTestLanes {
lane('loadLane') {
performance()
sourceSet = 'performance'
description = 'Measures a resource bound.'
%s
}
}
"""
.formatted(extraDeclarations);
}
private static String junitDependencies() {
return """
dependencies {