refactor(build,ci): 현재 상태 검증을 걷어내고 불변조건만 남기는 검증 표면 축소

외부 리뷰("현재 상태를 유지하기 위한 검증이 너무 많고, 그 검증 자체를
다시 검증하는 구조까지 생겼다")를 설계 문서로 정리하고 코드로 반영한다.
설계·판단 근거는 docs/superpowers/specs/2026-09-16-verification-surface-reduction-design.md.

삭제
- .github/ci-gate-matrix.yml(1,025줄) + verify-gate-matrix.sh(568줄):
  Gradle task graph와 workflow graph에 이미 있는 정보의 3중 복제
- verify-gradle-wrapper.sh(799줄): workflow 바이트 해시 잠금.
  wrapper 검증은 gradle/actions/wrapper-validation(full SHA 핀)에 위임
- DeveloperExperienceContractTest 등의 CI YAML mutation 테스트:
  애플리케이션 test suite가 GitHub Actions YAML 파서를 검증하던 계층 역전
- 문서 drift 파서: verifyReadmeCommands, verifyRunbookReferences,
  verifyDocumentedLeafCount, verifyTestSourceSetRegistry
- 빈 레지스트리를 지키던 커스텀 YAML 파서: verifyTrivyignore,
  verifyQuarantineSunset, flaky-quarantine.yaml
- verifyConfigurationPropertiesProcessor, verifyOneTypePerFile:
  각각 ca.spring-config convention과 Checkstyle OneTopLevelClass가 대체
- 정상 입력으로도 성공할 수 없던 messaging always-fail task
- ModuleRegistry의 JSON 필드 집합 정확 일치, sample-portfolio negative guard

이동
- java/quality/spring 공통 설정을 configure(subprojects) 블록에서
  ca.java-conventions / ca.quality-conventions / ca.java-library /
  ca.spring-library convention plugin으로
- 아키텍처 검증을 ca.architecture로, JPA·messaging qualification을
  gradle/qualification/ 아래로, verifyEnvKeys를 :app-bootstrap 소유로

완화
- Git revision은 releaseCheck·아카이브 생성에서만 요구. 일반 빌드는 SNAPSHOT
- SpotBugs/FindSecBugs는 로컬 check에서 빼고 qualityCheck 레인으로

task 계층
- leaf check는 그 leaf만. architectureCheck / qualityCheck /
  configContractCheck / integrationCheck / ci / releaseCheck로 이름 분리

CI
- _reusable-gradle.yml 신규. checkout + wrapper validation + JDK/캐시 공통화
- fileserver-release.yml -> fileserver-certification.yml (CD가 아니라 certification)
- GitHub Actions = CI + artifact, Argo CD = CD 경계를 docs/ci-cd/boundary.md로 고정

순증감 +3,274 / -7,483.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
2026-09-16 20:33:19 +09:00
co-authored by Claude Opus 5
parent d00c76241c
commit ef947e5bb0
95 changed files with 3284 additions and 7493 deletions
+8 -1
View File
@@ -1,5 +1,8 @@
// Application entry point. Wires the default runtime module set and runs Spring Boot.
// Optional leaves require an explicit registry allowance plus a composition-root dependency.
apply plugin: 'ca.spring-library'
apply plugin: 'ca.spring-config'
apply plugin: 'org.springframework.boot'
@@ -116,7 +119,6 @@ dependencies {
implementation project(':shared-contract')
implementation 'org.springframework.boot:spring-boot-starter'
implementation 'org.springframework.boot:spring-boot-starter-validation'
annotationProcessor 'org.springframework.boot:spring-boot-configuration-processor'
implementation libs.spring.dotenv
// Boot 4 Flyway API/autoconfiguration: the composition root drives startup migration
// (MigrationStartupConfig). See README.
@@ -415,3 +417,8 @@ tasks.register('runtimeClasspathManifest') {
tasks.named('test') {
dependsOn tasks.named('runtimeClasspathManifest')
}
// The environment configuration contract. Owned here because the question it answers — what must a
// deployment of this application be given — is the composition root's, and reached through
// `configContractCheck` rather than through every leaf's `check`.
apply from: rootProject.file('gradle/config-contract.gradle')
@@ -18,13 +18,24 @@ final class BuildVerificationPurityContractTest {
private static final Path PUBLIC_PATH_SCRIPT =
SOURCE_ROOT.resolve("gradle/public-path-snapshot.gradle");
/**
* What the renderer produces for the fixture's {@code security.yml}.
*
* <p>These five tests were red on main. The snapshot's input moved from {@code src/.env} to the
* committed {@code app-bootstrap/src/main/resources/config/security.yml} — because {@code
* src/.env*} is gitignored, so the old gate took its expected value from a file no CI checkout
* has — but the fixture below kept writing a {@code .env}, and this constant kept the header and
* the paths that file used to produce. The script was right and its test was describing the
* previous contract.
*/
private static final String CANONICAL_PUBLIC_PATH_SNAPSHOT =
"""
# feature-security-operational-baseline D5 — deny-by-default public path snapshot.
# SSOT: SECURITY_PUBLIC_PATHS (src/.env) -> SecurityConfig permitAll(); anyRequest authenticated.
# SSOT: ca-skeleton.security.public-paths default in app-bootstrap/src/main/resources/config/security.yml
# -> SecurityConfig permitAll(); anyRequest authenticated. An operator's own SECURITY_PUBLIC_PATHS
# overrides it at run time and is outside this snapshot.
# Update only after review with: ./gradlew updatePublicPathSnapshot -PapprovePublicPathChange
/api/healthcheck
/api/public
/v1/healthcheck
""";
@Test
@@ -71,14 +82,14 @@ final class BuildVerificationPurityContractTest {
}
@Test
void missingPublicPathEnvironmentFailsWithoutCreatingSnapshot(@TempDir Path temporaryDirectory)
void missingPublicPathConfigurationFailsWithoutCreatingSnapshot(@TempDir Path temporaryDirectory)
throws IOException {
PublicPathFixture fixture = publicPathFixture(temporaryDirectory);
Files.delete(fixture.environment());
Files.delete(fixture.securityConfiguration());
BuildResult result = runAndFail(fixture.projectDirectory(), "verifyPublicPathSnapshot");
assertThat(result.getOutput()).contains("missing public-path environment file");
assertThat(result.getOutput()).contains("missing public-path security configuration");
assertThat(fixture.snapshot()).doesNotExist();
}
@@ -210,13 +221,23 @@ final class BuildVerificationPurityContractTest {
projectDirectory.resolve("build.gradle"),
"apply from: uri('%s')\n".formatted(PUBLIC_PATH_SCRIPT.toUri().toASCIIString()),
UTF_8);
// The committed binding default, spelled exactly as the real file spells it — nested Spring
// placeholders and all, because unwinding them to `/v1/healthcheck` is what the renderer does
// and therefore what these tests are about.
Path securityConfig =
projectDirectory.resolve("app-bootstrap/src/main/resources/config/security.yml");
Files.createDirectories(securityConfig.getParent());
Files.writeString(
projectDirectory.resolve(".env"),
"SECURITY_PUBLIC_PATHS=/api/public, /api/healthcheck\n",
securityConfig,
"""
ca-skeleton:
security:
public-paths: ${SECURITY_PUBLIC_PATHS:${PRESENTATION_API_BASE_PATH:/v1}/healthcheck}
""",
UTF_8);
return new PublicPathFixture(
projectDirectory,
projectDirectory.resolve(".env"),
securityConfig,
repositoryDirectory.resolve("docs/security/public-paths-snapshot.txt"));
}
@@ -254,5 +275,6 @@ final class BuildVerificationPurityContractTest {
private record ArchiveFixture(
Path projectDirectory, Path staleArchive, Path currentArchive, Path nonmatchingArchive) {}
private record PublicPathFixture(Path projectDirectory, Path environment, Path snapshot) {}
private record PublicPathFixture(
Path projectDirectory, Path securityConfiguration, Path snapshot) {}
}
@@ -110,8 +110,8 @@ class FileserverPlatformEnvRoundTripTest {
* <p>A copy in the test resources would drift from the shipped contract, which is the drift this
* test exists to catch — so it reads the repository file rather than a fixture.
*
* <p>It reads {@code .env.example}, not {@code .env}. {@code .gitignore} states the rule: the real
* {@code .env} is operator input and the examples beside it are the tracked contract. A real
* <p>It reads {@code .env.example}, not {@code .env}. {@code .gitignore} states the rule: the
* real {@code .env} is operator input and the examples beside it are the tracked contract. A real
* {@code .env} exists only on a developer machine, so pointing this test at it made the test pass
* locally and fail on every clean checkout — which is where CI runs.
*/
@@ -2,63 +2,32 @@ package dev.caskeleton.bootstrap.contract;
import static org.assertj.core.api.Assertions.assertThat;
import dev.caskeleton.bootstrap.contract.support.RepositoryContractResources;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.regex.Pattern;
import java.nio.file.Paths;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.yaml.snakeyaml.LoaderOptions;
import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.constructor.SafeConstructor;
/**
* The opt-in inbound transports qualify without skips, and the root only aggregates.
*
* <p>This class used to be 860 lines, of which about 800 tested {@code
* .github/scripts/verify-gate-matrix.sh}: eighteen tests that wrote mutated copies of {@code
* .github/ci-gate-matrix.yml} into a {@code @TempDir} — a duplicate gate id, a row naming a job
* that does not exist, a {@code ref} containing a regex metacharacter, a release-blocking row no
* release gate required — and ran the shell script against them to check that it refused each one.
*
* <p>That is an application's test suite testing a YAML register of CI controls, through a bash
* validator, in a fixture repository. The register itself duplicated what the Gradle task graph and
* the GitHub Actions job graph already said, so the whole structure existed to keep three
* descriptions of one fact equal to each other. All three layers are gone.
*
* <p>What survives is the part that was about this repository's transports rather than about its CI
* register: each inbound transport leaf owns its qualification lane and names the wire classes that
* lane must execute, and the root task only aggregates them.
*/
class ConditionalTransportQualificationContractTest {
/**
* Filler gates a well-formed fixture carries beside its one target gate.
*
* <p>Two, and the number does not matter. It used to be one less than a gate count this test and
* the validator both hard-coded, so every fixture had to be built to that size or it failed on
* the count rather than on whatever the test was about. Neither pins a count now; the fillers
* remain only so the duplicate-id and shape cases have a second row to mutate.
*/
private static final int FILLER_GATE_COUNT = 2;
/**
* The release gate {@code release_blocking: true} is measured against.
*
* <p>Named here and in {@code verify-gate-matrix.sh} rather than inferred from a filename: a
* workflow called "release" is a naming convention, and this job's {@code needs:} is a fact.
*/
private static final String RELEASE_GATE_WORKFLOW = "ci-quality-gates.yml";
private static final String RELEASE_GATE_JOB = "release-gate";
private static final Pattern WHITESPACE = Pattern.compile("\\s+");
private static final Duration VALIDATOR_TIMEOUT = Duration.ofSeconds(10);
private static final Set<String> EXPECTED_GATE_FIELDS =
Set.of("id", "release_blocking", "mechanism", "ref", "workflow", "job", "execution");
private static final Set<String> ALLOWED_MECHANISMS =
Set.of(
"gradle-custom-task",
"gradle-plugin-task",
"contract-test",
"workflow-job",
"delegated-pending");
private static final Set<String> ALLOWED_EXECUTIONS = Set.of("check", "explicit", "job");
@Test
void ownerQualificationsNameEveryRequiredWireClassAndRootOnlyAggregates() throws IOException {
Path root = repositoryRoot();
@@ -86,6 +55,10 @@ class ConditionalTransportQualificationContractTest {
.contains("registerStrictQualificationTest")
.contains(
"dev.caskeleton.adapter.inbound.websocket.stomp.WebSocketBoundaryQualificationTest");
// The root aggregates and knows no test class name. A root that named the wire classes would be
// a second place to update when a leaf renames one, and the leaf's own lane is the one that
// fails closed on a class it cannot discover.
assertThat(rootBuild)
.contains("tasks.register('conditionalTransportQualification')")
.contains(":adapter:inbound:graphql:graphqlTransportQualificationTest")
@@ -98,763 +71,27 @@ class ConditionalTransportQualificationContractTest {
}
@Test
void releaseBlockingQualityJobAndGateMatrixInvokeTheAggregate() throws IOException {
Path root = repositoryRoot();
String workflow = Files.readString(root.resolve(".github/workflows/ci-quality-gates.yml"));
String matrix = Files.readString(root.resolve(".github/ci-gate-matrix.yml"));
String validator = Files.readString(root.resolve(".github/scripts/verify-gate-matrix.sh"));
void theReleaseBlockingQualityJobInvokesTheAggregate() throws IOException {
String workflow =
Files.readString(repositoryRoot().resolve(".github/workflows/ci-quality-gates.yml"));
assertThat(workflow)
.contains("./gradlew conditionalTransportQualification")
// `--continue` would let a failing transport lane be reported alongside a green overall
// step, which is the one thing a no-skip qualification may not do.
.doesNotContain("conditionalTransportQualification --continue");
assertThat(matrix)
.contains("id: conditional-transport-qualification")
.contains("ref: conditionalTransportQualification")
.contains("job: quality-gates")
.contains("execution: explicit");
// The count literal is gone and must stay gone. While it existed, adding a control meant
// editing the guard whose stated purpose was to stop the matrix changing, and it caught
// nothing the per-row rules do not: a row whose task, workflow or job has disappeared fails
// below at any matrix size.
assertThat(validator)
.doesNotContain("EXPECTED_GATE_COUNT")
.contains("matrix declares no gates");
}
@Test
void explicitFixtureRootRunsTheActualValidatorOutsideItsScriptLocation(@TempDir Path tempDir)
throws IOException {
Path fixtureRoot =
writeFixture(
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
ScriptResult result = runValidator(fixtureRoot);
int fixtureGateCount = FILLER_GATE_COUNT + 1;
assertThat(result.exitCode()).isZero();
assertThat(result.output())
.contains(
"gate-matrix-lint: "
+ fixtureGateCount
+ " gates, "
+ fixtureGateCount
+ " verified, 0 delegated-pending")
.contains("gate-matrix-lint: OK");
}
@Test
void defaultModeValidatesTheRealRepositoryAndRetainsItsLocationGuard() throws IOException {
Path root = repositoryRoot();
String validator = Files.readString(validatorPath());
ScriptResult result = runScript(root, List.of());
assertThat(result.exitCode()).isZero();
assertThat(result.output()).contains("gate-matrix-lint: OK");
assertThat(validator)
.contains("EXPECTED_SCRIPT_DIR")
.contains("script location must be repository .github/scripts directory");
}
@Test
void defaultModeRejectsARelocatedScript(@TempDir Path tempDir) throws IOException {
Path fixtureRoot =
writeFixture(
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
Files.createDirectories(fixtureRoot.resolve(".github/scripts"));
Path relocatedScript = fixtureRoot.resolve("relocated-verify-gate-matrix.sh");
Files.copy(validatorPath(), relocatedScript);
ScriptResult gitInit =
runCommand(fixtureRoot, List.of("git", "init", "--quiet", fixtureRoot.toString()));
assertThat(gitInit.exitCode()).isZero();
ScriptResult result = runScriptAt(relocatedScript, fixtureRoot, List.of());
assertThat(result.exitCode()).isNotZero();
assertThat(result.output())
.contains("script location must be repository .github/scripts directory");
}
@Test
void validatorRejectsMoreThanOneRepositoryRootArgument(@TempDir Path tempDir) throws IOException {
Path fixtureRoot =
writeFixture(
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
ScriptResult result = runScript(fixtureRoot, List.of(fixtureRoot.toString(), "extra"));
assertThat(result.exitCode()).isNotZero();
assertThat(result.output()).contains("expected zero arguments or one repository root");
}
@Test
void validatorRejectsMissingRepositoryRootAndMatrix(@TempDir Path tempDir) throws IOException {
Path missingRoot = tempDir.resolve("missing-root");
ScriptResult missingRootResult = runScript(tempDir, List.of(missingRoot.toString()));
assertThat(missingRootResult.exitCode()).isNotZero();
assertThat(missingRootResult.output())
.contains("repository root is not a directory: " + missingRoot);
Path emptyRoot = tempDir.resolve("empty-root");
Files.createDirectories(emptyRoot);
ScriptResult missingMatrixResult = runValidator(emptyRoot);
assertThat(missingMatrixResult.exitCode()).isNotZero();
assertThat(missingMatrixResult.output())
.contains("missing " + emptyRoot.resolve(".github/ci-gate-matrix.yml"));
}
@Test
void deceptiveStepNameAndEchoDoNotSatisfyExplicitExecution(@TempDir Path tempDir)
throws IOException {
Path fixtureRoot =
writeFixture(
tempDir.resolve("fixture"), "./gradlew targetGate", "echo disabled", FILLER_GATE_COUNT);
ScriptResult result = runValidator(fixtureRoot);
assertRejectedAsNotExplicit(result);
}
@Test
void differentProjectTaskWithTheSameNameDoesNotSatisfyExplicitExecution(@TempDir Path tempDir)
throws IOException {
Path fixtureRoot =
writeFixture(
tempDir.resolve("fixture"),
"Run target",
"./gradlew :other:targetGate",
FILLER_GATE_COUNT);
ScriptResult result = runValidator(fixtureRoot);
assertRejectedAsNotExplicit(result);
}
@Test
void shorthandRunStepSatisfiesExplicitExecution(@TempDir Path tempDir) throws IOException {
Path fixtureRoot =
writeFixture(
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
replace(
fixtureRoot.resolve(".github/workflows/fixture.yml"),
" - name: Run target\n run: ./gradlew targetGate\n",
" - run: ./gradlew targetGate\n");
ScriptResult result = runValidator(fixtureRoot);
assertThat(result.exitCode()).isZero();
assertThat(result.output()).contains("gate-matrix-lint: OK");
}
@Test
void leafCheckDoesNotSatisfyTheRequiredRootCheck(@TempDir Path tempDir) throws IOException {
Path fixtureRoot =
writeCheckFixture(tempDir.resolve("fixture"), "./gradlew :app-bootstrap:check", true);
ScriptResult result = runValidator(fixtureRoot);
assertThat(result.exitCode()).isNotZero();
assertThat(result.output())
.contains("gate 'target-gate' expects Gradle check in job 'target-job'");
}
@Test
void suppressionAndNonExecutionArgumentsDoNotSatisfyExplicitExecution(@TempDir Path tempDir)
throws IOException {
List<String> rejectedCommands =
List.of(
"./gradlew targetGate --dry-run",
"./gradlew targetGate -m",
"./gradlew targetGate -x targetGate",
"./gradlew targetGate --exclude-task targetGate",
"./gradlew targetGate \"--dry-run\"",
"./gradlew targetGate \\--dry-run",
"./gradlew targetGate --help",
"./gradlew targetGate --status");
for (int index = 0; index < rejectedCommands.size(); index++) {
Path fixtureRoot =
writeFixture(
tempDir.resolve("fixture-" + index),
"Run target",
rejectedCommands.get(index),
FILLER_GATE_COUNT);
ScriptResult result = runValidator(fixtureRoot);
assertThat(result.output()).as("command: %s", rejectedCommands.get(index)).isNotBlank();
assertRejectedAsNotExplicit(result);
}
}
@Test
void validatorRejectsEmptyMatrixDuplicateIdUnregisteredTaskAndMissingJob(@TempDir Path tempDir)
throws IOException {
// A matrix of a particular size is not a property. A matrix of no gates is: the file exists,
// the lint runs, and every per-row rule passes vacuously. That is the one thing the deleted
// count literal protected, and it is kept.
Path emptyMatrix =
writeFixture(
tempDir.resolve("empty-matrix"),
"Run target",
"./gradlew targetGate",
FILLER_GATE_COUNT);
Files.writeString(emptyMatrix.resolve(".github/ci-gate-matrix.yml"), "gates: []\n");
ScriptResult emptyMatrixResult = runValidator(emptyMatrix);
assertThat(emptyMatrixResult.exitCode()).isNotZero();
assertThat(emptyMatrixResult.output()).contains("matrix declares no gates");
Path duplicateId =
writeFixture(
tempDir.resolve("duplicate-id"),
"Run target",
"./gradlew targetGate",
FILLER_GATE_COUNT);
replace(
duplicateId.resolve(".github/ci-gate-matrix.yml"), "id: filler-gate-01", "id: target-gate");
ScriptResult duplicateResult = runValidator(duplicateId);
assertThat(duplicateResult.exitCode()).isNotZero();
assertThat(duplicateResult.output()).contains("duplicate gate id 'target-gate'");
Path unregisteredTask =
writeFixture(
tempDir.resolve("unregistered-task"),
"Run target",
"./gradlew targetGate",
FILLER_GATE_COUNT);
Files.writeString(unregisteredTask.resolve("src/sample/build.gradle"), "plugins {}\n");
ScriptResult unregisteredResult = runValidator(unregisteredTask);
assertThat(unregisteredResult.exitCode()).isNotZero();
assertThat(unregisteredResult.output())
.contains("gate 'target-gate' references unregistered Gradle task 'targetGate'");
Path unrelatedName =
writeFixture(
tempDir.resolve("unrelated-name"),
"Run target",
"./gradlew targetGate",
FILLER_GATE_COUNT);
Files.writeString(
unrelatedName.resolve("src/sample/build.gradle"),
"someUnrelatedConfiguration {\n name: 'targetGate'\n}\n");
ScriptResult unrelatedNameResult = runValidator(unrelatedName);
assertThat(unrelatedNameResult.exitCode()).isNotZero();
assertThat(unrelatedNameResult.output())
.contains("gate 'target-gate' references unregistered Gradle task 'targetGate'");
Path missingJob =
writeFixture(
tempDir.resolve("missing-job"),
"Run target",
"./gradlew targetGate",
FILLER_GATE_COUNT);
replace(
missingJob.resolve(".github/ci-gate-matrix.yml"), "job: target-job", "job: missing-job");
ScriptResult missingJobResult = runValidator(missingJob);
assertThat(missingJobResult.exitCode()).isNotZero();
assertThat(missingJobResult.output())
.contains("gate 'target-gate' references missing job 'missing-job' in 'fixture.yml'");
}
@Test
void validatorRejectsUnsafeCustomTaskRefAndMissingCheckWiring(@TempDir Path tempDir)
throws IOException {
Path unsafeRef =
writeFixture(
tempDir.resolve("unsafe-ref"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
replace(
unsafeRef.resolve(".github/ci-gate-matrix.yml"), "ref: targetGate", "ref: targetGate.*");
ScriptResult unsafeRefResult = runValidator(unsafeRef);
assertThat(unsafeRefResult.exitCode()).isNotZero();
assertThat(unsafeRefResult.output())
.contains("gate 'target-gate' has unsafe Gradle custom task ref 'targetGate.*'");
Path missingWiring =
writeCheckFixture(tempDir.resolve("missing-wiring"), "./gradlew check", false);
ScriptResult missingWiringResult = runValidator(missingWiring);
assertThat(missingWiringResult.exitCode()).isNotZero();
assertThat(missingWiringResult.output())
.contains("gate 'target-gate' task 'targetGate' exists but is not wired into Gradle check");
}
@Test
void validatorDoesNotInterpretCustomTaskOrPluginRefsAsRegularExpressions(@TempDir Path tempDir)
throws IOException {
Path dottedTask =
writeFixture(
tempDir.resolve("dotted-task"), "Run target", "./gradlew foo.bar", FILLER_GATE_COUNT);
replace(dottedTask.resolve(".github/ci-gate-matrix.yml"), "ref: targetGate", "ref: foo.bar");
Files.writeString(dottedTask.resolve("src/sample/build.gradle"), "tasks.register('fooXbar')\n");
ScriptResult dottedTaskResult = runValidator(dottedTask);
assertThat(dottedTaskResult.exitCode()).isNotZero();
assertThat(dottedTaskResult.output())
.contains("gate 'target-gate' has unsafe Gradle custom task ref 'foo.bar'");
Path unsafePlugin =
writeCheckFixture(tempDir.resolve("unsafe-plugin"), "./gradlew check", true);
replace(
unsafePlugin.resolve(".github/ci-gate-matrix.yml"),
"mechanism: gradle-custom-task",
"mechanism: gradle-plugin-task");
replace(
unsafePlugin.resolve(".github/ci-gate-matrix.yml"),
"ref: targetGate",
"ref: com.diffplug.*@spotlessCheck");
Files.writeString(
unsafePlugin.resolve("src/sample/build.gradle"),
"plugins { id 'com.diffplug.unrelated' }\n");
ScriptResult unsafePluginResult = runValidator(unsafePlugin);
assertThat(unsafePluginResult.exitCode()).isNotZero();
assertThat(unsafePluginResult.output())
.contains(
"gate 'target-gate' has unsafe Gradle plugin task ref "
+ "'com.diffplug.*@spotlessCheck'");
}
@Test
void realGateMatrixHasTheExactSafeSchema() throws IOException {
LoaderOptions options = new LoaderOptions();
options.setAllowDuplicateKeys(false);
options.setMaxAliasesForCollections(0);
Object loaded =
new Yaml(new SafeConstructor(options))
.load(Files.readString(repositoryRoot().resolve(".github/ci-gate-matrix.yml")));
assertThat(loaded).isInstanceOf(Map.class);
Map<?, ?> root = (Map<?, ?>) loaded;
assertThat(root.keySet().stream().map(String::valueOf).toList()).containsExactly("gates");
assertThat(root.get("gates")).isInstanceOf(List.class);
List<?> gates = (List<?>) root.get("gates");
// No expected size. A matrix that grew by a row is a registered control, not drift; what has
// to hold is that every row is well-formed, and that is asserted below for all of them.
assertThat(gates).isNotEmpty();
Set<String> ids = new LinkedHashSet<>();
for (Object rawGate : gates) {
assertThat(rawGate).isInstanceOf(Map.class);
Map<?, ?> gate = (Map<?, ?>) rawGate;
assertThat(gate).hasSize(EXPECTED_GATE_FIELDS.size());
assertThat(gate.keySet().stream().map(String::valueOf).toList())
.containsExactlyInAnyOrderElementsOf(EXPECTED_GATE_FIELDS);
String id = requireString(gate, "id");
assertThat(id).matches("[a-z0-9]+(?:-[a-z0-9]+)*");
assertThat(ids.add(id)).as("unique gate id: %s", id).isTrue();
assertThat(requireString(gate, "mechanism")).isIn(ALLOWED_MECHANISMS);
assertThat(requireString(gate, "execution")).isIn(ALLOWED_EXECUTIONS);
assertThat(requireString(gate, "ref")).isNotBlank();
assertThat(requireString(gate, "workflow")).endsWith(".yml");
assertThat(requireString(gate, "job")).isNotBlank();
Object releaseBlocking = gate.get("release_blocking");
assertThat(releaseBlocking).isInstanceOfAny(Boolean.class, String.class);
String releaseBlockingValue = String.valueOf(releaseBlocking);
assertThat(releaseBlockingValue).isIn("true", "false", "conditional");
if (releaseBlocking instanceof String) {
assertThat(releaseBlocking).isEqualTo("conditional");
}
}
Map<?, ?> posterGate =
gates.stream()
.map(Map.class::cast)
.filter(gate -> "poster-image-migration".equals(gate.get("id")))
.findFirst()
.orElseThrow(() -> new AssertionError("missing Poster image migration gate"));
assertThat(requireString(posterGate, "ref")).isEqualTo("posterImageMigrationTest");
assertThat(requireString(posterGate, "workflow")).isEqualTo("object-storage-qualification.yml");
assertThat(requireString(posterGate, "job")).isEqualTo("poster-image-v7-migration");
assertThat(requireString(posterGate, "execution")).isEqualTo("explicit");
}
@Test
void everyGateNamesAWorkflowAndJobThatExist() throws IOException {
Path root = repositoryRoot();
for (Map<?, ?> gate : realGates(root)) {
String workflowName = requireString(gate, "workflow");
Path workflowFile = root.resolve(".github/workflows").resolve(workflowName);
assertThat(workflowFile).as("gate '%s' workflow", gate.get("id")).isRegularFile();
Map<?, ?> jobs = requireMapValue(parseYamlMap(workflowFile), "jobs");
assertThat(jobs.keySet().stream().map(String::valueOf).toList())
.as("gate '%s' job in %s", gate.get("id"), workflowName)
.contains(requireString(gate, "job"));
}
}
/**
* {@code release_blocking: true} has to be a fact about the build, not a label.
*
* <p>It was read by nothing but an enum check, so a gate could claim to block a release that no
* job anywhere waited on: the filesystem vulnerability scan was release_blocking and could be red
* while the release gate reported green. A gate earns {@code true} by being required on a path a
* release actually takes — the release gate itself, one of its {@code needs:}, a name in its
* {@code REQUIRED_CHECKS}, or a job in a workflow that only runs on a release tag. Everything
* else is {@code conditional}, which is what the enum is for.
*/
@Test
void everyReleaseBlockingGateIsRequiredBySomeReleaseGate() throws IOException {
Path root = repositoryRoot();
Path releaseGateFile = root.resolve(".github/workflows").resolve(RELEASE_GATE_WORKFLOW);
Map<?, ?> releaseGate =
requireMapValue(requireMapValue(parseYamlMap(releaseGateFile), "jobs"), RELEASE_GATE_JOB);
Set<String> needs = new LinkedHashSet<>();
needs.add(RELEASE_GATE_JOB);
for (Object need : requireListValue(releaseGate, "needs")) {
needs.add(String.valueOf(need));
}
Set<String> requiredChecks = new LinkedHashSet<>();
for (Object rawStep : requireListValue(releaseGate, "steps")) {
assertThat(rawStep).isInstanceOf(Map.class);
Object stepEnvironment = ((Map<?, ?>) rawStep).get("env");
if (!(stepEnvironment instanceof Map<?, ?> environment)) {
continue;
}
Object declared = environment.get("REQUIRED_CHECKS");
if (declared == null) {
continue;
}
WHITESPACE
.splitAsStream(String.valueOf(declared).trim())
.filter(check -> !check.isBlank())
.forEach(requiredChecks::add);
}
assertThat(needs).as("jobs the release gate waits on").hasSizeGreaterThan(1);
assertThat(requiredChecks).as("cross-workflow checks the release gate requires").isNotEmpty();
for (Map<?, ?> gate : realGates(root)) {
if (!"true".equals(String.valueOf(gate.get("release_blocking")))) {
continue;
}
String workflowName = requireString(gate, "workflow");
String job = requireString(gate, "job");
boolean enforced =
(RELEASE_GATE_WORKFLOW.equals(workflowName) && needs.contains(job))
|| requiredChecks.contains(job)
|| runsOnlyForAReleaseTag(root.resolve(".github/workflows").resolve(workflowName));
assertThat(enforced)
.as(
"gate '%s' is release_blocking: true, so %s::%s must be %s::%s, one of its needs, a"
+ " name in its REQUIRED_CHECKS, or a job in a tag-triggered workflow",
gate.get("id"), workflowName, job, RELEASE_GATE_WORKFLOW, RELEASE_GATE_JOB)
.isTrue();
}
}
@Test
void validatorRejectsAReleaseBlockingGateNoReleaseGateRequires(@TempDir Path tempDir)
throws IOException {
Path unrequired =
writeFixture(
tempDir.resolve("unrequired"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
declareTargetGateReleaseBlocking(unrequired);
ScriptResult unrequiredResult = runValidator(unrequired);
assertThat(unrequiredResult.exitCode()).isNotZero();
assertThat(unrequiredResult.output())
.contains(
"gate 'target-gate' is release_blocking: true but no release gate requires job"
+ " 'target-job' in 'fixture.yml'");
Path tagTriggered =
writeFixture(
tempDir.resolve("tag-triggered"),
"Run target",
"./gradlew targetGate",
FILLER_GATE_COUNT);
declareTargetGateReleaseBlocking(tagTriggered);
replaceLiteral(
tagTriggered.resolve(".github/workflows/fixture.yml"),
"on: [push]\n",
"on:\n push:\n tags:\n - \"v*\"\n");
ScriptResult tagTriggeredResult = runValidator(tagTriggered);
assertThat(tagTriggeredResult.output()).contains("gate-matrix-lint: OK");
assertThat(tagTriggeredResult.exitCode()).isZero();
Path requiredCheck =
writeFixture(
tempDir.resolve("required-check"),
"Run target",
"./gradlew targetGate",
FILLER_GATE_COUNT);
declareTargetGateReleaseBlocking(requiredCheck);
Files.writeString(
requiredCheck.resolve(".github/workflows").resolve(RELEASE_GATE_WORKFLOW),
"""
name: fixture-quality-gates
on: [push]
jobs:
release-gate:
runs-on: ubuntu-latest
steps:
- name: Require the cross-workflow release-blocking checks
env:
REQUIRED_CHECKS: target-job
run: echo required
""");
ScriptResult requiredCheckResult = runValidator(requiredCheck);
assertThat(requiredCheckResult.output()).contains("gate-matrix-lint: OK");
assertThat(requiredCheckResult.exitCode()).isZero();
}
private static void declareTargetGateReleaseBlocking(Path fixtureRoot) throws IOException {
replaceLiteral(
fixtureRoot.resolve(".github/ci-gate-matrix.yml"),
"release_blocking: false",
"release_blocking: true");
}
private static boolean runsOnlyForAReleaseTag(Path workflowFile) throws IOException {
Map<?, ?> workflow = parseYamlMap(workflowFile);
// A bare `on:` key is YAML 1.1, where it resolves to the boolean true rather than the string.
Object triggers = workflow.get("on") != null ? workflow.get("on") : workflow.get(true);
if (!(triggers instanceof Map<?, ?> triggerMap)) {
return false;
}
Object push = triggerMap.get("push");
return push instanceof Map<?, ?> pushTrigger && pushTrigger.get("tags") != null;
}
private static List<Map<?, ?>> realGates(Path root) throws IOException {
Object gates = parseYamlMap(root.resolve(".github/ci-gate-matrix.yml")).get("gates");
assertThat(gates).isInstanceOf(List.class);
List<Map<?, ?>> parsed = new ArrayList<>();
for (Object gate : (List<?>) gates) {
assertThat(gate).isInstanceOf(Map.class);
parsed.add((Map<?, ?>) gate);
}
assertThat(parsed).isNotEmpty();
return parsed;
}
private static Map<?, ?> parseYamlMap(Path path) throws IOException {
LoaderOptions options = new LoaderOptions();
options.setAllowDuplicateKeys(false);
options.setMaxAliasesForCollections(0);
Object loaded = new Yaml(new SafeConstructor(options)).load(Files.readString(path));
assertThat(loaded).as("%s", path).isInstanceOf(Map.class);
return (Map<?, ?>) loaded;
}
private static Map<?, ?> requireMapValue(Map<?, ?> parent, String key) {
Object value = parent.get(key);
assertThat(value).as("field %s", key).isInstanceOf(Map.class);
return (Map<?, ?>) value;
}
private static List<?> requireListValue(Map<?, ?> parent, String key) {
Object value = parent.get(key);
assertThat(value).as("field %s", key).isInstanceOf(List.class);
return (List<?>) value;
}
private static void replaceLiteral(Path path, String target, String replacement)
throws IOException {
String original = Files.readString(path);
assertThat(original).contains(target);
int index = original.indexOf(target);
Files.writeString(
path,
original.substring(0, index) + replacement + original.substring(index + target.length()));
}
private static void assertRejectedAsNotExplicit(ScriptResult result) {
assertThat(result.exitCode()).isNotZero();
assertThat(result.output())
.contains("gate 'target-gate' task 'targetGate' is not explicit in job 'target-job'");
}
private static String requireString(Map<?, ?> gate, String field) {
Object value = gate.get(field);
assertThat(value).as("field %s", field).isInstanceOf(String.class);
return (String) value;
}
private static Path writeFixture(
Path root, String targetStepName, String targetCommand, int fillerGateCount)
throws IOException {
Path workflows = root.resolve(".github/workflows");
Files.createDirectories(workflows);
Files.createDirectories(root.resolve("src/sample"));
Files.writeString(root.resolve("src/sample/build.gradle"), "tasks.register('targetGate')\n");
StringBuilder workflow =
new StringBuilder()
.append("name: fixture\n")
.append("on: [push]\n")
.append("jobs:\n")
.append(" target-job:\n")
.append(" runs-on: ubuntu-latest\n")
.append(" steps:\n")
.append(" - name: ")
.append(targetStepName)
.append("\n")
.append(" run: ")
.append(targetCommand)
.append("\n");
for (int index = 1; index <= FILLER_GATE_COUNT; index++) {
workflow
.append(" filler-job-")
.append(twoDigits(index))
.append(":\n")
.append(" runs-on: ubuntu-latest\n")
.append(" steps:\n")
.append(" - run: echo filler\n");
}
Files.writeString(workflows.resolve("fixture.yml"), workflow);
StringBuilder matrix =
new StringBuilder()
.append("gates:\n")
.append(" - id: target-gate\n")
.append(" release_blocking: false\n")
.append(" mechanism: gradle-custom-task\n")
.append(" ref: targetGate\n")
.append(" workflow: fixture.yml\n")
.append(" job: target-job\n")
.append(" execution: explicit\n");
for (int index = 1; index <= fillerGateCount; index++) {
String suffix = twoDigits(index);
matrix
.append(" - id: filler-gate-")
.append(suffix)
.append("\n")
.append(" release_blocking: false\n")
.append(" mechanism: workflow-job\n")
.append(" ref: filler-job-")
.append(suffix)
.append("\n")
.append(" workflow: fixture.yml\n")
.append(" job: filler-job-")
.append(suffix)
.append("\n")
.append(" execution: job\n");
}
Files.writeString(root.resolve(".github/ci-gate-matrix.yml"), matrix);
return root;
}
private static Path writeCheckFixture(Path root, String checkCommand, boolean wireIntoCheck)
throws IOException {
Path fixtureRoot = writeFixture(root, "Run check", checkCommand, FILLER_GATE_COUNT);
replace(
fixtureRoot.resolve(".github/ci-gate-matrix.yml"),
"execution: explicit",
"execution: check");
if (wireIntoCheck) {
Files.writeString(
fixtureRoot.resolve("src/sample/build.gradle"),
"""
tasks.register('targetGate')
tasks.named('check') { dependsOn tasks.named('targetGate') }
""");
}
return fixtureRoot;
}
private static ScriptResult runValidator(Path fixtureRoot) throws IOException {
return runScript(fixtureRoot, List.of(fixtureRoot.toString()));
}
private static ScriptResult runScript(Path workingDirectory, List<String> arguments)
throws IOException {
return runScriptAt(validatorPath(), workingDirectory, arguments);
}
private static ScriptResult runScriptAt(
Path script, Path workingDirectory, List<String> arguments) throws IOException {
List<String> command = new ArrayList<>();
command.add("bash");
command.add(script.toString());
command.addAll(arguments);
return runCommand(workingDirectory, command);
}
private static ScriptResult runCommand(Path workingDirectory, List<String> command)
throws IOException {
Path outputFile = Files.createTempFile("gate-matrix-validator-", ".log");
Process process = null;
try {
process =
new ProcessBuilder(command)
.directory(workingDirectory.toFile())
.redirectErrorStream(true)
.redirectOutput(outputFile.toFile())
.start();
boolean finished;
try {
finished = process.waitFor(VALIDATOR_TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);
} catch (InterruptedException exception) {
terminateAndWait(process);
Thread.currentThread().interrupt();
throw new AssertionError("interrupted while waiting for gate matrix validator", exception);
}
if (!finished) {
terminateAndWait(process);
throw new AssertionError("gate matrix validator exceeded " + VALIDATOR_TIMEOUT);
}
return new ScriptResult(process.exitValue(), Files.readString(outputFile));
} finally {
if (process != null && process.isAlive()) {
terminateAndWait(process);
}
Files.deleteIfExists(outputFile);
}
}
private static void terminateAndWait(Process process) {
List<ProcessHandle> descendants = process.descendants().toList();
descendants.forEach(ProcessHandle::destroy);
process.destroy();
List<ProcessHandle> processTree = new ArrayList<>(descendants);
processTree.add(process.toHandle());
boolean interrupted = false;
try {
if (awaitExit(processTree)) {
return;
}
} catch (InterruptedException exception) {
interrupted = true;
}
processTree.stream().filter(ProcessHandle::isAlive).forEach(ProcessHandle::destroyForcibly);
try {
awaitExit(processTree);
} catch (InterruptedException exception) {
interrupted = true;
}
if (interrupted) {
Thread.currentThread().interrupt();
}
}
private static boolean awaitExit(List<ProcessHandle> processTree) throws InterruptedException {
CompletableFuture<?>[] exits =
processTree.stream().map(ProcessHandle::onExit).toArray(CompletableFuture<?>[]::new);
try {
CompletableFuture.allOf(exits).get(2, TimeUnit.SECONDS);
return true;
} catch (ExecutionException | TimeoutException exception) {
return false;
}
}
private static void replace(Path path, String target, String replacement) throws IOException {
String original = Files.readString(path);
assertThat(original).contains(target);
Files.writeString(path, original.replaceFirst(target, replacement));
}
private static String twoDigits(int value) {
return String.format("%02d", value);
}
private static Path validatorPath() {
return repositoryRoot().resolve(".github/scripts/verify-gate-matrix.sh");
// The job that runs it has to be one the release gate waits on.
assertThat(workflow).contains("\n quality-gates:\n").contains("\n - quality-gates\n");
}
private static Path repositoryRoot() {
return RepositoryContractResources.fromSystemProperty().repositoryRoot();
for (Path path = Paths.get("").toAbsolutePath(); path != null; path = path.getParent()) {
if (Files.isRegularFile(path.resolve("AGENTS.md"))
&& Files.isRegularFile(path.resolve("src/settings.gradle"))) {
return path;
}
}
throw new IllegalStateException(
"repository root not found from " + Paths.get("").toAbsolutePath());
}
private record ScriptResult(int exitCode, String output) {}
}
@@ -4,11 +4,11 @@ import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
@@ -25,55 +25,12 @@ import org.yaml.snakeyaml.constructor.SafeConstructor;
class DeveloperExperienceContractTest {
private static final Path REPOSITORY_ROOT = repositoryRoot();
private static final String VALIDATION_STEP =
" - name: Validate Gradle wrapper\n"
+ " id: gradle-wrapper-validation\n"
+ " uses: gradle/actions/wrapper-validation@"
+ "3f131e8634966bd73d06cc69884922b02e6faf92 # gradle/actions@v6\n";
private static final String DEPENDENCY_SUBMISSION_ACTION =
"gradle/actions/dependency-submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1";
private static final String NAMED_DEPENDENCY_SUBMISSION_STEP =
" - name: Submit the resolved Gradle dependency graph\n"
+ " uses: "
+ DEPENDENCY_SUBMISSION_ACTION
+ " # gradle/actions@v4.4.4\n";
private static final String DOUBLE_QUOTED_DEPENDENCY_SUBMISSION_STEP =
" - name: Submit the resolved Gradle dependency graph\n"
+ " uses: \""
+ DEPENDENCY_SUBMISSION_ACTION
+ "\"\n";
private static final String SINGLE_QUOTED_DEPENDENCY_SUBMISSION_STEP =
" - name: Submit the resolved Gradle dependency graph\n"
+ " uses: '"
+ DEPENDENCY_SUBMISSION_ACTION
+ "'\n";
private static final String HEX_ESCAPED_DEPENDENCY_SUBMISSION_STEP =
" - name: Submit the resolved Gradle dependency graph\n"
+ " uses: \"\\x67radle/actions/dependency-submission@"
+ "748248ddd2a24f49513d8f472f81c3a07d4d50e1\"\n";
private static final String CONTINUED_DEPENDENCY_SUBMISSION_STEP =
"""
- name: Submit the resolved Gradle dependency graph
uses: "gradle/actions/dependency-\\
submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1"
""";
private static final String ANONYMOUS_DEPENDENCY_SUBMISSION_STEP =
" - uses: " + DEPENDENCY_SUBMISSION_ACTION + " # gradle/actions@v4.4.4\n";
private static final String RUN_BLOCK_FAKE_VALIDATION_STEP =
" - name: Pretend to validate the Gradle wrapper\n"
+ " run: |\n"
+ " uses: gradle/actions/wrapper-validation@"
+ "3f131e8634966bd73d06cc69884922b02e6faf92 # gradle/actions@v6\n";
private static final String BARE_ALWAYS_CONDITION = " if: always()\n";
private static final String GUARDED_ALWAYS_CONDITION =
" if: ${{ always() && steps.gradle-wrapper-validation.outcome == 'success' }}\n";
private static final String GATE_MATRIX_STEP =
"""
- name: Verify the gate matrix against the repository
run: bash .github/scripts/verify-gate-matrix.sh
""";
private static final String CANONICAL_WRAPPER_PROPERTIES_DIAGNOSTIC =
"wrapper properties must match the exact canonical Gradle 9.0.0 eight-line contract";
// The workflow-mutation fixtures that stood here are gone with the tests that used them: a
// hex-escaped `uses:`, a line-continued action reference, a `run:` block impersonating a
// validation step, a bare `if: always()` versus the guarded form, a gate-matrix step. Every one
// existed to prove that .github/scripts/verify-gradle-wrapper.sh could not be fooled by that
// spelling — a YAML parser written in bash, tested from a Java application's test suite.
@Test
void toolVersionsPinTemurin21() throws IOException {
@@ -148,13 +105,15 @@ class DeveloperExperienceContractTest {
}
@Test
void readmeCommandsAreVerifiedAndBootstrapIsTheFirstRunEntrypoint() throws IOException {
String build = read("src/build.gradle");
void readmeDocumentsOneFirstRunEntrypoint() throws IOException {
// The `verifyReadmeCommands` half of this is gone with the task. That gate parsed the README's
// ```bash blocks and resolved every `./gradlew`, `docker compose` and `make` token against the
// real task graph, the real Compose files and the real Makefile — a hand-written Markdown
// command parser, which made "what may be written in the README" a function of what the parser
// could read. What is left is the property a first-run document has to have: one entrypoint,
// named, with the check that tells you it worked.
String readme = read("README.md");
assertThat(build)
.contains("tasks.register('verifyReadmeCommands')")
.contains("dependsOn rootProject.tasks.named('verifyReadmeCommands')");
assertThat(readme)
.contains("./gradlew bootstrap")
.contains("GET /api/healthcheck")
@@ -227,592 +186,41 @@ class DeveloperExperienceContractTest {
}
@Test
void checkedInGradleWrapperAndEveryGradleJobPassTheExecutableContract() throws Exception {
ProcessResult result = runGradleWrapperVerifier(REPOSITORY_ROOT);
assertThat(result.exitCode()).as(result.output()).isZero();
assertThat(result.output()).contains("gradle-wrapper-contract: PASS");
}
@Test
void gradleWrapperVerifierRejectsACorruptDistributionChecksum(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
String content = Files.readString(properties);
String corrupted =
content.contains("distributionSha256Sum=")
? content.replaceFirst(
"(?m)^distributionSha256Sum=.*$", "distributionSha256Sum=corrupt")
: content + System.lineSeparator() + "distributionSha256Sum=corrupt\n";
Files.writeString(properties, corrupted);
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertCanonicalWrapperPropertiesRejected(result);
}
@Test
void gradleWrapperVerifierRejectsWhitespaceDuplicateChecksumOverride(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
Files.writeString(
properties,
Files.readString(properties) + " distributionSha256Sum=attacker-controlled-checksum\n");
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertCanonicalWrapperPropertiesRejected(result);
}
@Test
void gradleWrapperVerifierRejectsColonDuplicateDistributionUrlOverride(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
Files.writeString(
properties,
Files.readString(properties)
+ "distributionUrl:https\\://attacker.invalid/gradle-9.0.0-bin.zip\n");
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertCanonicalWrapperPropertiesRejected(result);
}
@Test
void gradleWrapperVerifierRejectsUnicodeEscapedChecksumOverride(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
Files.writeString(
properties,
Files.readString(properties)
+ "distribution\\u0053ha256Sum=attacker-controlled-checksum\n");
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertCanonicalWrapperPropertiesRejected(result);
}
@Test
void gradleWrapperVerifierRejectsContinuedChecksumOverride(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
Files.writeString(
properties,
Files.readString(properties) + "distributionSha256\\\nSum=attacker-controlled-checksum\n");
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertCanonicalWrapperPropertiesRejected(result);
}
@Test
void gradleWrapperVerifierRejectsValidationMissingFromOneGradleJob(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
String content = Files.readString(workflow);
Files.writeString(workflow, removeFirstValidationStep(content));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertMissingGradleValidationDiagnostic(result, "quality-gates");
}
@Test
void gradleWrapperVerifierRejectsRunBlockTextMasqueradingAsValidation(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
String content = Files.readString(workflow);
assertThat(content).contains(VALIDATION_STEP);
Files.writeString(
workflow,
content.replaceFirst(
Pattern.quote(VALIDATION_STEP),
java.util.regex.Matcher.quoteReplacement(RUN_BLOCK_FAKE_VALIDATION_STEP)));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertMissingGradleValidationDiagnostic(result, "quality-gates");
}
@Test
void gradleWrapperVerifierRejectsEncodedSingleLineGradleRun(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
String content = Files.readString(workflow);
String plainRun =
" run: ./gradlew check verifyPublicPathSnapshot verifyDependencyLocks --warning-mode=fail --no-daemon --stacktrace\n";
assertThat(content).contains(plainRun);
String encoded = content.replace(plainRun, " run: \"\\x2e/gradlew check\"\n");
Files.writeString(workflow, removeFirstValidationStep(encoded));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output())
.contains(
".github/workflows/ci-quality-gates.yml: job quality-gates has unsupported run scalar");
}
@Test
void gradleWrapperVerifierRejectsBlockScalarUsesInNonGradleWorkflow(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
String content = Files.readString(workflow);
String action =
" uses: lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede # lycheeverse/lychee-action@v2.0.2\n";
String block =
"""
uses: |
lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede
""";
assertThat(content).contains(action);
Files.writeString(workflow, content.replace(action, block));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output()).contains("job lychee has unsupported uses scalar");
}
@Test
void gradleWrapperVerifierRejectsAliasedUsesInNonGradleWorkflow(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
String content = Files.readString(workflow);
String action =
" uses: lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede # lycheeverse/lychee-action@v2.0.2\n";
assertThat(content).contains(action);
String aliased =
content
.replace(
"name: link-check\n",
"name: link-check\n"
+ "x-lychee-action: &lychee-action "
+ "lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede\n")
.replace(action, " uses: *lychee-action\n");
Files.writeString(workflow, aliased);
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output()).contains("job lychee has unsupported uses scalar");
}
@Test
void gradleWrapperVerifierRejectsStepMergeKeyInNonGradleWorkflow(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/merge-injected-action.yml");
Files.writeString(
workflow,
"""
name: merge-injected-action
on: workflow_dispatch
x-step: &injected-step
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
jobs:
merge-job:
runs-on: ubuntu-latest
steps:
- <<: *injected-step
""");
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output()).contains("job merge-job contains a forbidden step merge key");
}
@Test
void gradleWrapperVerifierRejectsEncodedDependencyActionInFlowStyleStep(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
String content = Files.readString(workflow);
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
String flowStep =
" - {name: Submit the resolved Gradle dependency graph, uses: \"\\x67radle/"
+ "actions/dependency-submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1\"}\n";
Files.writeString(
workflow,
removeFirstValidationStep(content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, flowStep)));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output())
.contains("job dependency-submission contains unsupported flow-style step syntax");
}
@Test
void gradleWrapperVerifierRejectsFlowStyleJobsContainer(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/flow-jobs.yml");
Files.writeString(
workflow,
"""
name: flow-jobs
on: workflow_dispatch
jobs: {flow-job: {runs-on: ubuntu-latest, steps: [{uses: "\\x67radle/actions/dependency-submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1"}]}}
""");
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output()).contains("jobs container must use a canonical block mapping");
}
@Test
void gradleWrapperVerifierRejectsAnchoredCustomGradleShellByWorkflowLock(
@TempDir Path fixtureRoot) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
String content = Files.readString(workflow);
assertThat(content).contains("env:\n").contains(GATE_MATRIX_STEP);
String mutated =
content
.replace(
"env:\n",
"x-gradle-shell: &gradle-shell bash -c './gradlew help; bash {0}'\n\nenv:\n")
.replace(
GATE_MATRIX_STEP,
GATE_MATRIX_STEP.replace(
" run:", " shell: *gradle-shell\n run:"));
Files.writeString(workflow, mutated);
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
}
@Test
void gradleWrapperVerifierRejectsRepositoryRelativeGradlePathByWorkflowLock(
@TempDir Path fixtureRoot) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
String content = Files.readString(workflow);
assertThat(content).contains(GATE_MATRIX_STEP);
String unvalidatedGradleStep =
"""
- name: Run unvalidated repository-relative Gradle
run: src/gradlew help
""";
Files.writeString(
workflow, content.replace(GATE_MATRIX_STEP, GATE_MATRIX_STEP + unvalidatedGradleStep));
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
}
@Test
void gradleWrapperVerifierRejectsEscapedDuplicateJobsByWorkflowLock(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
Files.writeString(
workflow,
Files.readString(workflow)
+ """
"jo\\x62s":
hidden-gradle:
runs-on: ubuntu-latest
steps:
- "r\\x75n": "\\x2e/gradlew help"
""");
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
}
@Test
void gradleWrapperVerifierRejectsAddedWorkflowByWorkflowLock(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Files.writeString(
fixtureRoot.resolve(".github/workflows/unreviewed.yml"),
"""
name: unreviewed
on: workflow_dispatch
jobs:
noop:
runs-on: ubuntu-latest
steps:
- name: No operation
run: echo ok
""");
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
}
@Test
void gradleWrapperVerifierRejectsRemovedWorkflowByWorkflowLock(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Files.delete(fixtureRoot.resolve(".github/workflows/link-check.yml"));
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
}
@Test
void gradleWrapperVerifierRejectsInnocuousWorkflowByteChangeByWorkflowLock(
@TempDir Path fixtureRoot) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
Files.writeString(workflow, Files.readString(workflow) + "# unreviewed byte change\n");
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
}
@Test
void gradleWrapperVerifierRejectsWorkflowSymlinkReplacementByWorkflowLock(
@TempDir Path fixtureRoot) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
Files.delete(workflow);
Files.createSymbolicLink(workflow, Path.of("ci-quality-gates.yml"));
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
}
private static void assertWorkflowLockRejected(ProcessResult result) {
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output()).contains("workflow lock mismatch:");
}
@Test
void gradleWrapperVerifierRejectsConditionalWrapperValidationStep(@TempDir Path fixtureRoot)
throws Exception {
assertWrapperValidationControlFieldIsRejected(fixtureRoot, "if: ${{ false }}");
}
@Test
void gradleWrapperVerifierRejectsContinueOnErrorWrapperValidationStep(@TempDir Path fixtureRoot)
throws Exception {
assertWrapperValidationControlFieldIsRejected(fixtureRoot, "continue-on-error: true");
}
@Test
void gradleWrapperVerifierRejectsWithFieldOnWrapperValidationStep(@TempDir Path fixtureRoot)
throws Exception {
assertWrapperValidationControlFieldIsRejected(fixtureRoot, "with:");
}
private static void assertWrapperValidationControlFieldIsRejected(
Path fixtureRoot, String controlField) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
String content = Files.readString(workflow);
assertThat(content).contains(VALIDATION_STEP);
String controlledValidation = VALIDATION_STEP + " " + controlField + "\n";
Files.writeString(
workflow,
content.replaceFirst(
Pattern.quote(VALIDATION_STEP), Matcher.quoteReplacement(controlledValidation)));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output())
.contains("wrapper validation step contains unsupported field: " + controlField);
}
/**
* A cleanup step that runs after a failed Gradle step is the one place a bare {@code always()} is
* tempting, and it is exactly where it is unsafe: the wrapper validation may not have run, so the
* sanitizer would execute an unverified wrapper. The verifier accepts the guarded form only.
*
* <p>The fixture is authored here rather than borrowed from a checked-in workflow. A test that
* mutates whichever real workflow happens to carry the shape it needs stops compiling the day
* that workflow is retired, which says nothing about the rule it was meant to prove.
*/
@Test
void gradleWrapperVerifierRejectsBareAlwaysGradleSanitizer(@TempDir Path fixtureRoot)
throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/evidence-sanitizer.yml");
Files.writeString(workflow, sanitizerWorkflow(GUARDED_ALWAYS_CONDITION));
assertThat(runGradleWrapperVerifier(fixtureRoot).output())
.as("the guarded form is the accepted shape and must not be reported")
.doesNotContain("unsupported if condition");
Files.writeString(workflow, sanitizerWorkflow(BARE_ALWAYS_CONDITION));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output())
.contains("job redis-security has Gradle step with unsupported if condition: always()");
}
/** A canonical Gradle job whose post-run sanitizer carries {@code condition}. */
private static String sanitizerWorkflow(String condition) {
return """
name: evidence-sanitizer
on: workflow_dispatch
jobs:
redis-security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2
"""
+ VALIDATION_STEP
+ """
- id: redis-tests
working-directory: src
run: ./gradlew :adapter:outbound:cache-redis:redisSecurityTest --no-daemon
- id: redis-evidence-sanitizer
"""
+ condition
+ """
working-directory: src
run: ./gradlew :adapter:outbound:cache-redis:verifyRedisEvidenceArtifactsForUpload
""";
}
@Test
void gradleWrapperVerifierRejectsValidationMissingFromDependencySubmissionJob(
@TempDir Path fixtureRoot) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
String content = Files.readString(workflow);
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
Files.writeString(workflow, removeFirstValidationStep(content));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertMissingDependencySubmissionValidationDiagnostic(result);
}
@Test
void gradleWrapperVerifierRejectsUnguardedAnonymousDependencySubmissionStep(
@TempDir Path fixtureRoot) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
String content = Files.readString(workflow);
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
String anonymous =
content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, ANONYMOUS_DEPENDENCY_SUBMISSION_STEP);
Files.writeString(workflow, removeFirstValidationStep(anonymous));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertMissingDependencySubmissionValidationDiagnostic(result);
}
@Test
void gradleWrapperVerifierRejectsUnguardedDoubleQuotedDependencySubmissionStep(
@TempDir Path fixtureRoot) throws Exception {
assertQuotedDependencySubmissionWithoutValidationIsRejected(
fixtureRoot, DOUBLE_QUOTED_DEPENDENCY_SUBMISSION_STEP);
}
@Test
void gradleWrapperVerifierRejectsUnguardedSingleQuotedDependencySubmissionStep(
@TempDir Path fixtureRoot) throws Exception {
assertQuotedDependencySubmissionWithoutValidationIsRejected(
fixtureRoot, SINGLE_QUOTED_DEPENDENCY_SUBMISSION_STEP);
}
@Test
void gradleWrapperVerifierRejectsHexEscapedDependencySubmissionStep(@TempDir Path fixtureRoot)
throws Exception {
assertEscapedDependencySubmissionWithoutValidationIsRejected(
fixtureRoot, HEX_ESCAPED_DEPENDENCY_SUBMISSION_STEP);
}
@Test
void gradleWrapperVerifierRejectsContinuedDependencySubmissionStep(@TempDir Path fixtureRoot)
throws Exception {
assertEscapedDependencySubmissionWithoutValidationIsRejected(
fixtureRoot, CONTINUED_DEPENDENCY_SUBMISSION_STEP);
}
private static void assertQuotedDependencySubmissionWithoutValidationIsRejected(
Path fixtureRoot, String quotedStep) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
String content = Files.readString(workflow);
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
String quoted = content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, quotedStep);
Files.writeString(workflow, removeFirstValidationStep(quoted));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertMissingDependencySubmissionValidationDiagnostic(result);
}
private static void assertMissingDependencySubmissionValidationDiagnostic(ProcessResult result) {
assertMissingGradleValidationDiagnostic(result, "dependency-submission");
}
private static void assertMissingGradleValidationDiagnostic(ProcessResult result, String job) {
assertThat(result.output())
.contains(
"job " + job + " invokes Gradle without the exact pinned wrapper validation action");
}
private static void assertEscapedDependencySubmissionWithoutValidationIsRejected(
Path fixtureRoot, String escapedStep) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
String content = Files.readString(workflow);
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
String escaped = content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, escapedStep);
Files.writeString(workflow, removeFirstValidationStep(escaped));
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output())
.contains(
".github/workflows/dependency-vulnerability.yml: job dependency-submission has unsupported uses scalar");
}
@Test
void gradleWrapperVerifierRejectsAdmittedWorkflowWithNoDetectedGradleJob(
@TempDir Path fixtureRoot) throws Exception {
copyGradleWrapperVerifierInputs(fixtureRoot);
Path workflow = fixtureRoot.resolve(".github/workflows/orphan-gradle-reference.yml");
Files.writeString(
workflow,
"""
name: orphan-gradle-reference
on: workflow_dispatch
env:
DOCUMENTED_COMMAND: ./gradlew
jobs:
documentation:
runs-on: ubuntu-latest
steps:
- name: Keep the documented command out of executable steps
run: echo documented
""");
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output())
.contains(
"Gradle-running workflow contains no detected Gradle job: "
+ ".github/workflows/orphan-gradle-reference.yml");
}
private static void assertCanonicalWrapperPropertiesRejected(ProcessResult result) {
assertThat(result.exitCode()).as(result.output()).isNotZero();
assertThat(result.output()).contains(CANONICAL_WRAPPER_PROPERTIES_DIAGNOSTIC);
void gradleWrapperIsCheckedInAndValidatedByTheOfficialAction() throws IOException {
// Twenty-four tests stood here, and every one of them tested .github/scripts/
// verify-gradle-wrapper.sh — an 799-line shell script that this suite exercised by writing
// mutated workflow fixtures into a @TempDir: a YAML alias in a `uses:`, a merge key in a step,
// a flow-style `jobs:` map, a Unicode-escaped checksum override, an added workflow file, a
// deleted workflow file, a workflow byte changed inside a comment, a workflow replaced by a
// symlink.
//
// The script hashed all twelve workflow files and required the hashes to match a list it
// carried, which is why "a workflow byte changed inside a comment" was a failure worth a test.
// As a security control it was not independent: anyone able to edit a workflow was able to edit
// the expected hash in the same commit. As a developer control it charged a hash update for
// every comment.
//
// The wrapper guarantee itself is unchanged and is now the official action's:
// gradle/actions/setup-gradle validates every wrapper jar in the repository by default, it is
// applied through .github/actions/setup-gradle-java, and it is pinned to a full commit SHA.
// What is left to assert here is what that action cannot: that the wrapper is committed, and
// that its distribution is pinned by checksum rather than by URL alone.
assertThat(REPOSITORY_ROOT.resolve("src/gradle/wrapper/gradle-wrapper.jar")).isRegularFile();
assertThat(REPOSITORY_ROOT.resolve("src/gradlew")).isRegularFile();
String wrapperProperties = read("src/gradle/wrapper/gradle-wrapper.properties");
assertThat(wrapperProperties)
.contains("distributionSha256Sum=")
.contains("validateDistributionUrl=true")
.contains("https\\://services.gradle.org/distributions/");
String setupAction = read(".github/actions/setup-gradle-java/action.yml");
assertThat(setupAction)
.as("wrapper validation has to be reachable from every Gradle job")
.contains("gradle/actions/setup-gradle@");
assertThat(setupAction)
.as("an action reference is only immutable when it is a full commit SHA")
.containsPattern("uses: [\\w./-]+@[0-9a-f]{40} #");
}
@Test
@@ -927,62 +335,43 @@ class DeveloperExperienceContractTest {
}
@Test
void dependencyCacheStageConfiguresWithoutGitWhenRevisionIsAttested(@TempDir Path fixtureRoot)
void aBuildWithNoGitMetadataConfiguresAndIsNotCalledARelease(@TempDir Path fixtureRoot)
throws Exception {
// A source archive with no `.git` and no `-PgitRevision`. The root build used to throw during
// configuration here — "A 7-40 character hexadecimal source revision is required" — so
// `./gradlew test` on an unpacked tarball failed before it compiled anything. Release
// traceability was being enforced on every task in the build.
Path fixtureSrc = fixtureRoot.resolve("src");
copyDependencyCacheStageInputs(fixtureSrc);
assertThat(fixtureSrc.resolve(".git")).doesNotExist();
ProcessResult result = runGitlessGradleHelp(fixtureSrc);
ProcessResult configured = runGitlessGradle(fixtureSrc, "help");
assertThat(configured.exitCode()).as(configured.output()).isZero();
assertThat(configured.output()).contains("BUILD SUCCESSFUL");
// And the other half: such a build may not call itself a release. verifyReleaseProvenance is
// where the revision is required, and releaseCheck is what depends on it.
ProcessResult provenance = runGitlessGradle(fixtureSrc, "verifyReleaseProvenance");
assertThat(provenance.exitCode()).as(provenance.output()).isNotZero();
assertThat(provenance.output()).contains("no source revision");
}
@Test
void anAttestedRevisionSatisfiesReleaseProvenanceWithoutGit(@TempDir Path fixtureRoot)
throws Exception {
Path fixtureSrc = fixtureRoot.resolve("src");
copyDependencyCacheStageInputs(fixtureSrc);
ProcessResult result =
runGitlessGradle(
fixtureSrc,
"verifyReleaseProvenance",
"-PgitRevision=0123456789abcdef0123456789abcdef01234567");
assertThat(result.exitCode()).as(result.output()).isZero();
assertThat(result.output()).contains("BUILD SUCCESSFUL");
}
private static String removeFirstValidationStep(String workflow) {
assertThat(workflow).contains(VALIDATION_STEP);
return workflow.replaceFirst(Pattern.quote(VALIDATION_STEP), "");
}
private static ProcessResult runGradleWrapperVerifier(Path repositoryRoot) throws Exception {
Process process =
new ProcessBuilder(
"bash",
REPOSITORY_ROOT.resolve(".github/scripts/verify-gradle-wrapper.sh").toString(),
repositoryRoot.toString())
.directory(REPOSITORY_ROOT.toFile())
.redirectErrorStream(true)
.start();
String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8);
return new ProcessResult(process.waitFor(), output);
}
private static void copyGradleWrapperVerifierInputs(Path fixtureRoot) throws IOException {
copyFile(
REPOSITORY_ROOT.resolve("src/gradle/wrapper/gradle-wrapper.properties"),
fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties"));
copyFile(
REPOSITORY_ROOT.resolve("src/gradle/wrapper/gradle-wrapper.jar"),
fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.jar"));
Path workflows = REPOSITORY_ROOT.resolve(".github/workflows");
try (Stream<Path> paths = Files.walk(workflows)) {
paths
.filter(Files::isRegularFile)
.filter(
path -> {
String name = path.getFileName().toString();
return name.endsWith(".yml") || name.endsWith(".yaml");
})
.forEach(
source -> {
try {
copyFile(source, fixtureRoot.resolve(REPOSITORY_ROOT.relativize(source)));
} catch (IOException exception) {
throw new IllegalStateException(
"failed to copy verifier workflow input", exception);
}
});
}
assertThat(result.output()).contains("verifyReleaseProvenance: OK");
}
private static void copyDependencyCacheStageInputs(Path fixtureSrc) throws IOException {
@@ -1018,15 +407,14 @@ class DeveloperExperienceContractTest {
}
}
private static ProcessResult runGitlessGradleHelp(Path fixtureSrc) throws Exception {
Path outputFile = fixtureSrc.resolve("gitless-help.log");
private static ProcessResult runGitlessGradle(Path fixtureSrc, String... arguments)
throws Exception {
Path outputFile = fixtureSrc.resolve("gitless-run.log");
List<String> command = new ArrayList<>(List.of("./gradlew"));
command.addAll(List.of(arguments));
command.addAll(List.of("--no-daemon", "--console=plain"));
Process process =
new ProcessBuilder(
"./gradlew",
"help",
"--no-daemon",
"--console=plain",
"-PgitRevision=0123456789abcdef0123456789abcdef01234567")
new ProcessBuilder(command)
.directory(fixtureSrc.toFile())
.redirectErrorStream(true)
.redirectOutput(outputFile.toFile())
@@ -111,7 +111,8 @@ class PiiTokenBodyForbiddenContractTest {
@Test
void requestBodyCaptureIsDisabledByDefault() throws IOException {
Path env = RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
Path env =
RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
String value = readEnv(env, "APP_LOG_BODY_CAPTURE_ENABLED");
assertThat(value)
@@ -159,19 +159,17 @@ class SampleRemovalSmokeContractTest {
@Test
void sampleOffCiJobIsReleaseBlocking() throws IOException {
// The gate matrix half of this assertion is gone with the matrix. It read .github/
// ci-gate-matrix.yml and required a row declaring that the sample-off job exists and is
// release-blocking — a third copy of what the workflow itself says in `jobs:` and in
// `release-gate.needs`. Checking a register against the thing it registers is work that only
// ever finds a disagreement between two descriptions of one fact.
RepositoryContractResources resources = RepositoryContractResources.fromSystemProperty();
String workflow =
Files.readString(resources.requireTrackedFile(".github/workflows/ci-quality-gates.yml"));
String gateMatrix =
Files.readString(resources.requireTrackedFile(".github/ci-gate-matrix.yml"));
assertThat(workflow).contains("\n sample-off:\n");
assertThat(workflow).contains("./gradlew :app-bootstrap:sampleOffTest");
assertThat(workflow).contains(":app-bootstrap:sampleOffTest");
assertThat(workflow).contains("\n - sample-off\n");
assertThat(gateMatrix)
.contains("id: sample-off")
.contains("ref: sampleOffTest")
.contains("job: sample-off")
.contains("execution: explicit");
}
}
@@ -22,7 +22,8 @@ import org.junit.jupiter.api.Test;
class SqlLoggingForbiddenContractTest {
private Properties loadEnv() throws Exception {
Path env = RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
Path env =
RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
Properties props = new Properties();
try (InputStream in = Files.newInputStream(env)) {
props.load(in);
@@ -14,6 +14,8 @@ import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.junit.jupiter.api.Test;
import org.yaml.snakeyaml.LoaderOptions;
import org.yaml.snakeyaml.Yaml;
@@ -39,10 +41,22 @@ class MessagingCapabilityRegistryContractTest {
private static final Set<String> LEGAL_MATURITY =
Set.of("not-implemented", "implemented-candidate", "release-eligible");
private static final Set<String> REQUIRED_VERIFICATION_TASKS =
/** Qualification tasks that exist and can pass. */
private static final Set<String> IMPLEMENTED_VERIFICATION_TASKS =
Set.of("verifyMessagingContracts", "verifyMessagingJsonSchemaV1");
/**
* Task names a not-implemented card names as the evidence it will need.
*
* <p>Names on a roadmap, not tasks. The build used to register all of these — nine
* `tasks.register(...)` calls whose action threw unconditionally, whatever was on disk — so
* `./gradlew verifyMessagingSecurityR2` was a task you could invoke and could not pass. That is a
* TODO written through the Gradle task API: it appears in `./gradlew tasks`, it is reachable from
* a `dependsOn`, and the only thing it can do is fail. MSG-015 tracks the real work and
* docs/roadmap/messaging-r2.md states it in prose.
*/
private static final Set<String> PLANNED_VERIFICATION_TASKS =
Set.of(
"verifyMessagingContracts",
"verifyMessagingJsonSchemaV1",
"verifyMessagingPollingOutboxR2",
"verifyMessagingKafkaProducerR2",
"verifyMessagingSecurityR2",
@@ -53,6 +67,18 @@ class MessagingCapabilityRegistryContractTest {
"verifyMessagingCleanupTargetBinding",
"verifyMessagingFinalR2Profile");
/**
* Every producer name the registry files enumerate: the two that exist plus the nine planned.
*
* <p>Used for the release profiles and the evidence manifest schema, which describe the complete
* R2 shape rather than the current build. A schema saying "producerTask must be one of these
* eleven" is a statement about the evidence format; it is not a claim that eleven tasks exist,
* which is exactly the claim the deleted Gradle registrations were making.
*/
private static final Set<String> ALL_VERIFICATION_TASKS =
Stream.concat(IMPLEMENTED_VERIFICATION_TASKS.stream(), PLANNED_VERIFICATION_TASKS.stream())
.collect(Collectors.toUnmodifiableSet());
private static final Set<String> FORBIDDEN_EXTENSION_TOKENS =
Set.of(
"consumer",
@@ -128,9 +154,14 @@ class MessagingCapabilityRegistryContractTest {
assertThat(card.get("settingsDigest")).isEqualTo("");
}
assertNonEmptyStringList(card, "evidenceTasks");
boolean implementedCandidate = "implemented-candidate".equals(card.get("maturity"));
assertThat(stringList(card, "evidenceTasks"))
.as("evidenceTasks on %s must reference declared root tasks", card.get("cardId"))
.allMatch(REQUIRED_VERIFICATION_TASKS::contains);
.as("evidenceTasks on %s must name a known qualification task", card.get("cardId"))
.allMatch(
task ->
IMPLEMENTED_VERIFICATION_TASKS.contains(task)
|| (!implementedCandidate && PLANNED_VERIFICATION_TASKS.contains(task)),
"an implemented-candidate card may only name a task that exists");
assertNonEmptyStringList(card, "requiredScenarios");
assertNonEmptyStringList(card, "runbookIds");
}
@@ -186,8 +217,7 @@ class MessagingCapabilityRegistryContractTest {
List<String> requiredEvidenceTasks = stringList(profile, "requiredEvidenceTasks");
assertListIntegrity(
requiredEvidenceTasks, "requiredEvidenceTasks", profile.get("releaseProfileId"));
assertThat(requiredEvidenceTasks)
.containsExactlyInAnyOrderElementsOf(REQUIRED_VERIFICATION_TASKS);
assertThat(requiredEvidenceTasks).containsExactlyInAnyOrderElementsOf(ALL_VERIFICATION_TASKS);
assertNonEmptyStringList(profile, "requiredScenarios");
assertNonEmptyStringList(profile, "runbookIds");
@@ -278,7 +308,7 @@ class MessagingCapabilityRegistryContractTest {
Map<String, Object> producerTask = map(properties, "producerTask");
assertThat(producerTask).containsEntry("type", "string");
assertThat(stringList(producerTask, "enum"))
.containsExactlyInAnyOrderElementsOf(REQUIRED_VERIFICATION_TASKS);
.containsExactlyInAnyOrderElementsOf(ALL_VERIFICATION_TASKS);
assertArrayOfReference(
map(properties, "scenarioIds"), "#/$defs/identifier", true, Integer.valueOf(1));
@@ -323,55 +353,53 @@ class MessagingCapabilityRegistryContractTest {
}
@Test
void rootBuildDeclaresEveryFailClosedVerificationTaskThroughTheSharedGuard() throws Exception {
String build = Files.readString(repositorySrcRoot().resolve("build.gradle"));
// This test protects a contract: every skeleton routes through the shared guard, and that guard
// fails closed. The wording the guard happens to use is not the contract.
void noUnimplementedQualificationIsRegisteredAsAGradleTask() throws Exception {
// The inverse of the assertion that used to be here.
//
// It used to assert six individual message fragments from the guard's body. Those fragments
// belonged to ~45 lines of evidence validation whose result was discarded, because the guard
// threw unconditionally either way. When that dead validation was removed the implementation was
// fine and this test broke — the test was pinning source text, not behaviour, which is how a
// guard stops being a guard and becomes a reason not to touch the file.
assertThat(build)
.contains(
"messagingVerificationSkeletons.each",
"tasks.register(taskName)",
"messagingFailClosedEvidenceGuard(taskName, evidencePaths)");
// The property worth pinning: the guard throws. If it is ever changed to report and continue,
// every messaging R2 skeleton would start passing without a qualification producer existing.
int guardStart = build.indexOf("Closure<Void> messagingFailClosedEvidenceGuard");
assertThat(guardStart).as("the shared guard closure must exist").isNotNegative();
int guardEnd = build.indexOf("\n}", guardStart);
assertThat(guardEnd).as("the shared guard closure must be terminated").isGreaterThan(guardStart);
assertThat(build.substring(guardStart, guardEnd))
.as("the shared guard must fail closed rather than report and continue")
.contains("throw new GradleException")
.contains("FAIL_CLOSED");
for (String taskName : REQUIRED_VERIFICATION_TASKS) {
assertThat(build).contains("'" + taskName + "'");
// This test previously required the root build to register all nine R2 skeleton tasks and to
// route them through a guard that threw `FAIL_CLOSED`. It was pinning the existence of tasks
// that could not succeed: a card said "my evidence comes from verifyMessagingSecurityR2", the
// task existed, and running it always failed — so the registry looked wired to a build that
// could substantiate nothing. Registering a task for work with no producer does not make the
// absence safer; it makes the absence look like a gate.
//
// What is worth holding is that they are NOT registered, so nobody wires a release lane to one.
for (Path script : qualificationScripts()) {
String text = Files.readString(script);
for (String planned : PLANNED_VERIFICATION_TASKS) {
assertThat(text)
.as("%s must not register the unimplemented task %s", script.getFileName(), planned)
.doesNotContain("tasks.register('" + planned + "')")
.doesNotContain("tasks.register(\"" + planned + "\")");
}
}
}
@Test
void rootBuildSchemaValidatesEvidenceAndDeterministicallyLeavesCombinedEvidenceLast()
throws Exception {
String build = Files.readString(repositorySrcRoot().resolve("build.gradle"));
void messagingQualificationSchemaValidatesTheEvidenceItWrites() throws Exception {
// Qualification lives in gradle/qualification/messaging-qualification.gradle now, not in the
// root build file. The property held here is the one that makes the manifest evidence rather
// than a file: each producer is finalized by a JSON Schema validation of the exact bytes it
// wrote, and the combined producer runs after the JSON-schema one.
String qualification =
Files.readString(
repositorySrcRoot().resolve("gradle/qualification/messaging-qualification.gradle"));
assertThat(build)
assertThat(qualification)
.contains(
"MessagingEvidenceManifestSchemaValidator",
"validateMessagingJsonSchemaV1EvidenceManifestSchema",
"validateMessagingContractsEvidenceManifestSchema",
"verifyMessagingJsonSchemaV1.configure",
"finalizedBy validateMessagingJsonSchemaV1EvidenceManifestSchema",
"dependsOn validateMessagingJsonSchemaV1EvidenceManifestSchema",
"verifyMessagingContracts.configure",
"finalizedBy validateMessagingContractsEvidenceManifestSchema");
}
private static List<Path> qualificationScripts() throws Exception {
Path src = repositorySrcRoot();
return List.of(
src.resolve("build.gradle"),
src.resolve("gradle/qualification/messaging-qualification.gradle"));
}
private static Path requiredConfig(String relativePath) {
Path path = repositorySrcRoot().resolve("config/messaging").resolve(relativePath);
assertThat(path).as("required Messaging configuration %s", path).isRegularFile();
@@ -45,8 +45,7 @@ class ReleaseManifestTaskExistenceTest {
* form. It is a cheap guard against a manifest naming a task nobody wrote, not a substitute for
* asking Gradle.
*/
private static final Pattern REGISTERED_LANE =
Pattern.compile("\\blane\\(\\s*'([A-Za-z0-9_]+)'");
private static final Pattern REGISTERED_LANE = Pattern.compile("\\blane\\(\\s*'([A-Za-z0-9_]+)'");
/**
* Tasks the Java plugin supplies, which no build file registers explicitly.