refactor(build,ci): 현재 상태 검증을 걷어내고 불변조건만 남기는 검증 표면 축소
외부 리뷰("현재 상태를 유지하기 위한 검증이 너무 많고, 그 검증 자체를
다시 검증하는 구조까지 생겼다")를 설계 문서로 정리하고 코드로 반영한다.
설계·판단 근거는 docs/superpowers/specs/2026-09-16-verification-surface-reduction-design.md.
삭제
- .github/ci-gate-matrix.yml(1,025줄) + verify-gate-matrix.sh(568줄):
Gradle task graph와 workflow graph에 이미 있는 정보의 3중 복제
- verify-gradle-wrapper.sh(799줄): workflow 바이트 해시 잠금.
wrapper 검증은 gradle/actions/wrapper-validation(full SHA 핀)에 위임
- DeveloperExperienceContractTest 등의 CI YAML mutation 테스트:
애플리케이션 test suite가 GitHub Actions YAML 파서를 검증하던 계층 역전
- 문서 drift 파서: verifyReadmeCommands, verifyRunbookReferences,
verifyDocumentedLeafCount, verifyTestSourceSetRegistry
- 빈 레지스트리를 지키던 커스텀 YAML 파서: verifyTrivyignore,
verifyQuarantineSunset, flaky-quarantine.yaml
- verifyConfigurationPropertiesProcessor, verifyOneTypePerFile:
각각 ca.spring-config convention과 Checkstyle OneTopLevelClass가 대체
- 정상 입력으로도 성공할 수 없던 messaging always-fail task
- ModuleRegistry의 JSON 필드 집합 정확 일치, sample-portfolio negative guard
이동
- java/quality/spring 공통 설정을 configure(subprojects) 블록에서
ca.java-conventions / ca.quality-conventions / ca.java-library /
ca.spring-library convention plugin으로
- 아키텍처 검증을 ca.architecture로, JPA·messaging qualification을
gradle/qualification/ 아래로, verifyEnvKeys를 :app-bootstrap 소유로
완화
- Git revision은 releaseCheck·아카이브 생성에서만 요구. 일반 빌드는 SNAPSHOT
- SpotBugs/FindSecBugs는 로컬 check에서 빼고 qualityCheck 레인으로
task 계층
- leaf check는 그 leaf만. architectureCheck / qualityCheck /
configContractCheck / integrationCheck / ci / releaseCheck로 이름 분리
CI
- _reusable-gradle.yml 신규. checkout + wrapper validation + JDK/캐시 공통화
- fileserver-release.yml -> fileserver-certification.yml (CD가 아니라 certification)
- GitHub Actions = CI + artifact, Argo CD = CD 경계를 docs/ci-cd/boundary.md로 고정
순증감 +3,274 / -7,483.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
d00c76241c
commit
ef947e5bb0
@@ -1,5 +1,8 @@
|
||||
// Application entry point. Wires the default runtime module set and runs Spring Boot.
|
||||
// Optional leaves require an explicit registry allowance plus a composition-root dependency.
|
||||
apply plugin: 'ca.spring-library'
|
||||
apply plugin: 'ca.spring-config'
|
||||
|
||||
apply plugin: 'org.springframework.boot'
|
||||
|
||||
|
||||
@@ -116,7 +119,6 @@ dependencies {
|
||||
implementation project(':shared-contract')
|
||||
implementation 'org.springframework.boot:spring-boot-starter'
|
||||
implementation 'org.springframework.boot:spring-boot-starter-validation'
|
||||
annotationProcessor 'org.springframework.boot:spring-boot-configuration-processor'
|
||||
implementation libs.spring.dotenv
|
||||
// Boot 4 Flyway API/autoconfiguration: the composition root drives startup migration
|
||||
// (MigrationStartupConfig). See README.
|
||||
@@ -415,3 +417,8 @@ tasks.register('runtimeClasspathManifest') {
|
||||
tasks.named('test') {
|
||||
dependsOn tasks.named('runtimeClasspathManifest')
|
||||
}
|
||||
|
||||
// The environment configuration contract. Owned here because the question it answers — what must a
|
||||
// deployment of this application be given — is the composition root's, and reached through
|
||||
// `configContractCheck` rather than through every leaf's `check`.
|
||||
apply from: rootProject.file('gradle/config-contract.gradle')
|
||||
|
||||
+32
-10
@@ -18,13 +18,24 @@ final class BuildVerificationPurityContractTest {
|
||||
private static final Path PUBLIC_PATH_SCRIPT =
|
||||
SOURCE_ROOT.resolve("gradle/public-path-snapshot.gradle");
|
||||
|
||||
/**
|
||||
* What the renderer produces for the fixture's {@code security.yml}.
|
||||
*
|
||||
* <p>These five tests were red on main. The snapshot's input moved from {@code src/.env} to the
|
||||
* committed {@code app-bootstrap/src/main/resources/config/security.yml} — because {@code
|
||||
* src/.env*} is gitignored, so the old gate took its expected value from a file no CI checkout
|
||||
* has — but the fixture below kept writing a {@code .env}, and this constant kept the header and
|
||||
* the paths that file used to produce. The script was right and its test was describing the
|
||||
* previous contract.
|
||||
*/
|
||||
private static final String CANONICAL_PUBLIC_PATH_SNAPSHOT =
|
||||
"""
|
||||
# feature-security-operational-baseline D5 — deny-by-default public path snapshot.
|
||||
# SSOT: SECURITY_PUBLIC_PATHS (src/.env) -> SecurityConfig permitAll(); anyRequest authenticated.
|
||||
# SSOT: ca-skeleton.security.public-paths default in app-bootstrap/src/main/resources/config/security.yml
|
||||
# -> SecurityConfig permitAll(); anyRequest authenticated. An operator's own SECURITY_PUBLIC_PATHS
|
||||
# overrides it at run time and is outside this snapshot.
|
||||
# Update only after review with: ./gradlew updatePublicPathSnapshot -PapprovePublicPathChange
|
||||
/api/healthcheck
|
||||
/api/public
|
||||
/v1/healthcheck
|
||||
""";
|
||||
|
||||
@Test
|
||||
@@ -71,14 +82,14 @@ final class BuildVerificationPurityContractTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
void missingPublicPathEnvironmentFailsWithoutCreatingSnapshot(@TempDir Path temporaryDirectory)
|
||||
void missingPublicPathConfigurationFailsWithoutCreatingSnapshot(@TempDir Path temporaryDirectory)
|
||||
throws IOException {
|
||||
PublicPathFixture fixture = publicPathFixture(temporaryDirectory);
|
||||
Files.delete(fixture.environment());
|
||||
Files.delete(fixture.securityConfiguration());
|
||||
|
||||
BuildResult result = runAndFail(fixture.projectDirectory(), "verifyPublicPathSnapshot");
|
||||
|
||||
assertThat(result.getOutput()).contains("missing public-path environment file");
|
||||
assertThat(result.getOutput()).contains("missing public-path security configuration");
|
||||
assertThat(fixture.snapshot()).doesNotExist();
|
||||
}
|
||||
|
||||
@@ -210,13 +221,23 @@ final class BuildVerificationPurityContractTest {
|
||||
projectDirectory.resolve("build.gradle"),
|
||||
"apply from: uri('%s')\n".formatted(PUBLIC_PATH_SCRIPT.toUri().toASCIIString()),
|
||||
UTF_8);
|
||||
// The committed binding default, spelled exactly as the real file spells it — nested Spring
|
||||
// placeholders and all, because unwinding them to `/v1/healthcheck` is what the renderer does
|
||||
// and therefore what these tests are about.
|
||||
Path securityConfig =
|
||||
projectDirectory.resolve("app-bootstrap/src/main/resources/config/security.yml");
|
||||
Files.createDirectories(securityConfig.getParent());
|
||||
Files.writeString(
|
||||
projectDirectory.resolve(".env"),
|
||||
"SECURITY_PUBLIC_PATHS=/api/public, /api/healthcheck\n",
|
||||
securityConfig,
|
||||
"""
|
||||
ca-skeleton:
|
||||
security:
|
||||
public-paths: ${SECURITY_PUBLIC_PATHS:${PRESENTATION_API_BASE_PATH:/v1}/healthcheck}
|
||||
""",
|
||||
UTF_8);
|
||||
return new PublicPathFixture(
|
||||
projectDirectory,
|
||||
projectDirectory.resolve(".env"),
|
||||
securityConfig,
|
||||
repositoryDirectory.resolve("docs/security/public-paths-snapshot.txt"));
|
||||
}
|
||||
|
||||
@@ -254,5 +275,6 @@ final class BuildVerificationPurityContractTest {
|
||||
private record ArchiveFixture(
|
||||
Path projectDirectory, Path staleArchive, Path currentArchive, Path nonmatchingArchive) {}
|
||||
|
||||
private record PublicPathFixture(Path projectDirectory, Path environment, Path snapshot) {}
|
||||
private record PublicPathFixture(
|
||||
Path projectDirectory, Path securityConfiguration, Path snapshot) {}
|
||||
}
|
||||
|
||||
+2
-2
@@ -110,8 +110,8 @@ class FileserverPlatformEnvRoundTripTest {
|
||||
* <p>A copy in the test resources would drift from the shipped contract, which is the drift this
|
||||
* test exists to catch — so it reads the repository file rather than a fixture.
|
||||
*
|
||||
* <p>It reads {@code .env.example}, not {@code .env}. {@code .gitignore} states the rule: the real
|
||||
* {@code .env} is operator input and the examples beside it are the tracked contract. A real
|
||||
* <p>It reads {@code .env.example}, not {@code .env}. {@code .gitignore} states the rule: the
|
||||
* real {@code .env} is operator input and the examples beside it are the tracked contract. A real
|
||||
* {@code .env} exists only on a developer machine, so pointing this test at it made the test pass
|
||||
* locally and fail on every clean checkout — which is where CI runs.
|
||||
*/
|
||||
|
||||
+38
-801
@@ -2,63 +2,32 @@ package dev.caskeleton.bootstrap.contract;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
import dev.caskeleton.bootstrap.contract.support.RepositoryContractResources;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
import java.util.concurrent.ExecutionException;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.TimeoutException;
|
||||
import java.util.regex.Pattern;
|
||||
import java.nio.file.Paths;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.yaml.snakeyaml.LoaderOptions;
|
||||
import org.yaml.snakeyaml.Yaml;
|
||||
import org.yaml.snakeyaml.constructor.SafeConstructor;
|
||||
|
||||
/**
|
||||
* The opt-in inbound transports qualify without skips, and the root only aggregates.
|
||||
*
|
||||
* <p>This class used to be 860 lines, of which about 800 tested {@code
|
||||
* .github/scripts/verify-gate-matrix.sh}: eighteen tests that wrote mutated copies of {@code
|
||||
* .github/ci-gate-matrix.yml} into a {@code @TempDir} — a duplicate gate id, a row naming a job
|
||||
* that does not exist, a {@code ref} containing a regex metacharacter, a release-blocking row no
|
||||
* release gate required — and ran the shell script against them to check that it refused each one.
|
||||
*
|
||||
* <p>That is an application's test suite testing a YAML register of CI controls, through a bash
|
||||
* validator, in a fixture repository. The register itself duplicated what the Gradle task graph and
|
||||
* the GitHub Actions job graph already said, so the whole structure existed to keep three
|
||||
* descriptions of one fact equal to each other. All three layers are gone.
|
||||
*
|
||||
* <p>What survives is the part that was about this repository's transports rather than about its CI
|
||||
* register: each inbound transport leaf owns its qualification lane and names the wire classes that
|
||||
* lane must execute, and the root task only aggregates them.
|
||||
*/
|
||||
class ConditionalTransportQualificationContractTest {
|
||||
|
||||
/**
|
||||
* Filler gates a well-formed fixture carries beside its one target gate.
|
||||
*
|
||||
* <p>Two, and the number does not matter. It used to be one less than a gate count this test and
|
||||
* the validator both hard-coded, so every fixture had to be built to that size or it failed on
|
||||
* the count rather than on whatever the test was about. Neither pins a count now; the fillers
|
||||
* remain only so the duplicate-id and shape cases have a second row to mutate.
|
||||
*/
|
||||
private static final int FILLER_GATE_COUNT = 2;
|
||||
|
||||
/**
|
||||
* The release gate {@code release_blocking: true} is measured against.
|
||||
*
|
||||
* <p>Named here and in {@code verify-gate-matrix.sh} rather than inferred from a filename: a
|
||||
* workflow called "release" is a naming convention, and this job's {@code needs:} is a fact.
|
||||
*/
|
||||
private static final String RELEASE_GATE_WORKFLOW = "ci-quality-gates.yml";
|
||||
|
||||
private static final String RELEASE_GATE_JOB = "release-gate";
|
||||
|
||||
private static final Pattern WHITESPACE = Pattern.compile("\\s+");
|
||||
|
||||
private static final Duration VALIDATOR_TIMEOUT = Duration.ofSeconds(10);
|
||||
private static final Set<String> EXPECTED_GATE_FIELDS =
|
||||
Set.of("id", "release_blocking", "mechanism", "ref", "workflow", "job", "execution");
|
||||
private static final Set<String> ALLOWED_MECHANISMS =
|
||||
Set.of(
|
||||
"gradle-custom-task",
|
||||
"gradle-plugin-task",
|
||||
"contract-test",
|
||||
"workflow-job",
|
||||
"delegated-pending");
|
||||
private static final Set<String> ALLOWED_EXECUTIONS = Set.of("check", "explicit", "job");
|
||||
|
||||
@Test
|
||||
void ownerQualificationsNameEveryRequiredWireClassAndRootOnlyAggregates() throws IOException {
|
||||
Path root = repositoryRoot();
|
||||
@@ -86,6 +55,10 @@ class ConditionalTransportQualificationContractTest {
|
||||
.contains("registerStrictQualificationTest")
|
||||
.contains(
|
||||
"dev.caskeleton.adapter.inbound.websocket.stomp.WebSocketBoundaryQualificationTest");
|
||||
|
||||
// The root aggregates and knows no test class name. A root that named the wire classes would be
|
||||
// a second place to update when a leaf renames one, and the leaf's own lane is the one that
|
||||
// fails closed on a class it cannot discover.
|
||||
assertThat(rootBuild)
|
||||
.contains("tasks.register('conditionalTransportQualification')")
|
||||
.contains(":adapter:inbound:graphql:graphqlTransportQualificationTest")
|
||||
@@ -98,763 +71,27 @@ class ConditionalTransportQualificationContractTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
void releaseBlockingQualityJobAndGateMatrixInvokeTheAggregate() throws IOException {
|
||||
Path root = repositoryRoot();
|
||||
String workflow = Files.readString(root.resolve(".github/workflows/ci-quality-gates.yml"));
|
||||
String matrix = Files.readString(root.resolve(".github/ci-gate-matrix.yml"));
|
||||
String validator = Files.readString(root.resolve(".github/scripts/verify-gate-matrix.sh"));
|
||||
void theReleaseBlockingQualityJobInvokesTheAggregate() throws IOException {
|
||||
String workflow =
|
||||
Files.readString(repositoryRoot().resolve(".github/workflows/ci-quality-gates.yml"));
|
||||
|
||||
assertThat(workflow)
|
||||
.contains("./gradlew conditionalTransportQualification")
|
||||
// `--continue` would let a failing transport lane be reported alongside a green overall
|
||||
// step, which is the one thing a no-skip qualification may not do.
|
||||
.doesNotContain("conditionalTransportQualification --continue");
|
||||
assertThat(matrix)
|
||||
.contains("id: conditional-transport-qualification")
|
||||
.contains("ref: conditionalTransportQualification")
|
||||
.contains("job: quality-gates")
|
||||
.contains("execution: explicit");
|
||||
// The count literal is gone and must stay gone. While it existed, adding a control meant
|
||||
// editing the guard whose stated purpose was to stop the matrix changing, and it caught
|
||||
// nothing the per-row rules do not: a row whose task, workflow or job has disappeared fails
|
||||
// below at any matrix size.
|
||||
assertThat(validator)
|
||||
.doesNotContain("EXPECTED_GATE_COUNT")
|
||||
.contains("matrix declares no gates");
|
||||
}
|
||||
|
||||
@Test
|
||||
void explicitFixtureRootRunsTheActualValidatorOutsideItsScriptLocation(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
Path fixtureRoot =
|
||||
writeFixture(
|
||||
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
|
||||
|
||||
ScriptResult result = runValidator(fixtureRoot);
|
||||
|
||||
int fixtureGateCount = FILLER_GATE_COUNT + 1;
|
||||
assertThat(result.exitCode()).isZero();
|
||||
assertThat(result.output())
|
||||
.contains(
|
||||
"gate-matrix-lint: "
|
||||
+ fixtureGateCount
|
||||
+ " gates, "
|
||||
+ fixtureGateCount
|
||||
+ " verified, 0 delegated-pending")
|
||||
.contains("gate-matrix-lint: OK");
|
||||
}
|
||||
|
||||
@Test
|
||||
void defaultModeValidatesTheRealRepositoryAndRetainsItsLocationGuard() throws IOException {
|
||||
Path root = repositoryRoot();
|
||||
String validator = Files.readString(validatorPath());
|
||||
|
||||
ScriptResult result = runScript(root, List.of());
|
||||
|
||||
assertThat(result.exitCode()).isZero();
|
||||
assertThat(result.output()).contains("gate-matrix-lint: OK");
|
||||
assertThat(validator)
|
||||
.contains("EXPECTED_SCRIPT_DIR")
|
||||
.contains("script location must be repository .github/scripts directory");
|
||||
}
|
||||
|
||||
@Test
|
||||
void defaultModeRejectsARelocatedScript(@TempDir Path tempDir) throws IOException {
|
||||
Path fixtureRoot =
|
||||
writeFixture(
|
||||
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
|
||||
Files.createDirectories(fixtureRoot.resolve(".github/scripts"));
|
||||
Path relocatedScript = fixtureRoot.resolve("relocated-verify-gate-matrix.sh");
|
||||
Files.copy(validatorPath(), relocatedScript);
|
||||
ScriptResult gitInit =
|
||||
runCommand(fixtureRoot, List.of("git", "init", "--quiet", fixtureRoot.toString()));
|
||||
assertThat(gitInit.exitCode()).isZero();
|
||||
|
||||
ScriptResult result = runScriptAt(relocatedScript, fixtureRoot, List.of());
|
||||
|
||||
assertThat(result.exitCode()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains("script location must be repository .github/scripts directory");
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatorRejectsMoreThanOneRepositoryRootArgument(@TempDir Path tempDir) throws IOException {
|
||||
Path fixtureRoot =
|
||||
writeFixture(
|
||||
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
|
||||
|
||||
ScriptResult result = runScript(fixtureRoot, List.of(fixtureRoot.toString(), "extra"));
|
||||
|
||||
assertThat(result.exitCode()).isNotZero();
|
||||
assertThat(result.output()).contains("expected zero arguments or one repository root");
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatorRejectsMissingRepositoryRootAndMatrix(@TempDir Path tempDir) throws IOException {
|
||||
Path missingRoot = tempDir.resolve("missing-root");
|
||||
ScriptResult missingRootResult = runScript(tempDir, List.of(missingRoot.toString()));
|
||||
assertThat(missingRootResult.exitCode()).isNotZero();
|
||||
assertThat(missingRootResult.output())
|
||||
.contains("repository root is not a directory: " + missingRoot);
|
||||
|
||||
Path emptyRoot = tempDir.resolve("empty-root");
|
||||
Files.createDirectories(emptyRoot);
|
||||
ScriptResult missingMatrixResult = runValidator(emptyRoot);
|
||||
assertThat(missingMatrixResult.exitCode()).isNotZero();
|
||||
assertThat(missingMatrixResult.output())
|
||||
.contains("missing " + emptyRoot.resolve(".github/ci-gate-matrix.yml"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void deceptiveStepNameAndEchoDoNotSatisfyExplicitExecution(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
Path fixtureRoot =
|
||||
writeFixture(
|
||||
tempDir.resolve("fixture"), "./gradlew targetGate", "echo disabled", FILLER_GATE_COUNT);
|
||||
|
||||
ScriptResult result = runValidator(fixtureRoot);
|
||||
|
||||
assertRejectedAsNotExplicit(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void differentProjectTaskWithTheSameNameDoesNotSatisfyExplicitExecution(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
Path fixtureRoot =
|
||||
writeFixture(
|
||||
tempDir.resolve("fixture"),
|
||||
"Run target",
|
||||
"./gradlew :other:targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
|
||||
ScriptResult result = runValidator(fixtureRoot);
|
||||
|
||||
assertRejectedAsNotExplicit(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void shorthandRunStepSatisfiesExplicitExecution(@TempDir Path tempDir) throws IOException {
|
||||
Path fixtureRoot =
|
||||
writeFixture(
|
||||
tempDir.resolve("fixture"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
|
||||
replace(
|
||||
fixtureRoot.resolve(".github/workflows/fixture.yml"),
|
||||
" - name: Run target\n run: ./gradlew targetGate\n",
|
||||
" - run: ./gradlew targetGate\n");
|
||||
|
||||
ScriptResult result = runValidator(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).isZero();
|
||||
assertThat(result.output()).contains("gate-matrix-lint: OK");
|
||||
}
|
||||
|
||||
@Test
|
||||
void leafCheckDoesNotSatisfyTheRequiredRootCheck(@TempDir Path tempDir) throws IOException {
|
||||
Path fixtureRoot =
|
||||
writeCheckFixture(tempDir.resolve("fixture"), "./gradlew :app-bootstrap:check", true);
|
||||
|
||||
ScriptResult result = runValidator(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains("gate 'target-gate' expects Gradle check in job 'target-job'");
|
||||
}
|
||||
|
||||
@Test
|
||||
void suppressionAndNonExecutionArgumentsDoNotSatisfyExplicitExecution(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
List<String> rejectedCommands =
|
||||
List.of(
|
||||
"./gradlew targetGate --dry-run",
|
||||
"./gradlew targetGate -m",
|
||||
"./gradlew targetGate -x targetGate",
|
||||
"./gradlew targetGate --exclude-task targetGate",
|
||||
"./gradlew targetGate \"--dry-run\"",
|
||||
"./gradlew targetGate \\--dry-run",
|
||||
"./gradlew targetGate --help",
|
||||
"./gradlew targetGate --status");
|
||||
|
||||
for (int index = 0; index < rejectedCommands.size(); index++) {
|
||||
Path fixtureRoot =
|
||||
writeFixture(
|
||||
tempDir.resolve("fixture-" + index),
|
||||
"Run target",
|
||||
rejectedCommands.get(index),
|
||||
FILLER_GATE_COUNT);
|
||||
|
||||
ScriptResult result = runValidator(fixtureRoot);
|
||||
|
||||
assertThat(result.output()).as("command: %s", rejectedCommands.get(index)).isNotBlank();
|
||||
assertRejectedAsNotExplicit(result);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatorRejectsEmptyMatrixDuplicateIdUnregisteredTaskAndMissingJob(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
// A matrix of a particular size is not a property. A matrix of no gates is: the file exists,
|
||||
// the lint runs, and every per-row rule passes vacuously. That is the one thing the deleted
|
||||
// count literal protected, and it is kept.
|
||||
Path emptyMatrix =
|
||||
writeFixture(
|
||||
tempDir.resolve("empty-matrix"),
|
||||
"Run target",
|
||||
"./gradlew targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
Files.writeString(emptyMatrix.resolve(".github/ci-gate-matrix.yml"), "gates: []\n");
|
||||
ScriptResult emptyMatrixResult = runValidator(emptyMatrix);
|
||||
assertThat(emptyMatrixResult.exitCode()).isNotZero();
|
||||
assertThat(emptyMatrixResult.output()).contains("matrix declares no gates");
|
||||
|
||||
Path duplicateId =
|
||||
writeFixture(
|
||||
tempDir.resolve("duplicate-id"),
|
||||
"Run target",
|
||||
"./gradlew targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
replace(
|
||||
duplicateId.resolve(".github/ci-gate-matrix.yml"), "id: filler-gate-01", "id: target-gate");
|
||||
ScriptResult duplicateResult = runValidator(duplicateId);
|
||||
assertThat(duplicateResult.exitCode()).isNotZero();
|
||||
assertThat(duplicateResult.output()).contains("duplicate gate id 'target-gate'");
|
||||
|
||||
Path unregisteredTask =
|
||||
writeFixture(
|
||||
tempDir.resolve("unregistered-task"),
|
||||
"Run target",
|
||||
"./gradlew targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
Files.writeString(unregisteredTask.resolve("src/sample/build.gradle"), "plugins {}\n");
|
||||
ScriptResult unregisteredResult = runValidator(unregisteredTask);
|
||||
assertThat(unregisteredResult.exitCode()).isNotZero();
|
||||
assertThat(unregisteredResult.output())
|
||||
.contains("gate 'target-gate' references unregistered Gradle task 'targetGate'");
|
||||
|
||||
Path unrelatedName =
|
||||
writeFixture(
|
||||
tempDir.resolve("unrelated-name"),
|
||||
"Run target",
|
||||
"./gradlew targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
Files.writeString(
|
||||
unrelatedName.resolve("src/sample/build.gradle"),
|
||||
"someUnrelatedConfiguration {\n name: 'targetGate'\n}\n");
|
||||
ScriptResult unrelatedNameResult = runValidator(unrelatedName);
|
||||
assertThat(unrelatedNameResult.exitCode()).isNotZero();
|
||||
assertThat(unrelatedNameResult.output())
|
||||
.contains("gate 'target-gate' references unregistered Gradle task 'targetGate'");
|
||||
|
||||
Path missingJob =
|
||||
writeFixture(
|
||||
tempDir.resolve("missing-job"),
|
||||
"Run target",
|
||||
"./gradlew targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
replace(
|
||||
missingJob.resolve(".github/ci-gate-matrix.yml"), "job: target-job", "job: missing-job");
|
||||
ScriptResult missingJobResult = runValidator(missingJob);
|
||||
assertThat(missingJobResult.exitCode()).isNotZero();
|
||||
assertThat(missingJobResult.output())
|
||||
.contains("gate 'target-gate' references missing job 'missing-job' in 'fixture.yml'");
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatorRejectsUnsafeCustomTaskRefAndMissingCheckWiring(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
Path unsafeRef =
|
||||
writeFixture(
|
||||
tempDir.resolve("unsafe-ref"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
|
||||
replace(
|
||||
unsafeRef.resolve(".github/ci-gate-matrix.yml"), "ref: targetGate", "ref: targetGate.*");
|
||||
ScriptResult unsafeRefResult = runValidator(unsafeRef);
|
||||
assertThat(unsafeRefResult.exitCode()).isNotZero();
|
||||
assertThat(unsafeRefResult.output())
|
||||
.contains("gate 'target-gate' has unsafe Gradle custom task ref 'targetGate.*'");
|
||||
|
||||
Path missingWiring =
|
||||
writeCheckFixture(tempDir.resolve("missing-wiring"), "./gradlew check", false);
|
||||
ScriptResult missingWiringResult = runValidator(missingWiring);
|
||||
assertThat(missingWiringResult.exitCode()).isNotZero();
|
||||
assertThat(missingWiringResult.output())
|
||||
.contains("gate 'target-gate' task 'targetGate' exists but is not wired into Gradle check");
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatorDoesNotInterpretCustomTaskOrPluginRefsAsRegularExpressions(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
Path dottedTask =
|
||||
writeFixture(
|
||||
tempDir.resolve("dotted-task"), "Run target", "./gradlew foo.bar", FILLER_GATE_COUNT);
|
||||
replace(dottedTask.resolve(".github/ci-gate-matrix.yml"), "ref: targetGate", "ref: foo.bar");
|
||||
Files.writeString(dottedTask.resolve("src/sample/build.gradle"), "tasks.register('fooXbar')\n");
|
||||
ScriptResult dottedTaskResult = runValidator(dottedTask);
|
||||
assertThat(dottedTaskResult.exitCode()).isNotZero();
|
||||
assertThat(dottedTaskResult.output())
|
||||
.contains("gate 'target-gate' has unsafe Gradle custom task ref 'foo.bar'");
|
||||
|
||||
Path unsafePlugin =
|
||||
writeCheckFixture(tempDir.resolve("unsafe-plugin"), "./gradlew check", true);
|
||||
replace(
|
||||
unsafePlugin.resolve(".github/ci-gate-matrix.yml"),
|
||||
"mechanism: gradle-custom-task",
|
||||
"mechanism: gradle-plugin-task");
|
||||
replace(
|
||||
unsafePlugin.resolve(".github/ci-gate-matrix.yml"),
|
||||
"ref: targetGate",
|
||||
"ref: com.diffplug.*@spotlessCheck");
|
||||
Files.writeString(
|
||||
unsafePlugin.resolve("src/sample/build.gradle"),
|
||||
"plugins { id 'com.diffplug.unrelated' }\n");
|
||||
ScriptResult unsafePluginResult = runValidator(unsafePlugin);
|
||||
assertThat(unsafePluginResult.exitCode()).isNotZero();
|
||||
assertThat(unsafePluginResult.output())
|
||||
.contains(
|
||||
"gate 'target-gate' has unsafe Gradle plugin task ref "
|
||||
+ "'com.diffplug.*@spotlessCheck'");
|
||||
}
|
||||
|
||||
@Test
|
||||
void realGateMatrixHasTheExactSafeSchema() throws IOException {
|
||||
LoaderOptions options = new LoaderOptions();
|
||||
options.setAllowDuplicateKeys(false);
|
||||
options.setMaxAliasesForCollections(0);
|
||||
Object loaded =
|
||||
new Yaml(new SafeConstructor(options))
|
||||
.load(Files.readString(repositoryRoot().resolve(".github/ci-gate-matrix.yml")));
|
||||
|
||||
assertThat(loaded).isInstanceOf(Map.class);
|
||||
Map<?, ?> root = (Map<?, ?>) loaded;
|
||||
assertThat(root.keySet().stream().map(String::valueOf).toList()).containsExactly("gates");
|
||||
assertThat(root.get("gates")).isInstanceOf(List.class);
|
||||
List<?> gates = (List<?>) root.get("gates");
|
||||
// No expected size. A matrix that grew by a row is a registered control, not drift; what has
|
||||
// to hold is that every row is well-formed, and that is asserted below for all of them.
|
||||
assertThat(gates).isNotEmpty();
|
||||
|
||||
Set<String> ids = new LinkedHashSet<>();
|
||||
for (Object rawGate : gates) {
|
||||
assertThat(rawGate).isInstanceOf(Map.class);
|
||||
Map<?, ?> gate = (Map<?, ?>) rawGate;
|
||||
assertThat(gate).hasSize(EXPECTED_GATE_FIELDS.size());
|
||||
assertThat(gate.keySet().stream().map(String::valueOf).toList())
|
||||
.containsExactlyInAnyOrderElementsOf(EXPECTED_GATE_FIELDS);
|
||||
|
||||
String id = requireString(gate, "id");
|
||||
assertThat(id).matches("[a-z0-9]+(?:-[a-z0-9]+)*");
|
||||
assertThat(ids.add(id)).as("unique gate id: %s", id).isTrue();
|
||||
assertThat(requireString(gate, "mechanism")).isIn(ALLOWED_MECHANISMS);
|
||||
assertThat(requireString(gate, "execution")).isIn(ALLOWED_EXECUTIONS);
|
||||
assertThat(requireString(gate, "ref")).isNotBlank();
|
||||
assertThat(requireString(gate, "workflow")).endsWith(".yml");
|
||||
assertThat(requireString(gate, "job")).isNotBlank();
|
||||
|
||||
Object releaseBlocking = gate.get("release_blocking");
|
||||
assertThat(releaseBlocking).isInstanceOfAny(Boolean.class, String.class);
|
||||
String releaseBlockingValue = String.valueOf(releaseBlocking);
|
||||
assertThat(releaseBlockingValue).isIn("true", "false", "conditional");
|
||||
if (releaseBlocking instanceof String) {
|
||||
assertThat(releaseBlocking).isEqualTo("conditional");
|
||||
}
|
||||
}
|
||||
|
||||
Map<?, ?> posterGate =
|
||||
gates.stream()
|
||||
.map(Map.class::cast)
|
||||
.filter(gate -> "poster-image-migration".equals(gate.get("id")))
|
||||
.findFirst()
|
||||
.orElseThrow(() -> new AssertionError("missing Poster image migration gate"));
|
||||
assertThat(requireString(posterGate, "ref")).isEqualTo("posterImageMigrationTest");
|
||||
assertThat(requireString(posterGate, "workflow")).isEqualTo("object-storage-qualification.yml");
|
||||
assertThat(requireString(posterGate, "job")).isEqualTo("poster-image-v7-migration");
|
||||
assertThat(requireString(posterGate, "execution")).isEqualTo("explicit");
|
||||
}
|
||||
|
||||
@Test
|
||||
void everyGateNamesAWorkflowAndJobThatExist() throws IOException {
|
||||
Path root = repositoryRoot();
|
||||
|
||||
for (Map<?, ?> gate : realGates(root)) {
|
||||
String workflowName = requireString(gate, "workflow");
|
||||
Path workflowFile = root.resolve(".github/workflows").resolve(workflowName);
|
||||
assertThat(workflowFile).as("gate '%s' workflow", gate.get("id")).isRegularFile();
|
||||
Map<?, ?> jobs = requireMapValue(parseYamlMap(workflowFile), "jobs");
|
||||
assertThat(jobs.keySet().stream().map(String::valueOf).toList())
|
||||
.as("gate '%s' job in %s", gate.get("id"), workflowName)
|
||||
.contains(requireString(gate, "job"));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* {@code release_blocking: true} has to be a fact about the build, not a label.
|
||||
*
|
||||
* <p>It was read by nothing but an enum check, so a gate could claim to block a release that no
|
||||
* job anywhere waited on: the filesystem vulnerability scan was release_blocking and could be red
|
||||
* while the release gate reported green. A gate earns {@code true} by being required on a path a
|
||||
* release actually takes — the release gate itself, one of its {@code needs:}, a name in its
|
||||
* {@code REQUIRED_CHECKS}, or a job in a workflow that only runs on a release tag. Everything
|
||||
* else is {@code conditional}, which is what the enum is for.
|
||||
*/
|
||||
@Test
|
||||
void everyReleaseBlockingGateIsRequiredBySomeReleaseGate() throws IOException {
|
||||
Path root = repositoryRoot();
|
||||
Path releaseGateFile = root.resolve(".github/workflows").resolve(RELEASE_GATE_WORKFLOW);
|
||||
Map<?, ?> releaseGate =
|
||||
requireMapValue(requireMapValue(parseYamlMap(releaseGateFile), "jobs"), RELEASE_GATE_JOB);
|
||||
|
||||
Set<String> needs = new LinkedHashSet<>();
|
||||
needs.add(RELEASE_GATE_JOB);
|
||||
for (Object need : requireListValue(releaseGate, "needs")) {
|
||||
needs.add(String.valueOf(need));
|
||||
}
|
||||
Set<String> requiredChecks = new LinkedHashSet<>();
|
||||
for (Object rawStep : requireListValue(releaseGate, "steps")) {
|
||||
assertThat(rawStep).isInstanceOf(Map.class);
|
||||
Object stepEnvironment = ((Map<?, ?>) rawStep).get("env");
|
||||
if (!(stepEnvironment instanceof Map<?, ?> environment)) {
|
||||
continue;
|
||||
}
|
||||
Object declared = environment.get("REQUIRED_CHECKS");
|
||||
if (declared == null) {
|
||||
continue;
|
||||
}
|
||||
WHITESPACE
|
||||
.splitAsStream(String.valueOf(declared).trim())
|
||||
.filter(check -> !check.isBlank())
|
||||
.forEach(requiredChecks::add);
|
||||
}
|
||||
assertThat(needs).as("jobs the release gate waits on").hasSizeGreaterThan(1);
|
||||
assertThat(requiredChecks).as("cross-workflow checks the release gate requires").isNotEmpty();
|
||||
|
||||
for (Map<?, ?> gate : realGates(root)) {
|
||||
if (!"true".equals(String.valueOf(gate.get("release_blocking")))) {
|
||||
continue;
|
||||
}
|
||||
String workflowName = requireString(gate, "workflow");
|
||||
String job = requireString(gate, "job");
|
||||
boolean enforced =
|
||||
(RELEASE_GATE_WORKFLOW.equals(workflowName) && needs.contains(job))
|
||||
|| requiredChecks.contains(job)
|
||||
|| runsOnlyForAReleaseTag(root.resolve(".github/workflows").resolve(workflowName));
|
||||
assertThat(enforced)
|
||||
.as(
|
||||
"gate '%s' is release_blocking: true, so %s::%s must be %s::%s, one of its needs, a"
|
||||
+ " name in its REQUIRED_CHECKS, or a job in a tag-triggered workflow",
|
||||
gate.get("id"), workflowName, job, RELEASE_GATE_WORKFLOW, RELEASE_GATE_JOB)
|
||||
.isTrue();
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatorRejectsAReleaseBlockingGateNoReleaseGateRequires(@TempDir Path tempDir)
|
||||
throws IOException {
|
||||
Path unrequired =
|
||||
writeFixture(
|
||||
tempDir.resolve("unrequired"), "Run target", "./gradlew targetGate", FILLER_GATE_COUNT);
|
||||
declareTargetGateReleaseBlocking(unrequired);
|
||||
ScriptResult unrequiredResult = runValidator(unrequired);
|
||||
assertThat(unrequiredResult.exitCode()).isNotZero();
|
||||
assertThat(unrequiredResult.output())
|
||||
.contains(
|
||||
"gate 'target-gate' is release_blocking: true but no release gate requires job"
|
||||
+ " 'target-job' in 'fixture.yml'");
|
||||
|
||||
Path tagTriggered =
|
||||
writeFixture(
|
||||
tempDir.resolve("tag-triggered"),
|
||||
"Run target",
|
||||
"./gradlew targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
declareTargetGateReleaseBlocking(tagTriggered);
|
||||
replaceLiteral(
|
||||
tagTriggered.resolve(".github/workflows/fixture.yml"),
|
||||
"on: [push]\n",
|
||||
"on:\n push:\n tags:\n - \"v*\"\n");
|
||||
ScriptResult tagTriggeredResult = runValidator(tagTriggered);
|
||||
assertThat(tagTriggeredResult.output()).contains("gate-matrix-lint: OK");
|
||||
assertThat(tagTriggeredResult.exitCode()).isZero();
|
||||
|
||||
Path requiredCheck =
|
||||
writeFixture(
|
||||
tempDir.resolve("required-check"),
|
||||
"Run target",
|
||||
"./gradlew targetGate",
|
||||
FILLER_GATE_COUNT);
|
||||
declareTargetGateReleaseBlocking(requiredCheck);
|
||||
Files.writeString(
|
||||
requiredCheck.resolve(".github/workflows").resolve(RELEASE_GATE_WORKFLOW),
|
||||
"""
|
||||
name: fixture-quality-gates
|
||||
on: [push]
|
||||
jobs:
|
||||
release-gate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Require the cross-workflow release-blocking checks
|
||||
env:
|
||||
REQUIRED_CHECKS: target-job
|
||||
run: echo required
|
||||
""");
|
||||
ScriptResult requiredCheckResult = runValidator(requiredCheck);
|
||||
assertThat(requiredCheckResult.output()).contains("gate-matrix-lint: OK");
|
||||
assertThat(requiredCheckResult.exitCode()).isZero();
|
||||
}
|
||||
|
||||
private static void declareTargetGateReleaseBlocking(Path fixtureRoot) throws IOException {
|
||||
replaceLiteral(
|
||||
fixtureRoot.resolve(".github/ci-gate-matrix.yml"),
|
||||
"release_blocking: false",
|
||||
"release_blocking: true");
|
||||
}
|
||||
|
||||
private static boolean runsOnlyForAReleaseTag(Path workflowFile) throws IOException {
|
||||
Map<?, ?> workflow = parseYamlMap(workflowFile);
|
||||
// A bare `on:` key is YAML 1.1, where it resolves to the boolean true rather than the string.
|
||||
Object triggers = workflow.get("on") != null ? workflow.get("on") : workflow.get(true);
|
||||
if (!(triggers instanceof Map<?, ?> triggerMap)) {
|
||||
return false;
|
||||
}
|
||||
Object push = triggerMap.get("push");
|
||||
return push instanceof Map<?, ?> pushTrigger && pushTrigger.get("tags") != null;
|
||||
}
|
||||
|
||||
private static List<Map<?, ?>> realGates(Path root) throws IOException {
|
||||
Object gates = parseYamlMap(root.resolve(".github/ci-gate-matrix.yml")).get("gates");
|
||||
assertThat(gates).isInstanceOf(List.class);
|
||||
List<Map<?, ?>> parsed = new ArrayList<>();
|
||||
for (Object gate : (List<?>) gates) {
|
||||
assertThat(gate).isInstanceOf(Map.class);
|
||||
parsed.add((Map<?, ?>) gate);
|
||||
}
|
||||
assertThat(parsed).isNotEmpty();
|
||||
return parsed;
|
||||
}
|
||||
|
||||
private static Map<?, ?> parseYamlMap(Path path) throws IOException {
|
||||
LoaderOptions options = new LoaderOptions();
|
||||
options.setAllowDuplicateKeys(false);
|
||||
options.setMaxAliasesForCollections(0);
|
||||
Object loaded = new Yaml(new SafeConstructor(options)).load(Files.readString(path));
|
||||
assertThat(loaded).as("%s", path).isInstanceOf(Map.class);
|
||||
return (Map<?, ?>) loaded;
|
||||
}
|
||||
|
||||
private static Map<?, ?> requireMapValue(Map<?, ?> parent, String key) {
|
||||
Object value = parent.get(key);
|
||||
assertThat(value).as("field %s", key).isInstanceOf(Map.class);
|
||||
return (Map<?, ?>) value;
|
||||
}
|
||||
|
||||
private static List<?> requireListValue(Map<?, ?> parent, String key) {
|
||||
Object value = parent.get(key);
|
||||
assertThat(value).as("field %s", key).isInstanceOf(List.class);
|
||||
return (List<?>) value;
|
||||
}
|
||||
|
||||
private static void replaceLiteral(Path path, String target, String replacement)
|
||||
throws IOException {
|
||||
String original = Files.readString(path);
|
||||
assertThat(original).contains(target);
|
||||
int index = original.indexOf(target);
|
||||
Files.writeString(
|
||||
path,
|
||||
original.substring(0, index) + replacement + original.substring(index + target.length()));
|
||||
}
|
||||
|
||||
private static void assertRejectedAsNotExplicit(ScriptResult result) {
|
||||
assertThat(result.exitCode()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains("gate 'target-gate' task 'targetGate' is not explicit in job 'target-job'");
|
||||
}
|
||||
|
||||
private static String requireString(Map<?, ?> gate, String field) {
|
||||
Object value = gate.get(field);
|
||||
assertThat(value).as("field %s", field).isInstanceOf(String.class);
|
||||
return (String) value;
|
||||
}
|
||||
|
||||
private static Path writeFixture(
|
||||
Path root, String targetStepName, String targetCommand, int fillerGateCount)
|
||||
throws IOException {
|
||||
Path workflows = root.resolve(".github/workflows");
|
||||
Files.createDirectories(workflows);
|
||||
Files.createDirectories(root.resolve("src/sample"));
|
||||
Files.writeString(root.resolve("src/sample/build.gradle"), "tasks.register('targetGate')\n");
|
||||
|
||||
StringBuilder workflow =
|
||||
new StringBuilder()
|
||||
.append("name: fixture\n")
|
||||
.append("on: [push]\n")
|
||||
.append("jobs:\n")
|
||||
.append(" target-job:\n")
|
||||
.append(" runs-on: ubuntu-latest\n")
|
||||
.append(" steps:\n")
|
||||
.append(" - name: ")
|
||||
.append(targetStepName)
|
||||
.append("\n")
|
||||
.append(" run: ")
|
||||
.append(targetCommand)
|
||||
.append("\n");
|
||||
for (int index = 1; index <= FILLER_GATE_COUNT; index++) {
|
||||
workflow
|
||||
.append(" filler-job-")
|
||||
.append(twoDigits(index))
|
||||
.append(":\n")
|
||||
.append(" runs-on: ubuntu-latest\n")
|
||||
.append(" steps:\n")
|
||||
.append(" - run: echo filler\n");
|
||||
}
|
||||
Files.writeString(workflows.resolve("fixture.yml"), workflow);
|
||||
|
||||
StringBuilder matrix =
|
||||
new StringBuilder()
|
||||
.append("gates:\n")
|
||||
.append(" - id: target-gate\n")
|
||||
.append(" release_blocking: false\n")
|
||||
.append(" mechanism: gradle-custom-task\n")
|
||||
.append(" ref: targetGate\n")
|
||||
.append(" workflow: fixture.yml\n")
|
||||
.append(" job: target-job\n")
|
||||
.append(" execution: explicit\n");
|
||||
for (int index = 1; index <= fillerGateCount; index++) {
|
||||
String suffix = twoDigits(index);
|
||||
matrix
|
||||
.append(" - id: filler-gate-")
|
||||
.append(suffix)
|
||||
.append("\n")
|
||||
.append(" release_blocking: false\n")
|
||||
.append(" mechanism: workflow-job\n")
|
||||
.append(" ref: filler-job-")
|
||||
.append(suffix)
|
||||
.append("\n")
|
||||
.append(" workflow: fixture.yml\n")
|
||||
.append(" job: filler-job-")
|
||||
.append(suffix)
|
||||
.append("\n")
|
||||
.append(" execution: job\n");
|
||||
}
|
||||
Files.writeString(root.resolve(".github/ci-gate-matrix.yml"), matrix);
|
||||
return root;
|
||||
}
|
||||
|
||||
private static Path writeCheckFixture(Path root, String checkCommand, boolean wireIntoCheck)
|
||||
throws IOException {
|
||||
Path fixtureRoot = writeFixture(root, "Run check", checkCommand, FILLER_GATE_COUNT);
|
||||
replace(
|
||||
fixtureRoot.resolve(".github/ci-gate-matrix.yml"),
|
||||
"execution: explicit",
|
||||
"execution: check");
|
||||
if (wireIntoCheck) {
|
||||
Files.writeString(
|
||||
fixtureRoot.resolve("src/sample/build.gradle"),
|
||||
"""
|
||||
tasks.register('targetGate')
|
||||
tasks.named('check') { dependsOn tasks.named('targetGate') }
|
||||
""");
|
||||
}
|
||||
return fixtureRoot;
|
||||
}
|
||||
|
||||
private static ScriptResult runValidator(Path fixtureRoot) throws IOException {
|
||||
return runScript(fixtureRoot, List.of(fixtureRoot.toString()));
|
||||
}
|
||||
|
||||
private static ScriptResult runScript(Path workingDirectory, List<String> arguments)
|
||||
throws IOException {
|
||||
return runScriptAt(validatorPath(), workingDirectory, arguments);
|
||||
}
|
||||
|
||||
private static ScriptResult runScriptAt(
|
||||
Path script, Path workingDirectory, List<String> arguments) throws IOException {
|
||||
List<String> command = new ArrayList<>();
|
||||
command.add("bash");
|
||||
command.add(script.toString());
|
||||
command.addAll(arguments);
|
||||
return runCommand(workingDirectory, command);
|
||||
}
|
||||
|
||||
private static ScriptResult runCommand(Path workingDirectory, List<String> command)
|
||||
throws IOException {
|
||||
Path outputFile = Files.createTempFile("gate-matrix-validator-", ".log");
|
||||
Process process = null;
|
||||
try {
|
||||
process =
|
||||
new ProcessBuilder(command)
|
||||
.directory(workingDirectory.toFile())
|
||||
.redirectErrorStream(true)
|
||||
.redirectOutput(outputFile.toFile())
|
||||
.start();
|
||||
boolean finished;
|
||||
try {
|
||||
finished = process.waitFor(VALIDATOR_TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);
|
||||
} catch (InterruptedException exception) {
|
||||
terminateAndWait(process);
|
||||
Thread.currentThread().interrupt();
|
||||
throw new AssertionError("interrupted while waiting for gate matrix validator", exception);
|
||||
}
|
||||
if (!finished) {
|
||||
terminateAndWait(process);
|
||||
throw new AssertionError("gate matrix validator exceeded " + VALIDATOR_TIMEOUT);
|
||||
}
|
||||
return new ScriptResult(process.exitValue(), Files.readString(outputFile));
|
||||
} finally {
|
||||
if (process != null && process.isAlive()) {
|
||||
terminateAndWait(process);
|
||||
}
|
||||
Files.deleteIfExists(outputFile);
|
||||
}
|
||||
}
|
||||
|
||||
private static void terminateAndWait(Process process) {
|
||||
List<ProcessHandle> descendants = process.descendants().toList();
|
||||
descendants.forEach(ProcessHandle::destroy);
|
||||
process.destroy();
|
||||
List<ProcessHandle> processTree = new ArrayList<>(descendants);
|
||||
processTree.add(process.toHandle());
|
||||
boolean interrupted = false;
|
||||
try {
|
||||
if (awaitExit(processTree)) {
|
||||
return;
|
||||
}
|
||||
} catch (InterruptedException exception) {
|
||||
interrupted = true;
|
||||
}
|
||||
processTree.stream().filter(ProcessHandle::isAlive).forEach(ProcessHandle::destroyForcibly);
|
||||
try {
|
||||
awaitExit(processTree);
|
||||
} catch (InterruptedException exception) {
|
||||
interrupted = true;
|
||||
}
|
||||
if (interrupted) {
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean awaitExit(List<ProcessHandle> processTree) throws InterruptedException {
|
||||
CompletableFuture<?>[] exits =
|
||||
processTree.stream().map(ProcessHandle::onExit).toArray(CompletableFuture<?>[]::new);
|
||||
try {
|
||||
CompletableFuture.allOf(exits).get(2, TimeUnit.SECONDS);
|
||||
return true;
|
||||
} catch (ExecutionException | TimeoutException exception) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private static void replace(Path path, String target, String replacement) throws IOException {
|
||||
String original = Files.readString(path);
|
||||
assertThat(original).contains(target);
|
||||
Files.writeString(path, original.replaceFirst(target, replacement));
|
||||
}
|
||||
|
||||
private static String twoDigits(int value) {
|
||||
return String.format("%02d", value);
|
||||
}
|
||||
|
||||
private static Path validatorPath() {
|
||||
return repositoryRoot().resolve(".github/scripts/verify-gate-matrix.sh");
|
||||
// The job that runs it has to be one the release gate waits on.
|
||||
assertThat(workflow).contains("\n quality-gates:\n").contains("\n - quality-gates\n");
|
||||
}
|
||||
|
||||
private static Path repositoryRoot() {
|
||||
return RepositoryContractResources.fromSystemProperty().repositoryRoot();
|
||||
for (Path path = Paths.get("").toAbsolutePath(); path != null; path = path.getParent()) {
|
||||
if (Files.isRegularFile(path.resolve("AGENTS.md"))
|
||||
&& Files.isRegularFile(path.resolve("src/settings.gradle"))) {
|
||||
return path;
|
||||
}
|
||||
}
|
||||
throw new IllegalStateException(
|
||||
"repository root not found from " + Paths.get("").toAbsolutePath());
|
||||
}
|
||||
|
||||
private record ScriptResult(int exitCode, String output) {}
|
||||
}
|
||||
|
||||
+86
-698
@@ -4,11 +4,11 @@ import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.nio.file.StandardCopyOption;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
@@ -25,55 +25,12 @@ import org.yaml.snakeyaml.constructor.SafeConstructor;
|
||||
class DeveloperExperienceContractTest {
|
||||
|
||||
private static final Path REPOSITORY_ROOT = repositoryRoot();
|
||||
private static final String VALIDATION_STEP =
|
||||
" - name: Validate Gradle wrapper\n"
|
||||
+ " id: gradle-wrapper-validation\n"
|
||||
+ " uses: gradle/actions/wrapper-validation@"
|
||||
+ "3f131e8634966bd73d06cc69884922b02e6faf92 # gradle/actions@v6\n";
|
||||
private static final String DEPENDENCY_SUBMISSION_ACTION =
|
||||
"gradle/actions/dependency-submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1";
|
||||
private static final String NAMED_DEPENDENCY_SUBMISSION_STEP =
|
||||
" - name: Submit the resolved Gradle dependency graph\n"
|
||||
+ " uses: "
|
||||
+ DEPENDENCY_SUBMISSION_ACTION
|
||||
+ " # gradle/actions@v4.4.4\n";
|
||||
private static final String DOUBLE_QUOTED_DEPENDENCY_SUBMISSION_STEP =
|
||||
" - name: Submit the resolved Gradle dependency graph\n"
|
||||
+ " uses: \""
|
||||
+ DEPENDENCY_SUBMISSION_ACTION
|
||||
+ "\"\n";
|
||||
private static final String SINGLE_QUOTED_DEPENDENCY_SUBMISSION_STEP =
|
||||
" - name: Submit the resolved Gradle dependency graph\n"
|
||||
+ " uses: '"
|
||||
+ DEPENDENCY_SUBMISSION_ACTION
|
||||
+ "'\n";
|
||||
private static final String HEX_ESCAPED_DEPENDENCY_SUBMISSION_STEP =
|
||||
" - name: Submit the resolved Gradle dependency graph\n"
|
||||
+ " uses: \"\\x67radle/actions/dependency-submission@"
|
||||
+ "748248ddd2a24f49513d8f472f81c3a07d4d50e1\"\n";
|
||||
private static final String CONTINUED_DEPENDENCY_SUBMISSION_STEP =
|
||||
"""
|
||||
- name: Submit the resolved Gradle dependency graph
|
||||
uses: "gradle/actions/dependency-\\
|
||||
submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1"
|
||||
""";
|
||||
private static final String ANONYMOUS_DEPENDENCY_SUBMISSION_STEP =
|
||||
" - uses: " + DEPENDENCY_SUBMISSION_ACTION + " # gradle/actions@v4.4.4\n";
|
||||
private static final String RUN_BLOCK_FAKE_VALIDATION_STEP =
|
||||
" - name: Pretend to validate the Gradle wrapper\n"
|
||||
+ " run: |\n"
|
||||
+ " uses: gradle/actions/wrapper-validation@"
|
||||
+ "3f131e8634966bd73d06cc69884922b02e6faf92 # gradle/actions@v6\n";
|
||||
private static final String BARE_ALWAYS_CONDITION = " if: always()\n";
|
||||
private static final String GUARDED_ALWAYS_CONDITION =
|
||||
" if: ${{ always() && steps.gradle-wrapper-validation.outcome == 'success' }}\n";
|
||||
private static final String GATE_MATRIX_STEP =
|
||||
"""
|
||||
- name: Verify the gate matrix against the repository
|
||||
run: bash .github/scripts/verify-gate-matrix.sh
|
||||
""";
|
||||
private static final String CANONICAL_WRAPPER_PROPERTIES_DIAGNOSTIC =
|
||||
"wrapper properties must match the exact canonical Gradle 9.0.0 eight-line contract";
|
||||
|
||||
// The workflow-mutation fixtures that stood here are gone with the tests that used them: a
|
||||
// hex-escaped `uses:`, a line-continued action reference, a `run:` block impersonating a
|
||||
// validation step, a bare `if: always()` versus the guarded form, a gate-matrix step. Every one
|
||||
// existed to prove that .github/scripts/verify-gradle-wrapper.sh could not be fooled by that
|
||||
// spelling — a YAML parser written in bash, tested from a Java application's test suite.
|
||||
|
||||
@Test
|
||||
void toolVersionsPinTemurin21() throws IOException {
|
||||
@@ -148,13 +105,15 @@ class DeveloperExperienceContractTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
void readmeCommandsAreVerifiedAndBootstrapIsTheFirstRunEntrypoint() throws IOException {
|
||||
String build = read("src/build.gradle");
|
||||
void readmeDocumentsOneFirstRunEntrypoint() throws IOException {
|
||||
// The `verifyReadmeCommands` half of this is gone with the task. That gate parsed the README's
|
||||
// ```bash blocks and resolved every `./gradlew`, `docker compose` and `make` token against the
|
||||
// real task graph, the real Compose files and the real Makefile — a hand-written Markdown
|
||||
// command parser, which made "what may be written in the README" a function of what the parser
|
||||
// could read. What is left is the property a first-run document has to have: one entrypoint,
|
||||
// named, with the check that tells you it worked.
|
||||
String readme = read("README.md");
|
||||
|
||||
assertThat(build)
|
||||
.contains("tasks.register('verifyReadmeCommands')")
|
||||
.contains("dependsOn rootProject.tasks.named('verifyReadmeCommands')");
|
||||
assertThat(readme)
|
||||
.contains("./gradlew bootstrap")
|
||||
.contains("GET /api/healthcheck")
|
||||
@@ -227,592 +186,41 @@ class DeveloperExperienceContractTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
void checkedInGradleWrapperAndEveryGradleJobPassTheExecutableContract() throws Exception {
|
||||
ProcessResult result = runGradleWrapperVerifier(REPOSITORY_ROOT);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isZero();
|
||||
assertThat(result.output()).contains("gradle-wrapper-contract: PASS");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsACorruptDistributionChecksum(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
|
||||
String content = Files.readString(properties);
|
||||
String corrupted =
|
||||
content.contains("distributionSha256Sum=")
|
||||
? content.replaceFirst(
|
||||
"(?m)^distributionSha256Sum=.*$", "distributionSha256Sum=corrupt")
|
||||
: content + System.lineSeparator() + "distributionSha256Sum=corrupt\n";
|
||||
Files.writeString(properties, corrupted);
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertCanonicalWrapperPropertiesRejected(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsWhitespaceDuplicateChecksumOverride(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
|
||||
Files.writeString(
|
||||
properties,
|
||||
Files.readString(properties) + " distributionSha256Sum=attacker-controlled-checksum\n");
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertCanonicalWrapperPropertiesRejected(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsColonDuplicateDistributionUrlOverride(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
|
||||
Files.writeString(
|
||||
properties,
|
||||
Files.readString(properties)
|
||||
+ "distributionUrl:https\\://attacker.invalid/gradle-9.0.0-bin.zip\n");
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertCanonicalWrapperPropertiesRejected(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsUnicodeEscapedChecksumOverride(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
|
||||
Files.writeString(
|
||||
properties,
|
||||
Files.readString(properties)
|
||||
+ "distribution\\u0053ha256Sum=attacker-controlled-checksum\n");
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertCanonicalWrapperPropertiesRejected(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsContinuedChecksumOverride(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path properties = fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties");
|
||||
Files.writeString(
|
||||
properties,
|
||||
Files.readString(properties) + "distributionSha256\\\nSum=attacker-controlled-checksum\n");
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertCanonicalWrapperPropertiesRejected(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsValidationMissingFromOneGradleJob(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
|
||||
String content = Files.readString(workflow);
|
||||
Files.writeString(workflow, removeFirstValidationStep(content));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertMissingGradleValidationDiagnostic(result, "quality-gates");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsRunBlockTextMasqueradingAsValidation(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(VALIDATION_STEP);
|
||||
Files.writeString(
|
||||
workflow,
|
||||
content.replaceFirst(
|
||||
Pattern.quote(VALIDATION_STEP),
|
||||
java.util.regex.Matcher.quoteReplacement(RUN_BLOCK_FAKE_VALIDATION_STEP)));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertMissingGradleValidationDiagnostic(result, "quality-gates");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsEncodedSingleLineGradleRun(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
|
||||
String content = Files.readString(workflow);
|
||||
String plainRun =
|
||||
" run: ./gradlew check verifyPublicPathSnapshot verifyDependencyLocks --warning-mode=fail --no-daemon --stacktrace\n";
|
||||
assertThat(content).contains(plainRun);
|
||||
String encoded = content.replace(plainRun, " run: \"\\x2e/gradlew check\"\n");
|
||||
Files.writeString(workflow, removeFirstValidationStep(encoded));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains(
|
||||
".github/workflows/ci-quality-gates.yml: job quality-gates has unsupported run scalar");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsBlockScalarUsesInNonGradleWorkflow(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
|
||||
String content = Files.readString(workflow);
|
||||
String action =
|
||||
" uses: lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede # lycheeverse/lychee-action@v2.0.2\n";
|
||||
String block =
|
||||
"""
|
||||
uses: |
|
||||
lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede
|
||||
""";
|
||||
assertThat(content).contains(action);
|
||||
Files.writeString(workflow, content.replace(action, block));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output()).contains("job lychee has unsupported uses scalar");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsAliasedUsesInNonGradleWorkflow(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
|
||||
String content = Files.readString(workflow);
|
||||
String action =
|
||||
" uses: lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede # lycheeverse/lychee-action@v2.0.2\n";
|
||||
assertThat(content).contains(action);
|
||||
String aliased =
|
||||
content
|
||||
.replace(
|
||||
"name: link-check\n",
|
||||
"name: link-check\n"
|
||||
+ "x-lychee-action: &lychee-action "
|
||||
+ "lycheeverse/lychee-action@7cd0af4c74a61395d455af97419279d86aafaede\n")
|
||||
.replace(action, " uses: *lychee-action\n");
|
||||
Files.writeString(workflow, aliased);
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output()).contains("job lychee has unsupported uses scalar");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsStepMergeKeyInNonGradleWorkflow(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/merge-injected-action.yml");
|
||||
Files.writeString(
|
||||
workflow,
|
||||
"""
|
||||
name: merge-injected-action
|
||||
on: workflow_dispatch
|
||||
x-step: &injected-step
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||
jobs:
|
||||
merge-job:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- <<: *injected-step
|
||||
""");
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output()).contains("job merge-job contains a forbidden step merge key");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsEncodedDependencyActionInFlowStyleStep(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
|
||||
String flowStep =
|
||||
" - {name: Submit the resolved Gradle dependency graph, uses: \"\\x67radle/"
|
||||
+ "actions/dependency-submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1\"}\n";
|
||||
Files.writeString(
|
||||
workflow,
|
||||
removeFirstValidationStep(content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, flowStep)));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains("job dependency-submission contains unsupported flow-style step syntax");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsFlowStyleJobsContainer(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/flow-jobs.yml");
|
||||
Files.writeString(
|
||||
workflow,
|
||||
"""
|
||||
name: flow-jobs
|
||||
on: workflow_dispatch
|
||||
jobs: {flow-job: {runs-on: ubuntu-latest, steps: [{uses: "\\x67radle/actions/dependency-submission@748248ddd2a24f49513d8f472f81c3a07d4d50e1"}]}}
|
||||
""");
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output()).contains("jobs container must use a canonical block mapping");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsAnchoredCustomGradleShellByWorkflowLock(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains("env:\n").contains(GATE_MATRIX_STEP);
|
||||
String mutated =
|
||||
content
|
||||
.replace(
|
||||
"env:\n",
|
||||
"x-gradle-shell: &gradle-shell bash -c './gradlew help; bash {0}'\n\nenv:\n")
|
||||
.replace(
|
||||
GATE_MATRIX_STEP,
|
||||
GATE_MATRIX_STEP.replace(
|
||||
" run:", " shell: *gradle-shell\n run:"));
|
||||
Files.writeString(workflow, mutated);
|
||||
|
||||
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsRepositoryRelativeGradlePathByWorkflowLock(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(GATE_MATRIX_STEP);
|
||||
String unvalidatedGradleStep =
|
||||
"""
|
||||
- name: Run unvalidated repository-relative Gradle
|
||||
run: src/gradlew help
|
||||
""";
|
||||
Files.writeString(
|
||||
workflow, content.replace(GATE_MATRIX_STEP, GATE_MATRIX_STEP + unvalidatedGradleStep));
|
||||
|
||||
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsEscapedDuplicateJobsByWorkflowLock(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
|
||||
Files.writeString(
|
||||
workflow,
|
||||
Files.readString(workflow)
|
||||
+ """
|
||||
|
||||
"jo\\x62s":
|
||||
hidden-gradle:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- "r\\x75n": "\\x2e/gradlew help"
|
||||
""");
|
||||
|
||||
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsAddedWorkflowByWorkflowLock(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Files.writeString(
|
||||
fixtureRoot.resolve(".github/workflows/unreviewed.yml"),
|
||||
"""
|
||||
name: unreviewed
|
||||
on: workflow_dispatch
|
||||
jobs:
|
||||
noop:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: No operation
|
||||
run: echo ok
|
||||
""");
|
||||
|
||||
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsRemovedWorkflowByWorkflowLock(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Files.delete(fixtureRoot.resolve(".github/workflows/link-check.yml"));
|
||||
|
||||
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsInnocuousWorkflowByteChangeByWorkflowLock(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
|
||||
Files.writeString(workflow, Files.readString(workflow) + "# unreviewed byte change\n");
|
||||
|
||||
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsWorkflowSymlinkReplacementByWorkflowLock(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/link-check.yml");
|
||||
Files.delete(workflow);
|
||||
Files.createSymbolicLink(workflow, Path.of("ci-quality-gates.yml"));
|
||||
|
||||
assertWorkflowLockRejected(runGradleWrapperVerifier(fixtureRoot));
|
||||
}
|
||||
|
||||
private static void assertWorkflowLockRejected(ProcessResult result) {
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output()).contains("workflow lock mismatch:");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsConditionalWrapperValidationStep(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
assertWrapperValidationControlFieldIsRejected(fixtureRoot, "if: ${{ false }}");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsContinueOnErrorWrapperValidationStep(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
assertWrapperValidationControlFieldIsRejected(fixtureRoot, "continue-on-error: true");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsWithFieldOnWrapperValidationStep(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
assertWrapperValidationControlFieldIsRejected(fixtureRoot, "with:");
|
||||
}
|
||||
|
||||
private static void assertWrapperValidationControlFieldIsRejected(
|
||||
Path fixtureRoot, String controlField) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/ci-quality-gates.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(VALIDATION_STEP);
|
||||
String controlledValidation = VALIDATION_STEP + " " + controlField + "\n";
|
||||
Files.writeString(
|
||||
workflow,
|
||||
content.replaceFirst(
|
||||
Pattern.quote(VALIDATION_STEP), Matcher.quoteReplacement(controlledValidation)));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains("wrapper validation step contains unsupported field: " + controlField);
|
||||
}
|
||||
|
||||
/**
|
||||
* A cleanup step that runs after a failed Gradle step is the one place a bare {@code always()} is
|
||||
* tempting, and it is exactly where it is unsafe: the wrapper validation may not have run, so the
|
||||
* sanitizer would execute an unverified wrapper. The verifier accepts the guarded form only.
|
||||
*
|
||||
* <p>The fixture is authored here rather than borrowed from a checked-in workflow. A test that
|
||||
* mutates whichever real workflow happens to carry the shape it needs stops compiling the day
|
||||
* that workflow is retired, which says nothing about the rule it was meant to prove.
|
||||
*/
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsBareAlwaysGradleSanitizer(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/evidence-sanitizer.yml");
|
||||
Files.writeString(workflow, sanitizerWorkflow(GUARDED_ALWAYS_CONDITION));
|
||||
|
||||
assertThat(runGradleWrapperVerifier(fixtureRoot).output())
|
||||
.as("the guarded form is the accepted shape and must not be reported")
|
||||
.doesNotContain("unsupported if condition");
|
||||
|
||||
Files.writeString(workflow, sanitizerWorkflow(BARE_ALWAYS_CONDITION));
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains("job redis-security has Gradle step with unsupported if condition: always()");
|
||||
}
|
||||
|
||||
/** A canonical Gradle job whose post-run sanitizer carries {@code condition}. */
|
||||
private static String sanitizerWorkflow(String condition) {
|
||||
return """
|
||||
name: evidence-sanitizer
|
||||
on: workflow_dispatch
|
||||
jobs:
|
||||
redis-security:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # actions/checkout@v4.2.2
|
||||
"""
|
||||
+ VALIDATION_STEP
|
||||
+ """
|
||||
- id: redis-tests
|
||||
working-directory: src
|
||||
run: ./gradlew :adapter:outbound:cache-redis:redisSecurityTest --no-daemon
|
||||
- id: redis-evidence-sanitizer
|
||||
"""
|
||||
+ condition
|
||||
+ """
|
||||
working-directory: src
|
||||
run: ./gradlew :adapter:outbound:cache-redis:verifyRedisEvidenceArtifactsForUpload
|
||||
""";
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsValidationMissingFromDependencySubmissionJob(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
|
||||
Files.writeString(workflow, removeFirstValidationStep(content));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertMissingDependencySubmissionValidationDiagnostic(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsUnguardedAnonymousDependencySubmissionStep(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
|
||||
String anonymous =
|
||||
content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, ANONYMOUS_DEPENDENCY_SUBMISSION_STEP);
|
||||
Files.writeString(workflow, removeFirstValidationStep(anonymous));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertMissingDependencySubmissionValidationDiagnostic(result);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsUnguardedDoubleQuotedDependencySubmissionStep(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
assertQuotedDependencySubmissionWithoutValidationIsRejected(
|
||||
fixtureRoot, DOUBLE_QUOTED_DEPENDENCY_SUBMISSION_STEP);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsUnguardedSingleQuotedDependencySubmissionStep(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
assertQuotedDependencySubmissionWithoutValidationIsRejected(
|
||||
fixtureRoot, SINGLE_QUOTED_DEPENDENCY_SUBMISSION_STEP);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsHexEscapedDependencySubmissionStep(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
assertEscapedDependencySubmissionWithoutValidationIsRejected(
|
||||
fixtureRoot, HEX_ESCAPED_DEPENDENCY_SUBMISSION_STEP);
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsContinuedDependencySubmissionStep(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
assertEscapedDependencySubmissionWithoutValidationIsRejected(
|
||||
fixtureRoot, CONTINUED_DEPENDENCY_SUBMISSION_STEP);
|
||||
}
|
||||
|
||||
private static void assertQuotedDependencySubmissionWithoutValidationIsRejected(
|
||||
Path fixtureRoot, String quotedStep) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
|
||||
String quoted = content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, quotedStep);
|
||||
Files.writeString(workflow, removeFirstValidationStep(quoted));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertMissingDependencySubmissionValidationDiagnostic(result);
|
||||
}
|
||||
|
||||
private static void assertMissingDependencySubmissionValidationDiagnostic(ProcessResult result) {
|
||||
assertMissingGradleValidationDiagnostic(result, "dependency-submission");
|
||||
}
|
||||
|
||||
private static void assertMissingGradleValidationDiagnostic(ProcessResult result, String job) {
|
||||
assertThat(result.output())
|
||||
.contains(
|
||||
"job " + job + " invokes Gradle without the exact pinned wrapper validation action");
|
||||
}
|
||||
|
||||
private static void assertEscapedDependencySubmissionWithoutValidationIsRejected(
|
||||
Path fixtureRoot, String escapedStep) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/dependency-vulnerability.yml");
|
||||
String content = Files.readString(workflow);
|
||||
assertThat(content).contains(NAMED_DEPENDENCY_SUBMISSION_STEP);
|
||||
String escaped = content.replace(NAMED_DEPENDENCY_SUBMISSION_STEP, escapedStep);
|
||||
Files.writeString(workflow, removeFirstValidationStep(escaped));
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains(
|
||||
".github/workflows/dependency-vulnerability.yml: job dependency-submission has unsupported uses scalar");
|
||||
}
|
||||
|
||||
@Test
|
||||
void gradleWrapperVerifierRejectsAdmittedWorkflowWithNoDetectedGradleJob(
|
||||
@TempDir Path fixtureRoot) throws Exception {
|
||||
copyGradleWrapperVerifierInputs(fixtureRoot);
|
||||
Path workflow = fixtureRoot.resolve(".github/workflows/orphan-gradle-reference.yml");
|
||||
Files.writeString(
|
||||
workflow,
|
||||
"""
|
||||
name: orphan-gradle-reference
|
||||
on: workflow_dispatch
|
||||
env:
|
||||
DOCUMENTED_COMMAND: ./gradlew
|
||||
jobs:
|
||||
documentation:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Keep the documented command out of executable steps
|
||||
run: echo documented
|
||||
""");
|
||||
|
||||
ProcessResult result = runGradleWrapperVerifier(fixtureRoot);
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output())
|
||||
.contains(
|
||||
"Gradle-running workflow contains no detected Gradle job: "
|
||||
+ ".github/workflows/orphan-gradle-reference.yml");
|
||||
}
|
||||
|
||||
private static void assertCanonicalWrapperPropertiesRejected(ProcessResult result) {
|
||||
assertThat(result.exitCode()).as(result.output()).isNotZero();
|
||||
assertThat(result.output()).contains(CANONICAL_WRAPPER_PROPERTIES_DIAGNOSTIC);
|
||||
void gradleWrapperIsCheckedInAndValidatedByTheOfficialAction() throws IOException {
|
||||
// Twenty-four tests stood here, and every one of them tested .github/scripts/
|
||||
// verify-gradle-wrapper.sh — an 799-line shell script that this suite exercised by writing
|
||||
// mutated workflow fixtures into a @TempDir: a YAML alias in a `uses:`, a merge key in a step,
|
||||
// a flow-style `jobs:` map, a Unicode-escaped checksum override, an added workflow file, a
|
||||
// deleted workflow file, a workflow byte changed inside a comment, a workflow replaced by a
|
||||
// symlink.
|
||||
//
|
||||
// The script hashed all twelve workflow files and required the hashes to match a list it
|
||||
// carried, which is why "a workflow byte changed inside a comment" was a failure worth a test.
|
||||
// As a security control it was not independent: anyone able to edit a workflow was able to edit
|
||||
// the expected hash in the same commit. As a developer control it charged a hash update for
|
||||
// every comment.
|
||||
//
|
||||
// The wrapper guarantee itself is unchanged and is now the official action's:
|
||||
// gradle/actions/setup-gradle validates every wrapper jar in the repository by default, it is
|
||||
// applied through .github/actions/setup-gradle-java, and it is pinned to a full commit SHA.
|
||||
// What is left to assert here is what that action cannot: that the wrapper is committed, and
|
||||
// that its distribution is pinned by checksum rather than by URL alone.
|
||||
assertThat(REPOSITORY_ROOT.resolve("src/gradle/wrapper/gradle-wrapper.jar")).isRegularFile();
|
||||
assertThat(REPOSITORY_ROOT.resolve("src/gradlew")).isRegularFile();
|
||||
|
||||
String wrapperProperties = read("src/gradle/wrapper/gradle-wrapper.properties");
|
||||
assertThat(wrapperProperties)
|
||||
.contains("distributionSha256Sum=")
|
||||
.contains("validateDistributionUrl=true")
|
||||
.contains("https\\://services.gradle.org/distributions/");
|
||||
|
||||
String setupAction = read(".github/actions/setup-gradle-java/action.yml");
|
||||
assertThat(setupAction)
|
||||
.as("wrapper validation has to be reachable from every Gradle job")
|
||||
.contains("gradle/actions/setup-gradle@");
|
||||
assertThat(setupAction)
|
||||
.as("an action reference is only immutable when it is a full commit SHA")
|
||||
.containsPattern("uses: [\\w./-]+@[0-9a-f]{40} #");
|
||||
}
|
||||
|
||||
@Test
|
||||
@@ -927,62 +335,43 @@ class DeveloperExperienceContractTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
void dependencyCacheStageConfiguresWithoutGitWhenRevisionIsAttested(@TempDir Path fixtureRoot)
|
||||
void aBuildWithNoGitMetadataConfiguresAndIsNotCalledARelease(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
// A source archive with no `.git` and no `-PgitRevision`. The root build used to throw during
|
||||
// configuration here — "A 7-40 character hexadecimal source revision is required" — so
|
||||
// `./gradlew test` on an unpacked tarball failed before it compiled anything. Release
|
||||
// traceability was being enforced on every task in the build.
|
||||
Path fixtureSrc = fixtureRoot.resolve("src");
|
||||
copyDependencyCacheStageInputs(fixtureSrc);
|
||||
assertThat(fixtureSrc.resolve(".git")).doesNotExist();
|
||||
|
||||
ProcessResult result = runGitlessGradleHelp(fixtureSrc);
|
||||
ProcessResult configured = runGitlessGradle(fixtureSrc, "help");
|
||||
|
||||
assertThat(configured.exitCode()).as(configured.output()).isZero();
|
||||
assertThat(configured.output()).contains("BUILD SUCCESSFUL");
|
||||
|
||||
// And the other half: such a build may not call itself a release. verifyReleaseProvenance is
|
||||
// where the revision is required, and releaseCheck is what depends on it.
|
||||
ProcessResult provenance = runGitlessGradle(fixtureSrc, "verifyReleaseProvenance");
|
||||
|
||||
assertThat(provenance.exitCode()).as(provenance.output()).isNotZero();
|
||||
assertThat(provenance.output()).contains("no source revision");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anAttestedRevisionSatisfiesReleaseProvenanceWithoutGit(@TempDir Path fixtureRoot)
|
||||
throws Exception {
|
||||
Path fixtureSrc = fixtureRoot.resolve("src");
|
||||
copyDependencyCacheStageInputs(fixtureSrc);
|
||||
|
||||
ProcessResult result =
|
||||
runGitlessGradle(
|
||||
fixtureSrc,
|
||||
"verifyReleaseProvenance",
|
||||
"-PgitRevision=0123456789abcdef0123456789abcdef01234567");
|
||||
|
||||
assertThat(result.exitCode()).as(result.output()).isZero();
|
||||
assertThat(result.output()).contains("BUILD SUCCESSFUL");
|
||||
}
|
||||
|
||||
private static String removeFirstValidationStep(String workflow) {
|
||||
assertThat(workflow).contains(VALIDATION_STEP);
|
||||
return workflow.replaceFirst(Pattern.quote(VALIDATION_STEP), "");
|
||||
}
|
||||
|
||||
private static ProcessResult runGradleWrapperVerifier(Path repositoryRoot) throws Exception {
|
||||
Process process =
|
||||
new ProcessBuilder(
|
||||
"bash",
|
||||
REPOSITORY_ROOT.resolve(".github/scripts/verify-gradle-wrapper.sh").toString(),
|
||||
repositoryRoot.toString())
|
||||
.directory(REPOSITORY_ROOT.toFile())
|
||||
.redirectErrorStream(true)
|
||||
.start();
|
||||
String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8);
|
||||
return new ProcessResult(process.waitFor(), output);
|
||||
}
|
||||
|
||||
private static void copyGradleWrapperVerifierInputs(Path fixtureRoot) throws IOException {
|
||||
copyFile(
|
||||
REPOSITORY_ROOT.resolve("src/gradle/wrapper/gradle-wrapper.properties"),
|
||||
fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.properties"));
|
||||
copyFile(
|
||||
REPOSITORY_ROOT.resolve("src/gradle/wrapper/gradle-wrapper.jar"),
|
||||
fixtureRoot.resolve("src/gradle/wrapper/gradle-wrapper.jar"));
|
||||
Path workflows = REPOSITORY_ROOT.resolve(".github/workflows");
|
||||
try (Stream<Path> paths = Files.walk(workflows)) {
|
||||
paths
|
||||
.filter(Files::isRegularFile)
|
||||
.filter(
|
||||
path -> {
|
||||
String name = path.getFileName().toString();
|
||||
return name.endsWith(".yml") || name.endsWith(".yaml");
|
||||
})
|
||||
.forEach(
|
||||
source -> {
|
||||
try {
|
||||
copyFile(source, fixtureRoot.resolve(REPOSITORY_ROOT.relativize(source)));
|
||||
} catch (IOException exception) {
|
||||
throw new IllegalStateException(
|
||||
"failed to copy verifier workflow input", exception);
|
||||
}
|
||||
});
|
||||
}
|
||||
assertThat(result.output()).contains("verifyReleaseProvenance: OK");
|
||||
}
|
||||
|
||||
private static void copyDependencyCacheStageInputs(Path fixtureSrc) throws IOException {
|
||||
@@ -1018,15 +407,14 @@ class DeveloperExperienceContractTest {
|
||||
}
|
||||
}
|
||||
|
||||
private static ProcessResult runGitlessGradleHelp(Path fixtureSrc) throws Exception {
|
||||
Path outputFile = fixtureSrc.resolve("gitless-help.log");
|
||||
private static ProcessResult runGitlessGradle(Path fixtureSrc, String... arguments)
|
||||
throws Exception {
|
||||
Path outputFile = fixtureSrc.resolve("gitless-run.log");
|
||||
List<String> command = new ArrayList<>(List.of("./gradlew"));
|
||||
command.addAll(List.of(arguments));
|
||||
command.addAll(List.of("--no-daemon", "--console=plain"));
|
||||
Process process =
|
||||
new ProcessBuilder(
|
||||
"./gradlew",
|
||||
"help",
|
||||
"--no-daemon",
|
||||
"--console=plain",
|
||||
"-PgitRevision=0123456789abcdef0123456789abcdef01234567")
|
||||
new ProcessBuilder(command)
|
||||
.directory(fixtureSrc.toFile())
|
||||
.redirectErrorStream(true)
|
||||
.redirectOutput(outputFile.toFile())
|
||||
|
||||
+2
-1
@@ -111,7 +111,8 @@ class PiiTokenBodyForbiddenContractTest {
|
||||
|
||||
@Test
|
||||
void requestBodyCaptureIsDisabledByDefault() throws IOException {
|
||||
Path env = RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
|
||||
Path env =
|
||||
RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
|
||||
|
||||
String value = readEnv(env, "APP_LOG_BODY_CAPTURE_ENABLED");
|
||||
assertThat(value)
|
||||
|
||||
+6
-8
@@ -159,19 +159,17 @@ class SampleRemovalSmokeContractTest {
|
||||
|
||||
@Test
|
||||
void sampleOffCiJobIsReleaseBlocking() throws IOException {
|
||||
// The gate matrix half of this assertion is gone with the matrix. It read .github/
|
||||
// ci-gate-matrix.yml and required a row declaring that the sample-off job exists and is
|
||||
// release-blocking — a third copy of what the workflow itself says in `jobs:` and in
|
||||
// `release-gate.needs`. Checking a register against the thing it registers is work that only
|
||||
// ever finds a disagreement between two descriptions of one fact.
|
||||
RepositoryContractResources resources = RepositoryContractResources.fromSystemProperty();
|
||||
String workflow =
|
||||
Files.readString(resources.requireTrackedFile(".github/workflows/ci-quality-gates.yml"));
|
||||
String gateMatrix =
|
||||
Files.readString(resources.requireTrackedFile(".github/ci-gate-matrix.yml"));
|
||||
|
||||
assertThat(workflow).contains("\n sample-off:\n");
|
||||
assertThat(workflow).contains("./gradlew :app-bootstrap:sampleOffTest");
|
||||
assertThat(workflow).contains(":app-bootstrap:sampleOffTest");
|
||||
assertThat(workflow).contains("\n - sample-off\n");
|
||||
assertThat(gateMatrix)
|
||||
.contains("id: sample-off")
|
||||
.contains("ref: sampleOffTest")
|
||||
.contains("job: sample-off")
|
||||
.contains("execution: explicit");
|
||||
}
|
||||
}
|
||||
|
||||
+2
-1
@@ -22,7 +22,8 @@ import org.junit.jupiter.api.Test;
|
||||
class SqlLoggingForbiddenContractTest {
|
||||
|
||||
private Properties loadEnv() throws Exception {
|
||||
Path env = RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
|
||||
Path env =
|
||||
RepositoryContractResources.fromSystemProperty().requireTrackedFile("src/.env.example");
|
||||
Properties props = new Properties();
|
||||
try (InputStream in = Files.newInputStream(env)) {
|
||||
props.load(in);
|
||||
|
||||
+72
-44
@@ -14,6 +14,8 @@ import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.stream.Collectors;
|
||||
import java.util.stream.Stream;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.yaml.snakeyaml.LoaderOptions;
|
||||
import org.yaml.snakeyaml.Yaml;
|
||||
@@ -39,10 +41,22 @@ class MessagingCapabilityRegistryContractTest {
|
||||
private static final Set<String> LEGAL_MATURITY =
|
||||
Set.of("not-implemented", "implemented-candidate", "release-eligible");
|
||||
|
||||
private static final Set<String> REQUIRED_VERIFICATION_TASKS =
|
||||
/** Qualification tasks that exist and can pass. */
|
||||
private static final Set<String> IMPLEMENTED_VERIFICATION_TASKS =
|
||||
Set.of("verifyMessagingContracts", "verifyMessagingJsonSchemaV1");
|
||||
|
||||
/**
|
||||
* Task names a not-implemented card names as the evidence it will need.
|
||||
*
|
||||
* <p>Names on a roadmap, not tasks. The build used to register all of these — nine
|
||||
* `tasks.register(...)` calls whose action threw unconditionally, whatever was on disk — so
|
||||
* `./gradlew verifyMessagingSecurityR2` was a task you could invoke and could not pass. That is a
|
||||
* TODO written through the Gradle task API: it appears in `./gradlew tasks`, it is reachable from
|
||||
* a `dependsOn`, and the only thing it can do is fail. MSG-015 tracks the real work and
|
||||
* docs/roadmap/messaging-r2.md states it in prose.
|
||||
*/
|
||||
private static final Set<String> PLANNED_VERIFICATION_TASKS =
|
||||
Set.of(
|
||||
"verifyMessagingContracts",
|
||||
"verifyMessagingJsonSchemaV1",
|
||||
"verifyMessagingPollingOutboxR2",
|
||||
"verifyMessagingKafkaProducerR2",
|
||||
"verifyMessagingSecurityR2",
|
||||
@@ -53,6 +67,18 @@ class MessagingCapabilityRegistryContractTest {
|
||||
"verifyMessagingCleanupTargetBinding",
|
||||
"verifyMessagingFinalR2Profile");
|
||||
|
||||
/**
|
||||
* Every producer name the registry files enumerate: the two that exist plus the nine planned.
|
||||
*
|
||||
* <p>Used for the release profiles and the evidence manifest schema, which describe the complete
|
||||
* R2 shape rather than the current build. A schema saying "producerTask must be one of these
|
||||
* eleven" is a statement about the evidence format; it is not a claim that eleven tasks exist,
|
||||
* which is exactly the claim the deleted Gradle registrations were making.
|
||||
*/
|
||||
private static final Set<String> ALL_VERIFICATION_TASKS =
|
||||
Stream.concat(IMPLEMENTED_VERIFICATION_TASKS.stream(), PLANNED_VERIFICATION_TASKS.stream())
|
||||
.collect(Collectors.toUnmodifiableSet());
|
||||
|
||||
private static final Set<String> FORBIDDEN_EXTENSION_TOKENS =
|
||||
Set.of(
|
||||
"consumer",
|
||||
@@ -128,9 +154,14 @@ class MessagingCapabilityRegistryContractTest {
|
||||
assertThat(card.get("settingsDigest")).isEqualTo("");
|
||||
}
|
||||
assertNonEmptyStringList(card, "evidenceTasks");
|
||||
boolean implementedCandidate = "implemented-candidate".equals(card.get("maturity"));
|
||||
assertThat(stringList(card, "evidenceTasks"))
|
||||
.as("evidenceTasks on %s must reference declared root tasks", card.get("cardId"))
|
||||
.allMatch(REQUIRED_VERIFICATION_TASKS::contains);
|
||||
.as("evidenceTasks on %s must name a known qualification task", card.get("cardId"))
|
||||
.allMatch(
|
||||
task ->
|
||||
IMPLEMENTED_VERIFICATION_TASKS.contains(task)
|
||||
|| (!implementedCandidate && PLANNED_VERIFICATION_TASKS.contains(task)),
|
||||
"an implemented-candidate card may only name a task that exists");
|
||||
assertNonEmptyStringList(card, "requiredScenarios");
|
||||
assertNonEmptyStringList(card, "runbookIds");
|
||||
}
|
||||
@@ -186,8 +217,7 @@ class MessagingCapabilityRegistryContractTest {
|
||||
List<String> requiredEvidenceTasks = stringList(profile, "requiredEvidenceTasks");
|
||||
assertListIntegrity(
|
||||
requiredEvidenceTasks, "requiredEvidenceTasks", profile.get("releaseProfileId"));
|
||||
assertThat(requiredEvidenceTasks)
|
||||
.containsExactlyInAnyOrderElementsOf(REQUIRED_VERIFICATION_TASKS);
|
||||
assertThat(requiredEvidenceTasks).containsExactlyInAnyOrderElementsOf(ALL_VERIFICATION_TASKS);
|
||||
assertNonEmptyStringList(profile, "requiredScenarios");
|
||||
assertNonEmptyStringList(profile, "runbookIds");
|
||||
|
||||
@@ -278,7 +308,7 @@ class MessagingCapabilityRegistryContractTest {
|
||||
Map<String, Object> producerTask = map(properties, "producerTask");
|
||||
assertThat(producerTask).containsEntry("type", "string");
|
||||
assertThat(stringList(producerTask, "enum"))
|
||||
.containsExactlyInAnyOrderElementsOf(REQUIRED_VERIFICATION_TASKS);
|
||||
.containsExactlyInAnyOrderElementsOf(ALL_VERIFICATION_TASKS);
|
||||
|
||||
assertArrayOfReference(
|
||||
map(properties, "scenarioIds"), "#/$defs/identifier", true, Integer.valueOf(1));
|
||||
@@ -323,55 +353,53 @@ class MessagingCapabilityRegistryContractTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
void rootBuildDeclaresEveryFailClosedVerificationTaskThroughTheSharedGuard() throws Exception {
|
||||
String build = Files.readString(repositorySrcRoot().resolve("build.gradle"));
|
||||
|
||||
// This test protects a contract: every skeleton routes through the shared guard, and that guard
|
||||
// fails closed. The wording the guard happens to use is not the contract.
|
||||
void noUnimplementedQualificationIsRegisteredAsAGradleTask() throws Exception {
|
||||
// The inverse of the assertion that used to be here.
|
||||
//
|
||||
// It used to assert six individual message fragments from the guard's body. Those fragments
|
||||
// belonged to ~45 lines of evidence validation whose result was discarded, because the guard
|
||||
// threw unconditionally either way. When that dead validation was removed the implementation was
|
||||
// fine and this test broke — the test was pinning source text, not behaviour, which is how a
|
||||
// guard stops being a guard and becomes a reason not to touch the file.
|
||||
assertThat(build)
|
||||
.contains(
|
||||
"messagingVerificationSkeletons.each",
|
||||
"tasks.register(taskName)",
|
||||
"messagingFailClosedEvidenceGuard(taskName, evidencePaths)");
|
||||
|
||||
// The property worth pinning: the guard throws. If it is ever changed to report and continue,
|
||||
// every messaging R2 skeleton would start passing without a qualification producer existing.
|
||||
int guardStart = build.indexOf("Closure<Void> messagingFailClosedEvidenceGuard");
|
||||
assertThat(guardStart).as("the shared guard closure must exist").isNotNegative();
|
||||
int guardEnd = build.indexOf("\n}", guardStart);
|
||||
assertThat(guardEnd).as("the shared guard closure must be terminated").isGreaterThan(guardStart);
|
||||
assertThat(build.substring(guardStart, guardEnd))
|
||||
.as("the shared guard must fail closed rather than report and continue")
|
||||
.contains("throw new GradleException")
|
||||
.contains("FAIL_CLOSED");
|
||||
for (String taskName : REQUIRED_VERIFICATION_TASKS) {
|
||||
assertThat(build).contains("'" + taskName + "'");
|
||||
// This test previously required the root build to register all nine R2 skeleton tasks and to
|
||||
// route them through a guard that threw `FAIL_CLOSED`. It was pinning the existence of tasks
|
||||
// that could not succeed: a card said "my evidence comes from verifyMessagingSecurityR2", the
|
||||
// task existed, and running it always failed — so the registry looked wired to a build that
|
||||
// could substantiate nothing. Registering a task for work with no producer does not make the
|
||||
// absence safer; it makes the absence look like a gate.
|
||||
//
|
||||
// What is worth holding is that they are NOT registered, so nobody wires a release lane to one.
|
||||
for (Path script : qualificationScripts()) {
|
||||
String text = Files.readString(script);
|
||||
for (String planned : PLANNED_VERIFICATION_TASKS) {
|
||||
assertThat(text)
|
||||
.as("%s must not register the unimplemented task %s", script.getFileName(), planned)
|
||||
.doesNotContain("tasks.register('" + planned + "')")
|
||||
.doesNotContain("tasks.register(\"" + planned + "\")");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void rootBuildSchemaValidatesEvidenceAndDeterministicallyLeavesCombinedEvidenceLast()
|
||||
throws Exception {
|
||||
String build = Files.readString(repositorySrcRoot().resolve("build.gradle"));
|
||||
void messagingQualificationSchemaValidatesTheEvidenceItWrites() throws Exception {
|
||||
// Qualification lives in gradle/qualification/messaging-qualification.gradle now, not in the
|
||||
// root build file. The property held here is the one that makes the manifest evidence rather
|
||||
// than a file: each producer is finalized by a JSON Schema validation of the exact bytes it
|
||||
// wrote, and the combined producer runs after the JSON-schema one.
|
||||
String qualification =
|
||||
Files.readString(
|
||||
repositorySrcRoot().resolve("gradle/qualification/messaging-qualification.gradle"));
|
||||
|
||||
assertThat(build)
|
||||
assertThat(qualification)
|
||||
.contains(
|
||||
"MessagingEvidenceManifestSchemaValidator",
|
||||
"validateMessagingJsonSchemaV1EvidenceManifestSchema",
|
||||
"validateMessagingContractsEvidenceManifestSchema",
|
||||
"verifyMessagingJsonSchemaV1.configure",
|
||||
"finalizedBy validateMessagingJsonSchemaV1EvidenceManifestSchema",
|
||||
"dependsOn validateMessagingJsonSchemaV1EvidenceManifestSchema",
|
||||
"verifyMessagingContracts.configure",
|
||||
"finalizedBy validateMessagingContractsEvidenceManifestSchema");
|
||||
}
|
||||
|
||||
private static List<Path> qualificationScripts() throws Exception {
|
||||
Path src = repositorySrcRoot();
|
||||
return List.of(
|
||||
src.resolve("build.gradle"),
|
||||
src.resolve("gradle/qualification/messaging-qualification.gradle"));
|
||||
}
|
||||
|
||||
private static Path requiredConfig(String relativePath) {
|
||||
Path path = repositorySrcRoot().resolve("config/messaging").resolve(relativePath);
|
||||
assertThat(path).as("required Messaging configuration %s", path).isRegularFile();
|
||||
|
||||
+1
-2
@@ -45,8 +45,7 @@ class ReleaseManifestTaskExistenceTest {
|
||||
* form. It is a cheap guard against a manifest naming a task nobody wrote, not a substitute for
|
||||
* asking Gradle.
|
||||
*/
|
||||
private static final Pattern REGISTERED_LANE =
|
||||
Pattern.compile("\\blane\\(\\s*'([A-Za-z0-9_]+)'");
|
||||
private static final Pattern REGISTERED_LANE = Pattern.compile("\\blane\\(\\s*'([A-Za-z0-9_]+)'");
|
||||
|
||||
/**
|
||||
* Tasks the Java plugin supplies, which no build file registers explicitly.
|
||||
|
||||
Reference in New Issue
Block a user