donghyeon.kang/tech-log-backend
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Files
48517b9e65f816155135bb36eb5c650e4cd05240
tech-log-backend/docs/superpowers/specs/2026-07-28-production-grade-test-architecture-environment-design.md
T

69 KiB
Raw Blame History

Production-grade test architecture and environment design

  • Status: proposed design; implementation not started
  • Date: 2026-07-28
  • Repository baseline: Java 21, Spring Boot 4.0.0, Gradle 9.0.0, 19 registered leaf modules
  • Scope: test taxonomy, Clean Architecture boundary ownership, Gradle source sets/tasks, service provisioning, data isolation, CI/release environments, non-functional qualification, evidence and rollout
  • Out of scope: implementing the suites, choosing an organization's SLO numbers, provisioning production cloud accounts, or changing the 19-leaf production dependency graph

1. Executive decision

This repository does not need a larger undifferentiated test pyramid. It needs a fail-closed evidence ladder in which each result says which boundary and environment it actually exercised.

The proposed destination is:

source/static evidence
  -> deterministic leaf tests
  -> real-provider integration tests
  -> migration compatibility tests
  -> immutable-artifact system/smoke tests
  -> scheduled operational qualification
  -> post-deploy synthetic evidence

The core decisions are:

  1. Classify a test on three independent axes:
    • execution boundary: unit, component/slice, integration, system;
    • verification purpose: behavior, contract, architecture, migration, security, resilience, performance, smoke;
    • environment: in-memory, loopback, disposable container, provider sandbox, deployed environment.
  2. Keep src/test deterministic and independent of external infrastructure. It may use an in-process Spring context or an ephemeral loopback port, but no Docker daemon, Internet service, manually prestarted database, or fixed host port.
  3. Put real PostgreSQL, Redis, MongoDB, Kafka, MinIO, SMTP, and fault-proxy tests in an owning leaf's src/integrationTest. Selecting that lane is a promise to execute it: missing Docker, zero discovered tests, or an unexpected skip is a failure, not a green build.
  4. Add migrationTest only where its historical fixtures, destructive lifecycle, and release frequency differ materially from ordinary integration tests.
  5. Reserve systemTest/E2E for black-box tests of the built boot JAR or OCI image. A @WebMvcTest, mocked repository, or custom test-only boot application is not E2E.
  6. Keep tests in their owning registered leaf. Do not add a twentieth production test-support module. Reusable port contract kits use Gradle test fixtures and test-only dependencies.
  7. Use explicit Gradle SourceSet + Test tasks for the first implementation. Gradle's JVM Test Suite model fits the problem, but remains incubating; this long-lived template favors the stable mechanism already used by sampleOffTest.
  8. Use Testcontainers for leaf/provider integration, a CI-specific disposable Compose override for candidate-image system tests, and protected provider sandboxes only for compatibility claims that local substitutes cannot establish.
  9. Keep check Docker-free. CI explicitly fans in deterministic quality, mandatory provider integration, sample-off, and candidate-image smoke jobs.
  10. Treat PR-head artifacts as unpromotable. Reverify the protected main merge commit, build its boot JAR once, copy that exact JAR into the canonical OCI image, and promote only that image digest.
  11. Do not adopt an arbitrary repository-wide coverage percentage as the definition of quality. Record coverage first, require explicit risk scenarios, then ratchet changed-code coverage and use mutation testing selectively for pure domain/application policy.

2. Why this is the right problem

The repository already has extensive tests, architecture guards, snapshots, Testcontainers, and a flaky-test quarantine. The problem is not a lack of test classes. The problem is that materially different evidence is mixed under the same test task, while some provider evidence is outside CI or can disappear through conditional skip.

A test called ContractTest can be a pure port contract, a real PostgreSQL contract, or an HTTP schema contract. A test called E2ETest can still be a MockMvc slice. Names alone therefore cannot route release evidence.

The design must answer four questions for every important claim:

  • What production boundary was crossed?
  • What was real and what was replaced?
  • In which environment did it run?
  • What can still be false even when the test passes?

3. Evidence and audit scope

3.1 Repository evidence

The audit read:

  • AGENTS.md, root and leaf CLAUDE.md files;
  • src/config/architecture/modules.json, src/settings.gradle, root and leaf Gradle files;
  • all src/test/java and src/test/groovy trees;
  • Dockerfiles and Compose files;
  • .github/ci-gate-matrix.yml and GitHub Actions workflows;
  • existing production-capability and CI design documents.

The inspected worktree was main with pre-existing local changes. Those changes were preserved. Counts below describe that worktree, not a clean historical commit.

3.2 Fresh executable baseline

The following command was run with task outputs forced to rerun:

cd src
./gradlew test --rerun-tasks --console=plain --no-daemon

Result:

  • build successful in 2m 26s;
  • 1,661 JUnit/Jqwik/Spock test invocations;
  • 8 skipped invocations;
  • 0 failures and 0 errors;
  • 275 emitted JUnit XML suites;
  • 372 Java test-tree source files and 3 Groovy test-tree source files;
  • domain-core:test was NO-SOURCE;
  • the explicit Redis service lane was not part of this result.

Docker was available during this run, so current PostgreSQL and MinIO tests ran. The successful run does not prove they would fail closed on a runner without Docker.

3.3 Evidence grades used in this design

Grade Meaning
A Fresh executable repository evidence or deterministic source/config inspection
B Official product/framework documentation
C Design inference tailored to this repository
D Organization-specific value still requiring an owner decision or measured baseline

Provider support versions, CI duration budgets, production SLO thresholds, and cloud topology are grade D until the adopting service declares them. This design does not fabricate those values.

4. Current state

4.1 Strong foundations to retain

  • Exactly 19 production leaves and allowed dependency edges are fail-closed in modules.json, settings, and Gradle verification.
  • ArchUnit has positive controls rather than only vacuous clean checks.
  • Core modules use framework-light JUnit/AssertJ dependencies.
  • There are MVC, JPA, GraphQL, gRPC, full-context, OpenAPI snapshot, property-based, PostgreSQL, MinIO, and concurrency tests.
  • sampleOffTest proves production can be compiled and tested without the example portfolio.
  • quarantineTest is non-blocking, while registry drift and a 14-day sunset remain blocking.
  • Dependency locks, format/static analysis, environment registry, public path snapshot, and security scanning already feed CI.
  • The Docker runtime already has non-root, read-only filesystem, memory, shutdown, and health contracts that a future artifact lane can exercise.

4.2 Material gaps

Gap Repository evidence False conclusion it permits
Mixed execution environments Unit, slice, Spring context, PostgreSQL, and MinIO tests mostly share test "test is fast and hermetic" or "test always verifies providers"
Conditional provider skip disabledWithoutDocker and Assumptions.assumeTrue(Docker...) are present "CI verified PostgreSQL/MinIO" when it may have skipped
Redis lane outside CI redisServiceTest is excluded from default test; CI never invokes it "Redis scripts/TTL work on a real server"
Provider tests owned by composition root Several PostgreSQL semantics tests live under app-bootstrap Leaf capability and cross-leaf wiring evidence are conflated
No candidate artifact gate Local bootstrapSmoke exists but is absent from required CI "The Docker image boots with release configuration"
Artifact identity split src/Dockerfile invokes bootJar again during image build "The JAR/class output already tested is byte-identical to the JAR inside the image"
Split CI fan-in release-gate can needs only same-workflow jobs, while trivy-fs runs in dependency-vulnerability.yml "One release-gate currently proves every blocking check across workflows"
Non-hermetic local smoke Persistent DB volume, fixed port 8080, local .env, restart policy Local success is not reproducible CI evidence
Misclassified names A @WebMvcTest is named WorkLogAuthorizationE2ETest A slice is mistaken for artifact E2E
Incomplete transport evidence GraphQL tests bypass HTTP; WebSocket tests mock the broadcaster; gRPC has only built-in health/reflection Feature transport/security/flow-control claims exceed evidence
No common test runtime policy UTC is not applied to every normal Test; no repository timeout/parallel policy Host locale/timezone or leaked state can create flakiness
Broad Boot test classpaths Root adds spring-boot-starter-test and webmvc-test to every non-core leaf Outbound leaves see irrelevant web test infrastructure
Test dependency graph not governed Production configurations are checked; test configurations are not equivalently registered Cross-leaf fixtures or test helpers can silently couple modules
Assumption-heavy contracts About 51 assumption calls across roughly 24 bootstrap test files Missing paths/resources can be reported as skip rather than failure
Java-only quarantine scan verifyQuarantineSunset scans src/test/java A future Groovy quarantined test can evade registry drift checks
No coverage/mutation baseline No JaCoCo or mutation configuration Untested change risk is not visible, though a percentage alone would not fix it
No provider/version/fault matrix Only selected local substitutes and versions exist Managed-service, topology, upgrade, or failover readiness is overstated

4.3 Current test execution by leaf

The fresh XML baseline was:

Leaf Suites Invocations Skipped Current emphasis
domain-core 0 0 0 Production domain is intentionally skeletal
shared-contract 18 202 0 Values, metrics, tracing, operation/error contracts
application-core 19 123 0 Ports, idempotency, outbox, transaction intent
adapter:outbound:support 1 4 0 Support behavior
adapter:outbound:persistence-jpa 11 63 0 Mapping, translation, configuration, transaction units
adapter:outbound:persistence-mongo 1 4 0 Mock-client configuration context
adapter:outbound:identifier 2 16 0 UUID/HMAC pure behavior
adapter:outbound:fileserver 7 79 0 Temp-filesystem publication/recovery
adapter:outbound:objectstorage 3 15 0 Filesystem, mocked S3, MinIO
adapter:outbound:cache-redis 12 49 0 Fake/runtime/settings; real Redis excluded
adapter:outbound:httpclient 17 140 0 Loopback/fake resilience and lifecycle
adapter:outbound:messaging 6 20 0 Fake publisher/settings/log contracts
adapter:outbound:notification 2 19 0 Routing/provider fake
adapter:inbound:web 35 180 0 MVC/standalone transport policies
adapter:inbound:grpc 4 18 0 Mapping plus real loopback health/reflection
adapter:inbound:graphql 2 14 0 Execution service/schema, no HTTP
adapter:inbound:websocket 1 2 0 Broadcaster with mocked messaging template
app-bootstrap 91 537 8 Architecture, settings, contracts, PostgreSQL integration
sample-portfolio 43 176 0 Reference vertical slice and real PostgreSQL

The eight fresh skips were optional-adapter conditional-execution meta-tests and a sample-off-only assertion. This is why the new policy must reject unexpected skips per required suite rather than naïvely requiring zero skips repository-wide.

5. Taxonomy: execution boundary and purpose are different axes

5.1 Execution boundary

Boundary Real collaborators Typical location Expected runtime
Unit One object; hand values/fakes only src/test milliseconds
Component One use case or adapter plus in-process collaborators/ephemeral loopback src/test milliseconds to seconds
Spring slice Selected auto-configuration and boundary components src/test seconds
Provider integration Production adapter plus real disposable service/driver/protocol src/integrationTest seconds to minutes
Migration integration Historical schema/data plus production migration engine and DB src/migrationTest when needed minutes
System Built boot JAR/OCI image reached only through public ports src/systemTest or external harness minutes
Provider qualification Candidate adapter against protected real provider/topology protected scheduled job minutes to hours
Post-deploy synthetic Deployed artifact through ingress/control plane deployment pipeline/monitoring continuously or per deploy

5.2 Verification purpose

contract, architecture, security, resilience, performance, and smoke describe why a test exists, not automatically how far it runs.

Examples:

  • a port contract can run against a hand fake in test and against PostgreSQL in integrationTest;
  • authorization has domain/application unit tests, MVC slice tests, full filter-chain system tests, and security DAST;
  • resilience includes a pure backoff-policy unit test and a Toxiproxy socket integration test;
  • a smoke test can target a loopback embedded server or the candidate image, but only the latter is artifact smoke.

5.3 Naming policy

The source set is the execution SSOT; suffixes are human navigation aids.

Source set/harness Naming
test *Test, *ContractTest, *ArchitectureTest, *ComponentTest
integrationTest *IntegrationTest
migrationTest *MigrationTest
systemTest *SystemTest or narrowly *E2ETest
JMH *Benchmark

IT is accepted during migration but converges to IntegrationTest. E2E is forbidden outside the system source set. Proposed first renames include:

  • WorkLogAuthorizationE2ETest → WorkLogAuthorizationWebMvcContractTest;
  • VirtualThreadMdcE2ETest → VirtualThreadMdcHttpComponentTest;
  • EnvelopeMetaIntegrationTest → EnvelopeMetaComponentTest;
  • S3ObjectStorageAdapterIT → S3ObjectStorageAdapterIntegrationTest.

6. Confidence ladder and honest limits

client contract
    |
    |  transport slice/component
    v
inbound adapter ------ actual socket only in component/system evidence
    |
    |  DTO -> command/result mapping
    v
application use case - fake ports prove policy, not provider behavior
    |
    |  reusable port contract
    v
outbound adapter ----- real service only in integration evidence
    |
    |  driver/protocol
    v
provider ------------- managed topology only in provider qualification

composition root + packaged runtime cross the whole vertical path only in system evidence
Test type It can establish It cannot establish by itself
Architecture/static Registered project edges, bytecode imports, package/layer rules, schema/snapshot drift Reflection/string lookup, runtime bean selection, business correctness
Unit Invariant, state transition, decision table, application sequencing Spring proxies, transaction semantics, serialization, provider behavior
Component/slice A selected adapter/context, request mapping, validation, error/security mapping Excluded filters/configuration, real server/network, complete bean graph
Provider integration Driver/protocol, vendor constraint, atomicity, serialization, TTL, basic concurrency Managed IAM/KMS, multi-AZ failover, real quotas/latency, production data scale
Contract Declared port/schema/consumer examples remain compatible Undeclared consumers, full workflow, deployment, latency
System/E2E Candidate artifact, runtime configuration, public network path, critical vertical flow Exhaustive business cases, every fault, production capacity
Migration Empty install and named previous-release upgrade/compatibility paths All production data distributions, lock duration at full scale, guaranteed downgrade
Resilience Behavior under explicitly injected latency/reset/restart/duplicates Unknown compound failures or regional disaster
Security Named controls, negative cases, known scanner rules Absence of vulnerabilities or business-abuse paths
Performance Thresholds for a fixed artifact/workload/environment Production capacity when the environment or workload differs
Smoke The artifact starts, becomes ready, and serves a few critical probes Functional completeness or SLO compliance

7. Clean Architecture ownership by leaf

Leaf Deterministic test responsibility Real integration/qualification responsibility Passing still does not prove
domain-core Pure invariant/value/state/event/property tests when behavior exists None Persistence, transport, framework behavior
shared-contract Framework-free value, redaction, metric, trace, error compatibility None; each consuming adapter owns serialization compatibility for shared types External serializer/collector/dashboard/consumer behavior
application-core Use cases with hand fakes; authorization, idempotency, transaction intent, compensation Reusable port contract kit, consumed by adapters Actual DB/broker/cache transaction or concurrency
adapter:outbound:support Deadline/logging/failure helper behavior Only if a real runtime backend is part of its public capability Provider-specific semantics
adapter:outbound:persistence-jpa Mapper, SQL-state translation, transaction-template and configuration component tests Real PostgreSQL JPA/query/constraint/lock/outbox/idempotency integration Managed DB topology and bootstrap wiring
adapter:outbound:persistence-mongo Mapping/configuration/codec tests Real MongoDB indexes, queries and, if claimed, replica-set transactions/change streams Atlas IAM/control plane, sharding/failover
adapter:outbound:identifier Deterministic codecs, vectors, validation, pseudonymization KMS/provider rotation only when such an adapter exists Global uniqueness or cryptographic safety from small samples
adapter:outbound:fileserver @TempDir path safety, journal/recovery, limits, atomic local publication Real mount permissions, process crash, disk-full; supported filesystem-specific qualification NFS/cross-node fencing unless explicitly tested
adapter:outbound:objectstorage Filesystem and mocked SDK mapping MinIO protocol baseline; protected AWS sandbox for IAM/versioning/checksum/multipart claims MinIO equivalence to AWS S3 control plane
adapter:outbound:cache-redis Key/codec/program/policy/settings tests Mandatory real Redis standalone; Sentinel/Cluster/TLS/ACL/restart/eviction only for claimed readiness HA from a standalone test
adapter:outbound:httpclient Request/response mapping, budget, retry/circuit, loopback behavior TCP/TLS/proxy/DNS/pool/cancellation with real sockets and fault proxy; provider sandbox where needed External provider semantics from a mock server
adapter:outbound:messaging Envelope, routing, disabled mode, fake publisher Real broker ack, partition ordering, duplicate/redelivery, DLT and schema compatibility End-to-end consumer processing if no inbound consumer exists
adapter:outbound:notification Routing/template/provider mapping with fake client SMTP/webhook sandbox, throttling, timeout, retry and receipt mapping Deliverability or provider reputation
adapter:inbound:web DTO, validation, mapper, error envelope, MVC/security/filter slice Real embedded-server component tests for servlet/network behavior Proxy/TLS/ingress or full application composition
adapter:inbound:grpc Status mapping/interceptors; loopback health/reflection Feature RPC, TLS/mTLS, auth, deadline/cancellation/backpressure Production load balancer behavior
adapter:inbound:graphql Schema/resolver/error/complexity component tests HTTP/WebSocket transport, auth, subscription, N+1 behavior Client compatibility not represented by schema checks
adapter:inbound:websocket Destination mapping and broadcaster policy Real handshake, STOMP framing, origin/auth, reconnect, broker relay/backpressure Durable delivery
app-bootstrap Architecture, configuration binding/validation, conditional beans, composition contracts Full production application with real required providers; packaged artifact smoke/system Domain correctness
sample-portfolio Reference domain/use-case/transport tests and sample isolation Reference PostgreSQL/full-stack sample acceptance Any downstream project's domain quality

domain-core having no tests today is not automatically a defect: its current production content is mostly marker/port abstraction and the example domain is intentionally isolated in sample-portfolio. The gate is behavioral: when a real invariant or value behavior enters domain-core, a Spring-free test enters with it.

8. Gradle execution model

8.1 Alternatives considered

Alternative Advantages Failure mode in this repository Decision
JUnit tags inside the current test source set Small initial diff; familiar filtering All dependencies remain visible; an untagged container test silently returns to test; environment policy cannot be enforced from the classpath Reject as the primary boundary; keep tags for orthogonal purpose/resource labels
Gradle JVM Test Suite plugin Declarative suites and useful future model The API remains incubating and would become a template-wide build convention commitment Reconsider after one explicit-source-set implementation is stable
Explicit SourceSet plus typed Test tasks Stable Gradle mechanism; isolates classpaths, discovery, reports, and environment policy More build code and registry validation are required Adopt
A new Gradle test-support or acceptance-test project Strong classpath isolation Creates a twentieth production leaf or ambiguous cross-leaf ownership and weakens the current registry model Reject

The first implementation should use a small convention plugin or root build helper backed by explicit source sets, not copy-pasted task definitions in 19 leaf builds. It must not add a production dependency edge to modules.json.

8.2 Source layout

Only source sets that a leaf actually needs are created.

src/<leaf>/
  src/main/java/                 production
  src/test/java/                 deterministic unit/component/slice/architecture
  src/testFixtures/java/         optional reusable test-only port contract kit
  src/integrationTest/java/      disposable real service/driver/protocol
  src/migrationTest/java/        optional historical migration lifecycle

src/app-bootstrap/
  src/systemTest/java/           optional black-box harness with no production classes

qa/
  performance/                   k6/JMH entrypoints and versioned workloads
  security/                      DAST configuration and safe target policy
  system/                        shell/container harness when Java adds no value

qa/ is verification infrastructure, not a Gradle production module and not a Clean Architecture leaf. A Java systemTest must deliberately avoid sourceSets.main.output on its compile/runtime classpath; it can know public HTTP/gRPC schemas or generated client contracts, but it must not call an application bean, repository, controller method, or test-only boot class.

8.3 Task contracts

Task Allowed infrastructure Failure contract
<leaf>:test Heap/in-process resources, @TempDir, ephemeral loopback server Fails on Docker/Internet/manual-service dependency, fixed port, zero discovery where the leaf declares required tests, or ordinary assertion failure
<leaf>:integrationTest Testcontainers with pinned provider images and optional Toxiproxy Fails before discovery if Docker is unavailable; fails on unexpected skip, zero discovery, leaked container, or provider failure
<leaf>:migrationTest Disposable DB plus checked-in versioned fixtures Fails on empty install, supported-version upgrade, validation, data invariant, lock/error-budget, or cleanup failure
app-bootstrap:systemTest Built candidate JAR/image and disposable external services Fails if the exact candidate cannot start, become ready, expose its build identity, or complete critical black-box probes
root test Lifecycle aggregation only Depends on every registered deterministic leaf task and remains Docker-free
root integrationTest Lifecycle aggregation only Depends on every suite marked required for the selected profile
root releaseCandidateCheck Local/single-runner lifecycle for quality, required integrations, and candidate smoke Reproduces the repository verification bundle; it cannot aggregate independent CI jobs or workflows

The root lifecycle tasks are not themselves Test tasks and must not manufacture an empty green report. Local developers may select a leaf lane, while CI selects the registered aggregate. releaseCandidateCheck is a local Gradle lifecycle only. A GitHub Actions release-gate becomes the CI fan-in only after every blocking workflow is exposed through workflow_call and invoked as a job in one caller workflow, because needs cannot reference a job in another workflow. Until that migration is complete, the branch-protection required-check union—not the current release-gate alone—is the enforcement authority.

8.4 Test-suite registry

src/config/architecture/modules.json remains the production dependency SSOT. Add a separate, fail-closed src/config/testing/test-suites.json for verification execution. It references module IDs from the architecture registry rather than duplicating their paths or Gradle coordinates.

Conceptual shape:

{
  "schema_version": 1,
  "allowed_test_dependencies": [
    {
      "from_module_id": "adapter-outbound-persistence-jpa",
      "to_module_id": "application-core",
      "scope": "test-fixtures"
    }
  ],
  "providers": {
    "postgresql-16": {
      "container_image": "postgres:<approved-tag>@sha256:<declared-digest>",
      "topologies": ["standalone"]
    }
  },
  "suites": [
    {
      "id": "persistence-jpa-postgresql",
      "owner_module_id": "adapter-outbound-persistence-jpa",
      "task": "integrationTest",
      "source_set": "integrationTest",
      "boundary": "provider-integration",
      "purposes": ["contract", "transaction", "migration-baseline"],
      "required_services": ["postgresql-16"],
      "required_in": ["pull-request", "main-merge"],
      "skip_policy": "none"
    }
  ]
}

The digest above is intentionally a placeholder, not a proposed image selection. Implementation resolves and reviews actual supported image tags/digests. A movable tag such as postgres:16-alpine is insufficient release evidence because its bits can change without a source change.

verifyTestSuiteRegistry must fail when:

  • an owner module is unknown or lacks the declared source root/task;
  • a suite ID, provider alias, boundary, purpose, or required cadence is unknown;
  • a required suite discovers zero tests or has an unapproved skip;
  • a required CI suite has no corresponding job/fan-in entry;
  • a registered source set has no registry entry, or a registry entry is never selected;
  • production configurations depend on a test fixture;
  • test-only cross-leaf dependencies are not explicitly declared and allowed;
  • quarantine scanning omits Java/Groovy or a registered source set.

The suite registry describes execution, not individual test classes. Class discovery counts and skip outcomes are emitted at runtime and compared with the suite policy.

The suite registry is the SSOT for suite ownership, test-only edges, and required cadence. The existing gate matrix remains the SSOT for the complete set of blocking CI jobs, including non-Gradle and cross-workflow security jobs. CI configuration maps suite IDs to jobs; verifyTestSuiteRegistry checks both directions so neither authority can silently drift.

8.5 Dependency policy for Spring Boot 4

Spring Boot 4 modularized starters and test infrastructure. Each leaf declares only the focused test support it owns—for example, MVC test support for the web leaf and data-JPA test support for the JPA leaf—instead of the root adding Web MVC test infrastructure to every non-core leaf. The classic umbrella test starter may remain during migration, but the destination classpath is capability-specific.

Baseline rules:

  • domain-core: JUnit/Jqwik/AssertJ only as behavior requires;
  • application-core and shared-contract: JUnit/Jqwik/AssertJ and hand-written fakes;
  • Spring slices: owning Boot 4 focused test starter only;
  • Testcontainers modules and provider drivers: integrationTestImplementation only;
  • system harness: protocol client/assertion libraries only, never production output;
  • strict dependency locks include every new resolvable test configuration.

This repository is pinned to Spring Boot 4.0.0, whose dependency management currently resolves JUnit 6.0.1 and Testcontainers 2.0.2. Examples written for Boot 3 or Testcontainers 1.x are not copied mechanically. A Boot patch uplift is a separate compatibility change with its own evidence.

9. Boundary contracts and reusable fixtures

9.1 Contract-kit pattern

When application-core owns a port, it may publish a test-only abstract contract from testFixtures. Each outbound adapter supplies a factory and runs the same behavioral examples against its implementation.

application-core test fixture: CachePortContract
  -> in-memory fake contract test
  -> Redis adapter deterministic codec test
  -> Redis adapter real-provider integration test

The contract kit should express only application-visible semantics: key validity, idempotency, not-found behavior, expiry guarantees, error categories, and concurrency expectations that the port actually promises. It must not import Redis, JPA, HTTP, Spring MVC, or persistence entity types.

Provider-specific behavior remains in the adapter suite. For example, a generic repository port contract does not replace PostgreSQL tests for unique constraints, isolation, SQL-state translation, locking, or outbox atomicity.

9.2 Contract testing across process boundaries

Use checked-in schema/snapshot compatibility while this repository is a single template:

  • OpenAPI and GraphQL schema snapshots;
  • protobuf descriptor/binary compatibility;
  • event schema/envelope compatibility;
  • notification/webhook example payloads.

Serialization tests for a shared-contract type live in the consuming web, gRPC, GraphQL, messaging, or other adapter and use that adapter's production serializer. shared-contract stays framework-free and does not acquire Jackson, Spring, transport, or provider test dependencies.

Introduce one consumer-driven contract framework only when independently released consumers and provider verification workflows actually exist. Spring Cloud Contract or Pact can distribute examples, but neither proves a complete workflow, deployment correctness, authorization, or provider performance. Running both without distinct consumers would add ceremony rather than evidence.

9.3 Test-double vocabulary

Use names according to behavior:

  • dummy: required but unused value;
  • stub: fixed answers;
  • spy: records interaction;
  • fake: working but simplified implementation;
  • mock: expectation-driven interaction verifier;
  • simulator: protocol-level substitute such as MinIO or a loopback HTTP server.

Tests and reports must say which substitute was used. Calling MinIO “S3 E2E” or a hand fake “Redis integration” is prohibited.

10. Environment topology

10.1 Environment matrix

Environment Artifact/target Data and services Blocking purpose Explicit limitation
Developer/PR deterministic Compiled classes Heap, temp directory, ephemeral loopback Fast behavior, slice, architecture, schema No real provider or packaged artifact
PR provider integration Compiled owning adapter Disposable Testcontainers per job/run Required provider/driver/transaction contract Local container topology only
PR candidate system Exact boot JAR/OCI digest built once Disposable CI Compose stack, random ports Startup, readiness, configuration, critical black-box path Not managed cloud or production ingress
Main/merge candidate Canonical OCI digest built from the protected merge commit Required disposable providers plus system stack Reverify the merged revision and create the only promotable candidate A passing PR head is not evidence for a different merge commit
Nightly compatibility Same candidate or main artifact Version/topology matrix, faults, historical migration fixtures Upgrade, resilience, supported-version breadth Longer cadence delays feedback
Protected provider sandbox Candidate adapter/application Real cloud/provider tenant with least privilege IAM/KMS/TLS/quota/control-plane compatibility Still not production scale/data/topology
Staging/pre-production Promotable immutable digest Production-like deployment and sanitized synthetic data Ingress, rollout, observability, safe DAST/load smoke Configuration and traffic never perfectly equal production
Production Promoted digest Production control plane Read-only/idempotent synthetic and rollback signal No destructive test, load test, or chaos by default

An environment name is not evidence by itself. Each run records artifact digest, source revision, suite registry version, provider/image versions, topology, feature flags, and sanitized configuration fingerprint.

10.2 Provisioning rules

  • Testcontainers owns leaf integration dependencies and their lifecycle.
  • Spring Boot service connections are preferred when a supported container module exists; a GenericContainer needs an explicit connection name or configuration mapping.
  • Containers use exact reviewed tags and, for release evidence, digests.
  • Testcontainers reusable-container mode is developer-only: it is experimental and must be off in CI.
  • CI workers use a supported Docker environment. “Docker unavailable” is a preflight failure for a selected integration job, never a JUnit assumption.
  • Compose is reserved for multi-process candidate-image/system evidence, not as the ordinary leaf test dependency mechanism.
  • Provider credentials are short-lived, least-privilege, masked, and issued only to protected jobs. Fork pull requests cannot reach provider sandboxes.

11. Test data, time, identity, and cleanup

Every run receives a non-secret TEST_RUN_ID; parallel workers additionally include the Gradle worker ID. That identity scopes every mutable resource:

Resource Isolation key and cleanup
PostgreSQL Database or schema per run/worker; explicit truncation or container disposal after commit/concurrency tests
MongoDB Database per run/class; drop on completion
Redis Run-specific key prefix; bounded TTL; never FLUSHALL on a shared target
Kafka/broker Unique topic and consumer-group suffix; delete where supported or let disposable broker die
S3/MinIO Bucket or prefix per run; version/delete markers included in cleanup
Filesystem JUnit @TempDir; no repository-relative or developer-home mutable paths
HTTP/provider sandbox Idempotency key and tenant/run namespace; compensating cleanup with an audit trail

Rules:

  1. Unit/component tests receive an injected fixed Clock, seeded random source, and deterministic identifier generator where time/identity affects behavior.
  2. Property-test and randomized-concurrency seeds are printed in XML/log evidence and can be replayed.
  3. Generated credentials exist only for the run, never in source, fixtures, snapshots, command output, or uploaded logs.
  4. Tests never use production data. A release migration rehearsal may use an approved, de-identified, access-controlled snapshot with owner, retention, deletion, and audit policy.
  5. Cleanup runs in finally/post-job even after failure. Cleanup failure is visible and does not erase the original test failure.
  6. Polling uses a bounded condition with a diagnostic timeout. Thread.sleep is not a synchronization protocol.
  7. A repository test never requires a developer's .env.local, persistent Compose volume, or pre-existing localhost service.

Rollback is an optimization, not a universal isolation guarantee. Tests that verify commit-time constraints, listeners, outbox records, retries, concurrent transactions, or a real HTTP server must commit deliberately and clean their data explicitly.

12. Spring and JUnit execution hazards

12.1 Transaction truth

  • @SpringBootTest(webEnvironment = RANDOM_PORT) runs server work on another thread and transaction; a transaction on the test method does not roll back writes made by the server.
  • A default @DataJpaTest rollback can hide deferred constraints, commit callbacks, and outbox behavior. Important JPA tests explicitly flush, clear the persistence context, reload state, and use an explicit commit test where the contract is commit-time.
  • Preemptive timeout mechanisms can execute work on a different thread from Spring's thread-bound test transaction and accidentally commit it. Use framework-aware/non-preemptive timeouts for transactional tests, and capture a thread dump when a process-level deadline fires.

12.2 Context cache

Spring's test context cache is static within one JVM and has a finite default maximum. Excess profiles, unique dynamic property functions, mock-bean declarations, and @DirtiesContext create distinct cache keys or evict contexts. Forked JVMs cannot share the cache.

The implementation should:

  • define a small named set of test context archetypes;
  • reuse configuration and dynamic properties within an archetype;
  • prefer hand fakes or explicit test configuration over per-class context mutation;
  • report context cache statistics during optimization;
  • avoid @DirtiesContext unless the test truly corrupts shared context state.

Context-start failures must fail quickly rather than repeating the same expensive failure across hundreds of classes.

12.3 Parallel execution

Parallelism is opt-in by suite:

  • pure unit/property tests may run concurrently after shared-static-state review;
  • Spring tests using @DirtiesContext, per-test mock-bean mutation, shared database state, or ordered lifecycle remain sequential;
  • Testcontainers' JUnit integration does not promise parallel execution safety, so stateful provider suites start sequentially and parallelize at the CI job/provider level first;
  • JUnit resource locks protect unavoidable JVM-global resources such as timezone, system properties, and singleton registries;
  • all Test tasks receive an explicit timezone/locale, bounded task timeout, heap policy, and deterministic parallel configuration.

Static Testcontainers fields also need lifecycle review: a container stopped after a class can leave a cached Spring context pointing at a dead service. Context-managed container beans or well-scoped shared fixtures are safer when the context is reused.

13. Provider qualification matrix

The support policy must distinguish a protocol baseline from a production-readiness claim.

Capability Required integration baseline Conditional qualification for a claimed feature Not established locally
PostgreSQL/JPA Pinned PostgreSQL, Flyway, mappings, constraints, SQL-state translation, transaction/outbox/idempotency, representative query plan Each supported major upgrade, lock/concurrency and managed-provider TLS/IAM Multi-AZ failover, production cardinality/IO
MongoDB Pinned real MongoDB, codecs, indexes, queries Replica set for transactions/change streams; supported upgrade path Atlas control plane, sharding/failover unless targeted
Redis Pinned standalone Redis, Lua/functions, TTL, serialization, atomicity ACL/TLS, Sentinel/Cluster, eviction, restart/failover when advertised HA or cluster safety from standalone
HTTP client Real loopback sockets, TLS fixture, pool/cancellation/deadline, Toxiproxy latency/reset Corporate proxy/DNS/provider sandbox and supported JDK matrix External API correctness from WireMock/stub
Messaging Real supported broker, ack, ordering boundary, duplicate/redelivery, retry/DLT, schema Broker version/topology, auth/TLS, restart/partition fault End-user completion without a real consumer flow
Object storage MinIO S3 protocol baseline, multipart/checksum/error mapping AWS sandbox for IAM, KMS, versioning, presigned URL, lifecycle, throttling AWS control-plane equivalence from MinIO
File server Temp filesystem for path/journal rules Each supported mount/filesystem, permissions, disk-full, crash recovery, multi-process fencing Distributed consistency from local disk
Notification Fake routing/template plus local SMTP/webhook receiver Provider sandbox for auth, throttling, timeout, receipt/webhook mapping Human inbox placement/deliverability
Web MockMvc slice plus real embedded HTTP component Proxy headers, TLS, compression, body/connection limits Ingress/WAF behavior
gRPC Real loopback feature RPC with auth/deadline/cancellation TLS/mTLS, proxy/load balancer, streaming/backpressure Mesh/provider behavior
GraphQL Schema/resolver/security/complexity tests Real HTTP/WebSocket transport, subscriptions, DataLoader query count Arbitrary client query safety
WebSocket Real handshake/STOMP/origin/auth/reconnect Broker relay, slow consumer/backpressure, proxy idle timeout Durable exactly-once delivery

This matrix becomes executable only after the adopting project declares which optional capability and topology it supports. Unclaimed optional features remain documented exclusions rather than permanently skipped tests.

14. Migration verification

Database migration evidence has three distinct paths:

  1. Empty install: an empty supported database migrates to current and the application starts.
  2. Upgrade: a checked-in fixture from every supported upgrade baseline migrates to current, preserves named invariants, and passes Flyway validation.
  3. Compatibility window: when rolling deployment is supported, old and new application versions can coexist through the declared expand/contract window.

Migration fixtures contain structure and synthetic boundary data, not copied production records. Each fixture declares source application and schema versions, source release artifact digest, database engine and migration-tool versions, reproducible generation command, fixture and migration checksums, invariant manifest, and retirement rule. Upgrade fixtures are generated or verified against the immutable historical release's migration artifacts; a hand-edited dump without that provenance is not release evidence.

The release lane also measures migration duration and lock behavior on a representative synthetic scale. Its threshold is derived from the deployment error budget. A successful small-container migration cannot establish production lock duration or permit an automatic downgrade. Rollback is usually application roll-forward plus data repair; destructive database downgrade requires a separately designed and tested policy.

migrationTest should not be created merely to rename existing JPA integration tests. Add it when historical fixtures or destructive lifecycle require separate retention, permissions, cadence, or timeouts.

15. Candidate artifact and system environment

15.1 Build once, test the promotable bits

For each source revision, CI creates the boot JAR once after the required deterministic and provider verification for that revision. The image build copies that exact prebuilt JAR rather than invoking bootJar again, records both SHA-256 digests and their provenance relationship, and treats the OCI digest as the canonical promotable artifact. System, security, staging, and promotion reuse that digest; a later job must not rebuild “equivalent” bits.

The current src/Dockerfile runs bootJar inside the image build, so this guarantee does not exist yet. Phase 3 must refactor the Docker build input or explicitly choose an image-only build pipeline before claiming artifact identity.

The current local bootstrap/bootstrapSmoke workflow is useful developer evidence but cannot be the required system gate unchanged. Add a CI-only docker-compose.test.yml or generated override:

  • unique Compose project name derived from TEST_RUN_ID;
  • generated credentials and random host ports;
  • no .env.local, developer secrets, named persistent volumes, or restart policy;
  • read-only/non-root runtime constraints retained;
  • deterministic health/readiness deadlines;
  • logs, inspect output, resource usage, and sanitized environment captured before teardown;
  • down --volumes --remove-orphans in an unconditional cleanup step.

15.2 Minimum black-box probes

The system harness observes application behavior only through public network interfaces. It may use the Docker/orchestrator control plane for process lifecycle, signal delivery, dependency fault injection, digest inspection, and diagnostic collection; it must never call internal beans, controllers, or repositories. It verifies:

  • process/container starts under production-like profile and filesystem/user constraints;
  • liveness and readiness have distinct semantics and readiness waits for required dependencies;
  • build revision/image digest and effective non-secret feature profile are observable;
  • malformed, unauthenticated, unauthorized, oversized, and unsupported-content requests fail with the public error contract and no sensitive disclosure;
  • graceful shutdown removes readiness first, drains bounded in-flight work, and exits within the declared platform budget;
  • one representative critical flow is exercised for each enabled inbound protocol, with state verified through a public read path or provider observation rather than an internal repository;
  • required migration and dependency-loss behavior match the declared startup/readiness policy.

The representative flow is not selected by the template in the abstract. An adopting application must name business-critical journeys and their data cleanup contract.

16. Security, resilience, observability, and performance

16.1 Security

Security evidence is layered:

  • unit tests for authorization policy and redaction;
  • transport slice tests for authentication mapping, CSRF/CORS/origin rules, validation, and error disclosure;
  • system negative tests through the full filter chain and candidate runtime;
  • dependency/secret/container/static scanning;
  • authenticated DAST against an ephemeral or staging target;
  • manual threat-model and abuse-case review for controls scanners cannot infer.

The control catalog maps to a selected OWASP ASVS version; test techniques may reference OWASP WSTG. Scanner success is not a proof that the application has no vulnerability. Production security synthetics are non-destructive and explicitly allowlisted.

16.2 Resilience

Pure tests verify retry budgets, backoff calculations, idempotency decisions, circuit-state transitions, and cancellation propagation. Provider integration injects explicit socket latency, connection reset, timeout, dependency restart, duplicate delivery, and partial response using a fault proxy or provider control.

Every scenario asserts both the caller result and bounded side effects:

  • total attempts and elapsed budget;
  • no retry of forbidden/non-idempotent operations;
  • connection/thread/resource recovery;
  • correct metrics/traces/log redaction;
  • readiness degradation or continued service according to policy;
  • no duplicate durable outcome where idempotency is promised.

Chaos is not a synonym for randomness. Fault, scope, duration, expected steady state, abort condition, and cleanup are versioned inputs. Broad production chaos is out of scope until the organization has an owner and safety process.

16.3 Observability

Tests use Micrometer's observation test facilities or an in-memory registry to assert semantic names, low-cardinality tags, error/timeout status, trace propagation, and secret/PII exclusion. Candidate system tests confirm actuator exposure policy and correlation across a real inbound to outbound call.

They cannot establish dashboard correctness, alert routing, collector capacity, or production cardinality. A staging/post-deploy observability check must inject a known signal and confirm it reaches the configured backend/alert path.

16.4 Performance

  • JMH is used for microbenchmarks of isolated CPU/allocation-sensitive algorithms only.
  • k6 or an equivalent external driver targets the immutable system artifact for latency, throughput, and error-rate thresholds.
  • thresholds are derived from an agreed SLO and workload model, not invented from a shared PR runner.
  • PR may run a small non-gating regression smoke; blocking load/soak runs on controlled, comparable runners nightly or before release.
  • reports record warm-up, JVM flags, CPU/memory limits, dataset/cardinality, concurrency, request mix, duration, provider topology, and artifact digest.

JUnit wall-clock assertions on a busy shared runner do not qualify as performance tests. A passing small load test does not prove maximum production capacity.

17. Determinism, flaky tests, and diagnostics

The existing 14-day quarantine remains an emergency containment mechanism, not a second backlog. Extend it to every registered source set and Java/Groovy. A quarantine entry requires owner, tracking issue, symptom, first/last observed time, deterministic reproduction evidence, and expiry.

Policy:

  • a required gate never converts an infrastructure error or unexpected skip into quarantine;
  • blind auto-retry cannot turn the first failure green;
  • one diagnostic rerun may be retained, but the job remains failed and preserves both attempts;
  • repeatedly failing tests are fixed or removed only with a replacement evidence argument;
  • clock, random seed, port, ordering, locale, timezone, thread scheduling, and external resource ownership are controlled explicitly;
  • process-level timeouts collect thread dump, test task state, container state, and last logs before termination.

Each failed provider/system job uploads:

  • JUnit XML and HTML report;
  • source revision, artifact and provider image digests;
  • suite ID, seed, timezone/locale, Java/Gradle/OS/Docker fingerprint;
  • sanitized application/container logs and container inspection;
  • thread dump and resource snapshot on hang/timeout;
  • migration/provider diagnostics relevant to the owning suite.

Secrets and payloads are redacted before artifact upload. Retention follows the repository's security and incident policy.

18. Coverage and test effectiveness

Add separate JaCoCo execution data and aggregate reports for deterministic and integration lanes. Do not merge them so early that a provider test hides a missing unit-level decision test.

Adoption sequence:

  1. publish a baseline without a blocking percentage;
  2. inspect packages/classes with meaningful production behavior but no exercised branch;
  3. require named risk scenarios for changed domain/application policy and changed adapter boundaries;
  4. introduce a changed-code coverage ratchet once the baseline is stable;
  5. apply package-specific floors only when owners understand generated code, DTOs, configuration, and unavoidable branches;
  6. run mutation analysis nightly on pure domain-core/application-core policy, not on the whole Spring/container stack.

Coverage means code was executed; it does not prove the assertion would detect a defect. Mutation survival is stronger diagnostic evidence but still does not replace missing business scenarios, contract examples, or production topology tests.

19. CI and release graph

registry / compile / static / dependency / taxonomy preflight
                    |
                    v
       deterministic test + architecture + snapshots
          |              |                 |
          |              +--> sample-off --+
          |                                |
          +--> provider integration matrix-+
                                           v
                         build main/merge candidate once
                                           |
                         +-----------------+------------------+
                         v                                    v
                 candidate image smoke              migration compatibility
                         |                                    |
                         +-----------------+------------------+
                                           v
                     GitHub release-gate (caller-workflow fan-in)
                                           |
                       scheduled/provider/staging qualification
                                           |
                                  immutable promotion
                                           |
                              post-deploy safe synthetic

19.1 Pull-request blocking lanes

  1. Preflight/control: suite/module registries, format/static, dependency locks, environment keys, taxonomy, quarantine drift, compile.
  2. Deterministic quality: all leaf test, ArchUnit, public-path/schema snapshots, sample-on and sampleOffTest.
  3. Provider matrix: required PostgreSQL, Redis, MongoDB, broker, and MinIO suites according to enabled capability registry; jobs parallelize by provider but suites remain fail-closed.
  4. Candidate build: boot JAR/image plus SBOM/provenance/digest.
  5. Candidate smoke: disposable Compose and black-box minimum probes.
  6. Gate fan-in: add a .github/workflows/ci-release-candidate.yml caller that invokes ci-quality-gates.yml through workflow_call as one job and dependency-vulnerability.yml—including trivy-fs—as another reusable-workflow job, then uses needs from its release-gate to those jobs and the candidate jobs in that caller. The gate matrix remains the complete blocking-check inventory; its validator fails when the caller mapping, reusable workflow, or required-check identity is missing or renamed.

Do not add path-based job skipping initially. This repository is small enough that correctness of the evidence graph is more valuable. Optimize only from measured duration/cache data and keep a periodic full run.

PR artifacts are diagnostic and unpromotable. A protected main/merge lane reruns all required deterministic and provider suites against the actual merge commit, builds the canonical JAR/image once, runs migration and candidate smoke against that digest, and retains its provenance. Only that merge-commit candidate can advance to staging or release.

19.2 Main/merge candidate lane

  1. Revalidate registries, dependency locks, deterministic tests, and required provider suites on the protected merge commit.
  2. Build the preverified JAR once, copy it into the OCI image, and publish immutable provenance.
  3. Run migration compatibility and black-box system smoke against the published digest.
  4. Let the caller workflow's release-gate fan in every required reusable/candidate job; retain the successful digest as the only promotable candidate.

19.3 Scheduled/release lanes

  • supported provider and version topology matrix;
  • historical migration and rolling-compatibility tests;
  • fault/restart/network resilience;
  • authenticated DAST;
  • controlled load, soak, and resource-leak tests;
  • protected real-provider sandbox qualification;
  • optional mutation analysis and dependency upgrade compatibility.

Scheduled failure creates an owned signal and blocks release according to capability policy; it is not an informational dashboard that can remain red indefinitely.

GitHub Actions service containers are acceptable for job-level utilities, but Testcontainers remains the leaf integration mechanism because lifecycle, network endpoint, and image selection stay close to the test. The candidate application itself is tested as an image in the system lane.

20. Evidence ledger and claim discipline

Every externally meaningful capability should have a short ledger entry in generated test documentation:

Field Example kind of value
Claim “Repository save and idempotency are atomic on supported PostgreSQL”
Owner persistence-jpa
Evidence suites unit port contract, PostgreSQL integration, migration path, candidate smoke
Real/replaced real PostgreSQL; application transport may be replaced in leaf integration
Environment/version image digest/topology or provider sandbox identifier
Last result/artifact CI run and immutable report link
Known exclusions managed failover, production cardinality
Expiry/requalification provider/app version or time-based trigger

The following phrases are forbidden unless the corresponding evidence exists:

  • “E2E tested” for a slice, mocked port, or test-only application;
  • “production-ready Redis” after only a fake or standalone path when Cluster/Sentinel is claimed;
  • “S3 compatible” from SDK mocks alone, or “AWS verified” from MinIO;
  • “migration safe” after only empty-database startup;
  • “performance proven” without an artifact, workload, controlled environment, and threshold;
  • “secure” because scanners are green;
  • “all tests passed” when a selected required suite skipped, discovered zero tests, or did not run.

21. Incremental rollout

Phase 0 — classify and freeze the baseline

  • approve this taxonomy and capability/support claims;
  • add the test-suite registry schema and verification task;
  • record current task/class/discovery/skip/duration/context-cache baseline;
  • rename misleading E2E/IT classes without changing behavior;
  • declare intentional skip reasons and owners.

Exit: every current suite is assigned an owner, boundary, environment, purpose, and CI policy.

Phase 1 — separate deterministic and provider lanes

  • create explicit integrationTest convention/source set;
  • move existing PostgreSQL and MinIO tests without changing assertions;
  • migrate the Redis real-service lane into the same model and make it CI-required when Redis is an enabled capability;
  • remove Docker assumptions/disabledWithoutDocker from required provider suites;
  • make test Docker-free and add discovery/skip enforcement;
  • update strict dependency locks.

Exit: test succeeds on a runner with no Docker, while selected integrationTest fails preflight without Docker and executes real providers when Docker exists.

Phase 2 — close capability gaps

  • move provider semantics from app-bootstrap to owning leaves;
  • add real MongoDB/broker and missing feature-transport integrations for enabled capabilities;
  • create application-owned reusable port contract fixtures;
  • add provider/version/topology policy and fault cases;
  • keep composition-only checks in app-bootstrap.

Exit: each enabled production capability has deterministic contract evidence and at least its declared provider baseline.

Phase 3 — immutable candidate system gate

  • build candidate once;
  • add disposable CI Compose/system harness and production-like health/readiness/shutdown checks;
  • name critical reference/sample journeys;
  • feed all blocking jobs into the existing gate matrix.

Exit: CI proves the promotable image boots and crosses its declared public/provider boundaries.

Phase 4 — migration and non-functional qualification

  • add historical migration fixtures and rolling compatibility where relevant;
  • add security mapping/DAST, Toxiproxy faults, observation assertions, controlled load/soak;
  • publish split JaCoCo baseline and targeted mutation reports;
  • add protected provider sandboxes only for advertised managed capabilities.

Exit: release claims have owned, reproducible evidence and explicit exclusions.

Phase 5 — measured optimization

  • analyze duration, context-cache churn, container startup, and runner utilization;
  • tune job sharding and safe unit parallelism;
  • introduce changed-code coverage ratchets and evidence expiry;
  • consider Gradle JVM Test Suite adoption only if it materially simplifies the proven model.

Exit: optimization preserves the fail-closed evidence graph and is backed by before/after data.

22. Acceptance criteria

The test-environment implementation is complete only when all of the following hold:

  1. modules.json still registers exactly 19 production leaves and no new forbidden production dependency edge exists.
  2. Every test suite is registered to one owning leaf, one execution environment, and at least one verification purpose.
  3. ./gradlew test is deterministic and succeeds without Docker, Internet, manual services, .env.local, persistent volume, or fixed port.
  4. Selecting a required integration suite with Docker unavailable fails before JUnit discovery.
  5. Required suites fail on zero discovery and unexpected skip, while explicitly approved conditional/meta-test skips remain visible.
  6. Testcontainers dependencies are absent from ordinary deterministic source-set classpaths.
  7. Provider images/versions/topologies and compatibility exclusions are declared and emitted in reports.
  8. Mutable data is namespaced per run/worker and cleanup is verified after success and failure.
  9. Candidate system tests observe application behavior only through public ports; lifecycle and fault controls stay in the external orchestrator plane. The canonical OCI provenance identifies the exact prebuilt JAR it contains.
  10. Migration evidence includes empty install and every declared upgrade baseline; rolling compatibility is tested if advertised, and historical fixtures have immutable release provenance.
  11. All blocking work is represented inside one caller as ordinary or reusable-workflow jobs; its release-gate cannot stay green if one is missing or renamed, and the only promotable candidate was built and reverified from the protected main merge commit.
  12. Reports preserve first failure, seed, environment fingerprint, artifact/provider digests, and sanitized diagnostics.
  13. Security, resilience, observability, and performance claims list the specific environment and exclusions they cover.
  14. Test fixtures cannot become production dependencies, and adapter/provider types cannot leak into core contract kits.
  15. Focused leaf verification, aggregate deterministic/integration checks, candidate smoke, and architecture gates have fresh executable evidence.
  16. The LLM Wiki branch note records implementation decisions, commands, failures, and evidence grade before completion is claimed.

23. Expected implementation impact

The design anticipates changes in these areas; this document does not yet authorize or implement them:

  • src/config/testing/test-suites.json and its schema;
  • src/build.gradle or a build-logic convention for source sets/tasks/verification;
  • focused leaf Gradle dependencies and lockfiles;
  • relocation/rename of existing tests without changing their initial behavior;
  • provider fixtures and port contract test fixtures;
  • a new .github/workflows/ci-release-candidate.yml caller/fan-in workflow;
  • workflow_call entrypoints in ci-quality-gates.yml and dependency-vulnerability.yml;
  • .github/ci-gate-matrix.yml plus its caller/reusable-job mapping validation;
  • src/Dockerfile or its build context so the image consumes the exact prebuilt boot JAR;
  • a CI-only disposable Compose override/system harness;
  • qa/security, qa/performance, and evidence/report publishing;
  • contributor documentation for choosing an owner, source set, and local command.

Explicit non-goals for the first implementation:

  • adding a twentieth production module;
  • changing a business port or production dependency direction merely for testing convenience;
  • immediately supporting every optional provider topology;
  • enforcing an arbitrary global coverage percentage;
  • running destructive DAST/load/chaos against production;
  • using cloud credentials in untrusted pull-request jobs;
  • replacing focused leaf tests with a single slow system suite.

24. Owner decisions required before implementation

The architecture can be implemented incrementally, but these values cannot be derived honestly from the skeleton:

  1. Which adapters are mandatory in the default template CI versus optional capability profiles?
  2. Which PostgreSQL, MongoDB, Redis, broker, object-storage, and JDK versions/topologies are supported?
  3. What are the maximum PR and release-lane budgets, runner topology, artifact retention, and quarantine service-level agreement?
  4. Which public journeys are release-critical when the sample portfolio is disabled?
  5. Is rolling application/schema compatibility promised, and for how many released versions?
  6. Which managed-provider sandboxes exist, who owns cost/credentials/cleanup, and which claims do they qualify?
  7. What SLO/workload/error budget defines readiness, graceful shutdown, migration, and performance thresholds?
  8. Which ASVS level/control set and DAST target policy does the adopting organization require?

Default pending those decisions:

  • all capabilities included in the ordinary production composition are required PR integrations;
  • optional/sample-only capabilities are explicitly non-required, not assumption-skipped;
  • test remains Docker-free and provider jobs fail closed;
  • managed topology, production capacity, and rolling compatibility are not claimed;
  • current CI has no invented numeric performance or duration gate.

25. Official references

The design is based on repository evidence plus the following primary documentation:

Versioned Spring Boot /4.0/ documentation can reflect a later 4.0.x patch than this repository's exact 4.0.0 baseline. Any API or dependency not verified against the locked build remains a proposal until implementation tests it.

Reference in New Issue View Git Blame Copy Permalink