Files
clean-architecture-backend-…/scripts/tests/test_compose_contract_verdict.py
DongHyeonka 474bd0462c
pr / done (pull_request) Successful in 1m42s
fix(compose): 계약 검사의 판정을 시험 있는 스크립트로 옮기고 빈 입력 통과를 막는다
scripts/verify-compose-profile-contracts.sh는 prCheck의 게이트인데 판정
열 가지가 셸 안의 jq라 시험이 없다. 고장 난 입력을 넣어 보면 넷이 통과한다.
- 계약의 lanes가 비면 lane 반복이 돌지 않고 "all 0 lanes match"로 끝난다.
- persistence-jpa를 켠 lane에 expectedPersistenceVendor가 없으면 통과한다.
  smoke는 보고된 vendor를 그 칸과 비교하니 그 lane은 매번 실패한다.
- heap dump 경로는 짧은 `tmpfs:` 목록의 같은 항목만 비교한다. 위 디렉터리의
  tmpfs(/var/tmp)나 long syntax `type: tmpfs` volume은 통과한다.
- Redis 계정을 이름 그대로가 아니라 정규식으로 찾아 `.`이 아무 글자와 맞는다.
  `ca.skeleton-application`을 가리키는 lane이 `ca-skeleton-application` 줄로
  통과한다.

판정을 scripts/compose-contract-verdict.py로 옮긴다. 셸은 docker만 부른다.
`version`으로 Compose 하한을 먼저 보고, `lanes`가 준 lane별 인자로
`config --services`와 `config --format json`을 돌려 출력과 rc를 파일에 쓴다.
rc는 조건 밖에서 읽는다. 그다음 `judge`가 그 디렉터리를 읽는다. 판정은 위
넷을 실패로 보고, 파일 없는 lane, 겹치는 id, smoke가 읽는 목록이 없거나 그
목록에 이름이 아닌 원소가 든 lane, 셸이 렌더 기록을 남기지 않은 lane도 실패로
본다. docker 호출의 stdin은 /dev/null이라 lane 목록을 먹지 않는다.
scripts/tests/test_compose_contract_verdict.py가 판정마다 고장 난 입력과 정상
입력을 보고, 셸을 가짜 docker로 돌려 lane 인자·rc 기록·판정 rc를 본다.
prCheck의 testDocumentChecks가 돌린다. 루트 build.gradle의 주석은 PR 단계
Exec 태스크마다 시험 파일을 적어 사실과 맞춘다.

버린 안: 모델 JSON의 service key로 `config --services`를 대신하기. 렌더를
한 번 줄이지만 지금 lane 결과와 같은지 따로 보여야 해서 두 호출을 그대로 둔다.
bats는 desktop과 runner 둘 다에 새 도구를 깔아야 한다.

위험·한계: 판정이 jq와 다른 곳은 위 넷 말고도 있다. main이 통과시키고 이제
실패로 보는 것: 같은 사용자를 두 줄에서 켠 ACL(Redis가 거절한다), 값이
null·false인 realm의 `_` key, 최상위 값이 둘인 realm, LANE_ID 꼴이 아닌 id,
빈 springRuntime, `files`가 없는 lane, 이름이 아닌 목록 원소, 읽을 수 없는
Compose 버전. main이 실패시키고 이제 통과하는 것: ACL 줄의 겹친 공백과 CRLF,
끝에 `/`가 붙은 bind target, 로캘에 따라 달라지던 service 정렬, list 꼴
environment, 파일 하나로 bind mount한 ACL(target이 --aclfile 경로와 같음).
Redis가 ACL 줄의 `\r`와 겹친 공백을 받아들인다는 것은 acl.c를 열어 보지 않은
판단이다. 실제 계약 18 lane에서는 main과 이 트리의 출력 줄이 같다(PR 본문).

검증: 노트북 `python3 -m unittest discover -s scripts/tests` 275개 통과. 판정의
빈 lane 거절·vendor 검사·heap 상위 tmpfs·long syntax tmpfs·비밀번호 글자 비교·
정확한 service 집합·사용자 이름 비교·렌더 기록 없음·파일 없는 lane·목록 원소
검사를 되돌리거나, judge_lane에서 판정 하나를 빼거나 realm 판정을 빼거나, 셸의
두 렌더 rc를 `|| true`로 삼키거나, docker가 stdin을 읽게 두거나, judge 호출을
지우거나 `|| true`로 바꾸거나, version 실패를 무시하거나, profile 인자를 빼면
새 시험이 실패한다(변이 23개 모두). desktop RED·GREEN과 prCheck 결과는 PR
본문에 있다.

Refs: REV-BE-0016
2026-11-22 01:10:29 +09:00

830 lines
36 KiB
Python

"""scripts/compose-contract-verdict.py의 시험. Compose 계약의 정적 판정이 고장 난 입력을 실패로 보는지 본다:
lane이 없는 계약, 파일이 없는 lane, 렌더 실패와 셸이 남기지 않은 렌더 기록, service 집합·Spring runtime·mount 중복,
role 분할, 기대 vendor가 없는 JPA lane, Redis 계정 대조(비밀번호의 `*`를 포함해), tmpfs 위의 heap dump 경로,
Keycloak realm의 `_` key. 정상 입력은 통과하는지도 같이 본다. 마지막으로
scripts/verify-compose-profile-contracts.sh를 가짜 `docker`로 돌려, 셸이 판정에 lane 인자와 렌더 결과를 넘기고
그 rc로 끝나는지 본다.
python3 -m unittest discover -s scripts/tests
"""
from __future__ import annotations
import importlib.util
import io
import json
import os
import shutil
import stat
import subprocess
import sys
import tempfile
import textwrap
import unittest
from contextlib import redirect_stderr, redirect_stdout
from pathlib import Path
SCRIPTS = Path(__file__).resolve().parent.parent
VERDICT = SCRIPTS / "compose-contract-verdict.py"
SHELL = SCRIPTS / "verify-compose-profile-contracts.sh"
_spec = importlib.util.spec_from_file_location("compose_contract_verdict", VERDICT)
verdict = importlib.util.module_from_spec(_spec)
sys.modules["compose_contract_verdict"] = verdict
_spec.loader.exec_module(verdict)
CONTRACT_PATH = "src/bootstrap/app/compose-lanes/compose-profile-contracts.json"
HEAP = "/var/tmp/heap"
ACL = "user default off\nuser app on >example-app-password ~prod:* +@all\nuser app-advanced on >example-advanced-password ~prod:* +@all\n"
def contract(*lanes: dict, **fields: object) -> dict:
document = {
"minimumComposeVersion": "2.24.4",
"composeFiles": {"base": "compose.yaml", "lane": "lane-services.yaml"},
"preStartServices": ["schema"],
"oneShotServices": ["smoke"],
"capabilities": ["persistence-jpa", "redis"],
"lanes": list(lanes),
}
document.update(fields)
return document
def lane(**fields: object) -> dict:
document = {
"id": "off-local", "composeProfiles": [], "files": ["base"], "springRuntime": "local",
"services": ["app"], "longRunningServices": ["app"], "oneShotServices": [], "blocking": True,
"activationEnv": {}, "expectedCapabilitiesOn": [], "expectedPersistenceVendor": "none",
}
document.update(fields)
return document
def app(**fields: object) -> dict:
service = {"environment": {"SPRING_PROFILES_ACTIVE": "local"}, "tmpfs": ["/tmp:mode=1777,size=128m"]}
service.update(fields)
return service
def model(**services: dict) -> dict:
return {"services": services or {"app": app()}}
class Repository:
"""A repository root with the files the verdict reads: compose files, a Dockerfile, an ACL file."""
def __init__(self) -> None:
self._directory = tempfile.TemporaryDirectory()
self.root = Path(self._directory.name)
self.write("compose.yaml", "services: {}\n")
self.write("lane-services.yaml", "services: {}\n")
self.write("Dockerfile", f'ENV JAVA_TOOL_OPTIONS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath={HEAP} \\\n')
self.write("compose/redis/acl/lane.acl", ACL)
def write(self, relative: str, text: str) -> Path:
path = self.root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(text, encoding="utf-8")
return path
def close(self) -> None:
self._directory.cleanup()
def rendered(services: str = "app\n", document: object = None, services_rc: int | None = 0,
model_rc: int | None = 0, error: str = "") -> verdict.Rendered:
return verdict.Rendered(
services_rc=services_rc, services=services, services_error=error,
model_rc=model_rc, model=json.dumps(model() if document is None else document),
)
class RepositoryTestCase(unittest.TestCase):
def setUp(self) -> None:
self.repository = Repository()
self.addCleanup(self.repository.close)
self.root = self.repository.root
class ContractShapeTest(RepositoryTestCase):
def test_a_list_entry_that_is_not_a_name_fails_the_lane(self) -> None:
# given: strings() drops such an entry, so the role and report checks would never see it
cases = {
"expectedCapabilitiesOn": [None],
"longRunningServices": ["app", 1],
"services": ["app", ""],
"composeProfiles": [["jpa"]],
"oneShotServices": [None],
"preStartServices": [False],
}
for field, entries in cases.items():
with self.subTest(field=field):
# when
renderable, problems = verdict.lanes(contract(lane(**{field: entries})), self.root)
# then
self.assertEqual(renderable, [])
self.assertEqual(problems, [f"off-local: '{field}' holds {json.dumps(entries)}, and every entry must be a name"])
def test_a_contract_with_lanes_that_name_existing_files_is_renderable(self) -> None:
# given
document = contract(lane(), lane(id="off-dev", files=["base", "lane"], composeProfiles=["jpa"]))
# when
renderable, problems = verdict.lanes(document, self.root)
# then
self.assertEqual(problems, [])
self.assertEqual(
verdict.compose_arguments(document, renderable[1], self.root),
["-f", str(self.root / "compose.yaml"), "-f", str(self.root / "lane-services.yaml"), "--profile", "jpa"],
)
def test_a_contract_with_no_lanes_fails_rather_than_matching_all_of_none(self) -> None:
for lanes in ({"lanes": []}, {"lanes": None}):
with self.subTest(lanes=lanes):
# given
document = contract(**lanes)
# when
renderable, problems = verdict.lanes(document, self.root)
# then
self.assertEqual(renderable, [])
self.assertEqual(problems, ["the contract declares no lanes, so nothing would be checked"])
def test_a_lane_without_files_fails_rather_than_rendering_whatever_compose_finds(self) -> None:
# given
document = contract(lane(files=[]))
# when
renderable, problems = verdict.lanes(document, self.root)
# then
self.assertEqual(renderable, [])
self.assertEqual(len(problems), 1)
self.assertTrue(problems[0].startswith("off-local: names no compose file under 'files'"), problems)
def test_a_lane_naming_an_unknown_or_missing_file_fails(self) -> None:
# given
(self.root / "lane-services.yaml").unlink()
document = contract(lane(files=["base", "tls", "lane"]))
# when
_, problems = verdict.lanes(document, self.root)
# then
self.assertEqual(problems, [
"off-local: names compose file 'tls', which composeFiles does not list",
"off-local: lane-services.yaml is missing",
])
def test_a_lane_without_the_lists_the_smoke_wrapper_reads_fails(self) -> None:
# given
broken = lane()
del broken["composeProfiles"], broken["expectedCapabilitiesOn"], broken["springRuntime"]
# when
_, problems = verdict.lanes(contract(broken), self.root)
# then
self.assertEqual(problems, [
"off-local: has no 'composeProfiles' list; the smoke wrapper reads it as one",
"off-local: has no 'expectedCapabilitiesOn' list; the smoke wrapper reads it as one",
"off-local: names no springRuntime",
])
def test_two_lanes_with_one_id_fail_because_their_records_would_overwrite_each_other(self) -> None:
# given
document = contract(lane(), lane())
# when
renderable, problems = verdict.lanes(document, self.root)
# then
self.assertEqual(len(renderable), 1)
self.assertEqual(problems, ["off-local: the id is used by more than one lane"])
def test_an_id_that_is_not_a_file_name_fails(self) -> None:
# given
document = contract(lane(id="../escape"))
# when
_, problems = verdict.lanes(document, self.root)
# then
self.assertEqual(problems, ["lane 0: id '../escape' is not a name such as off-local"])
class ComposeVersionTest(unittest.TestCase):
def test_a_compose_at_or_above_the_floor_passes(self) -> None:
for actual in ("2.24.4", "v2.29.1", "2.30.0-desktop.1", "3.0"):
with self.subTest(actual=actual):
self.assertEqual(verdict.compose_version(contract(), actual), [])
def test_a_compose_below_the_floor_fails(self) -> None:
for actual in ("2.24.3", "2.24", "1.29.2"):
with self.subTest(actual=actual):
problems = verdict.compose_version(contract(), actual)
self.assertEqual(problems[0], f"docker compose {actual} is below the required 2.24.4;")
def test_a_version_that_cannot_be_read_fails(self) -> None:
self.assertEqual(
verdict.compose_version(contract(), ""),
["docker compose reports '', which is not a version"],
)
self.assertEqual(
verdict.compose_version(contract(minimumComposeVersion=None), "2.29.1"),
["minimumComposeVersion '' is not a version such as 2.24.4"],
)
class RenderTest(RepositoryTestCase):
def judge(self, record: verdict.Rendered, subject: dict | None = None) -> list[str]:
subject = subject or lane()
return verdict.judge_lane(contract(subject), subject, record, self.root, HEAP)
def test_a_lane_that_renders_its_services_and_a_clean_model_passes(self) -> None:
self.assertEqual(self.judge(rendered()), [])
def test_a_stack_that_does_not_render_fails_with_what_compose_said(self) -> None:
# given
record = rendered(services="", services_rc=1, error="service \"app\" refers to\nundefined volume\n")
# when
problems = self.judge(record)
# then
self.assertEqual(problems, ["the stack does not render: service \"app\" refers to undefined volume"])
def test_a_lane_the_shell_left_no_record_for_fails(self) -> None:
for record in (rendered(services_rc=None), rendered(model_rc=None)):
with self.subTest(record=record):
self.assertEqual(self.judge(record), ["the shell left no record of rendering this lane"])
def test_a_superset_or_a_subset_of_the_services_fails(self) -> None:
for services in ("app\npostgres\n", ""):
with self.subTest(services=services):
problems = self.judge(rendered(services=services))
self.assertEqual(len(problems), 1)
self.assertTrue(problems[0].startswith("services are ["), problems)
def test_the_order_of_the_rendered_services_does_not_matter(self) -> None:
# given
subject = lane(services=["postgres", "app"], longRunningServices=["app", "postgres"])
document = model(app=app(), postgres={})
# then
self.assertEqual(self.judge(rendered(services="app\npostgres\n", document=document), subject), [])
def test_a_model_that_is_not_json_fails(self) -> None:
# given
record = verdict.Rendered(services_rc=0, services="app\n", services_error="", model_rc=0, model="services:\n")
# then
self.assertEqual(self.judge(record), ["the merged model does not render as JSON"])
self.assertEqual(self.judge(rendered(model_rc=15)), ["the merged model does not render as JSON"])
def test_another_spring_runtime_fails(self) -> None:
# given
document = model(app=app(environment={"SPRING_PROFILES_ACTIVE": "prod"}))
# then
self.assertEqual(
self.judge(rendered(document=document)),
["app renders SPRING_PROFILES_ACTIVE='prod', the contract says 'local'"],
)
def test_a_mount_target_declared_twice_fails_naming_the_service(self) -> None:
# given
document = model(app=app(volumes=[{"type": "bind", "source": "/data", "target": "/tmp"}]))
# then
self.assertEqual(self.judge(rendered(document=document)), ["a mount target is declared twice — app: /tmp"])
class RolePartitionTest(unittest.TestCase):
def test_a_service_filed_under_exactly_one_role_passes(self) -> None:
# given
subject = lane(services=["app", "schema", "smoke"], preStartServices=["schema"], oneShotServices=["smoke"])
# then
self.assertEqual(verdict.role_partition(contract(), subject), [])
def test_an_unfiled_a_not_rendered_and_a_twice_filed_service_fail(self) -> None:
# given
subject = lane(services=["app", "postgres"], longRunningServices=["app", "redis"], oneShotServices=["app"])
# when
problems = verdict.role_partition(contract(oneShotServices=["smoke", "app"]), subject)
# then
self.assertIn(
"services are not partitioned by role — unfiled=[postgres] not-rendered=[redis] twice=[app]",
problems,
)
def test_a_one_shot_listed_as_a_wait_target_fails(self) -> None:
# given
subject = lane(services=["app", "smoke"], longRunningServices=["app", "smoke"])
# then
self.assertEqual(
verdict.role_partition(contract(), subject),
["[smoke] are declared non-waiting but listed as --wait targets"],
)
def test_a_service_run_once_but_in_no_catalogue_fails(self) -> None:
# given
subject = lane(services=["app", "probe"], oneShotServices=["probe"])
# then
self.assertEqual(
verdict.role_partition(contract(), subject),
["[probe] are run with `run --rm` but are in neither top-level catalogue"],
)
class ApplicationReportTest(unittest.TestCase):
def test_a_vendor_that_agrees_with_persistence_jpa_passes(self) -> None:
on = lane(expectedCapabilitiesOn=["persistence-jpa"], expectedPersistenceVendor="postgresql")
self.assertEqual(verdict.application_report(contract(), on), [])
self.assertEqual(verdict.application_report(contract(), lane()), [])
def test_a_jpa_lane_without_an_expected_vendor_fails(self) -> None:
# given: the smoke wrapper compares the reported vendor with this field, so the lane fails every run
subject = lane(expectedCapabilitiesOn=["persistence-jpa"])
del subject["expectedPersistenceVendor"]
# when
problems = verdict.application_report(contract(), subject)
# then
self.assertEqual(problems, [
"the expected application report is unreachable — no expectedPersistenceVendor with persistence-jpa on",
])
def test_a_vendor_that_contradicts_persistence_jpa_fails(self) -> None:
cases = {
"none with jpa on": (lane(expectedCapabilitiesOn=["persistence-jpa"]), "vendor none with persistence-jpa on"),
"a vendor with jpa off": (lane(expectedPersistenceVendor="postgresql"), "vendor postgresql with persistence-jpa off"),
}
for case, (subject, reason) in cases.items():
with self.subTest(case=case):
self.assertEqual(
verdict.application_report(contract(), subject),
[f"the expected application report is unreachable — {reason}"],
)
def test_a_capability_outside_the_catalogue_fails(self) -> None:
# given
subject = lane(expectedCapabilitiesOn=["websocket"])
# then
self.assertEqual(
verdict.application_report(contract(), subject),
["the expected application report is unreachable — uncatalogued=[websocket]"],
)
class RedisAuthenticationTest(RepositoryTestCase):
def environment(self, **overrides: object) -> dict:
env = {
"APP_REDIS_ENABLED": "true",
"APP_REDIS_NAMESPACE_ENVIRONMENT": "prod",
"APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE": "secret://app@environment/REDIS_LANE_APPLICATION_PASSWORD",
"APP_REDIS_AUTHENTICATION_ADVANCED_CREDENTIAL_REFERENCE":
"secret://app-advanced@environment/REDIS_LANE_ADVANCED_PASSWORD",
"REDIS_LANE_APPLICATION_PASSWORD": "example-app-password",
"REDIS_LANE_ADVANCED_PASSWORD": "example-advanced-password",
}
env.update(overrides)
return {key: value for key, value in env.items() if value is not None}
def redis(self, **fields: object) -> dict:
service = {
"command": ["redis-server", "--appendonly", "no", "--aclfile", "/etc/redis/acl/lane.acl"],
"volumes": [{"type": "bind", "source": str(self.root / "compose/redis/acl"),
"target": "/etc/redis/acl", "read_only": True}],
"healthcheck": {"test": ["CMD-SHELL", "redis-cli --user app --pass \"$$P\" ping"]},
}
service.update(fields)
return service
def problems(self, env: dict | None = None, redis: dict | None = None) -> list[str]:
subject = lane(activationEnv=self.environment() if env is None else env)
document = model(app=app(), redis=self.redis() if redis is None else redis)
return verdict.redis_authentication(subject, document, self.root)
def test_a_lane_that_names_accounts_its_redis_provisions_passes(self) -> None:
self.assertEqual(self.problems(), [])
def test_a_lane_that_does_not_switch_redis_on_is_not_checked(self) -> None:
self.assertEqual(self.problems(env={"APP_REDIS_ENABLED": "false"}, redis={}), [])
def test_a_password_with_a_star_does_not_match_an_account_with_another_password(self) -> None:
# given: '*' is a character of the password, not a wildcard
env = self.environment(REDIS_LANE_APPLICATION_PASSWORD="*")
# then
self.assertEqual(
self.problems(env=env),
["REDIS_LANE_APPLICATION_PASSWORD is not the password compose/redis/acl/lane.acl gives app"],
)
def test_a_wrong_password_fails(self) -> None:
self.assertEqual(
self.problems(env=self.environment(REDIS_LANE_ADVANCED_PASSWORD="example-app-password")),
["REDIS_LANE_ADVANCED_PASSWORD is not the password compose/redis/acl/lane.acl gives app-advanced"],
)
def test_a_user_name_with_a_dot_matches_only_itself(self) -> None:
# given: a regular expression read 'app.x' as 'app' plus any character plus 'x'
self.repository.write("compose/redis/acl/lane.acl", ACL + "user appXx on >example-x-password ~prod:* +@all\n")
env = self.environment(APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE="secret://app.x@environment/X",
X="example-x-password")
# then
self.assertEqual(
self.problems(env=env),
["APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE names app.x, which compose/redis/acl/lane.acl does not enable"],
)
def test_a_namespace_the_account_may_not_touch_fails(self) -> None:
problems = self.problems(env=self.environment(APP_REDIS_NAMESPACE_ENVIRONMENT="local"))
self.assertEqual(problems, [
"app may not touch local:*, the namespace this lane writes",
"app-advanced may not touch local:*, the namespace this lane writes",
])
def test_each_way_a_lane_falls_short_is_named(self) -> None:
cases = {
"anonymous": (self.environment(APP_REDIS_AUTHENTICATION_ANONYMOUS_ACCESS_ACCEPTED=True), None,
"accepts anonymous Redis access"),
"no namespace": (self.environment(APP_REDIS_NAMESPACE_ENVIRONMENT=None), None,
"sets no APP_REDIS_NAMESPACE_ENVIRONMENT"),
"no aclfile": (None, self.redis(command=["redis-server"]), "starts Redis without --aclfile"),
"aclfile is the last word": (None, self.redis(command=["redis-server", "--aclfile"]),
"starts Redis without --aclfile"),
"writable mount": (None, self.redis(volumes=[{"type": "bind", "source": str(self.root / "compose/redis/acl"),
"target": "/etc/redis/acl"}]),
"which no read-only bind mount of a repository file provides"),
"healthcheck as default user": (None, self.redis(healthcheck={"test": ["CMD", "redis-cli", "ping"]}),
"the Redis healthcheck does not authenticate"),
"malformed reference": (self.environment(APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE="secret://env/X"),
None, "is 'secret://env/X', not secret://<account>@environment/<variable>"),
"unset variable": (self.environment(REDIS_LANE_APPLICATION_PASSWORD=None), None,
"reads REDIS_LANE_APPLICATION_PASSWORD, which the lane does not supply"),
}
for case, (env, redis, reason) in cases.items():
with self.subTest(case=case):
problems = self.problems(env=env, redis=redis)
self.assertTrue(any(reason in problem for problem in problems), problems)
def test_an_acl_file_that_leaves_the_default_user_on_or_enables_a_user_twice_fails(self) -> None:
# given
self.repository.write("compose/redis/acl/lane.acl", ACL.replace("user default off\n", "")
+ "user app on >example-app-password ~prod:* +@all\n")
# then
self.assertEqual(self.problems(), [
"compose/redis/acl/lane.acl leaves the default user on, which makes every account restriction optional",
"compose/redis/acl/lane.acl enables app on more than one line, which Redis refuses to load",
])
def test_an_acl_file_mounted_as_a_file_is_read(self) -> None:
# given
mount = {"type": "bind", "source": str(self.root / "compose/redis/acl/lane.acl"),
"target": "/etc/redis/acl/lane.acl", "read_only": True}
# then
self.assertEqual(self.problems(redis=self.redis(volumes=[mount])), [])
class HeapDumpTest(unittest.TestCase):
def test_a_tmpfs_away_from_the_heap_dump_path_passes(self) -> None:
self.assertEqual(verdict.heap_dump(model(), HEAP), [])
self.assertEqual(verdict.heap_dump(model(app=app(tmpfs=["/var/tmp/heapless"])), HEAP), [])
def test_a_tmpfs_at_the_path_or_a_directory_above_it_fails(self) -> None:
# given: only an exact entry of the short list was compared
cases = {
"short syntax at the path": app(tmpfs=["/tmp", f"{HEAP}:size=1g"]),
"short syntax above the path": app(tmpfs=["/var/tmp:size=1g"]),
"trailing slash above the path": app(tmpfs=["/var/tmp/"]),
"long syntax at the path": app(volumes=[{"type": "tmpfs", "target": HEAP}]),
"long syntax above the path": app(volumes=[{"type": "tmpfs", "target": "/var"}]),
}
for case, service in cases.items():
with self.subTest(case=case):
problems = verdict.heap_dump(model(app=service), HEAP)
self.assertEqual(len(problems), 1)
self.assertIn(f"which holds the heap dump path {HEAP}", problems[0])
def test_a_volume_or_bind_mount_at_the_heap_dump_path_is_not_a_tmpfs(self) -> None:
for kind, source in (("volume", "dumps"), ("bind", "./build/heap")):
with self.subTest(kind=kind):
service = app(volumes=[{"type": kind, "source": source, "target": HEAP}])
self.assertEqual(verdict.heap_dump(model(app=service), HEAP), [])
class MemoryLimitTest(unittest.TestCase):
def test_a_compose_memory_size_or_none_passes(self) -> None:
self.assertEqual(verdict.memory_limit(lane(appMemoryLimit="1g")), [])
self.assertEqual(verdict.memory_limit(lane()), [])
def test_a_size_compose_does_not_read_fails(self) -> None:
for limit in ("1G", "1.5g", "1024", "0m"):
with self.subTest(limit=limit):
self.assertEqual(
verdict.memory_limit(lane(appMemoryLimit=limit)),
[f"appMemoryLimit '{limit}' is not a Compose memory size such as 1g"],
)
class KeycloakRealmTest(RepositoryTestCase):
def test_no_realm_or_a_realm_without_annotations_passes(self) -> None:
self.assertEqual(verdict.keycloak_realm(self.root), [])
self.repository.write(verdict.REALM, json.dumps({"realm": "ca", "clients": [{"clientId": "web"}]}))
self.assertEqual(verdict.keycloak_realm(self.root), [])
def test_an_annotation_key_anywhere_in_the_tree_fails(self) -> None:
# given
self.repository.write(verdict.REALM, json.dumps({"_comment": "x", "clients": [{"_flowComment": "y"}]}))
# then
self.assertEqual(
verdict.keycloak_realm(self.root),
["the Keycloak realm carries key(s) Keycloak refuses to deserialize: _comment, _flowComment"],
)
def test_a_realm_that_is_not_json_fails(self) -> None:
self.repository.write(verdict.REALM, "{")
self.assertEqual(verdict.keycloak_realm(self.root), [f"{verdict.REALM} is not valid JSON"])
class CommandLineTest(RepositoryTestCase):
def record(self, directory: Path, lane_id: str, services: str = "app\n", document: object = None) -> None:
(directory / f"{lane_id}.services").write_text(services, encoding="utf-8")
(directory / f"{lane_id}.services.err").write_text("", encoding="utf-8")
(directory / f"{lane_id}.services.rc").write_text("0\n", encoding="utf-8")
(directory / f"{lane_id}.json").write_text(json.dumps(document or model()), encoding="utf-8")
(directory / f"{lane_id}.json.rc").write_text("0\n", encoding="utf-8")
def run_judge(self, document: dict) -> tuple[int, str, str]:
path = self.repository.write(CONTRACT_PATH, json.dumps(document))
stdout, stderr = io.StringIO(), io.StringIO()
with redirect_stdout(stdout), redirect_stderr(stderr):
rc = verdict.main(["--repo-root", str(self.root), "judge", str(path), str(self.root / "rendered")])
return rc, stdout.getvalue(), stderr.getvalue()
def setUp(self) -> None:
super().setUp()
(self.root / "rendered").mkdir()
def test_every_lane_matching_exits_zero_naming_each(self) -> None:
# given
self.record(self.root / "rendered", "off-local")
# when
rc, stdout, stderr = self.run_judge(contract(lane()))
# then
self.assertEqual((rc, stderr), (0, ""))
self.assertEqual(stdout.splitlines(), [
"off-local: app @ local",
f"verify-compose-profile-contracts: all 1 lanes match {CONTRACT_PATH}",
])
def test_each_lane_judgement_fails_the_command_on_its_own(self) -> None:
# given: one fault per case, each seen by one judgement only, so a judgement left out of
# judge_lane passes its case
no_vendor = lane(expectedCapabilitiesOn=["persistence-jpa"])
del no_vendor["expectedPersistenceVendor"]
redis_on = {"APP_REDIS_ENABLED": "true", "APP_REDIS_NAMESPACE_ENVIRONMENT": "prod"}
cases = {
"role partition": (lane(longRunningServices=[]), model(),
"services are not partitioned by role — unfiled=[app]"),
"application report": (no_vendor, model(),
"no expectedPersistenceVendor with persistence-jpa on"),
"redis authentication": (lane(activationEnv=redis_on), model(),
"starts Redis without --aclfile"),
"memory limit": (lane(appMemoryLimit="1G"), model(),
"appMemoryLimit '1G' is not a Compose memory size such as 1g"),
"heap dump": (lane(), model(app=app(tmpfs=["/var/tmp:size=64m"])),
f"which holds the heap dump path {HEAP}"),
}
for case, (subject, document, reason) in cases.items():
with self.subTest(case=case):
self.record(self.root / "rendered", "off-local", document=document)
# when
rc, stdout, stderr = self.run_judge(contract(subject))
# then
self.assertEqual((rc, stdout), (1, ""))
failures = [line for line in stderr.splitlines() if line.startswith(" FAIL: ")]
self.assertEqual(len(failures), 1, stderr)
self.assertTrue(failures[0].startswith(" FAIL: off-local: "), failures[0])
self.assertIn(reason, failures[0])
def test_a_realm_annotation_fails_the_command_although_every_lane_matches(self) -> None:
# given
self.record(self.root / "rendered", "off-local")
self.repository.write(verdict.REALM, json.dumps({"_comment": "x"}))
# when
rc, stdout, stderr = self.run_judge(contract(lane()))
# then
self.assertEqual(rc, 1)
self.assertEqual(stdout, "off-local: app @ local\n")
self.assertIn(" FAIL: the Keycloak realm carries key(s) Keycloak refuses to deserialize: _comment\n", stderr)
def test_a_failing_lane_exits_one_and_the_other_lanes_are_still_judged(self) -> None:
# given: off-dev has no record at all, and off-local renders a service too many
self.record(self.root / "rendered", "off-local", services="app\nextra\n")
# when
rc, stdout, stderr = self.run_judge(contract(lane(), lane(id="off-dev")))
# then
self.assertEqual(rc, 1)
self.assertEqual(stdout, "")
self.assertEqual(stderr.splitlines(), [
" FAIL: off-local: services are [app,extra], the contract says [app]",
" FAIL: off-dev: the shell left no record of rendering this lane",
f"verify-compose-profile-contracts: 2 problem(s) against {CONTRACT_PATH}",
])
def test_a_dockerfile_without_a_heap_dump_path_is_a_setup_error(self) -> None:
# given
self.repository.write("Dockerfile", "FROM eclipse-temurin:21-jre\n")
# when
rc, _, stderr = self.run_judge(contract(lane()))
# then
self.assertEqual((rc, stderr), (78, "the Dockerfile sets no -XX:HeapDumpPath\n"))
def test_a_contract_that_is_not_json_fails(self) -> None:
# given
path = self.repository.write(CONTRACT_PATH, "{")
# when
with redirect_stderr(io.StringIO()) as stderr:
rc = verdict.main(["--repo-root", str(self.root), "lanes", str(path)])
# then
self.assertEqual(rc, 1)
self.assertIn("is not valid JSON", stderr.getvalue())
FAKE_DOCKER = """\
#!{python}
# Answers `docker compose ...` the way the contract script calls it, from files beside this script.
import pathlib, sys
here = pathlib.Path(__file__).resolve().parent
with open(here / "calls.log", "a", encoding="utf-8") as log:
log.write(" ".join(sys.argv[1:]) + "\\n")
args = sys.argv[1:]
sys.stdin.read() # as a call that reads stdin would: the shell's lane list must not be what it reads
if args[:3] == ["compose", "version", "--short"]:
print((here / "version").read_text().strip())
sys.exit(0)
stack = "+".join(pathlib.Path(args[i + 1]).name for i, a in enumerate(args) if a == "-f")
kind = "json" if "--format" in args else "services"
answer = here / f"{{stack}}.{{kind}}"
if not answer.exists():
print(f"no such stack: {{stack}}", file=sys.stderr)
sys.exit(15)
sys.stdout.write(answer.read_text())
status = here / f"{{stack}}.{{kind}}.rc"
sys.exit(int(status.read_text()) if status.exists() else 0)
"""
class ContractScriptTest(RepositoryTestCase):
# The judgements above are only a gate while the shell gives them what docker rendered and exits
# with their status. This runs the real shell against a repository whose `docker` is a fake.
def setUp(self) -> None:
super().setUp()
(self.root / "scripts").mkdir()
for script in (SHELL, VERDICT):
shutil.copy(script, self.root / "scripts" / script.name)
self.fake = self.root / "fake-bin"
self.fake.mkdir()
docker = self.fake / "docker"
docker.write_text(FAKE_DOCKER.format(python=sys.executable), encoding="utf-8")
docker.chmod(docker.stat().st_mode | stat.S_IXUSR)
(self.fake / "version").write_text("2.29.1\n", encoding="utf-8")
self.answer("compose.yaml", "app\n", model())
self.answer("compose.yaml+lane-services.yaml", "app\nschema\n", model(app=app(), schema={}))
def answer(self, stack: str, services: str, document: dict) -> None:
(self.fake / f"{stack}.services").write_text(services, encoding="utf-8")
(self.fake / f"{stack}.json").write_text(json.dumps(document), encoding="utf-8")
def run_script(self, document: dict) -> subprocess.CompletedProcess:
self.repository.write(CONTRACT_PATH, json.dumps(document))
env = dict(os.environ, PATH=f"{self.fake}{os.pathsep}{os.environ.get('PATH', '')}", TMPDIR=str(self.root))
return subprocess.run(["bash", str(self.root / "scripts" / SHELL.name)],
stdin=subprocess.DEVNULL, capture_output=True, text=True, env=env, check=False)
def calls(self) -> list[str]:
log = self.fake / "calls.log"
return log.read_text(encoding="utf-8").splitlines() if log.exists() else []
def two_lanes(self) -> dict:
return contract(lane(), lane(id="schema-lane", files=["base", "lane"], composeProfiles=["jpa"],
services=["app", "schema"], preStartServices=["schema"]))
def test_lanes_that_render_what_they_claim_pass_and_each_is_rendered_with_its_files_and_profiles(self) -> None:
# when
result = self.run_script(self.two_lanes())
# then
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.splitlines(), [
"compose version 2.29.1 >= 2.24.4",
"off-local: app @ local",
"schema-lane: app,schema @ local",
f"verify-compose-profile-contracts: all 2 lanes match {CONTRACT_PATH}",
])
base, extra = self.root / "compose.yaml", self.root / "lane-services.yaml"
self.assertEqual(self.calls(), [
"compose version --short",
f"compose -f {base} config --services",
f"compose -f {base} config --format json",
f"compose -f {base} -f {extra} --profile jpa config --services",
f"compose -f {base} -f {extra} --profile jpa config --format json",
])
self.assertEqual(sorted(path.name for path in self.root.glob("compose-contract.*")), [])
def test_a_lane_whose_stack_does_not_render_fails_and_the_next_lane_is_still_rendered(self) -> None:
# given
(self.fake / "compose.yaml.services").unlink()
# when
result = self.run_script(self.two_lanes())
# then
self.assertEqual(result.returncode, 1)
self.assertIn(" FAIL: off-local: the stack does not render: no such stack: compose.yaml\n", result.stderr)
self.assertIn("schema-lane: app,schema @ local\n", result.stdout)
def test_a_model_render_that_exits_non_zero_fails_although_it_printed_json(self) -> None:
# given: the JSON is a model that would pass; only the exit status says the render failed
(self.fake / "compose.yaml.json.rc").write_text("1\n", encoding="utf-8")
# when
result = self.run_script(self.two_lanes())
# then
self.assertEqual(result.returncode, 1)
self.assertIn(" FAIL: off-local: the merged model does not render as JSON\n", result.stderr)
self.assertIn("schema-lane: app,schema @ local\n", result.stdout)
def test_a_contract_with_no_lanes_fails(self) -> None:
# when
result = self.run_script(contract())
# then
self.assertEqual(result.returncode, 1)
self.assertIn("the contract declares no lanes", result.stderr)
def test_a_compose_below_the_floor_fails_before_any_lane_is_rendered(self) -> None:
# given
(self.fake / "version").write_text("2.20.0\n", encoding="utf-8")
# when
result = self.run_script(self.two_lanes())
# then
self.assertEqual(result.returncode, 1)
self.assertIn("docker compose 2.20.0 is below the required 2.24.4;", result.stderr)
self.assertEqual(self.calls(), ["compose version --short"])
def test_the_shell_keeps_no_judgement_of_its_own(self) -> None:
# Every judgement is in the verdict, where the cases above can reach it.
script = SHELL.read_text(encoding="utf-8")
commands = "\n".join(line for line in script.splitlines() if not line.lstrip().startswith("#"))
self.assertNotRegex(commands, r"\bjq\b")
self.assertRegex(script, r'\npython3 "\$\{VERDICT\}" judge "\$\{CONTRACTS\}" "\$\{RENDERED\}"\n$')
if __name__ == "__main__":
unittest.main()