pr / done (pull_request) Successful in 1m42s
scripts/verify-compose-profile-contracts.sh는 prCheck의 게이트인데 판정 열 가지가 셸 안의 jq라 시험이 없다. 고장 난 입력을 넣어 보면 넷이 통과한다. - 계약의 lanes가 비면 lane 반복이 돌지 않고 "all 0 lanes match"로 끝난다. - persistence-jpa를 켠 lane에 expectedPersistenceVendor가 없으면 통과한다. smoke는 보고된 vendor를 그 칸과 비교하니 그 lane은 매번 실패한다. - heap dump 경로는 짧은 `tmpfs:` 목록의 같은 항목만 비교한다. 위 디렉터리의 tmpfs(/var/tmp)나 long syntax `type: tmpfs` volume은 통과한다. - Redis 계정을 이름 그대로가 아니라 정규식으로 찾아 `.`이 아무 글자와 맞는다. `ca.skeleton-application`을 가리키는 lane이 `ca-skeleton-application` 줄로 통과한다. 판정을 scripts/compose-contract-verdict.py로 옮긴다. 셸은 docker만 부른다. `version`으로 Compose 하한을 먼저 보고, `lanes`가 준 lane별 인자로 `config --services`와 `config --format json`을 돌려 출력과 rc를 파일에 쓴다. rc는 조건 밖에서 읽는다. 그다음 `judge`가 그 디렉터리를 읽는다. 판정은 위 넷을 실패로 보고, 파일 없는 lane, 겹치는 id, smoke가 읽는 목록이 없거나 그 목록에 이름이 아닌 원소가 든 lane, 셸이 렌더 기록을 남기지 않은 lane도 실패로 본다. docker 호출의 stdin은 /dev/null이라 lane 목록을 먹지 않는다. scripts/tests/test_compose_contract_verdict.py가 판정마다 고장 난 입력과 정상 입력을 보고, 셸을 가짜 docker로 돌려 lane 인자·rc 기록·판정 rc를 본다. prCheck의 testDocumentChecks가 돌린다. 루트 build.gradle의 주석은 PR 단계 Exec 태스크마다 시험 파일을 적어 사실과 맞춘다. 버린 안: 모델 JSON의 service key로 `config --services`를 대신하기. 렌더를 한 번 줄이지만 지금 lane 결과와 같은지 따로 보여야 해서 두 호출을 그대로 둔다. bats는 desktop과 runner 둘 다에 새 도구를 깔아야 한다. 위험·한계: 판정이 jq와 다른 곳은 위 넷 말고도 있다. main이 통과시키고 이제 실패로 보는 것: 같은 사용자를 두 줄에서 켠 ACL(Redis가 거절한다), 값이 null·false인 realm의 `_` key, 최상위 값이 둘인 realm, LANE_ID 꼴이 아닌 id, 빈 springRuntime, `files`가 없는 lane, 이름이 아닌 목록 원소, 읽을 수 없는 Compose 버전. main이 실패시키고 이제 통과하는 것: ACL 줄의 겹친 공백과 CRLF, 끝에 `/`가 붙은 bind target, 로캘에 따라 달라지던 service 정렬, list 꼴 environment, 파일 하나로 bind mount한 ACL(target이 --aclfile 경로와 같음). Redis가 ACL 줄의 `\r`와 겹친 공백을 받아들인다는 것은 acl.c를 열어 보지 않은 판단이다. 실제 계약 18 lane에서는 main과 이 트리의 출력 줄이 같다(PR 본문). 검증: 노트북 `python3 -m unittest discover -s scripts/tests` 275개 통과. 판정의 빈 lane 거절·vendor 검사·heap 상위 tmpfs·long syntax tmpfs·비밀번호 글자 비교· 정확한 service 집합·사용자 이름 비교·렌더 기록 없음·파일 없는 lane·목록 원소 검사를 되돌리거나, judge_lane에서 판정 하나를 빼거나 realm 판정을 빼거나, 셸의 두 렌더 rc를 `|| true`로 삼키거나, docker가 stdin을 읽게 두거나, judge 호출을 지우거나 `|| true`로 바꾸거나, version 실패를 무시하거나, profile 인자를 빼면 새 시험이 실패한다(변이 23개 모두). desktop RED·GREEN과 prCheck 결과는 PR 본문에 있다. Refs: REV-BE-0016
830 lines
36 KiB
Python
830 lines
36 KiB
Python
"""scripts/compose-contract-verdict.py의 시험. Compose 계약의 정적 판정이 고장 난 입력을 실패로 보는지 본다:
|
|
lane이 없는 계약, 파일이 없는 lane, 렌더 실패와 셸이 남기지 않은 렌더 기록, service 집합·Spring runtime·mount 중복,
|
|
role 분할, 기대 vendor가 없는 JPA lane, Redis 계정 대조(비밀번호의 `*`를 포함해), tmpfs 위의 heap dump 경로,
|
|
Keycloak realm의 `_` key. 정상 입력은 통과하는지도 같이 본다. 마지막으로
|
|
scripts/verify-compose-profile-contracts.sh를 가짜 `docker`로 돌려, 셸이 판정에 lane 인자와 렌더 결과를 넘기고
|
|
그 rc로 끝나는지 본다.
|
|
|
|
python3 -m unittest discover -s scripts/tests
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import importlib.util
|
|
import io
|
|
import json
|
|
import os
|
|
import shutil
|
|
import stat
|
|
import subprocess
|
|
import sys
|
|
import tempfile
|
|
import textwrap
|
|
import unittest
|
|
from contextlib import redirect_stderr, redirect_stdout
|
|
from pathlib import Path
|
|
|
|
SCRIPTS = Path(__file__).resolve().parent.parent
|
|
VERDICT = SCRIPTS / "compose-contract-verdict.py"
|
|
SHELL = SCRIPTS / "verify-compose-profile-contracts.sh"
|
|
_spec = importlib.util.spec_from_file_location("compose_contract_verdict", VERDICT)
|
|
verdict = importlib.util.module_from_spec(_spec)
|
|
sys.modules["compose_contract_verdict"] = verdict
|
|
_spec.loader.exec_module(verdict)
|
|
|
|
CONTRACT_PATH = "src/bootstrap/app/compose-lanes/compose-profile-contracts.json"
|
|
HEAP = "/var/tmp/heap"
|
|
ACL = "user default off\nuser app on >example-app-password ~prod:* +@all\nuser app-advanced on >example-advanced-password ~prod:* +@all\n"
|
|
|
|
|
|
def contract(*lanes: dict, **fields: object) -> dict:
|
|
document = {
|
|
"minimumComposeVersion": "2.24.4",
|
|
"composeFiles": {"base": "compose.yaml", "lane": "lane-services.yaml"},
|
|
"preStartServices": ["schema"],
|
|
"oneShotServices": ["smoke"],
|
|
"capabilities": ["persistence-jpa", "redis"],
|
|
"lanes": list(lanes),
|
|
}
|
|
document.update(fields)
|
|
return document
|
|
|
|
|
|
def lane(**fields: object) -> dict:
|
|
document = {
|
|
"id": "off-local", "composeProfiles": [], "files": ["base"], "springRuntime": "local",
|
|
"services": ["app"], "longRunningServices": ["app"], "oneShotServices": [], "blocking": True,
|
|
"activationEnv": {}, "expectedCapabilitiesOn": [], "expectedPersistenceVendor": "none",
|
|
}
|
|
document.update(fields)
|
|
return document
|
|
|
|
|
|
def app(**fields: object) -> dict:
|
|
service = {"environment": {"SPRING_PROFILES_ACTIVE": "local"}, "tmpfs": ["/tmp:mode=1777,size=128m"]}
|
|
service.update(fields)
|
|
return service
|
|
|
|
|
|
def model(**services: dict) -> dict:
|
|
return {"services": services or {"app": app()}}
|
|
|
|
|
|
class Repository:
|
|
"""A repository root with the files the verdict reads: compose files, a Dockerfile, an ACL file."""
|
|
|
|
def __init__(self) -> None:
|
|
self._directory = tempfile.TemporaryDirectory()
|
|
self.root = Path(self._directory.name)
|
|
self.write("compose.yaml", "services: {}\n")
|
|
self.write("lane-services.yaml", "services: {}\n")
|
|
self.write("Dockerfile", f'ENV JAVA_TOOL_OPTIONS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath={HEAP} \\\n')
|
|
self.write("compose/redis/acl/lane.acl", ACL)
|
|
|
|
def write(self, relative: str, text: str) -> Path:
|
|
path = self.root / relative
|
|
path.parent.mkdir(parents=True, exist_ok=True)
|
|
path.write_text(text, encoding="utf-8")
|
|
return path
|
|
|
|
def close(self) -> None:
|
|
self._directory.cleanup()
|
|
|
|
|
|
def rendered(services: str = "app\n", document: object = None, services_rc: int | None = 0,
|
|
model_rc: int | None = 0, error: str = "") -> verdict.Rendered:
|
|
return verdict.Rendered(
|
|
services_rc=services_rc, services=services, services_error=error,
|
|
model_rc=model_rc, model=json.dumps(model() if document is None else document),
|
|
)
|
|
|
|
|
|
class RepositoryTestCase(unittest.TestCase):
|
|
|
|
def setUp(self) -> None:
|
|
self.repository = Repository()
|
|
self.addCleanup(self.repository.close)
|
|
self.root = self.repository.root
|
|
|
|
|
|
class ContractShapeTest(RepositoryTestCase):
|
|
|
|
def test_a_list_entry_that_is_not_a_name_fails_the_lane(self) -> None:
|
|
# given: strings() drops such an entry, so the role and report checks would never see it
|
|
cases = {
|
|
"expectedCapabilitiesOn": [None],
|
|
"longRunningServices": ["app", 1],
|
|
"services": ["app", ""],
|
|
"composeProfiles": [["jpa"]],
|
|
"oneShotServices": [None],
|
|
"preStartServices": [False],
|
|
}
|
|
for field, entries in cases.items():
|
|
with self.subTest(field=field):
|
|
# when
|
|
renderable, problems = verdict.lanes(contract(lane(**{field: entries})), self.root)
|
|
|
|
# then
|
|
self.assertEqual(renderable, [])
|
|
self.assertEqual(problems, [f"off-local: '{field}' holds {json.dumps(entries)}, and every entry must be a name"])
|
|
|
|
def test_a_contract_with_lanes_that_name_existing_files_is_renderable(self) -> None:
|
|
# given
|
|
document = contract(lane(), lane(id="off-dev", files=["base", "lane"], composeProfiles=["jpa"]))
|
|
|
|
# when
|
|
renderable, problems = verdict.lanes(document, self.root)
|
|
|
|
# then
|
|
self.assertEqual(problems, [])
|
|
self.assertEqual(
|
|
verdict.compose_arguments(document, renderable[1], self.root),
|
|
["-f", str(self.root / "compose.yaml"), "-f", str(self.root / "lane-services.yaml"), "--profile", "jpa"],
|
|
)
|
|
|
|
def test_a_contract_with_no_lanes_fails_rather_than_matching_all_of_none(self) -> None:
|
|
for lanes in ({"lanes": []}, {"lanes": None}):
|
|
with self.subTest(lanes=lanes):
|
|
# given
|
|
document = contract(**lanes)
|
|
|
|
# when
|
|
renderable, problems = verdict.lanes(document, self.root)
|
|
|
|
# then
|
|
self.assertEqual(renderable, [])
|
|
self.assertEqual(problems, ["the contract declares no lanes, so nothing would be checked"])
|
|
|
|
def test_a_lane_without_files_fails_rather_than_rendering_whatever_compose_finds(self) -> None:
|
|
# given
|
|
document = contract(lane(files=[]))
|
|
|
|
# when
|
|
renderable, problems = verdict.lanes(document, self.root)
|
|
|
|
# then
|
|
self.assertEqual(renderable, [])
|
|
self.assertEqual(len(problems), 1)
|
|
self.assertTrue(problems[0].startswith("off-local: names no compose file under 'files'"), problems)
|
|
|
|
def test_a_lane_naming_an_unknown_or_missing_file_fails(self) -> None:
|
|
# given
|
|
(self.root / "lane-services.yaml").unlink()
|
|
document = contract(lane(files=["base", "tls", "lane"]))
|
|
|
|
# when
|
|
_, problems = verdict.lanes(document, self.root)
|
|
|
|
# then
|
|
self.assertEqual(problems, [
|
|
"off-local: names compose file 'tls', which composeFiles does not list",
|
|
"off-local: lane-services.yaml is missing",
|
|
])
|
|
|
|
def test_a_lane_without_the_lists_the_smoke_wrapper_reads_fails(self) -> None:
|
|
# given
|
|
broken = lane()
|
|
del broken["composeProfiles"], broken["expectedCapabilitiesOn"], broken["springRuntime"]
|
|
|
|
# when
|
|
_, problems = verdict.lanes(contract(broken), self.root)
|
|
|
|
# then
|
|
self.assertEqual(problems, [
|
|
"off-local: has no 'composeProfiles' list; the smoke wrapper reads it as one",
|
|
"off-local: has no 'expectedCapabilitiesOn' list; the smoke wrapper reads it as one",
|
|
"off-local: names no springRuntime",
|
|
])
|
|
|
|
def test_two_lanes_with_one_id_fail_because_their_records_would_overwrite_each_other(self) -> None:
|
|
# given
|
|
document = contract(lane(), lane())
|
|
|
|
# when
|
|
renderable, problems = verdict.lanes(document, self.root)
|
|
|
|
# then
|
|
self.assertEqual(len(renderable), 1)
|
|
self.assertEqual(problems, ["off-local: the id is used by more than one lane"])
|
|
|
|
def test_an_id_that_is_not_a_file_name_fails(self) -> None:
|
|
# given
|
|
document = contract(lane(id="../escape"))
|
|
|
|
# when
|
|
_, problems = verdict.lanes(document, self.root)
|
|
|
|
# then
|
|
self.assertEqual(problems, ["lane 0: id '../escape' is not a name such as off-local"])
|
|
|
|
|
|
class ComposeVersionTest(unittest.TestCase):
|
|
|
|
def test_a_compose_at_or_above_the_floor_passes(self) -> None:
|
|
for actual in ("2.24.4", "v2.29.1", "2.30.0-desktop.1", "3.0"):
|
|
with self.subTest(actual=actual):
|
|
self.assertEqual(verdict.compose_version(contract(), actual), [])
|
|
|
|
def test_a_compose_below_the_floor_fails(self) -> None:
|
|
for actual in ("2.24.3", "2.24", "1.29.2"):
|
|
with self.subTest(actual=actual):
|
|
problems = verdict.compose_version(contract(), actual)
|
|
self.assertEqual(problems[0], f"docker compose {actual} is below the required 2.24.4;")
|
|
|
|
def test_a_version_that_cannot_be_read_fails(self) -> None:
|
|
self.assertEqual(
|
|
verdict.compose_version(contract(), ""),
|
|
["docker compose reports '', which is not a version"],
|
|
)
|
|
self.assertEqual(
|
|
verdict.compose_version(contract(minimumComposeVersion=None), "2.29.1"),
|
|
["minimumComposeVersion '' is not a version such as 2.24.4"],
|
|
)
|
|
|
|
|
|
class RenderTest(RepositoryTestCase):
|
|
|
|
def judge(self, record: verdict.Rendered, subject: dict | None = None) -> list[str]:
|
|
subject = subject or lane()
|
|
return verdict.judge_lane(contract(subject), subject, record, self.root, HEAP)
|
|
|
|
def test_a_lane_that_renders_its_services_and_a_clean_model_passes(self) -> None:
|
|
self.assertEqual(self.judge(rendered()), [])
|
|
|
|
def test_a_stack_that_does_not_render_fails_with_what_compose_said(self) -> None:
|
|
# given
|
|
record = rendered(services="", services_rc=1, error="service \"app\" refers to\nundefined volume\n")
|
|
|
|
# when
|
|
problems = self.judge(record)
|
|
|
|
# then
|
|
self.assertEqual(problems, ["the stack does not render: service \"app\" refers to undefined volume"])
|
|
|
|
def test_a_lane_the_shell_left_no_record_for_fails(self) -> None:
|
|
for record in (rendered(services_rc=None), rendered(model_rc=None)):
|
|
with self.subTest(record=record):
|
|
self.assertEqual(self.judge(record), ["the shell left no record of rendering this lane"])
|
|
|
|
def test_a_superset_or_a_subset_of_the_services_fails(self) -> None:
|
|
for services in ("app\npostgres\n", ""):
|
|
with self.subTest(services=services):
|
|
problems = self.judge(rendered(services=services))
|
|
self.assertEqual(len(problems), 1)
|
|
self.assertTrue(problems[0].startswith("services are ["), problems)
|
|
|
|
def test_the_order_of_the_rendered_services_does_not_matter(self) -> None:
|
|
# given
|
|
subject = lane(services=["postgres", "app"], longRunningServices=["app", "postgres"])
|
|
document = model(app=app(), postgres={})
|
|
|
|
# then
|
|
self.assertEqual(self.judge(rendered(services="app\npostgres\n", document=document), subject), [])
|
|
|
|
def test_a_model_that_is_not_json_fails(self) -> None:
|
|
# given
|
|
record = verdict.Rendered(services_rc=0, services="app\n", services_error="", model_rc=0, model="services:\n")
|
|
|
|
# then
|
|
self.assertEqual(self.judge(record), ["the merged model does not render as JSON"])
|
|
self.assertEqual(self.judge(rendered(model_rc=15)), ["the merged model does not render as JSON"])
|
|
|
|
def test_another_spring_runtime_fails(self) -> None:
|
|
# given
|
|
document = model(app=app(environment={"SPRING_PROFILES_ACTIVE": "prod"}))
|
|
|
|
# then
|
|
self.assertEqual(
|
|
self.judge(rendered(document=document)),
|
|
["app renders SPRING_PROFILES_ACTIVE='prod', the contract says 'local'"],
|
|
)
|
|
|
|
def test_a_mount_target_declared_twice_fails_naming_the_service(self) -> None:
|
|
# given
|
|
document = model(app=app(volumes=[{"type": "bind", "source": "/data", "target": "/tmp"}]))
|
|
|
|
# then
|
|
self.assertEqual(self.judge(rendered(document=document)), ["a mount target is declared twice — app: /tmp"])
|
|
|
|
|
|
class RolePartitionTest(unittest.TestCase):
|
|
|
|
def test_a_service_filed_under_exactly_one_role_passes(self) -> None:
|
|
# given
|
|
subject = lane(services=["app", "schema", "smoke"], preStartServices=["schema"], oneShotServices=["smoke"])
|
|
|
|
# then
|
|
self.assertEqual(verdict.role_partition(contract(), subject), [])
|
|
|
|
def test_an_unfiled_a_not_rendered_and_a_twice_filed_service_fail(self) -> None:
|
|
# given
|
|
subject = lane(services=["app", "postgres"], longRunningServices=["app", "redis"], oneShotServices=["app"])
|
|
|
|
# when
|
|
problems = verdict.role_partition(contract(oneShotServices=["smoke", "app"]), subject)
|
|
|
|
# then
|
|
self.assertIn(
|
|
"services are not partitioned by role — unfiled=[postgres] not-rendered=[redis] twice=[app]",
|
|
problems,
|
|
)
|
|
|
|
def test_a_one_shot_listed_as_a_wait_target_fails(self) -> None:
|
|
# given
|
|
subject = lane(services=["app", "smoke"], longRunningServices=["app", "smoke"])
|
|
|
|
# then
|
|
self.assertEqual(
|
|
verdict.role_partition(contract(), subject),
|
|
["[smoke] are declared non-waiting but listed as --wait targets"],
|
|
)
|
|
|
|
def test_a_service_run_once_but_in_no_catalogue_fails(self) -> None:
|
|
# given
|
|
subject = lane(services=["app", "probe"], oneShotServices=["probe"])
|
|
|
|
# then
|
|
self.assertEqual(
|
|
verdict.role_partition(contract(), subject),
|
|
["[probe] are run with `run --rm` but are in neither top-level catalogue"],
|
|
)
|
|
|
|
|
|
class ApplicationReportTest(unittest.TestCase):
|
|
|
|
def test_a_vendor_that_agrees_with_persistence_jpa_passes(self) -> None:
|
|
on = lane(expectedCapabilitiesOn=["persistence-jpa"], expectedPersistenceVendor="postgresql")
|
|
self.assertEqual(verdict.application_report(contract(), on), [])
|
|
self.assertEqual(verdict.application_report(contract(), lane()), [])
|
|
|
|
def test_a_jpa_lane_without_an_expected_vendor_fails(self) -> None:
|
|
# given: the smoke wrapper compares the reported vendor with this field, so the lane fails every run
|
|
subject = lane(expectedCapabilitiesOn=["persistence-jpa"])
|
|
del subject["expectedPersistenceVendor"]
|
|
|
|
# when
|
|
problems = verdict.application_report(contract(), subject)
|
|
|
|
# then
|
|
self.assertEqual(problems, [
|
|
"the expected application report is unreachable — no expectedPersistenceVendor with persistence-jpa on",
|
|
])
|
|
|
|
def test_a_vendor_that_contradicts_persistence_jpa_fails(self) -> None:
|
|
cases = {
|
|
"none with jpa on": (lane(expectedCapabilitiesOn=["persistence-jpa"]), "vendor none with persistence-jpa on"),
|
|
"a vendor with jpa off": (lane(expectedPersistenceVendor="postgresql"), "vendor postgresql with persistence-jpa off"),
|
|
}
|
|
for case, (subject, reason) in cases.items():
|
|
with self.subTest(case=case):
|
|
self.assertEqual(
|
|
verdict.application_report(contract(), subject),
|
|
[f"the expected application report is unreachable — {reason}"],
|
|
)
|
|
|
|
def test_a_capability_outside_the_catalogue_fails(self) -> None:
|
|
# given
|
|
subject = lane(expectedCapabilitiesOn=["websocket"])
|
|
|
|
# then
|
|
self.assertEqual(
|
|
verdict.application_report(contract(), subject),
|
|
["the expected application report is unreachable — uncatalogued=[websocket]"],
|
|
)
|
|
|
|
|
|
class RedisAuthenticationTest(RepositoryTestCase):
|
|
|
|
def environment(self, **overrides: object) -> dict:
|
|
env = {
|
|
"APP_REDIS_ENABLED": "true",
|
|
"APP_REDIS_NAMESPACE_ENVIRONMENT": "prod",
|
|
"APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE": "secret://app@environment/REDIS_LANE_APPLICATION_PASSWORD",
|
|
"APP_REDIS_AUTHENTICATION_ADVANCED_CREDENTIAL_REFERENCE":
|
|
"secret://app-advanced@environment/REDIS_LANE_ADVANCED_PASSWORD",
|
|
"REDIS_LANE_APPLICATION_PASSWORD": "example-app-password",
|
|
"REDIS_LANE_ADVANCED_PASSWORD": "example-advanced-password",
|
|
}
|
|
env.update(overrides)
|
|
return {key: value for key, value in env.items() if value is not None}
|
|
|
|
def redis(self, **fields: object) -> dict:
|
|
service = {
|
|
"command": ["redis-server", "--appendonly", "no", "--aclfile", "/etc/redis/acl/lane.acl"],
|
|
"volumes": [{"type": "bind", "source": str(self.root / "compose/redis/acl"),
|
|
"target": "/etc/redis/acl", "read_only": True}],
|
|
"healthcheck": {"test": ["CMD-SHELL", "redis-cli --user app --pass \"$$P\" ping"]},
|
|
}
|
|
service.update(fields)
|
|
return service
|
|
|
|
def problems(self, env: dict | None = None, redis: dict | None = None) -> list[str]:
|
|
subject = lane(activationEnv=self.environment() if env is None else env)
|
|
document = model(app=app(), redis=self.redis() if redis is None else redis)
|
|
return verdict.redis_authentication(subject, document, self.root)
|
|
|
|
def test_a_lane_that_names_accounts_its_redis_provisions_passes(self) -> None:
|
|
self.assertEqual(self.problems(), [])
|
|
|
|
def test_a_lane_that_does_not_switch_redis_on_is_not_checked(self) -> None:
|
|
self.assertEqual(self.problems(env={"APP_REDIS_ENABLED": "false"}, redis={}), [])
|
|
|
|
def test_a_password_with_a_star_does_not_match_an_account_with_another_password(self) -> None:
|
|
# given: '*' is a character of the password, not a wildcard
|
|
env = self.environment(REDIS_LANE_APPLICATION_PASSWORD="*")
|
|
|
|
# then
|
|
self.assertEqual(
|
|
self.problems(env=env),
|
|
["REDIS_LANE_APPLICATION_PASSWORD is not the password compose/redis/acl/lane.acl gives app"],
|
|
)
|
|
|
|
def test_a_wrong_password_fails(self) -> None:
|
|
self.assertEqual(
|
|
self.problems(env=self.environment(REDIS_LANE_ADVANCED_PASSWORD="example-app-password")),
|
|
["REDIS_LANE_ADVANCED_PASSWORD is not the password compose/redis/acl/lane.acl gives app-advanced"],
|
|
)
|
|
|
|
def test_a_user_name_with_a_dot_matches_only_itself(self) -> None:
|
|
# given: a regular expression read 'app.x' as 'app' plus any character plus 'x'
|
|
self.repository.write("compose/redis/acl/lane.acl", ACL + "user appXx on >example-x-password ~prod:* +@all\n")
|
|
env = self.environment(APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE="secret://app.x@environment/X",
|
|
X="example-x-password")
|
|
|
|
# then
|
|
self.assertEqual(
|
|
self.problems(env=env),
|
|
["APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE names app.x, which compose/redis/acl/lane.acl does not enable"],
|
|
)
|
|
|
|
def test_a_namespace_the_account_may_not_touch_fails(self) -> None:
|
|
problems = self.problems(env=self.environment(APP_REDIS_NAMESPACE_ENVIRONMENT="local"))
|
|
self.assertEqual(problems, [
|
|
"app may not touch local:*, the namespace this lane writes",
|
|
"app-advanced may not touch local:*, the namespace this lane writes",
|
|
])
|
|
|
|
def test_each_way_a_lane_falls_short_is_named(self) -> None:
|
|
cases = {
|
|
"anonymous": (self.environment(APP_REDIS_AUTHENTICATION_ANONYMOUS_ACCESS_ACCEPTED=True), None,
|
|
"accepts anonymous Redis access"),
|
|
"no namespace": (self.environment(APP_REDIS_NAMESPACE_ENVIRONMENT=None), None,
|
|
"sets no APP_REDIS_NAMESPACE_ENVIRONMENT"),
|
|
"no aclfile": (None, self.redis(command=["redis-server"]), "starts Redis without --aclfile"),
|
|
"aclfile is the last word": (None, self.redis(command=["redis-server", "--aclfile"]),
|
|
"starts Redis without --aclfile"),
|
|
"writable mount": (None, self.redis(volumes=[{"type": "bind", "source": str(self.root / "compose/redis/acl"),
|
|
"target": "/etc/redis/acl"}]),
|
|
"which no read-only bind mount of a repository file provides"),
|
|
"healthcheck as default user": (None, self.redis(healthcheck={"test": ["CMD", "redis-cli", "ping"]}),
|
|
"the Redis healthcheck does not authenticate"),
|
|
"malformed reference": (self.environment(APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE="secret://env/X"),
|
|
None, "is 'secret://env/X', not secret://<account>@environment/<variable>"),
|
|
"unset variable": (self.environment(REDIS_LANE_APPLICATION_PASSWORD=None), None,
|
|
"reads REDIS_LANE_APPLICATION_PASSWORD, which the lane does not supply"),
|
|
}
|
|
for case, (env, redis, reason) in cases.items():
|
|
with self.subTest(case=case):
|
|
problems = self.problems(env=env, redis=redis)
|
|
self.assertTrue(any(reason in problem for problem in problems), problems)
|
|
|
|
def test_an_acl_file_that_leaves_the_default_user_on_or_enables_a_user_twice_fails(self) -> None:
|
|
# given
|
|
self.repository.write("compose/redis/acl/lane.acl", ACL.replace("user default off\n", "")
|
|
+ "user app on >example-app-password ~prod:* +@all\n")
|
|
|
|
# then
|
|
self.assertEqual(self.problems(), [
|
|
"compose/redis/acl/lane.acl leaves the default user on, which makes every account restriction optional",
|
|
"compose/redis/acl/lane.acl enables app on more than one line, which Redis refuses to load",
|
|
])
|
|
|
|
def test_an_acl_file_mounted_as_a_file_is_read(self) -> None:
|
|
# given
|
|
mount = {"type": "bind", "source": str(self.root / "compose/redis/acl/lane.acl"),
|
|
"target": "/etc/redis/acl/lane.acl", "read_only": True}
|
|
|
|
# then
|
|
self.assertEqual(self.problems(redis=self.redis(volumes=[mount])), [])
|
|
|
|
|
|
class HeapDumpTest(unittest.TestCase):
|
|
|
|
def test_a_tmpfs_away_from_the_heap_dump_path_passes(self) -> None:
|
|
self.assertEqual(verdict.heap_dump(model(), HEAP), [])
|
|
self.assertEqual(verdict.heap_dump(model(app=app(tmpfs=["/var/tmp/heapless"])), HEAP), [])
|
|
|
|
def test_a_tmpfs_at_the_path_or_a_directory_above_it_fails(self) -> None:
|
|
# given: only an exact entry of the short list was compared
|
|
cases = {
|
|
"short syntax at the path": app(tmpfs=["/tmp", f"{HEAP}:size=1g"]),
|
|
"short syntax above the path": app(tmpfs=["/var/tmp:size=1g"]),
|
|
"trailing slash above the path": app(tmpfs=["/var/tmp/"]),
|
|
"long syntax at the path": app(volumes=[{"type": "tmpfs", "target": HEAP}]),
|
|
"long syntax above the path": app(volumes=[{"type": "tmpfs", "target": "/var"}]),
|
|
}
|
|
for case, service in cases.items():
|
|
with self.subTest(case=case):
|
|
problems = verdict.heap_dump(model(app=service), HEAP)
|
|
self.assertEqual(len(problems), 1)
|
|
self.assertIn(f"which holds the heap dump path {HEAP}", problems[0])
|
|
|
|
def test_a_volume_or_bind_mount_at_the_heap_dump_path_is_not_a_tmpfs(self) -> None:
|
|
for kind, source in (("volume", "dumps"), ("bind", "./build/heap")):
|
|
with self.subTest(kind=kind):
|
|
service = app(volumes=[{"type": kind, "source": source, "target": HEAP}])
|
|
self.assertEqual(verdict.heap_dump(model(app=service), HEAP), [])
|
|
|
|
|
|
class MemoryLimitTest(unittest.TestCase):
|
|
|
|
def test_a_compose_memory_size_or_none_passes(self) -> None:
|
|
self.assertEqual(verdict.memory_limit(lane(appMemoryLimit="1g")), [])
|
|
self.assertEqual(verdict.memory_limit(lane()), [])
|
|
|
|
def test_a_size_compose_does_not_read_fails(self) -> None:
|
|
for limit in ("1G", "1.5g", "1024", "0m"):
|
|
with self.subTest(limit=limit):
|
|
self.assertEqual(
|
|
verdict.memory_limit(lane(appMemoryLimit=limit)),
|
|
[f"appMemoryLimit '{limit}' is not a Compose memory size such as 1g"],
|
|
)
|
|
|
|
|
|
class KeycloakRealmTest(RepositoryTestCase):
|
|
|
|
def test_no_realm_or_a_realm_without_annotations_passes(self) -> None:
|
|
self.assertEqual(verdict.keycloak_realm(self.root), [])
|
|
self.repository.write(verdict.REALM, json.dumps({"realm": "ca", "clients": [{"clientId": "web"}]}))
|
|
self.assertEqual(verdict.keycloak_realm(self.root), [])
|
|
|
|
def test_an_annotation_key_anywhere_in_the_tree_fails(self) -> None:
|
|
# given
|
|
self.repository.write(verdict.REALM, json.dumps({"_comment": "x", "clients": [{"_flowComment": "y"}]}))
|
|
|
|
# then
|
|
self.assertEqual(
|
|
verdict.keycloak_realm(self.root),
|
|
["the Keycloak realm carries key(s) Keycloak refuses to deserialize: _comment, _flowComment"],
|
|
)
|
|
|
|
def test_a_realm_that_is_not_json_fails(self) -> None:
|
|
self.repository.write(verdict.REALM, "{")
|
|
self.assertEqual(verdict.keycloak_realm(self.root), [f"{verdict.REALM} is not valid JSON"])
|
|
|
|
|
|
class CommandLineTest(RepositoryTestCase):
|
|
|
|
def record(self, directory: Path, lane_id: str, services: str = "app\n", document: object = None) -> None:
|
|
(directory / f"{lane_id}.services").write_text(services, encoding="utf-8")
|
|
(directory / f"{lane_id}.services.err").write_text("", encoding="utf-8")
|
|
(directory / f"{lane_id}.services.rc").write_text("0\n", encoding="utf-8")
|
|
(directory / f"{lane_id}.json").write_text(json.dumps(document or model()), encoding="utf-8")
|
|
(directory / f"{lane_id}.json.rc").write_text("0\n", encoding="utf-8")
|
|
|
|
def run_judge(self, document: dict) -> tuple[int, str, str]:
|
|
path = self.repository.write(CONTRACT_PATH, json.dumps(document))
|
|
stdout, stderr = io.StringIO(), io.StringIO()
|
|
with redirect_stdout(stdout), redirect_stderr(stderr):
|
|
rc = verdict.main(["--repo-root", str(self.root), "judge", str(path), str(self.root / "rendered")])
|
|
return rc, stdout.getvalue(), stderr.getvalue()
|
|
|
|
def setUp(self) -> None:
|
|
super().setUp()
|
|
(self.root / "rendered").mkdir()
|
|
|
|
def test_every_lane_matching_exits_zero_naming_each(self) -> None:
|
|
# given
|
|
self.record(self.root / "rendered", "off-local")
|
|
|
|
# when
|
|
rc, stdout, stderr = self.run_judge(contract(lane()))
|
|
|
|
# then
|
|
self.assertEqual((rc, stderr), (0, ""))
|
|
self.assertEqual(stdout.splitlines(), [
|
|
"off-local: app @ local",
|
|
f"verify-compose-profile-contracts: all 1 lanes match {CONTRACT_PATH}",
|
|
])
|
|
|
|
def test_each_lane_judgement_fails_the_command_on_its_own(self) -> None:
|
|
# given: one fault per case, each seen by one judgement only, so a judgement left out of
|
|
# judge_lane passes its case
|
|
no_vendor = lane(expectedCapabilitiesOn=["persistence-jpa"])
|
|
del no_vendor["expectedPersistenceVendor"]
|
|
redis_on = {"APP_REDIS_ENABLED": "true", "APP_REDIS_NAMESPACE_ENVIRONMENT": "prod"}
|
|
cases = {
|
|
"role partition": (lane(longRunningServices=[]), model(),
|
|
"services are not partitioned by role — unfiled=[app]"),
|
|
"application report": (no_vendor, model(),
|
|
"no expectedPersistenceVendor with persistence-jpa on"),
|
|
"redis authentication": (lane(activationEnv=redis_on), model(),
|
|
"starts Redis without --aclfile"),
|
|
"memory limit": (lane(appMemoryLimit="1G"), model(),
|
|
"appMemoryLimit '1G' is not a Compose memory size such as 1g"),
|
|
"heap dump": (lane(), model(app=app(tmpfs=["/var/tmp:size=64m"])),
|
|
f"which holds the heap dump path {HEAP}"),
|
|
}
|
|
for case, (subject, document, reason) in cases.items():
|
|
with self.subTest(case=case):
|
|
self.record(self.root / "rendered", "off-local", document=document)
|
|
|
|
# when
|
|
rc, stdout, stderr = self.run_judge(contract(subject))
|
|
|
|
# then
|
|
self.assertEqual((rc, stdout), (1, ""))
|
|
failures = [line for line in stderr.splitlines() if line.startswith(" FAIL: ")]
|
|
self.assertEqual(len(failures), 1, stderr)
|
|
self.assertTrue(failures[0].startswith(" FAIL: off-local: "), failures[0])
|
|
self.assertIn(reason, failures[0])
|
|
|
|
def test_a_realm_annotation_fails_the_command_although_every_lane_matches(self) -> None:
|
|
# given
|
|
self.record(self.root / "rendered", "off-local")
|
|
self.repository.write(verdict.REALM, json.dumps({"_comment": "x"}))
|
|
|
|
# when
|
|
rc, stdout, stderr = self.run_judge(contract(lane()))
|
|
|
|
# then
|
|
self.assertEqual(rc, 1)
|
|
self.assertEqual(stdout, "off-local: app @ local\n")
|
|
self.assertIn(" FAIL: the Keycloak realm carries key(s) Keycloak refuses to deserialize: _comment\n", stderr)
|
|
|
|
def test_a_failing_lane_exits_one_and_the_other_lanes_are_still_judged(self) -> None:
|
|
# given: off-dev has no record at all, and off-local renders a service too many
|
|
self.record(self.root / "rendered", "off-local", services="app\nextra\n")
|
|
|
|
# when
|
|
rc, stdout, stderr = self.run_judge(contract(lane(), lane(id="off-dev")))
|
|
|
|
# then
|
|
self.assertEqual(rc, 1)
|
|
self.assertEqual(stdout, "")
|
|
self.assertEqual(stderr.splitlines(), [
|
|
" FAIL: off-local: services are [app,extra], the contract says [app]",
|
|
" FAIL: off-dev: the shell left no record of rendering this lane",
|
|
f"verify-compose-profile-contracts: 2 problem(s) against {CONTRACT_PATH}",
|
|
])
|
|
|
|
def test_a_dockerfile_without_a_heap_dump_path_is_a_setup_error(self) -> None:
|
|
# given
|
|
self.repository.write("Dockerfile", "FROM eclipse-temurin:21-jre\n")
|
|
|
|
# when
|
|
rc, _, stderr = self.run_judge(contract(lane()))
|
|
|
|
# then
|
|
self.assertEqual((rc, stderr), (78, "the Dockerfile sets no -XX:HeapDumpPath\n"))
|
|
|
|
def test_a_contract_that_is_not_json_fails(self) -> None:
|
|
# given
|
|
path = self.repository.write(CONTRACT_PATH, "{")
|
|
|
|
# when
|
|
with redirect_stderr(io.StringIO()) as stderr:
|
|
rc = verdict.main(["--repo-root", str(self.root), "lanes", str(path)])
|
|
|
|
# then
|
|
self.assertEqual(rc, 1)
|
|
self.assertIn("is not valid JSON", stderr.getvalue())
|
|
|
|
|
|
FAKE_DOCKER = """\
|
|
#!{python}
|
|
# Answers `docker compose ...` the way the contract script calls it, from files beside this script.
|
|
import pathlib, sys
|
|
here = pathlib.Path(__file__).resolve().parent
|
|
with open(here / "calls.log", "a", encoding="utf-8") as log:
|
|
log.write(" ".join(sys.argv[1:]) + "\\n")
|
|
args = sys.argv[1:]
|
|
sys.stdin.read() # as a call that reads stdin would: the shell's lane list must not be what it reads
|
|
if args[:3] == ["compose", "version", "--short"]:
|
|
print((here / "version").read_text().strip())
|
|
sys.exit(0)
|
|
stack = "+".join(pathlib.Path(args[i + 1]).name for i, a in enumerate(args) if a == "-f")
|
|
kind = "json" if "--format" in args else "services"
|
|
answer = here / f"{{stack}}.{{kind}}"
|
|
if not answer.exists():
|
|
print(f"no such stack: {{stack}}", file=sys.stderr)
|
|
sys.exit(15)
|
|
sys.stdout.write(answer.read_text())
|
|
status = here / f"{{stack}}.{{kind}}.rc"
|
|
sys.exit(int(status.read_text()) if status.exists() else 0)
|
|
"""
|
|
|
|
|
|
class ContractScriptTest(RepositoryTestCase):
|
|
# The judgements above are only a gate while the shell gives them what docker rendered and exits
|
|
# with their status. This runs the real shell against a repository whose `docker` is a fake.
|
|
|
|
def setUp(self) -> None:
|
|
super().setUp()
|
|
(self.root / "scripts").mkdir()
|
|
for script in (SHELL, VERDICT):
|
|
shutil.copy(script, self.root / "scripts" / script.name)
|
|
self.fake = self.root / "fake-bin"
|
|
self.fake.mkdir()
|
|
docker = self.fake / "docker"
|
|
docker.write_text(FAKE_DOCKER.format(python=sys.executable), encoding="utf-8")
|
|
docker.chmod(docker.stat().st_mode | stat.S_IXUSR)
|
|
(self.fake / "version").write_text("2.29.1\n", encoding="utf-8")
|
|
self.answer("compose.yaml", "app\n", model())
|
|
self.answer("compose.yaml+lane-services.yaml", "app\nschema\n", model(app=app(), schema={}))
|
|
|
|
def answer(self, stack: str, services: str, document: dict) -> None:
|
|
(self.fake / f"{stack}.services").write_text(services, encoding="utf-8")
|
|
(self.fake / f"{stack}.json").write_text(json.dumps(document), encoding="utf-8")
|
|
|
|
def run_script(self, document: dict) -> subprocess.CompletedProcess:
|
|
self.repository.write(CONTRACT_PATH, json.dumps(document))
|
|
env = dict(os.environ, PATH=f"{self.fake}{os.pathsep}{os.environ.get('PATH', '')}", TMPDIR=str(self.root))
|
|
return subprocess.run(["bash", str(self.root / "scripts" / SHELL.name)],
|
|
stdin=subprocess.DEVNULL, capture_output=True, text=True, env=env, check=False)
|
|
|
|
def calls(self) -> list[str]:
|
|
log = self.fake / "calls.log"
|
|
return log.read_text(encoding="utf-8").splitlines() if log.exists() else []
|
|
|
|
def two_lanes(self) -> dict:
|
|
return contract(lane(), lane(id="schema-lane", files=["base", "lane"], composeProfiles=["jpa"],
|
|
services=["app", "schema"], preStartServices=["schema"]))
|
|
|
|
def test_lanes_that_render_what_they_claim_pass_and_each_is_rendered_with_its_files_and_profiles(self) -> None:
|
|
# when
|
|
result = self.run_script(self.two_lanes())
|
|
|
|
# then
|
|
self.assertEqual(result.returncode, 0, result.stderr)
|
|
self.assertEqual(result.stdout.splitlines(), [
|
|
"compose version 2.29.1 >= 2.24.4",
|
|
"off-local: app @ local",
|
|
"schema-lane: app,schema @ local",
|
|
f"verify-compose-profile-contracts: all 2 lanes match {CONTRACT_PATH}",
|
|
])
|
|
base, extra = self.root / "compose.yaml", self.root / "lane-services.yaml"
|
|
self.assertEqual(self.calls(), [
|
|
"compose version --short",
|
|
f"compose -f {base} config --services",
|
|
f"compose -f {base} config --format json",
|
|
f"compose -f {base} -f {extra} --profile jpa config --services",
|
|
f"compose -f {base} -f {extra} --profile jpa config --format json",
|
|
])
|
|
self.assertEqual(sorted(path.name for path in self.root.glob("compose-contract.*")), [])
|
|
|
|
def test_a_lane_whose_stack_does_not_render_fails_and_the_next_lane_is_still_rendered(self) -> None:
|
|
# given
|
|
(self.fake / "compose.yaml.services").unlink()
|
|
|
|
# when
|
|
result = self.run_script(self.two_lanes())
|
|
|
|
# then
|
|
self.assertEqual(result.returncode, 1)
|
|
self.assertIn(" FAIL: off-local: the stack does not render: no such stack: compose.yaml\n", result.stderr)
|
|
self.assertIn("schema-lane: app,schema @ local\n", result.stdout)
|
|
|
|
def test_a_model_render_that_exits_non_zero_fails_although_it_printed_json(self) -> None:
|
|
# given: the JSON is a model that would pass; only the exit status says the render failed
|
|
(self.fake / "compose.yaml.json.rc").write_text("1\n", encoding="utf-8")
|
|
|
|
# when
|
|
result = self.run_script(self.two_lanes())
|
|
|
|
# then
|
|
self.assertEqual(result.returncode, 1)
|
|
self.assertIn(" FAIL: off-local: the merged model does not render as JSON\n", result.stderr)
|
|
self.assertIn("schema-lane: app,schema @ local\n", result.stdout)
|
|
|
|
def test_a_contract_with_no_lanes_fails(self) -> None:
|
|
# when
|
|
result = self.run_script(contract())
|
|
|
|
# then
|
|
self.assertEqual(result.returncode, 1)
|
|
self.assertIn("the contract declares no lanes", result.stderr)
|
|
|
|
def test_a_compose_below_the_floor_fails_before_any_lane_is_rendered(self) -> None:
|
|
# given
|
|
(self.fake / "version").write_text("2.20.0\n", encoding="utf-8")
|
|
|
|
# when
|
|
result = self.run_script(self.two_lanes())
|
|
|
|
# then
|
|
self.assertEqual(result.returncode, 1)
|
|
self.assertIn("docker compose 2.20.0 is below the required 2.24.4;", result.stderr)
|
|
self.assertEqual(self.calls(), ["compose version --short"])
|
|
|
|
def test_the_shell_keeps_no_judgement_of_its_own(self) -> None:
|
|
# Every judgement is in the verdict, where the cases above can reach it.
|
|
script = SHELL.read_text(encoding="utf-8")
|
|
commands = "\n".join(line for line in script.splitlines() if not line.lstrip().startswith("#"))
|
|
self.assertNotRegex(commands, r"\bjq\b")
|
|
self.assertRegex(script, r'\npython3 "\$\{VERDICT\}" judge "\$\{CONTRACTS\}" "\$\{RENDERED\}"\n$')
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|