pr / done (pull_request) Successful in 3m24s
scripts/run-compose-runtime-smoke.sh는 nightly 게이트인데 판정이 셸 분기라 시험이 없다. 「startup log is silent」 단계는 `docker compose logs app`을 `|| true`로 받고 시작 줄 앞의 WARN·ERROR만 찾는다. 그래서 `logs`가 실패해 오류 문구만 남거나 로그가 비어도 WARN이 없으니 통과한다. 같은 스크립트에서 빈 `ps`(d904d680)와 무시된 단계 실패(2dd4acc6)가 이미 같은 꼴로 초록을 냈다. 판정 셋을 scripts/compose-smoke-verdict.py로 옮긴다. 셸은 docker를 부르고 그 답을 증거 파일에 쓰기만 하고, 판정은 그 파일을 읽어 rc로 답한다. - startup-log: Spring Boot의 시작 줄(Started <앱> in <n> seconds)이 없으면 실패, 그 앞의 WARN·ERROR도 실패. `logs` 호출이 실패하면 판정 전에 lane이 실패한다. - restarts: 기록한 phase마다 컨테이너 줄이 하나 이상이고 재시작 0·OOM 아님. handoff lane은 1단계를 기록한 직후에도 본다. - matrix: 계약의 blocking lane마다 상태 줄이 정확히 하나이고 0. 빠지거나 겹치거나 계약 밖 lane이면 실패. scripts/tests/test_compose_smoke_verdict.py가 각 판정을 고장 난 입력과 정상 입력으로 보고, smoke 스크립트가 세 판정을 부르고 그 실패에서 멈추는지, `logs` 실패를 삼키지 않는지 본다. prCheck의 testDocumentChecks가 돌린다. 버린 안: bats는 desktop과 runner 둘 다에 새 도구를 깔아야 한다. 가짜 docker로 셸 전체를 돌리는 시험은 정적 계약 스크립트가 실제 렌더 결과를 요구해서, 게이트 스크립트에 시험용 분기를 넣어야 한다. 위험·한계: lane rc가 셸 안에서 0으로 바뀌는 일(2dd4acc6의 `if ( run_lane )`)은 matrix 판정이 볼 수 없다. 그 자리는 전처럼 subshell rc를 조건 밖에서 읽는다. 시작 줄 꼴은 Spring Boot 문구에 기댄다. 문구가 바뀌면 모든 lane이 시작 줄 없음으로 실패한다(조용히 통과하지 않는다). 검증: 노트북 `python3 -m unittest discover -s scripts/tests` 220개 통과. 판정의 시작 줄 검사·phase 검사·lane rc 검사를 지우거나 셸의 `logs`에 `|| true`를 되돌리거나 판정 호출을 `|| true`로 바꾸거나, 판정 호출을 자리마다 지우거나, lane rc 대신 0을 쓰거나 rc를 `set -e` 뒤에 읽으면 새 시험이 실패한다(변이 8개 모두). desktop RED·GREEN lane 변이와 nightly smoke matrix 결과는 PR 본문에 있다. Refs: REV-BE-0016
143 lines
6.5 KiB
Python
143 lines
6.5 KiB
Python
"""Judges the evidence a Compose runtime smoke lane leaves behind.
|
|
|
|
Called by scripts/run-compose-runtime-smoke.sh, which keeps the docker calls and writes their output
|
|
to files; this script reads those files and says pass or fail. The judgements live here rather than
|
|
in the shell because each of them once passed on input that should have failed it, and a shell
|
|
branch cannot be tried on a broken input without starting a lane (scripts/tests has the cases):
|
|
|
|
- startup-log: the application's log up to the line where it reports it has started carries no WARN
|
|
and no ERROR. A log without that line fails: an empty log, or one that `docker compose logs` never
|
|
wrote, has no WARN in it either, and passed as silent.
|
|
- restarts: every phase of the lane recorded at least one container, and none of them restarted or
|
|
was killed for memory. A phase with no line at all is a `ps` that found nothing, which left the
|
|
file empty and the check green (2026-11-13).
|
|
- matrix: every blocking lane reported exactly one status and every status is zero. A lane missing
|
|
from the list ran nothing. What this cannot see is a status the shell turned into zero before
|
|
writing it — all eighteen lanes reported green on code the build had refused because `set -e` was
|
|
off inside `if ( run_lane )` (2026-11-07) — so the shell reads each lane's status outside any
|
|
condition, and scripts/tests checks that it writes that status and not another.
|
|
|
|
Exit status 0 is a pass, 1 a failure with its reasons on stderr, 2 a call this script does not know.
|
|
|
|
python3 scripts/compose-smoke-verdict.py [--label <lane>] startup-log <startup.log>
|
|
python3 scripts/compose-smoke-verdict.py [--label <lane>] restarts <restarts.txt> <phase>...
|
|
python3 scripts/compose-smoke-verdict.py [--label <name>] matrix <statuses.txt> <lane>...
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import re
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
# Spring Boot's own line once the context is up: "Started CaSkeletonApplication in 4.2 seconds (process
|
|
# running for 5.0)". Any application name, so a service that renames the class keeps the check.
|
|
STARTED = re.compile(r"Started \S+ in \d+(\.\d+)? seconds")
|
|
NOISE = re.compile(r"WARN|ERROR")
|
|
# One line per container, written by `docker inspect --format` in record_restarts.
|
|
RESTART_LINE = re.compile(r"(?P<phase>\S+) (?P<name>\S+) restarts=(?P<restarts>\d+) oomKilled=(?P<oom>true|false)")
|
|
STATUS_LINE = re.compile(r"(?P<lane>\S+) (?P<rc>\d+)")
|
|
SHOWN = 20
|
|
|
|
|
|
def startup_log(text: str) -> list[str]:
|
|
"""The reasons the startup log is not silent; empty when it is."""
|
|
noise: list[str] = []
|
|
for line in text.splitlines():
|
|
if STARTED.search(line):
|
|
break
|
|
if NOISE.search(line):
|
|
noise.append(line)
|
|
else:
|
|
return [
|
|
"the log never reaches Spring Boot's 'Started <application> in <n> seconds' line, "
|
|
"so it does not show the startup it is meant to check"
|
|
]
|
|
if noise:
|
|
return ["the startup log is not silent:", *noise[:SHOWN]]
|
|
return []
|
|
|
|
|
|
def restarts(text: str, phases: list[str]) -> list[str]:
|
|
"""The reasons the lane's containers did not stay up; empty when every phase recorded a clean one."""
|
|
problems: list[str] = []
|
|
recorded: dict[str, int] = {}
|
|
for line in text.splitlines():
|
|
match = RESTART_LINE.fullmatch(line)
|
|
if match is None:
|
|
problems.append(f"unreadable line: {line!r}")
|
|
continue
|
|
recorded[match["phase"]] = recorded.get(match["phase"], 0) + 1
|
|
if match["restarts"] != "0" or match["oom"] != "false":
|
|
problems.append(f"a container restarted or was killed for memory: {line}")
|
|
for phase in phases:
|
|
if recorded.get(phase, 0) == 0:
|
|
problems.append(f"no container was recorded for phase '{phase}'")
|
|
return problems
|
|
|
|
|
|
def matrix(text: str, lanes: list[str]) -> list[str]:
|
|
"""The reasons the matrix did not pass; empty when every lane reported zero exactly once."""
|
|
problems: list[str] = []
|
|
statuses: dict[str, list[int]] = {}
|
|
for line in text.splitlines():
|
|
match = STATUS_LINE.fullmatch(line)
|
|
if match is None:
|
|
problems.append(f"unreadable line: {line!r}")
|
|
continue
|
|
statuses.setdefault(match["lane"], []).append(int(match["rc"]))
|
|
missing = [lane for lane in lanes if lane not in statuses]
|
|
if missing:
|
|
problems.append(
|
|
f"{len(lanes) - len(missing)} of {len(lanes)} lanes reported; a partial matrix is not a pass. "
|
|
f"Missing: {' '.join(missing)}"
|
|
)
|
|
unexpected = [lane for lane in statuses if lane not in lanes]
|
|
if unexpected:
|
|
problems.append(f"lanes that are not blocking lanes of the contract reported: {' '.join(unexpected)}")
|
|
repeated = [lane for lane in lanes if len(statuses.get(lane, [])) > 1]
|
|
if repeated:
|
|
problems.append(f"lanes that reported more than once: {' '.join(repeated)}")
|
|
failed = [lane for lane in lanes if any(rc != 0 for rc in statuses.get(lane, []))]
|
|
if failed:
|
|
problems.append(f"{len(failed)} lane(s) failed: {' '.join(failed)}")
|
|
return problems
|
|
|
|
|
|
def main(argv: list[str] | None = None) -> int:
|
|
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
|
parser.add_argument("--label", default="", help="put in front of every reason, such as the lane id")
|
|
commands = parser.add_subparsers(dest="command", required=True)
|
|
commands.add_parser("startup-log").add_argument("log", type=Path)
|
|
restart_command = commands.add_parser("restarts")
|
|
restart_command.add_argument("record", type=Path)
|
|
restart_command.add_argument("phases", nargs="+")
|
|
matrix_command = commands.add_parser("matrix")
|
|
matrix_command.add_argument("statuses", type=Path)
|
|
matrix_command.add_argument("lanes", nargs="+")
|
|
args = parser.parse_args(argv)
|
|
|
|
# A file the shell did not write is read as empty, which every judgement fails on its own terms.
|
|
if args.command == "startup-log":
|
|
problems = startup_log(read(args.log))
|
|
elif args.command == "restarts":
|
|
problems = restarts(read(args.record), args.phases)
|
|
else:
|
|
problems = matrix(read(args.statuses), args.lanes)
|
|
prefix = f"{args.label}: " if args.label else ""
|
|
for problem in problems:
|
|
print(f"{prefix}{problem}", file=sys.stderr)
|
|
return 1 if problems else 0
|
|
|
|
|
|
def read(path: Path) -> str:
|
|
try:
|
|
return path.read_text(encoding="utf-8", errors="replace")
|
|
except FileNotFoundError:
|
|
return ""
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|