문제: nightly의 redis-topology가 행렬 job 11개였다. 행마다 job을 새로 시작해 컨테이너·checkout·JDK·Gradle 시작과 구성을 다시 치렀다. 수동 실행 run 100에서 이 11개가 runner 135분 가운데 26분이었고, 그중 약 3분의 1이 시작 비용이었다. 고른 방법: job 하나에 행마다 step 하나를 둔다. 각 step은 지금과 같은 `bash scripts/ci/redis-topology.sh <topology> <version>` 한 줄이다. - `if: always()`로 한 행이 실패해도 나머지가 돈다. job이 빨갛게 되고, 실패한 step 이름이 행을 말한다. - 스크립트가 끝날 때 자기 topology를 내리므로 행끼리 서버를 나누지 않는다. - 같은 job 안에서 Gradle daemon과 구성 결과를 다시 쓴다. - timeout은 행 11개를 합친 시간에 맞춰 90분으로 둔다. 문서: 행렬이라고 적던 곳(gates.md, template-vs-consumer.md, redis-sdk compose README, 스크립트 머리 주석)을 step으로 고친다. 버린 대안: 버전마다 job 하나(3개). 시작 비용을 셋 더 치른다. 한 step에서 셸 반복문으로 돌기: 어느 행이 실패했는지 로그를 뒤져야 하고, 단계 job 규칙과 같은 이유로 셸 문법을 늘리지 않는다. 위험·한계: UI에서 행마다 따로 보이던 초록·빨강이 step 단위가 된다. job 하나가 시간 제한에 걸리면 남은 행이 함께 멈춘다. 검증: YAML을 읽어 step 13개(checkout, setup, 행 11)를 확인했다. runner nightly 측정은 PR 본문에 적는다.
95 lines
4.8 KiB
Bash
Executable File
95 lines
4.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# One Redis topology lane against a real server: start the Compose topology, run the leaves'
|
|
# `redisTopologyTest` for it, print the evidence manifest, stop the topology. Called by the
|
|
# `redis-topology` nightly job of .gitea/workflows/main.yml (one step per row; the script stops its
|
|
# topology on exit, so the rows do not share a server); a person can run the same line on a machine
|
|
# with Docker.
|
|
#
|
|
# bash scripts/ci/redis-topology.sh <standalone|sentinel|cluster|tls> <redis version>
|
|
#
|
|
# The lanes are explicitOnly: they need the endpoint this script passes, so no stage runs them
|
|
# (docs/redis/support-matrix.md records which lane produced which evidence). Each Gradle task fails
|
|
# closed on its own account: an unknown mode, a missing endpoint, a lane with no tagged test class
|
|
# and a run that executed zero tests are all errors.
|
|
set -euo pipefail
|
|
|
|
topology=${1:?topology}
|
|
version=${2:?redis version}
|
|
cd "$(dirname "${BASH_SOURCE[0]}")/../.."
|
|
compose=(docker compose -f "src/adapter/outbound/redis-sdk/compose/${topology}/compose.yml")
|
|
forwarder=''
|
|
stop() {
|
|
[ -z "${forwarder}" ] || kill "${forwarder}" 2>/dev/null || true
|
|
"${compose[@]}" down -v
|
|
}
|
|
trap stop EXIT
|
|
|
|
runtime=:adapter:outbound:redis-runtime:redisTopologyTest
|
|
sdk=:adapter:outbound:redis-sdk:redisTopologyTest
|
|
semantic=:adapter:outbound:cache-redis:redisTopologyTest
|
|
extra=()
|
|
case "${topology}" in
|
|
standalone) port=6379; ports=(6379); tasks=("${runtime}" "${sdk}" "${semantic}") ;;
|
|
sentinel) port=27010; ports=(7010 7011 27010 27011 27012); tasks=("${runtime}" "${sdk}"); extra=(-Predis.topology.master=skeleton) ;;
|
|
cluster) port=7100; ports=(7100 7101 7102 7103 7104 7105); tasks=("${runtime}" "${sdk}" "${semantic}") ;;
|
|
tls) port=6390; ports=(6390); tasks=("${runtime}") ;;
|
|
*) echo "redis-topology: unknown topology ${topology}" >&2; exit 2 ;;
|
|
esac
|
|
|
|
# Redis 8 ships the search module, and the command catalog makes its FT.INFO an admin diagnostic
|
|
# from 8.0 (ADMIN_ONLY). The committed fixture ACL cannot grant it: one file serves every version,
|
|
# and Redis 7 refuses to start on a rule that names a command it does not have. On 8 and later the
|
|
# lane mounts a copy whose admin line also grants +ft.info, made here from that one file so the two
|
|
# cannot drift (compose/redis/acl/README.md).
|
|
if [ "${version%%.*}" -ge 8 ]; then
|
|
acl_dir="${PWD}/src/adapter/outbound/redis-sdk/build/redis-topology-acl"
|
|
rm -rf "${acl_dir}"
|
|
mkdir -p "${acl_dir}"
|
|
sed '/^user ca-skeleton-admin-readonly /s/$/ +ft.info/' compose/redis/acl/lane-fixture-accounts.acl \
|
|
> "${acl_dir}/lane-fixture-accounts.acl"
|
|
grep -q '^user ca-skeleton-admin-readonly .* +ft\.info$' "${acl_dir}/lane-fixture-accounts.acl" \
|
|
|| { echo "redis-topology: the admin line of the fixture ACL was not found" >&2; exit 1; }
|
|
# Read by the image's redis user, which is not the user that made the copy.
|
|
chmod 0755 "${acl_dir}"
|
|
chmod 0644 "${acl_dir}/lane-fixture-accounts.acl"
|
|
export REDIS_ACL_DIR="${acl_dir}"
|
|
fi
|
|
|
|
REDIS_VERSION=${version} "${compose[@]}" up -d --wait
|
|
|
|
# Inside a CI job container the Docker daemon's ports are on the daemon host, not on localhost, and
|
|
# the lanes are built around localhost (cluster redirects announce 127.0.0.1, the TLS certificate
|
|
# names localhost). Forward them, so the lane runs exactly as it does on a workstation.
|
|
if [ -f /.dockerenv ]; then
|
|
# The default gateway of the job's network is the daemon host. /proc/net/route stores it as
|
|
# little-endian hex; awk reads "0x.." as 0 and would point the forwarder at itself.
|
|
daemon_host=$(python3 -c 'import socket, struct
|
|
for line in open("/proc/net/route").readlines()[1:]:
|
|
fields = line.split()
|
|
if fields[1] == "00000000":
|
|
print(socket.inet_ntoa(struct.pack("<L", int(fields[2], 16))))
|
|
break')
|
|
[ -n "${daemon_host}" ] && [ "${daemon_host}" != 0.0.0.0 ] \
|
|
|| { echo "redis-topology: no default gateway to reach the Docker daemon host" >&2; exit 1; }
|
|
python3 scripts/ci/forward-ports.py "${daemon_host}" "${ports[@]}" &
|
|
forwarder=$!
|
|
sleep 1
|
|
fi
|
|
|
|
# The tag says 7.4; the digest says which 7.4. Evidence that names only the tag cannot be reproduced
|
|
# once the tag moves, so a run that cannot name its digest has nothing to certify.
|
|
digest=$(docker image inspect --format '{{index .RepoDigests 0}}' "redis:${version}")
|
|
[ -n "${digest}" ] || { echo "redis-topology: no repository digest for redis:${version}" >&2; exit 1; }
|
|
if [ "${topology}" = tls ]; then
|
|
trust=$(mktemp "${RUNNER_TEMP:-/tmp}/redis-lane-ca.XXXXXX")
|
|
"${compose[@]}" cp redis:/tls/ca.crt "${trust}"
|
|
extra=(-Predis.topology.trust-material="${trust}")
|
|
fi
|
|
|
|
./gradlew "${tasks[@]}" --continue \
|
|
-Predis.topology.host=localhost -Predis.topology.port="${port}" -Predis.topology.mode="${topology}" \
|
|
"${extra[@]}"
|
|
|
|
printf 'evidence: commit=%s run=%s topology=%s redis_version=%s image_digest=%s\n' \
|
|
"$(git rev-parse HEAD)" "${GITHUB_RUN_ID:-local}" "${topology}" "${version}" "${digest}"
|