build: 템플릿 밖에서 혼자 빌드·배포되는 Gradle 루트를 만든다
옮겨 온 gRPC 모듈은 템플릿의 두 번째 빌드(optional build)였다. 그 빌드는 템플릿 build-logic·카탈로그·registry·config를 상대 경로로 읽었고, grpc-operation-ledger-jpa는 템플릿 persistence-jpa의 testFixtures 파일 셋을 빌드 때 복사해 왔다. 별도 repo에는 그 경로가 없다. repo 루트를 Gradle 루트로 둔다. - build-logic: 템플릿 22df3525의 src/build-logic을 복사하고 쓰지 않는 플러그인(bootrun-dotenv, redis topology, config contract, spring-library, optional registry, purity 태스크)을 뺐다. 출처와 고친 곳은 build-logic/UPSTREAM.md에 적었다(설계 10.3절 L1). - registry config/architecture/modules.json: 템플릿의 optional-grpc 9행, source_path는 Gradle 루트 기준. - 카탈로그·config: 이 repo가 쓰는 항목만. SpotBugs 제외는 이 repo 클래스와 일반 규칙만 남겼다. - group은 gradle.properties의 libraryGroup(com.hyeonworks.grpc)에서 읽는다. 버전은 -PreleaseVersion 말고 RELEASE_TAG(vX.Y.Z)에서도 읽고, 다른 모양의 태그는 빌드 전에 거부한다. - 배포: 새 ca.library-publishing이 모듈마다 Maven publication(sources· javadoc jar, versionMapping으로 해석된 버전)과 저장소 gitea·staging, SIGNING_KEY가 있으면 PGP 서명을 단다. 루트는 SBOM artifact grpc-library와 단계 태스크 ci·mainCheck·releaseCheck·release· releaseRehearsal을 가진다. Trivy·Buf는 scripts/tool.sh가 SHA-256으로 고정해 받는다. - testFixtures 결합(설계 4.6절 2번)은 integrationTest 안 사본으로 끊었다. - Buf lint·breaking은 Gradle 태스크(scripts/buf.sh)로 ci에 넣었다. 버린 대안: build-logic을 템플릿이 배포(L2, 순환), 공통 repo(L3, repo가 하나 더), 표준 플러그인만으로 새로 쓰기(L4, strict lane과 방향 검사를 잃음). SBOM은 CycloneDX Gradle 플러그인 대신 이미 고정해 쓰는 Trivy로 만든다(도구를 늘리지 않는다). 위험: strictlane 단계 검사는 included build 태스크를 따라가다 실패해서 그 태스크를 건너뛰게 고쳤다(UPSTREAM.md). Gitea에 실제로 올리는 경로는 비밀 값이 없어 이 커밋에서는 시험하지 못했다. 검증(desktop, 이 트리): ./gradlew ci rc=0, 테스트 739개(build-logic 포함) 중 첫 실행 실패 1개(템플릿 경로를 찾던 GrpcPackageOwnershipTest)를 고친 뒤 통과. ./gradlew verifyStrictLaneStages --dry-run rc=0. Refs: design-docs repo-structure/05-library-repos.md 4.6·7·10절
This commit is contained in:
Executable
+64
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env bash
|
||||
# Buf lint and breaking-change policy for the Stable gRPC schema (grpc/grpc-contract, buf.yaml there).
|
||||
#
|
||||
# scripts/buf.sh lint
|
||||
# scripts/buf.sh breaking
|
||||
#
|
||||
# protoc (the Gradle protobuf plugin) compiles the schema; Buf owns lint and breaking policy. The
|
||||
# root `ci` stage runs both through :grpc:grpc-contract:bufLint and :bufBreaking.
|
||||
#
|
||||
# breaking compares the working tree with a baseline commit:
|
||||
# BUF_AGAINST set → that ref
|
||||
# a pull request (GITHUB_BASE_REF) → the merge base with origin/<base>
|
||||
# otherwise → the merge base with origin/main, or HEAD^ when HEAD is on it
|
||||
# (or when a local clone has no origin/main)
|
||||
# A baseline without grpc/grpc-contract/buf.yaml has nothing to be compatible with, and the check
|
||||
# says so and passes. A baseline Git cannot find fails: a shallow clone cannot answer the question.
|
||||
set -euo pipefail
|
||||
|
||||
root=$(git rev-parse --show-toplevel)
|
||||
module=grpc/grpc-contract
|
||||
buf=$("$root/scripts/tool.sh" buf)
|
||||
cd "$root/$module"
|
||||
|
||||
case "${1:-}" in
|
||||
lint)
|
||||
"$buf" lint
|
||||
;;
|
||||
breaking)
|
||||
if [ -n "${BUF_AGAINST:-}" ]; then
|
||||
baseline=$BUF_AGAINST
|
||||
else
|
||||
base_branch="origin/${GITHUB_BASE_REF:-main}"
|
||||
if git rev-parse --verify --quiet "$base_branch^{commit}" >/dev/null; then
|
||||
baseline=$(git merge-base HEAD "$base_branch")
|
||||
elif [ "${GITHUB_ACTIONS:-}" = true ]; then
|
||||
# In CI the base branch must be there; a missing one means a shallow checkout, and comparing
|
||||
# with something else would answer a different question.
|
||||
echo "buf.sh: $base_branch is not in this clone; fetch it (actions/checkout fetch-depth: 0)." >&2
|
||||
exit 1
|
||||
else
|
||||
echo "buf.sh: no $base_branch here; comparing with HEAD^ (set BUF_AGAINST to choose)."
|
||||
baseline=$(git rev-parse HEAD)
|
||||
fi
|
||||
if [ -z "${GITHUB_BASE_REF:-}" ] && [ "$baseline" = "$(git rev-parse HEAD)" ]; then
|
||||
baseline=$(git rev-parse --verify --quiet 'HEAD^' || true)
|
||||
fi
|
||||
fi
|
||||
commit=$(git rev-parse --verify --quiet "${baseline}^{commit}" || true)
|
||||
if [ -z "$commit" ]; then
|
||||
echo "buf.sh: no baseline commit for '${baseline:-HEAD^}'; nothing to compare (first commit)." >&2
|
||||
[ -z "${baseline:-}" ] && exit 0 || exit 1
|
||||
fi
|
||||
if ! git cat-file -e "$commit:$module/buf.yaml" 2>/dev/null; then
|
||||
echo "buf.sh: baseline ${commit:0:12} has no $module/buf.yaml; breaking check skipped."
|
||||
exit 0
|
||||
fi
|
||||
echo "buf.sh: breaking against ${commit:0:12}"
|
||||
"$buf" breaking --against "$root/.git#ref=$commit,subdir=$module"
|
||||
;;
|
||||
*)
|
||||
echo "usage: scripts/buf.sh lint|breaking" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
@@ -0,0 +1,85 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Checks a staged release (build/staging-repository) file by file before anything is uploaded.
|
||||
|
||||
check-staging.py <repository dir> <group> <version> signed|unsigned <artifact>...
|
||||
|
||||
For every module: the jar, sources and javadoc jars, POM and Gradle module file, each with its
|
||||
.md5/.sha1/.sha256/.sha512 checksum (and .asc when signed), and every dependency in the POM and the
|
||||
module file carries a version. The last check is the publishing trap the design names: the modules
|
||||
declare most dependencies without a version and let a BOM fill it, and a published file written
|
||||
from the declarations would leave consumers nothing to resolve. For the SBOM artifact
|
||||
(grpc-library): its POM and the CycloneDX JSON.
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
import xml.etree.ElementTree as ET
|
||||
from pathlib import Path
|
||||
|
||||
CHECKSUMS = (".md5", ".sha1", ".sha256", ".sha512")
|
||||
POM = "{http://maven.apache.org/POM/4.0.0}"
|
||||
|
||||
|
||||
def expect(path: Path, signed: bool, problems: list[str]) -> None:
|
||||
if not path.is_file():
|
||||
problems.append(f"missing {path.name}")
|
||||
return
|
||||
for suffix in CHECKSUMS + ((".asc",) if signed else ()):
|
||||
if not path.with_name(path.name + suffix).is_file():
|
||||
problems.append(f"missing {path.name}{suffix}")
|
||||
|
||||
|
||||
def unversioned_pom(pom: Path) -> list[str]:
|
||||
root = ET.parse(pom).getroot()
|
||||
out = []
|
||||
for dependency in root.iter(f"{POM}dependency"):
|
||||
version = dependency.findtext(f"{POM}version")
|
||||
if not version or not version.strip():
|
||||
out.append(f"{dependency.findtext(f'{POM}groupId')}:{dependency.findtext(f'{POM}artifactId')}")
|
||||
return out
|
||||
|
||||
|
||||
def unversioned_module(module: Path) -> list[str]:
|
||||
data = json.loads(module.read_text(encoding="utf-8"))
|
||||
out = []
|
||||
for variant in data.get("variants", []):
|
||||
for dependency in variant.get("dependencies", []):
|
||||
version = dependency.get("version") or {}
|
||||
if not any(version.get(k) for k in ("requires", "strictly", "prefers")):
|
||||
out.append(f"{variant['name']}: {dependency['group']}:{dependency['module']}")
|
||||
return out
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) < 6:
|
||||
print(__doc__, file=sys.stderr)
|
||||
return 2
|
||||
repository, group, version, signing = Path(sys.argv[1]), sys.argv[2], sys.argv[3], sys.argv[4]
|
||||
signed = signing == "signed"
|
||||
failed = False
|
||||
for artifact in sys.argv[5:] + ["grpc-library"]:
|
||||
directory = repository / group.replace(".", "/") / artifact / version
|
||||
base = directory / f"{artifact}-{version}"
|
||||
problems: list[str] = []
|
||||
if artifact == "grpc-library":
|
||||
expect(base.with_name(base.name + ".pom"), signed, problems)
|
||||
expect(base.with_name(base.name + "-cyclonedx.json"), signed, problems)
|
||||
else:
|
||||
for suffix in (".jar", "-sources.jar", "-javadoc.jar", ".pom", ".module"):
|
||||
expect(base.with_name(base.name + suffix), signed, problems)
|
||||
pom, module = base.with_name(base.name + ".pom"), base.with_name(base.name + ".module")
|
||||
if pom.is_file():
|
||||
problems += [f"POM dependency without a version: {d}" for d in unversioned_pom(pom)]
|
||||
if module.is_file():
|
||||
problems += [f"module dependency without a version: {d}" for d in unversioned_module(module)]
|
||||
if problems:
|
||||
failed = True
|
||||
print(f"{group}:{artifact}:{version}", file=sys.stderr)
|
||||
for problem in problems:
|
||||
print(f" - {problem}", file=sys.stderr)
|
||||
else:
|
||||
print(f"staged: {group}:{artifact}:{version}{' (signed)' if signed else ''}")
|
||||
return 1 if failed else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+15
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env bash
|
||||
# The release's software bill of materials: one CycloneDX JSON for the version, from the committed
|
||||
# lockfiles of the published modules (build-logic is a build tool and is not shipped). Made before
|
||||
# the blocking vulnerability check, so a failed release can still say what it would have shipped.
|
||||
#
|
||||
# scripts/release/sbom.sh <output file>
|
||||
set -euo pipefail
|
||||
|
||||
root=$(git rev-parse --show-toplevel)
|
||||
output=${1:?usage: sbom.sh <output file>}
|
||||
mkdir -p "$(dirname "$output")"
|
||||
trivy=$("$root/scripts/tool.sh" trivy)
|
||||
cd "$root"
|
||||
"$trivy" fs --format cyclonedx --skip-dirs build-logic --output "$output" .
|
||||
python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d.get("bomFormat")=="CycloneDX" and d.get("components"), "empty SBOM"; print("sbom:", len(d["components"]), "components")' "$output"
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# After `publish`: every module's maven-metadata.xml, fetched anonymously, lists the new version.
|
||||
# Anonymous on purpose: consumers (the backend template, its image build) read without a token, so a
|
||||
# package only the owner can read is not published.
|
||||
#
|
||||
# scripts/release/verify-published.sh <repository url> <group> <version> <artifact>...
|
||||
set -euo pipefail
|
||||
|
||||
url=${1:?}; group=${2:?}; version=${3:?}; shift 3
|
||||
[ $# -gt 0 ] || { echo "verify-published: no artifacts named" >&2; exit 2; }
|
||||
path=${group//.//}
|
||||
missing=0
|
||||
for artifact in "$@"; do
|
||||
metadata=$(curl --fail --show-error --silent --location --retry 3 "$url/$path/$artifact/maven-metadata.xml" || true)
|
||||
if grep -q "<version>$version</version>" <<<"$metadata"; then
|
||||
echo "published: $group:$artifact:$version"
|
||||
else
|
||||
echo "verify-published: $group:$artifact:$version is not in $url/$path/$artifact/maven-metadata.xml" >&2
|
||||
missing=1
|
||||
fi
|
||||
done
|
||||
exit "$missing"
|
||||
Executable
+47
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env bash
|
||||
# The release's dependency gate over the committed lockfiles, the backend template's policy:
|
||||
# 1. HIGH and CRITICAL fail.
|
||||
# 2. MEDIUM and LOW are reported and do not fail.
|
||||
# 3. Any finding listed in the CISA Known Exploited Vulnerabilities catalog fails, whatever its
|
||||
# severity. An unreachable or malformed catalog fails too: a cross-check that did not run must
|
||||
# not read as "no match".
|
||||
# Suppressions: .trivyignore.yaml only (each entry with a statement and an expiry).
|
||||
#
|
||||
# scripts/release/vulnerability-scan.sh <report directory>
|
||||
set -euo pipefail
|
||||
|
||||
root=$(git rev-parse --show-toplevel)
|
||||
out=${1:?usage: vulnerability-scan.sh <report directory>}
|
||||
mkdir -p "$out"
|
||||
trivy=$("$root/scripts/tool.sh" trivy)
|
||||
cd "$root"
|
||||
|
||||
"$trivy" fs --scanners vuln --severity CRITICAL,HIGH --exit-code 1 --ignorefile .trivyignore.yaml .
|
||||
"$trivy" fs --scanners vuln --severity MEDIUM,LOW --exit-code 0 --ignorefile .trivyignore.yaml .
|
||||
"$trivy" fs --scanners vuln --severity CRITICAL,HIGH,MEDIUM,LOW,UNKNOWN --exit-code 0 \
|
||||
--ignorefile .trivyignore.yaml --format json --output "$out/trivy-all.json" .
|
||||
|
||||
kev_url=${KEV_FEED_URL:-https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json}
|
||||
if ! curl --fail --show-error --silent --location --retry 3 --proto '=https' --tlsv1.2 \
|
||||
"$kev_url" --output "$out/kev.json"; then
|
||||
echo "vulnerability-scan: KEV feed unavailable ($kev_url); set KEV_FEED_URL to an approved mirror." >&2
|
||||
exit 1
|
||||
fi
|
||||
python3 - "$out/trivy-all.json" "$out/kev.json" <<'PY'
|
||||
import json, re, sys
|
||||
findings, kev = (json.load(open(p, encoding="utf-8")) for p in sys.argv[1:3])
|
||||
ids = kev.get("vulnerabilities")
|
||||
if not isinstance(ids, list) or not ids or kev.get("count") != len(ids):
|
||||
sys.exit("vulnerability-scan: KEV feed is malformed, empty or count-inconsistent")
|
||||
catalog = {v.get("cveID") for v in ids}
|
||||
if not all(isinstance(c, str) and re.fullmatch(r"CVE-\d{4}-\d{4,}", c) for c in catalog):
|
||||
sys.exit("vulnerability-scan: KEV feed holds an invalid cveID")
|
||||
results = findings.get("Results")
|
||||
if not isinstance(results, list) or not results:
|
||||
sys.exit("vulnerability-scan: Trivy found no lockfile to scan")
|
||||
found = {v["VulnerabilityID"] for r in results for v in (r.get("Vulnerabilities") or [])}
|
||||
hits = sorted(found & catalog)
|
||||
if hits:
|
||||
sys.exit("vulnerability-scan: CISA KEV-listed vulnerability found regardless of severity: " + ", ".join(hits))
|
||||
print(f"vulnerability-scan: {len(found)} finding(s) below HIGH, none in the KEV catalog ({len(catalog)} entries).")
|
||||
PY
|
||||
Executable
+62
@@ -0,0 +1,62 @@
|
||||
#!/usr/bin/env bash
|
||||
# Prints the path of a pinned, checksum-verified tool binary, downloading it once into
|
||||
# ${TOOL_CACHE:-~/.cache/grpc-library}/<tool>-<version>/.
|
||||
#
|
||||
# scripts/tool.sh buf|trivy
|
||||
#
|
||||
# A tool is a native binary, not a Maven artifact, so Gradle cannot resolve it; this is the one place
|
||||
# that says which version and which bytes. A download whose SHA-256 differs from the one written here
|
||||
# fails and leaves nothing behind. Exit codes: 0 = path printed, 1 = download or checksum failed,
|
||||
# 2 = usage or unsupported machine.
|
||||
set -euo pipefail
|
||||
|
||||
tool=${1:-}
|
||||
case "$(uname -s)-$(uname -m)" in
|
||||
Linux-x86_64) arch=amd64 ;;
|
||||
Linux-aarch64 | Linux-arm64) arch=arm64 ;;
|
||||
*) echo "tool.sh: unsupported machine $(uname -s)-$(uname -m)" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
case "$tool" in
|
||||
buf)
|
||||
# Buf 1.73.0, the version the backend template's grpc-proto-buf job ran.
|
||||
version=1.73.0
|
||||
[ "$arch" = amd64 ] && asset=buf-Linux-x86_64 || asset=buf-Linux-aarch64
|
||||
[ "$arch" = amd64 ] \
|
||||
&& sha256=8f2986298ad08f0cc1bf999b9797b7c383adf32d7edf0f73d6f1e1a701baeac1 \
|
||||
|| sha256=902b75267db7f4391e99b7fa0756050e5354234cc0437ef50eee9c788950c7a3
|
||||
url="https://github.com/bufbuild/buf/releases/download/v${version}/${asset}"
|
||||
archive=0
|
||||
;;
|
||||
trivy)
|
||||
# Trivy 0.71.2, the version and checksums the backend template's dependency-vulnerability.yml pins.
|
||||
version=0.71.2
|
||||
[ "$arch" = amd64 ] && asset="trivy_${version}_Linux-64bit.tar.gz" || asset="trivy_${version}_Linux-ARM64.tar.gz"
|
||||
[ "$arch" = amd64 ] \
|
||||
&& sha256=0510e71e2fd39bf863856d499c8dc19feb4e7336546394c502a8f5cc7ab27460 \
|
||||
|| sha256=fe1c7106e15a5365d485b098a8c338f91e3b7ba71cb0e4963b98a3a098763cfc
|
||||
url="${TRIVY_DOWNLOAD_BASE_URL:-https://github.com/aquasecurity/trivy/releases/download/v${version}}/${asset}"
|
||||
archive=1
|
||||
;;
|
||||
*) echo "usage: scripts/tool.sh buf|trivy" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
dir="${TOOL_CACHE:-$HOME/.cache/grpc-library}/${tool}-${version}"
|
||||
binary="$dir/$tool"
|
||||
if [ ! -x "$binary" ]; then
|
||||
mkdir -p "$dir"
|
||||
tmp=$(mktemp "$dir/.download.XXXXXX")
|
||||
trap 'rm -f "$tmp"' EXIT
|
||||
curl --fail --show-error --silent --location --retry 3 --proto '=https' --tlsv1.2 "$url" --output "$tmp"
|
||||
if ! printf '%s %s\n' "$sha256" "$tmp" | sha256sum -c - >/dev/null 2>&1; then
|
||||
echo "tool.sh: $asset does not match the pinned SHA-256 $sha256" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$archive" = 1 ]; then
|
||||
tar -xzf "$tmp" -C "$dir" "$tool"
|
||||
else
|
||||
mv "$tmp" "$binary"
|
||||
fi
|
||||
chmod 0755 "$binary"
|
||||
fi
|
||||
printf '%s\n' "$binary"
|
||||
Reference in New Issue
Block a user