Commit Graph
5 Commits
Author SHA1 Message Date
DongHyeonka 5d58160c89 feat(observability): alert on the root disk before kubelet acts
The kubelet starts deleting unused images at 85% root use and evicts
pods at 90%. The root disk warning fired at 85% and the critical at
90%, so both arrived when the kubelet was already acting. On
2026-11-09 the warning fired for 33 hours, then an eviction stopped
the tech-log pods, the image GC deleted their imagePullPolicy: Never
images, and production went down.

Move the warning to 80% for 10m and the critical to 85% for 5m, and
add PlatformRootFilesystemFillingFast (critical, for 15m): the 6h
linear trend of free bytes falls below 10% of the disk, the eviction
point, within 6 hours. On 2026-11-07 the disk lost 17G in 9 hours;
this forecast would have fired hours before the 80% warning. The
info alert at 75% stays.

Add scripts/validate/test-platform-disk-alerts.sh, promtool unit
tests for the usage bands and the forecast with the pinned Prometheus
image. The existing rules test needs live metric inventories, so this
one runs offline.

Risk: the live rules change only when the observability access apply
runs; this commit does not touch the cluster. The forecast can fire
on a large build that the BuildKit GC removes minutes later; the 15m
hold and the 6h window damp that.

Verification: the new test failed against the old rules (no alert at
81%, 86% or for a 1-per-minute fill) and passes after the change;
promtool check rules finds 9 rules. test-observability-alerting.sh
passes its static contract, then stops at the helm render because
helm v3.19.4 is not installed on the desktop.
2026-11-11 21:51:59 +09:00
DongHyeonka a92e6b854b docs(gitea): record the Actions and package registry rollout
The Gitea README still said Actions and the package registry are
disabled on first deployment, and had no guidance for either.

Replace that decision with the 2026-11-07 state, add where packages
are stored and how much space they really have (a local PV does not
enforce the 50Gi request), the cleanup rules to use for Maven
SNAPSHOTs, and who may publish and read Maven packages. Record the
applied manifest SHA-256 and link the runner guide from the root
README.

Verification: statements checked against the Gitea 1.27 docs and
source (routers/api/packages/maven, modules/web/router.go), the live
app.ini, and API responses after the rollout.
2026-11-07 18:36:55 +09:00
DongHyeonka 87dee1f749 feat(gitea): enable the package registry
The backend template is moving its messaging and gRPC libraries into
separate repositories and will consume them as Maven versions from
Gitea Packages. [packages] ENABLED=false and repo.packages in
DISABLED_REPO_UNITS make the registry unavailable.

Turn on [packages] and leave DISABLED_REPO_UNITS as an explicit empty
value. The chart init container runs `gitea config edit-ini
--apply-env` against the app.ini on the data volume, so a removed key
would keep the old disabled list; an empty value overwrites it.
SSH and LFS stay off. This ships with the Actions change in the same
rollout, so Gitea restarts once.

Risk: packages are stored under /data/packages on the root SSD local
PV. The 50Gi PVC size is not enforced by a local PV; the real bound is
free space on the host root disk, shared with the host and the CI
runner. No per-owner size limit is set yet.

Verification: render-phase1.sh invariants pass; kubectl diff against
live shows only the inline-config keys actions, packages, repository
and the Deployment checksum/config annotation.

Refs: Gitea 1.27 config cheat sheet [packages], usage/packages
2026-11-07 18:22:23 +09:00
DongHyeonka 7cd4a419fd feat(gitea): enable Actions at the server level
The template repositories cannot run CI: [actions] ENABLED=false and
repo.actions in DISABLED_REPO_UNITS make Gitea ignore has_actions=true
on the repository API (200 response, value unchanged).

Turn on [actions] and drop only repo.actions from DISABLED_REPO_UNITS.
Packages stay disabled (packages.ENABLED=false, repo.packages kept).
Repositories created while the unit was disabled have no Actions unit
row, so every existing repository keeps has_actions=false until its
owner enables it. The values file is shared by the baseline and OIDC
build roots, so both renders change the same way.

Risk: new repositories get repo.actions by default (Gitea default
DEFAULT_REPO_UNITS). They run nothing without a runner in scope.
Gitea itself needs no new egress; runners connect inbound over the
existing Host Nginx -> Traefik HTTPS path.

Refs: Gitea 1.27 config cheat sheet [actions], [repository]
2026-11-07 17:37:52 +09:00
donghyeon-ka 16c337bcc9 Add platform infrastructure configuration 2026-08-28 17:35:41 +09:00