A backup nobody has restored is not known to work. The 2026-08 k3s
recovery bundle never got its isolated restore drill.
k3s-datastore-restore-drill.sh downloads the latest archive, checks its
sha256 and integrity, starts the same k3s version in a network-less
throwaway container on the copy, and checks the API comes up, the
Secret count is close to live and every Secret decrypts with the
token. gitea-backup-restore-drill.sh extracts the latest dump and
checks app.ini (INTERNAL_TOKEN, SECRET_KEY), git fsck on every
repository and the repository count against live Gitea. Both accept a
local copy and write a drill timestamp metric.
Risk: read-only for the cluster. The k3s drill runs the restored
control plane with --network none, so its controllers reach nothing.
Verification (desktop):
- test-k3s-datastore-restore-drill.sh: a fresh k3s with three test
Secrets (stored encrypted, k8s:enc:) restores with all Secrets
decrypted in 7s; before folding the WAL the test saw 2 of 3 rows,
which is why the real backup uses sqlite3 .backup.
- gitea-backup-restore-drill.sh on a live repository dump: 29
repositories, 0 fsck failures, 29 live; the dump was deleted after.