donghyeon.kang/tech-log-backend
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

feat: add production capability foundations

Browse Source
This commit is contained in:
donghyeon-ka
2026-07-31 23:50:44 +09:00
parent b3add0162d
commit 567422f2e5
757 changed files with 132385 additions and 2146 deletions
Download Patch File Download Diff File Expand all files Collapse all files
infra/redis-lab/README.md
+134
View File
@@ -0,0 +1,134 @@
# Disposable Redis qualification lab
This directory owns the lifecycle contract for the isolated three-node k3s lab. It does not contain
Redis workloads, credentials, certificates, or qualification evidence.
## Fixed topology
| Instance | CPU | Memory | Disk | Role |
| --- | ---: | ---: | ---: | --- |
| `ca-redis-lab-server` | 2 | 3G | 12G | k3s server |
| `ca-redis-lab-agent-1` | 2 | 2560M | 12G | k3s agent |
| `ca-redis-lab-agent-2` | 2 | 2560M | 12G | k3s agent |
The lab uses pod CIDR `10.52.0.0/16`, service CIDR `10.53.0.0/16`, and context
`ca-redis-lab`. `versions.env` pins the k3s version and Multipass image. Traefik and ServiceLB are
disabled.
## Safety model
All state, rendered cloud-init, kubeconfigs, tokens, and raw observations are mode-restricted
beneath the ignored `src/build/redis-lab` directory. Every canonical ancestor from the repository
root through `src/build/redis-lab`, plus runtime children, is validated before observation or
mutation; a symlink or real-path escape fails closed. The lifecycle never exports `KUBECONFIG`,
merges a kubeconfig, or writes the user's default kubeconfig.
Host observation and lab access deliberately use different explicit targets:
- host read-only queries copy the default kubeconfig into
`src/build/redis-lab/observations/host-kubeconfig` and use its unchanged original context;
- lab read-only queries use `src/build/redis-lab/kubeconfig` and exact context `ca-redis-lab`.
This split preserves the host context identity while ensuring no kubectl call relies on an implicit
target. Host kubectl mutations are not part of the lifecycle. The observation-only host copy is
removed after fingerprint and CIDR observation on success and every handled failure path.
Each exact name gets a private mode-`0600` rendered cloud-init file beneath
`src/build/redis-lab/cloud-init`. It writes only the non-secret ownership marker
`RUN_ID|VM_NAME` to `/var/lib/ca-redis-lab/ownership` as `root:root` mode `0600`; launch uses only
that rendered file.
Each name is then atomically reserved as `PENDING` in `run.state` before its bounded launch. The
state starts with an exact per-run identity, and each `PENDING`/`CREATED`/`RECONCILE` entry carries
that same identity. A successful launch becomes `CREATED` only after a bounded
`multipass exec <name> -- sudo cat /var/lib/ca-redis-lab/ownership` returns the exact marker.
Timeout, launch error, missing/foreign marker, signal, promotion failure, or uncertain cleanup
enters `RECONCILE`.
Cleanup transitions a recorded entry to `RECONCILE`, bounded-polls the exact instance and marker,
and issues `multipass delete --purge <exact-name>` only after the marker matches. It atomically
removes only an entry whose delete succeeded. A late-created matching instance is deleted; an
absent instance, unreadable marker, mismatched/foreign marker, or failed delete is retained as a
tombstone and fails closed without an unproven delete. Existing instance-bearing state blocks a
new `preflight`, `up`, or `run`; a rejected new run does not clean the prior run, and `down` is the
retry/reconciliation entry point. Existing
allowlisted names without owned state cause `up` to stop before reservation/launch and are never
adopted or deleted. Wildcards, `--all`, global purge, and discovered-instance deletion are
forbidden.
The lifecycle lock is nonblocking and exclusive. External children close its descriptor by
default, including detached infrastructure descendants and `run --` commands; only lock
acquisition retains descriptor 9.
Multipass list/launch/info/exec/transfer/delete, installation/join, and kubectl calls have fixed
time bounds. `up` succeeds only after the exact server and two agents all report `Ready=True`
within the bounded poll budget; incomplete or not-ready inventory enters marker-proven run-owned
cleanup.
The k3s runtime is amd64-only and fail-closed in this slice. `versions.env` pins the immutable
release URL and exact SHA-256 for `v1.33.3+k3s1`. The lifecycle performs a bounded host download,
verifies the digest, transfers the binary to each exact VM, verifies the transferred digest and
reported binary version inside each VM, and only then installs/starts it. It does not execute a
network installer or a `curl | sh` pipeline.
The generated lab kubeconfig is accepted only in the pinned single-cluster/single-context/
single-user block grammar. A tracked AWK state machine has one explicit transition for every
allowlisted line and publishes no output until the complete document reaches its exact final
state. It rejects missing, duplicate, reordered, unknown, whitespace-altered, quoted, tagged, or
explicit keys; anchors, aliases, merge keys, tabs, CRLF, document markers, trailing content, and
all flow collections except exact `preferences: {}`. Only the exact cluster/context/user identity,
`current-context`, and loopback API server are rewritten. CA data, client certificate/key data,
and an optional canonical namespace are byte-preserved.
Rendering uses a same-directory `kubeconfig.next`, applies mode `0600`, and replaces the
destination only after render and permission success. The renderer must be a readable regular
non-symlink file at its canonical tracked path, and both destination paths are protected by the
runtime symlink contract. Renderer, permission, or move failure removes both candidate and
destination, performs no lab `kubectl`, and enters exact marker-proven current-run cleanup.
Assigned Service ClusterIPs cannot prove the host service CIDR. When a host kubeconfig exists,
callers must supply one or more canonical, comma- or space-separated IPv4 CIDRs through
`REDIS_LAB_HOST_SERVICE_CIDRS`. Missing, malformed, or overlapping input fails before launch:
```bash
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 \
infra/redis-lab/bin/redis-lab preflight
```
## Commands
The real lifecycle is for a trusted local or dedicated runner only:
```bash
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 infra/redis-lab/bin/redis-lab preflight
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 infra/redis-lab/bin/redis-lab up
infra/redis-lab/bin/redis-lab down
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 infra/redis-lab/bin/redis-lab run -- command
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 \
infra/redis-lab/bin/redis-lab run --retain-on-failure -- command
```
`run` establishes its cleanup obligation before entering the inner `up`, keeps it through the
post-up/pre-command handoff and user command, then tears down after command success or failure and
compares the canonical pre/post host fingerprints after cleanup. A successful direct `up` retains
the lab by design. Local `--retain-on-failure` intentionally leaves the recorded lab for diagnosis
and skips an isolation-success claim; `CI=true` rejects that option before launch.
The blocking contract is VM-free:
```bash
cd src
./gradlew :adapter:outbound:cache-redis:redisLabContractTest --console=plain
```
It injects fake infrastructure commands. Hosted CI must run only this contract, never the real lab.
The contract executes a copied lifecycle in
`src/build/redis-lab-contract/repository`, seals `PATH` to explicit fakes/safe wrappers, and compares
a byte-level snapshot proving it did not modify the real repository's `src/build/redis-lab`. It
also exercises direct/run signal cleanup, rendered-child symlink rejection, successful and
late-create marker proof, absent/foreign-marker tombstones, second-run state preservation,
CREATED cleanup uncertainty, rejected-run preservation of prior `CREATED` and `RECONCILE` state,
the post-up/pre-command signal handoff, the canonical kubeconfig mutation matrix, missing/symlinked
renderer rejection, fail-closed `.next`/permission/move publication, and infrastructure/user
background-child lock non-inheritance. This is deterministic fake-runtime evidence only; it is not
live Multipass, k3s, kubectl, network, or host-isolation qualification.