135 lines
7.7 KiB
Markdown
135 lines
7.7 KiB
Markdown
# Disposable Redis qualification lab
|
|
|
|
This directory owns the lifecycle contract for the isolated three-node k3s lab. It does not contain
|
|
Redis workloads, credentials, certificates, or qualification evidence.
|
|
|
|
## Fixed topology
|
|
|
|
| Instance | CPU | Memory | Disk | Role |
|
|
| --- | ---: | ---: | ---: | --- |
|
|
| `ca-redis-lab-server` | 2 | 3G | 12G | k3s server |
|
|
| `ca-redis-lab-agent-1` | 2 | 2560M | 12G | k3s agent |
|
|
| `ca-redis-lab-agent-2` | 2 | 2560M | 12G | k3s agent |
|
|
|
|
The lab uses pod CIDR `10.52.0.0/16`, service CIDR `10.53.0.0/16`, and context
|
|
`ca-redis-lab`. `versions.env` pins the k3s version and Multipass image. Traefik and ServiceLB are
|
|
disabled.
|
|
|
|
## Safety model
|
|
|
|
All state, rendered cloud-init, kubeconfigs, tokens, and raw observations are mode-restricted
|
|
beneath the ignored `src/build/redis-lab` directory. Every canonical ancestor from the repository
|
|
root through `src/build/redis-lab`, plus runtime children, is validated before observation or
|
|
mutation; a symlink or real-path escape fails closed. The lifecycle never exports `KUBECONFIG`,
|
|
merges a kubeconfig, or writes the user's default kubeconfig.
|
|
|
|
Host observation and lab access deliberately use different explicit targets:
|
|
|
|
- host read-only queries copy the default kubeconfig into
|
|
`src/build/redis-lab/observations/host-kubeconfig` and use its unchanged original context;
|
|
- lab read-only queries use `src/build/redis-lab/kubeconfig` and exact context `ca-redis-lab`.
|
|
|
|
This split preserves the host context identity while ensuring no kubectl call relies on an implicit
|
|
target. Host kubectl mutations are not part of the lifecycle. The observation-only host copy is
|
|
removed after fingerprint and CIDR observation on success and every handled failure path.
|
|
|
|
Each exact name gets a private mode-`0600` rendered cloud-init file beneath
|
|
`src/build/redis-lab/cloud-init`. It writes only the non-secret ownership marker
|
|
`RUN_ID|VM_NAME` to `/var/lib/ca-redis-lab/ownership` as `root:root` mode `0600`; launch uses only
|
|
that rendered file.
|
|
|
|
Each name is then atomically reserved as `PENDING` in `run.state` before its bounded launch. The
|
|
state starts with an exact per-run identity, and each `PENDING`/`CREATED`/`RECONCILE` entry carries
|
|
that same identity. A successful launch becomes `CREATED` only after a bounded
|
|
`multipass exec <name> -- sudo cat /var/lib/ca-redis-lab/ownership` returns the exact marker.
|
|
Timeout, launch error, missing/foreign marker, signal, promotion failure, or uncertain cleanup
|
|
enters `RECONCILE`.
|
|
|
|
Cleanup transitions a recorded entry to `RECONCILE`, bounded-polls the exact instance and marker,
|
|
and issues `multipass delete --purge <exact-name>` only after the marker matches. It atomically
|
|
removes only an entry whose delete succeeded. A late-created matching instance is deleted; an
|
|
absent instance, unreadable marker, mismatched/foreign marker, or failed delete is retained as a
|
|
tombstone and fails closed without an unproven delete. Existing instance-bearing state blocks a
|
|
new `preflight`, `up`, or `run`; a rejected new run does not clean the prior run, and `down` is the
|
|
retry/reconciliation entry point. Existing
|
|
allowlisted names without owned state cause `up` to stop before reservation/launch and are never
|
|
adopted or deleted. Wildcards, `--all`, global purge, and discovered-instance deletion are
|
|
forbidden.
|
|
|
|
The lifecycle lock is nonblocking and exclusive. External children close its descriptor by
|
|
default, including detached infrastructure descendants and `run --` commands; only lock
|
|
acquisition retains descriptor 9.
|
|
|
|
Multipass list/launch/info/exec/transfer/delete, installation/join, and kubectl calls have fixed
|
|
time bounds. `up` succeeds only after the exact server and two agents all report `Ready=True`
|
|
within the bounded poll budget; incomplete or not-ready inventory enters marker-proven run-owned
|
|
cleanup.
|
|
|
|
The k3s runtime is amd64-only and fail-closed in this slice. `versions.env` pins the immutable
|
|
release URL and exact SHA-256 for `v1.33.3+k3s1`. The lifecycle performs a bounded host download,
|
|
verifies the digest, transfers the binary to each exact VM, verifies the transferred digest and
|
|
reported binary version inside each VM, and only then installs/starts it. It does not execute a
|
|
network installer or a `curl | sh` pipeline.
|
|
|
|
The generated lab kubeconfig is accepted only in the pinned single-cluster/single-context/
|
|
single-user block grammar. A tracked AWK state machine has one explicit transition for every
|
|
allowlisted line and publishes no output until the complete document reaches its exact final
|
|
state. It rejects missing, duplicate, reordered, unknown, whitespace-altered, quoted, tagged, or
|
|
explicit keys; anchors, aliases, merge keys, tabs, CRLF, document markers, trailing content, and
|
|
all flow collections except exact `preferences: {}`. Only the exact cluster/context/user identity,
|
|
`current-context`, and loopback API server are rewritten. CA data, client certificate/key data,
|
|
and an optional canonical namespace are byte-preserved.
|
|
|
|
Rendering uses a same-directory `kubeconfig.next`, applies mode `0600`, and replaces the
|
|
destination only after render and permission success. The renderer must be a readable regular
|
|
non-symlink file at its canonical tracked path, and both destination paths are protected by the
|
|
runtime symlink contract. Renderer, permission, or move failure removes both candidate and
|
|
destination, performs no lab `kubectl`, and enters exact marker-proven current-run cleanup.
|
|
|
|
Assigned Service ClusterIPs cannot prove the host service CIDR. When a host kubeconfig exists,
|
|
callers must supply one or more canonical, comma- or space-separated IPv4 CIDRs through
|
|
`REDIS_LAB_HOST_SERVICE_CIDRS`. Missing, malformed, or overlapping input fails before launch:
|
|
|
|
```bash
|
|
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 \
|
|
infra/redis-lab/bin/redis-lab preflight
|
|
```
|
|
|
|
## Commands
|
|
|
|
The real lifecycle is for a trusted local or dedicated runner only:
|
|
|
|
```bash
|
|
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 infra/redis-lab/bin/redis-lab preflight
|
|
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 infra/redis-lab/bin/redis-lab up
|
|
infra/redis-lab/bin/redis-lab down
|
|
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 infra/redis-lab/bin/redis-lab run -- command
|
|
REDIS_LAB_HOST_SERVICE_CIDRS=10.43.0.0/16 \
|
|
infra/redis-lab/bin/redis-lab run --retain-on-failure -- command
|
|
```
|
|
|
|
`run` establishes its cleanup obligation before entering the inner `up`, keeps it through the
|
|
post-up/pre-command handoff and user command, then tears down after command success or failure and
|
|
compares the canonical pre/post host fingerprints after cleanup. A successful direct `up` retains
|
|
the lab by design. Local `--retain-on-failure` intentionally leaves the recorded lab for diagnosis
|
|
and skips an isolation-success claim; `CI=true` rejects that option before launch.
|
|
|
|
The blocking contract is VM-free:
|
|
|
|
```bash
|
|
cd src
|
|
./gradlew :adapter:outbound:cache-redis:redisLabContractTest --console=plain
|
|
```
|
|
|
|
It injects fake infrastructure commands. Hosted CI must run only this contract, never the real lab.
|
|
The contract executes a copied lifecycle in
|
|
`src/build/redis-lab-contract/repository`, seals `PATH` to explicit fakes/safe wrappers, and compares
|
|
a byte-level snapshot proving it did not modify the real repository's `src/build/redis-lab`. It
|
|
also exercises direct/run signal cleanup, rendered-child symlink rejection, successful and
|
|
late-create marker proof, absent/foreign-marker tombstones, second-run state preservation,
|
|
CREATED cleanup uncertainty, rejected-run preservation of prior `CREATED` and `RECONCILE` state,
|
|
the post-up/pre-command signal handoff, the canonical kubeconfig mutation matrix, missing/symlinked
|
|
renderer rejection, fail-closed `.next`/permission/move publication, and infrastructure/user
|
|
background-child lock non-inheritance. This is deterministic fake-runtime evidence only; it is not
|
|
live Multipass, k3s, kubectl, network, or host-isolation qualification.
|