22 KiB
Project Infra Phase 1 dev GitOps Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use
superpowers:subagent-driven-development(recommended) orsuperpowers:executing-plansto execute this plan task by task. Usesuperpowers:test-driven-developmentfor behavior changes andsuperpowers:verification-before-completionbefore reporting success.
Goal: Rename the active rollout from lab to dev, make operator desired state render from Git, and correct the ForwardAuth/Keycloak ownership defects without changing a live cluster.
Architecture: The catalog remains ownership-first and environment-neutral in base, while overlays/dev contains dev-only values. gitops/clusters/dev/main is the only current deployable Kubernetes root. A shared render function always enables Kustomize Helm support. Operator charts live in independent platform units and are selected by a dedicated ordered stage. This plan only changes Git-owned source and local tests; it must not invoke kubectl apply, kubectl delete, or Helm mutation commands.
Tech Stack: Bash, Kustomize 5.8.1, Helm 3.19.4, Kubernetes/K3s manifests, repository shell test harness, rg.
Global constraints
- Read
/AGENTS.md, the nearest nestedAGENTS.md,docs/standards/infra/STYLE.md, and the directly relevant standards before each manifest task. - Preserve
mnt,minio-operator, andvault-secrets-operator-systemnamespace names in Phase 1. - Preserve
.local.testhostnames as dev-only values. - Never apply
gitops/clusters/dev/main/all; it is render/audit-only. - Do not run a live-cluster mutation command while executing this plan.
- Do not edit CI workflow definitions or the sibling
cicd-platformrepository. - Commit only the files listed by the current task and inspect
git status --shortbefore each commit.
Dependency and interface contract
This plan is executed before the automation-safety and validation-structure plans.
tests/run.sh [name-fragment]discovers executabletests/contracts/*-test.shfiles and optionally filters by basename.tests/lib/assert.shprovidesfail,assert_eq,assert_file_exists,assert_file_absent,assert_contains, andassert_not_contains.scripts/lib/kustomize.shexportskustomize_render ENTRYPOINT; rendered YAML is written only to stdout and diagnostics only to stderr.- Every later render path must call
kustomize_render; no later script may reimplementkustomize buildorkubectl kustomize. - The ordered dev stages are
00-platform,05-operators,10-vault,20-secrets,30-data,35-registry,40-operations, and50-apps.
Task 1: Add the contract-test harness and rename the active environment
Files:
- Create:
tests/lib/assert.sh - Create:
tests/run.sh - Create:
tests/contracts/dev-layout-test.sh - Move:
gitops/clusters/lab/main→gitops/clusters/dev/main - Move:
gitops/apps/auth-migration/overlays/lab→gitops/apps/auth-migration/overlays/dev - Move:
gitops/apps/auth-server/overlays/lab→gitops/apps/auth-server/overlays/dev - Move:
gitops/apps/identity-postgres/overlays/lab→gitops/apps/identity-postgres/overlays/dev - Move:
gitops/apps/keycloak-realm-import/overlays/lab→gitops/apps/keycloak-realm-import/overlays/dev - Move:
gitops/platform/cert-manager/overlays/lab→gitops/platform/cert-manager/overlays/dev - Move:
gitops/platform/forward-auth/overlays/lab→gitops/platform/forward-auth/overlays/dev - Move:
gitops/platform/keycloak-operator/overlays/lab→gitops/platform/keycloak-operator/overlays/dev - Move:
gitops/platform/keycloak/overlays/lab→gitops/platform/keycloak/overlays/dev - Move:
gitops/platform/minio/overlays/lab→gitops/platform/minio/overlays/dev - Move:
gitops/platform/registry/overlays/lab→gitops/platform/registry/overlays/dev - Move:
gitops/platform/secret-delivery/overlays/lab→gitops/platform/secret-delivery/overlays/dev - Move:
gitops/platform/traefik/overlays/lab→gitops/platform/traefik/overlays/dev - Move:
gitops/platform/vault/overlays/lab→gitops/platform/vault/overlays/dev - Move:
gitops/policies/baseline/overlays/lab→gitops/policies/baseline/overlays/dev - Modify: moved Kustomization files,
tests/kustomize-entrypoints.txt - Modify:
AGENTS.md,README.md,guide.md,gitops/AGENTS.md,gitops/PROJECT.md,gitops/clusters/AGENTS.md,gitops/clusters/README.md,gitops/clusters/dev/main/README.md,docs/architecture.md,docs/architecture/repository-structure.md,docs/guides/getting-started.md,docs/ingress-traefik.md,docs/operations.md,docs/security-hardening.md,docs/troubleshooting.md,docs/vault-vso.md,scripts/bin/bootstrap.sh,scripts/bin/teardown.sh,scripts/tasks/minio-provision-registry.sh,scripts/ci/validate-docs.sh,scripts/README.md,tests/README.md
Step 1: Write the failing layout contract
Use strict mode in every new script. The runner must execute each selected test in a fresh Bash process and return non-zero if any test fails:
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
filter="${1:-}"
failed=0
while IFS= read -r test_file; do
[[ -z "$filter" || "$(basename "$test_file")" == *"$filter"* ]] || continue
bash "$test_file" || failed=$((failed + 1))
done < <(find "$ROOT_DIR/tests/contracts" -type f -name '*-test.sh' -print | sort)
((failed == 0))
dev-layout-test.sh must assert all of these conditions:
gitops/clusters/dev/main exists
gitops/clusters/lab does not exist
every selected unit has overlays/dev and no overlays/lab
tests/kustomize-entrypoints.txt contains gitops/clusters/dev/main
tests/kustomize-entrypoints.txt does not contain gitops/clusters/lab
no deployable kustomization contains example.com/environment: lab
the active docs listed above do not instruct users to deploy lab
Step 2: Run the contract and verify that it fails for the current tree
Run:
bash tests/run.sh dev-layout
Expected: non-zero, with the first failure reporting the missing gitops/clusters/dev/main path.
Step 3: Move the directories with Git-aware renames
Run one git mv per source directory. Do not use recursive copy/delete. After the moves, replace only active environment references:
rg -l 'clusters/lab|overlays/lab|environment: lab|\blab\b' \
AGENTS.md README.md guide.md gitops scripts docs tests \
--glob '!docs/standards/**' \
--glob '!docs/examples/**' \
--glob '!docs/superpowers/**'
Review every hit and change current deployment semantics from lab to dev, including bootstrap/teardown accepted environment names and MinIO's dev-only HTTP default. Preserve historical text in approved design/spec documents and normative examples that intentionally compare environments. Change gitops/AGENTS.md base reuse wording to name dev, staging, and prod; it may still describe lab only as an optional disposable environment.
Step 4: Update the moved cluster root
All resource paths under gitops/clusters/dev/main/stages/*/kustomization.yaml must select overlays/dev. Every stage label must be:
labels:
- pairs:
example.com/environment: dev
example.com/owner-team: platform
includeSelectors: false
includeTemplates: true
Remove ../../../../../tenants/mnt/base from stages/00-platform; namespace ownership belongs only to the separate namespaces entrypoint.
Update tests/kustomize-entrypoints.txt to contain only dev cluster entrypoints. Keep all/ in the render inventory, but mark it audit-only in the cluster README and tests README.
Step 5: Run the test and documentation gate
Run:
bash tests/run.sh dev-layout
bash scripts/ci/validate-docs.sh
git diff --check
Expected: all commands exit 0.
Step 6: Commit
git add tests gitops AGENTS.md README.md guide.md docs scripts/bin/bootstrap.sh scripts/bin/teardown.sh scripts/tasks/minio-provision-registry.sh scripts/ci/validate-docs.sh scripts/README.md
git commit -m "refactor: dev 환경 경로로 전환"
Task 2: Introduce one Helm-aware Kustomize render interface
Files:
- Create:
scripts/lib/kustomize.sh - Create:
tests/contracts/kustomize-render-test.sh - Modify:
scripts/ci/validate.sh - Modify:
scripts/lib/common.shonly if it is needed to shareREPO_ROOTwithout changing cluster behavior
Step 1: Write the failing fake-command test
The test creates a temporary PATH with fake kustomize and kubectl executables. It must prove:
- when
kustomizeexists, the helper calls exactlykustomize build --enable-helm ENTRYPOINT; - when only
kubectlexists, it calls exactlykubectl kustomize ENTRYPOINT --enable-helm; - when neither exists, it exits non-zero;
- rendered YAML remains on stdout and the command trace remains outside stdout.
Run:
bash tests/run.sh kustomize-render
Expected: non-zero because scripts/lib/kustomize.sh does not yet exist.
Step 2: Implement the shared helper
Use this public interface:
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
kustomize_render() {
local entrypoint="${1:?entrypoint is required}"
if command -v kustomize >/dev/null 2>&1; then
kustomize build --enable-helm "$entrypoint"
elif command -v kubectl >/dev/null 2>&1; then
kubectl kustomize "$entrypoint" --enable-helm
else
printf 'ERROR: kustomize 또는 kubectl이 필요합니다.\n' >&2
return 1
fi
}
Source this file from scripts/ci/validate.sh and delete its local render_kustomization implementation. Change the validation call to:
kustomize_render "$entrypoint" >"$rendered"
Step 3: Verify the helper and current dev renders
Run:
bash tests/run.sh kustomize-render
VALIDATION_PROFILE=local bash scripts/ci/validate.sh
git diff --check
Expected: all commands exit 0; render log lines identify dev entrypoints.
Step 4: Commit
git add scripts/lib/kustomize.sh scripts/ci/validate.sh tests/contracts/kustomize-render-test.sh
git commit -m "refactor: Helm 지원 렌더 경로 통합"
Task 3: Add declarative operator namespaces and catalog units
Files:
- Create:
gitops/tenants/minio-operator/base/kustomization.yaml - Create:
gitops/tenants/minio-operator/base/namespace.yaml - Create:
gitops/tenants/vault-secrets-operator-system/base/kustomization.yaml - Create:
gitops/tenants/vault-secrets-operator-system/base/namespace.yaml - Create:
gitops/platform/minio-operator/base/kustomization.yaml - Create:
gitops/platform/minio-operator/overlays/dev/kustomization.yaml - Create:
gitops/platform/minio-operator/overlays/dev/values.yaml - Create:
gitops/platform/vault-secrets-operator/base/kustomization.yaml - Create:
gitops/platform/vault-secrets-operator/overlays/dev/kustomization.yaml - Move:
gitops/platform/secret-delivery/base/helm/values.yaml→gitops/platform/vault-secrets-operator/overlays/dev/values.yaml - Create:
gitops/clusters/dev/main/stages/05-operators/kustomization.yaml - Modify:
gitops/clusters/dev/main/namespaces/kustomization.yaml - Modify:
gitops/clusters/dev/main/all/kustomization.yaml - Modify:
tests/kustomize-entrypoints.txt - Create:
tests/contracts/operator-render-test.sh - Modify:
gitops/platform/README.md,gitops/tenants/README.md,gitops/clusters/dev/main/README.md
Step 1: Write the failing operator render contract
Render gitops/clusters/dev/main/stages/05-operators through kustomize_render. Assert:
the render exits 0
at least two Deployment documents exist
tenants.minio.min.io exists as a CustomResourceDefinition name
vaultstaticsecrets.secrets.hashicorp.com exists as a CustomResourceDefinition name
the namespaces entrypoint renders Namespace/minio-operator and Namespace/vault-secrets-operator-system
every namespaced resource in the operator stage targets minio-operator or vault-secrets-operator-system
no HelmRelease custom resource is present
chart versions 7.0.0 and 0.9.0 are pinned in source
includeCRDs: true is present in both dev overlay kustomizations
Run:
bash tests/run.sh operator-render
Expected: non-zero because the operator stage does not exist.
Step 2: Add namespace ownership units
Each Namespace must use the standard labels from gitops/tenants/mnt/base/namespace.yaml, with app.kubernetes.io/name and app.kubernetes.io/instance set to its namespace. Apply the restricted Pod Security labels. Each base Kustomization contains only its namespace.yaml.
Add both bases to gitops/clusters/dev/main/namespaces/kustomization.yaml after mnt.
Step 3: Add the MinIO Operator unit
The dev overlay Kustomization must be exactly shaped as follows:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
helmCharts:
- name: operator
repo: https://operator.min.io
version: 7.0.0
releaseName: minio-operator
namespace: minio-operator
includeCRDs: true
valuesFile: values.yaml
Use only chart-supported keys from the pinned MinIO Operator 7.0.0 values schema. values.yaml must be:
operator:
replicaCount: 1
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
fsGroup: 1000
seccompProfile:
type: RuntimeDefault
containerSecurityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
resources:
requests:
cpu: 200m
memory: 256Mi
ephemeral-storage: 500Mi
limits:
memory: 384Mi
ephemeral-storage: 1Gi
The single replica is a dev overlay decision. A later staging/prod overlay must reassess replicas, PDB, and topology from measured capacity.
Step 4: Add the Vault Secrets Operator unit
The dev overlay Kustomization must pin:
helmCharts:
- name: vault-secrets-operator
repo: https://helm.releases.hashicorp.com
version: 0.9.0
releaseName: vault-secrets-operator
namespace: vault-secrets-operator-system
includeCRDs: true
valuesFile: values.yaml
Move the existing hardened VSO values file unchanged first. Remove the now-empty gitops/platform/secret-delivery/base/helm directory from Git. Keep secret-delivery CRs in their existing unit; only the controller chart moves.
Step 5: Add the ordered stage and inventory
gitops/clusters/dev/main/stages/05-operators/kustomization.yaml contains exactly the two dev overlays and the normal dev/platform labels. Insert it between 00-platform and 10-vault in all/kustomization.yaml and in tests/kustomize-entrypoints.txt.
Do not add these operator overlays to 00-platform.
Step 6: Verify render, schema, and policy
Run:
bash tests/run.sh operator-render
VALIDATION_PROFILE=full bash scripts/ci/validate.sh
git diff --check
Expected: operator test exits 0; full validation reports no invalid schema or policy result. If chart download is unavailable, stop and report the external availability failure rather than committing an unrendered chart definition.
Step 7: Commit
git add gitops/tenants gitops/platform/minio-operator gitops/platform/vault-secrets-operator gitops/platform/secret-delivery gitops/clusters/dev/main tests
git commit -m "feat: operator desired state를 GitOps로 선언"
Task 4: Put ForwardAuth resources and patches under their real owners
Files:
- Move:
gitops/platform/forward-auth/component/oauth2-proxy-config.yaml→gitops/platform/forward-auth/overlays/dev/oauth2-proxy-config.yaml - Move:
gitops/platform/forward-auth/component/oauth2-proxy-ingress.yaml→gitops/platform/forward-auth/overlays/dev/oauth2-proxy-ingress.yaml - Move:
gitops/platform/forward-auth/component/oauth2-proxy-middleware.yaml→gitops/platform/forward-auth/overlays/dev/oauth2-proxy-middleware.yaml - Move:
gitops/platform/forward-auth/component/oauth2-proxy-networkpolicy.yaml→gitops/platform/forward-auth/overlays/dev/oauth2-proxy-networkpolicy.yaml - Delete:
gitops/platform/forward-auth/component/kustomization.yaml - Modify:
gitops/platform/forward-auth/overlays/dev/kustomization.yaml - Modify:
gitops/apps/auth-server/base/configmap.yaml - Modify:
gitops/apps/auth-server/overlays/dev/kustomization.yaml - Modify:
gitops/platform/keycloak/overlays/dev/kustomization.yaml - Delete:
gitops/platform/keycloak/overlays/dev/ingress-admin.yaml - Create:
tests/contracts/dev-ingress-render-test.sh
Step 1: Write the failing rendered-output assertions
Render gitops/clusters/dev/main/stages/50-apps and 00-platform. Extract resources by kind and metadata.name without assuming document order. Assert:
Ingress/auth-server annotation is exactly:
mnt-oauth2-proxy-errors@kubernetescrd,mnt-oauth2-proxy-auth@kubernetescrd,kube-system-security-headers@kubernetescrd
Ingress/keycloak exists
Ingress/keycloak-admin does not exist
ConfigMap/auth-server-config in the app stage contains SPRING_PROFILES_ACTIVE=dev
ConfigMap/auth-server-config rendered from gitops/apps/auth-server/base does not contain SPRING_PROFILES_ACTIVE
the 00-platform render contains oauth2-proxy ConfigMap, Ingress, Middleware resources, and NetworkPolicy
Run:
bash tests/run.sh dev-ingress-render
Expected: non-zero because the current auth-server annotation has only the security middleware.
Step 2: Refactor the ForwardAuth overlay
Change the dev overlay from a Component reference to normal ownership:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../base
- oauth2-proxy-config.yaml
- oauth2-proxy-ingress.yaml
- oauth2-proxy-middleware.yaml
- oauth2-proxy-networkpolicy.yaml
Delete the component directory after all resources are moved. The platform stage continues to select only forward-auth/overlays/dev.
Step 3: Move the auth-server patch to the auth-server owner
Add this patch to gitops/apps/auth-server/overlays/dev/kustomization.yaml:
- target:
kind: Ingress
name: auth-server
patch: |-
- op: replace
path: /metadata/annotations/traefik.ingress.kubernetes.io~1router.middlewares
value: mnt-oauth2-proxy-errors@kubernetescrd,mnt-oauth2-proxy-auth@kubernetescrd,kube-system-security-headers@kubernetescrd
Remove SPRING_PROFILES_ACTIVE from the base ConfigMap and add it through the existing dev ConfigMap patch.
Step 4: Remove the dev admin Ingress from desired state
Remove ingress-admin.yaml from the Keycloak dev Kustomization and delete the file. Do not run kubectl delete; live cleanup belongs to Phase 2.
Step 5: Verify exact rendered behavior
Run:
bash tests/run.sh dev-ingress-render
VALIDATION_PROFILE=full bash scripts/ci/validate.sh
git diff --check
Expected: all commands exit 0.
Step 6: Commit
git add gitops/platform/forward-auth gitops/apps/auth-server gitops/platform/keycloak tests/contracts/dev-ingress-render-test.sh
git commit -m "fix: dev ForwardAuth 소유 경계 수정"
Task 5: Record the environment and CI ownership decisions
Files:
- Create:
docs/adr/0002-dev-environment-and-external-ci-ownership.md - Modify:
docs/architecture/repository-structure.md - Modify:
README.md - Modify:
docs/superpowers/specs/2026-08-02-project-infra-dev-structure-refactor-design.md - Modify:
scripts/ci/validate-docs.sh
Step 1: Extend the docs contract so it fails first
Add assertions to scripts/ci/validate-docs.sh requiring the ADR to contain all of these literals:
Status: Accepted
gitops/clusters/dev/main
cicd-platform
delivery-platform.yaml
all/ is render/audit-only
Run:
bash scripts/ci/validate-docs.sh
Expected: non-zero because ADR 0002 does not exist.
Step 2: Write the ADR
Use sections Context, Decision, Consequences, and Phase 2 follow-up. Record:
- current rollout environment is
dev, notlab; - dev is controller-less and uses ordered bootstrap stages;
all/ is render/audit-onlyand is never an apply target;- project-infra owns local validation contracts;
cicd-platformowns workflow execution, authoritative tools, evidence, and the futuredelivery-platform.yamlconsumer onboarding;- existing workflows remain transitional and are not changed in Phase 1.
Verify that the approved design document remains marked Status: approved.
Step 3: Verify documentation
Run:
bash scripts/ci/validate-docs.sh
git diff --check
Expected: both commands exit 0.
Step 4: Commit
git add docs/adr docs/architecture/repository-structure.md README.md docs/superpowers/specs scripts/ci/validate-docs.sh
git commit -m "docs: dev 및 CI 소유권 결정 기록"
Task 6: Plan-level verification checkpoint
Files: No source changes expected.
Step 1: Prove no live mutation was introduced by this plan
Run:
rg -n 'kubectl[[:space:]]+(apply|delete|replace|patch)|helm[[:space:]]+(install|upgrade|uninstall)' tests scripts/ci scripts/lib
Expected: no new invocation in files created or modified by this plan. Existing task/bootstrap matches are handled by the next plan.
Step 2: Run the complete plan contract
Run:
bash tests/run.sh
VALIDATION_PROFILE=full bash scripts/ci/validate.sh
bash scripts/ci/validate-docs.sh
git diff --check
git status --short
Expected:
- tests and validation exit 0;
- every inventory entrypoint renders with Helm enabled;
- no active
labcluster or overlay path remains; git diff --checkemits nothing;- status contains no unrelated files.
Step 3: Inspect commits
git log --oneline --decorate -7
Confirm that each task is independently reviewable. Do not squash before the user reviews the phase.