init: 폴더구조 설계 및 인프라 설계

This commit is contained in:
DongHyeonka
2026-07-24 14:31:53 +09:00
parent 34ad612281
commit f9c463f87a
1839 changed files with 323096 additions and 1 deletions
+26
View File
@@ -0,0 +1,26 @@
# k8s AGENTS
Role:
- own Kubernetes/K3s source-of-truth manifests and Kustomize composition
- keep base resources, environment overlays, helper scripts, and Vault Secrets Operator assets separate
- make render / validate / diff / apply possible from Git without relying on live cluster state
Scope:
- `base/`: environment-neutral bases
- `overlays/<env>/`: environment-specific composition
- `scripts/`: helper automation
- `vso/`: Vault Secrets Operator assets
Read first:
- `/docs/standards/infra/architecture-environments.md`
- `/docs/standards/infra/kustomize.md`
- `/docs/standards/infra/k3s-specific.md`
- `/docs/standards/infra/operations-runbook-upgrade-rollback.md`
Rules:
- prefer `kubectl kustomize`, `kubectl diff -k`, and `kubectl apply -k`
- do not use server-local manifests as source of truth
- do not place environment differences in `base/`
- do not place base resource definitions directly in overlays unless the resource is environment-only by design
- do not commit production secret values
- keep scripts as helpers; manifests remain declarative source
+58
View File
@@ -0,0 +1,58 @@
# Kubernetes Packaging
This tree is organized for many independently owned workloads.
Entry points:
- `overlays/<env>/` renders the whole environment skeleton.
- `overlays/<env>/<domain>/` renders one namespace/domain boundary.
- `overlays/<env>/managing/vault-secrets-operator/` is applied after the VSO CRDs are installed.
- `scripts/env/` contains environment bootstrap and teardown orchestration.
- `scripts/` contains imperative bootstrap and break-glass operations only.
- `vso/` contains Vault Secrets Operator installation lifecycle files.
Scale rules:
- Do not place hundreds of workloads directly under one package.
- Add workloads under `base/app/units/<unit>/<domain>/<workload-kind>/<workload>/`.
- Add matching environment overrides under `overlays/<env>/app/units/<unit>/<domain>/<workload-kind>/<workload>/` only when that workload has environment-specific differences.
- A workload package owns its Kubernetes object files together, such as `deployment.yaml`, `service.yaml`, `configmap.yaml`, `cronjob.yaml`, `statefulset.yaml`, or `pvc.yaml`.
- Do not create kind-based package roots such as `deployments/`, `services/`, or `configmaps/`.
Example for a 1000-workload organization:
```text
base/app/units/
├── commerce/
│ ├── checkout/
│ │ ├── services/order-api/
│ │ ├── services/payment-api/
│ │ ├── workers/payment-settlement-worker/
│ │ ├── schedulers/cart-expiry-scheduler/
│ │ └── stateful/orders-postgres/
│ └── catalog/
│ ├── services/catalog-api/
│ ├── workers/search-index-worker/
│ └── jobs/catalog-backfill-job/
├── identity/
│ ├── auth/
│ │ ├── services/auth-api/
│ │ ├── services/token-api/
│ │ └── schedulers/token-cleanup-scheduler/
│ └── profile/
│ ├── services/profile-api/
│ └── workers/profile-event-worker/
├── media/
│ ├── playback/
│ │ ├── services/playback-api/
│ │ └── workers/session-event-worker/
│ └── recommendation/
│ ├── services/recommendation-api/
│ ├── workers/model-feature-worker/
│ └── jobs/model-refresh-job/
└── data/
├── ingestion/
│ ├── workers/event-ingest-worker/
│ └── stateful/ingest-kafka/
└── analytics/
├── schedulers/daily-report-scheduler/
└── jobs/monthly-rollup-job/
```
+38
View File
@@ -0,0 +1,38 @@
# k8s/base AGENTS
Role:
- own environment-neutral Kustomize base resources
- define reusable workload, namespace, service, storage, policy, and platform/plugin shapes
- keep environment-specific values out of base
Scope:
- `app/`: application-facing base units
- `managing/`: management and operational base units
- `plugins/`: platform/plugin base resources
Allowed:
- shared labels/selectors
- common workload shape
- common probe/resource shape
- common service/storage/policy shape
- unit composition through nested `kustomization.yaml`
Forbidden:
- environment-specific hostnames
- production-only replicas/resources
- environment-specific secret values
- direct references to a specific cluster context
- overlay-only patches masquerading as base manifests
Read first:
- `/docs/standards/infra/kustomize.md`
- `/docs/standards/infra/architecture-environments.md`
- `/docs/standards/infra/workload-selection.md`
- `/docs/standards/infra/config-and-secrets.md`
- `/docs/standards/infra/security-hardening.md`
Rules:
- base must be reusable by dev, staging, and prod overlays
- base may define default shape, but overlays own environment differences
- unit ownership should be visible in path names
- large fleets should stay navigable by role first, then domain/unit
+13
View File
@@ -0,0 +1,13 @@
# Base Packages
Base packages contain shared Kubernetes definitions.
Boundaries:
- `managing/` owns infrastructure management resources in the `mnt` namespace.
- `app/` owns application namespace resources and large-scale workload package contracts.
- `plugins/` owns plugin namespace resources.
Rules:
- Base packages must not contain environment-specific values.
- A base package can be rendered with `kubectl kustomize` through its own `kustomization.yaml`.
- Parent packages compose child packages; child packages own their internal Kubernetes object files.
+69
View File
@@ -0,0 +1,69 @@
# k8s/base/app AGENTS
Role:
- own application-facing infrastructure base units
- define environment-neutral app workload, service, config reference, secret reference, storage, network policy, probe, and resource shapes
- organize many services by domain/service path directly under `k8s/base/app/<domain>/`
Current structure:
- domains sit directly under `k8s/base/app/` (`identity/`, `storage/`, `test/`)
- inside each domain, separate by workload role (`stateful/`, `stateless/`)
- example: `identity/auth/stateful/identity-postgres/`, `identity/auth/stateless/auth-server/`
Allowed:
- Kustomize base edits for application units
- Deployment / StatefulSet / Job / CronJob base resources
- Service / NetworkPolicy / PDB / HPA base resources
- ConfigMap / Secret reference wiring without secret values
- probe / resource / PVC / StorageClass reference shape
- component-specific manifests aligned with standards
Forbidden:
- environment-specific values that belong in `k8s/overlays/<env>`
- editing packaged K3s component manifests directly
- treating server-local manifests as source of truth
- embedding production secret values in Git
- generating opaque YAML via script as the primary ownership path
- merging unrelated app rollout + migration + cluster upgrade into one hidden change
Read first:
- `/docs/standards/infra/architecture-environments.md`
- `/docs/standards/infra/kustomize.md`
- `/docs/standards/infra/config-and-secrets.md`
- `/docs/standards/infra/workload-selection.md`
- `/docs/standards/infra/storage-pvc.md`
- `/docs/standards/infra/network-ingress-tls.md`
- `/docs/standards/infra/resources-probes-availability.md`
- `/docs/standards/infra/security-hardening.md`
- `/docs/standards/infra/observability-health.md`
- `/docs/standards/infra/db-and-migration.md`
Component-specific routing:
- paths containing `keycloak` -> `/docs/standards/infra/keycloak.md`
- paths containing `vault` -> `/docs/standards/infra/vault.md`
- paths containing `minio` -> `/docs/standards/infra/minio.md`
- paths containing `flyway` or migration jobs -> `/docs/standards/infra/flyway.md`
Examples:
- `/docs/examples/infra/kustomize.md`
- `/docs/examples/infra/config-and-secrets.md`
- `/docs/examples/infra/workload-selection.md`
- `/docs/examples/infra/storage-pvc.md`
- `/docs/examples/infra/network-ingress-tls.md`
- `/docs/examples/infra/resources-probes-availability.md`
- `/docs/examples/infra/security-hardening.md`
- `/docs/examples/infra/observability-health.md`
- `/docs/examples/infra/db-and-migration.md`
- `/docs/examples/infra/keycloak.md`
- `/docs/examples/infra/vault.md`
- `/docs/examples/infra/minio.md`
- `/docs/examples/infra/flyway.md`
Rules:
- base must stay environment-neutral
- app services default to `ClusterIP`
- public exposure must be explicit and added through overlays/ingress policy
- health/metrics/admin endpoints stay non-public by default
- DB migration must stay separate from app startup
- stateful workloads must have explicit storage and restore reasoning
- domain/unit nesting should make ownership clear for large service counts
+23
View File
@@ -0,0 +1,23 @@
# Application Base
This package owns the `app` namespace and the contract for application workloads.
For large scale, workloads are grouped by unit, domain, workload kind, and workload:
```text
base/app/units/<unit>/<domain>/<workload-kind>/<workload>/
```
Examples:
- `base/app/units/commerce/checkout/services/order-api/`
- `base/app/units/commerce/checkout/workers/payment-settlement-worker/`
- `base/app/units/identity/auth/schedulers/token-cleanup-scheduler/`
- `base/app/units/data/storage/stateful/orders-postgres/`
Rules:
- The unit folder is the broad ownership boundary for a business unit, platform area, or organization.
- The domain folder is the bounded context inside a unit.
- The workload-kind folder groups similar operating models inside one domain.
- The workload folder is the smallest independently deployable package.
- Kubernetes object files stay together inside the workload package.
- Shared namespace-level objects go in `shared/`.
@@ -0,0 +1,30 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: identity-postgres-initdb
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
data:
01-create-databases.sh: |
#!/usr/bin/env bash
set -euo pipefail
KEYCLOAK_DB_PASSWORD="$(cat /run/secrets/keycloak-db/password)"
AUTH_SERVER_DB_PASSWORD="$(cat /run/secrets/auth-server-db/SPRING_DATASOURCE_PASSWORD)"
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-'SQL'
CREATE ROLE keycloak LOGIN;
CREATE DATABASE keycloak OWNER keycloak;
CREATE ROLE auth_server LOGIN;
CREATE DATABASE auth_server OWNER auth_server;
SQL
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" \
-c "ALTER ROLE keycloak PASSWORD '$(printf '%s' "$KEYCLOAK_DB_PASSWORD" | sed "s/'/''/g")';"
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" \
-c "ALTER ROLE auth_server PASSWORD '$(printf '%s' "$AUTH_SERVER_DB_PASSWORD" | sed "s/'/''/g")';"
@@ -0,0 +1,18 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- configmap-initdb.yaml
- service-headless.yaml
- service.yaml
- statefulset.yaml
labels:
- pairs:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
@@ -0,0 +1,22 @@
apiVersion: v1
kind: Service
metadata:
name: identity-postgres-headless
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
clusterIP: None
selector:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
ports:
- name: postgres
port: 5432
targetPort: postgres
protocol: TCP
appProtocol: postgresql
@@ -0,0 +1,22 @@
apiVersion: v1
kind: Service
metadata:
name: identity-postgres
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
ports:
- name: postgres
port: 5432
targetPort: postgres
protocol: TCP
appProtocol: postgresql
@@ -0,0 +1,12 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: identity-postgres-sa
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
automountServiceAccountToken: false
@@ -0,0 +1,174 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: identity-postgres
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
replicas: 1
serviceName: identity-postgres-headless
podManagementPolicy: OrderedReady
updateStrategy:
type: RollingUpdate
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
selector:
matchLabels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
template:
metadata:
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
serviceAccountName: identity-postgres-sa
automountServiceAccountToken: false
terminationGracePeriodSeconds: 60
securityContext:
runAsNonRoot: true
runAsUser: 999
runAsGroup: 999
fsGroup: 999
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
- name: postgres
image: postgres:16.4
imagePullPolicy: IfNotPresent
ports:
- name: postgres
containerPort: 5432
protocol: TCP
env:
- name: POSTGRES_DB
value: postgres
- name: POSTGRES_USER_FILE
value: /run/secrets/superuser/username
- name: POSTGRES_PASSWORD_FILE
value: /run/secrets/superuser/password
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: KEYCLOAK_DB_PASSWORD_FILE
value: /run/secrets/keycloak-db/password
- name: AUTH_SERVER_DB_PASSWORD_FILE
value: /run/secrets/auth-server-db/SPRING_DATASOURCE_PASSWORD
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: "1"
memory: 1Gi
startupProbe:
exec:
command:
- sh
- -c
- pg_isready -U "$(cat "$POSTGRES_USER_FILE")"
periodSeconds: 5
failureThreshold: 60
timeoutSeconds: 3
readinessProbe:
exec:
command:
- sh
- -c
- pg_isready -U "$(cat "$POSTGRES_USER_FILE")"
periodSeconds: 10
failureThreshold: 3
timeoutSeconds: 3
livenessProbe:
exec:
command:
- sh
- -c
- pg_isready -U "$(cat "$POSTGRES_USER_FILE")"
periodSeconds: 30
failureThreshold: 3
timeoutSeconds: 3
securityContext:
runAsNonRoot: true
runAsUser: 999
runAsGroup: 999
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
- name: initdb
mountPath: /docker-entrypoint-initdb.d
readOnly: true
- name: superuser
mountPath: /run/secrets/superuser
readOnly: true
- name: keycloak-db
mountPath: /run/secrets/keycloak-db
readOnly: true
- name: auth-server-db
mountPath: /run/secrets/auth-server-db
readOnly: true
- name: tmp
mountPath: /tmp
- name: run
mountPath: /var/run/postgresql
volumes:
- name: initdb
configMap:
name: identity-postgres-initdb
defaultMode: 0755
- name: superuser
secret:
secretName: identity-postgres-superuser
defaultMode: 0400
- name: keycloak-db
secret:
secretName: keycloak-db
defaultMode: 0400
- name: auth-server-db
secret:
secretName: auth-server-db
defaultMode: 0400
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 128Mi
- name: run
emptyDir:
medium: Memory
sizeLimit: 64Mi
volumeClaimTemplates:
- metadata:
name: data
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/version: "16.4"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
accessModes:
- ReadWriteOnce
storageClassName: local-path
volumeMode: Filesystem
resources:
requests:
storage: 5Gi
@@ -0,0 +1,19 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: auth-server-config
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
data:
SPRING_PROFILES_ACTIVE: dev
SPRING_DATASOURCE_URL: jdbc:postgresql://identity-postgres:5432/auth_server
APP_DATASOURCE_URL: jdbc:postgresql://identity-postgres:5432/auth_server
MANAGEMENT_SERVER_ADDRESS: 0.0.0.0
MANAGEMENT_SERVER_PORT: "8081"
MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE: health,prometheus
SERVER_PORT: "8080"
@@ -0,0 +1,128 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-server
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
replicas: 1
revisionHistoryLimit: 5
progressDeadlineSeconds: 600
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 25%
maxUnavailable: 0
selector:
matchLabels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
template:
metadata:
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
serviceAccountName: auth-server-sa
automountServiceAccountToken: false
terminationGracePeriodSeconds: 45
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
- name: auth-server
image: registry.example.com/auth-platform/auth-server:0.1.0
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 8080
protocol: TCP
- name: metrics
containerPort: 8081
protocol: TCP
envFrom:
- configMapRef:
name: auth-server-config
- secretRef:
name: auth-server-db
env:
- name: JAVA_TOOL_OPTIONS
value: "-XX:MaxRAMPercentage=75 -XX:+ExitOnOutOfMemoryError"
- name: SPRING_CONFIG_IMPORT
value: "optional:configtree:/etc/secrets/"
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
memory: 768Mi
startupProbe:
httpGet:
path: /actuator/health/liveness
port: metrics
periodSeconds: 5
failureThreshold: 24
timeoutSeconds: 3
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: metrics
periodSeconds: 5
failureThreshold: 3
timeoutSeconds: 2
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: metrics
periodSeconds: 15
failureThreshold: 3
timeoutSeconds: 3
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
volumeMounts:
- name: db-creds
mountPath: /etc/secrets
readOnly: true
- name: tmp
mountPath: /tmp
volumes:
- name: db-creds
secret:
secretName: auth-server-db
defaultMode: 0400
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 128Mi
@@ -0,0 +1,17 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- configmap.yaml
- deployment.yaml
- service.yaml
labels:
- pairs:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
@@ -0,0 +1,27 @@
apiVersion: v1
kind: Service
metadata:
name: auth-server
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
ports:
- name: http
port: 80
targetPort: http
protocol: TCP
appProtocol: http
- name: metrics
port: 8081
targetPort: metrics
protocol: TCP
appProtocol: http
@@ -0,0 +1,12 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: auth-server-sa
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
automountServiceAccountToken: false
@@ -0,0 +1,113 @@
apiVersion: k8s.keycloak.org/v2beta1
kind: Keycloak
metadata:
name: keycloak
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/version: "26.6.1"
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: keycloak-operator
spec:
instances: 2
db:
vendor: postgres
host: identity-postgres
port: 5432
database: keycloak
usernameSecret:
name: keycloak-db-operator
key: username
passwordSecret:
name: keycloak-db-operator
key: password
poolInitialSize: 5
poolMinSize: 5
poolMaxSize: 20
bootstrapAdmin:
user:
secret: keycloak-bootstrap-admin-operator # 어떤 시크릿에서 읽을지 지정하는 거
hostname:
strict: true # 명시된 hostname을 기준으로 URL을 만든다.
backchannelDynamic: false # 백URL도 프론트 hostname과 동일하게 고정한다.
http:
httpEnabled: true
httpPort: 8080
serviceHttpPort: 80
serviceName: keycloak
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: keycloak-operator
proxy:
headers: xforwarded # X-Forwarded-For X-Forwarded-Proto X-Forwarded-Host X-Forwarded-Port 해당 헤더들을 사용하겠다는 의미
ingress:
enabled: false
networkPolicy:
enabled: false
additionalOptions:
- name: cache
value: ispn # 애플리케이션의 캐시 엔진으로 ispn으로 사용
- name: cache-stack
value: jdbc-ping # 클러스터링된 서버끼리 서로를 찾는(Discovery) 방식을 JDBC-PING으로 정한 것
- name: health-enabled
value: "true" # 상태 확인(Health Check) 엔드포인트를 활성화
- name: metrics-enabled
value: "true" # 메트릭(성능 지표) 수집 기능을 켬
- name: log-console-output
value: json # 로그 형식을 JSON 구조로 변경
env:
- name: JAVA_OPTS_APPEND
value: "-XX:MaxRAMPercentage=70 -XX:InitialRAMPercentage=50 -XX:+ExitOnOutOfMemoryError"
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: "2"
memory: 2Gi
scheduling:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
affinity: # 특정 노드에 다른 pod에 관계에 따라 배치 위치를 결정하는 규칙을 정의한다
podAntiAffinity: # 특정 조건을 만족하는 포드와는 같은 장소에 있지 않는다.
preferredDuringSchedulingIgnoredDuringExecution: # 가급적 지켜줘
- weight: 100 # 가중치가 100임 매우 중요하다고 알림
podAffinityTerm: # 어떤 파드를 피해다닐지
topologyKey: kubernetes.io/hostname # 피할 기준은 호스트 네임으로 설정
labelSelector:
matchLabels:
app.kubernetes.io/instance: keycloak # keycloak과 피하겠다
app.kubernetes.io/managed-by: keycloak-operator # keycloak-operator에 의해 관리되는
unsupported:
podTemplate:
spec:
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
- name: keycloak
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
seccompProfile:
type: RuntimeDefault
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- keycloak.yaml
labels:
- pairs:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/version: "26.6.1"
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- tenant.yaml
labels:
- pairs:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/version: "2025.01.20"
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
@@ -0,0 +1,92 @@
apiVersion: minio.min.io/v2
kind: Tenant
metadata:
name: minio
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/version: "2025.01.20"
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
spec:
image: minio/minio:RELEASE.2025-01-20T14-49-07Z
imagePullPolicy: IfNotPresent
mountPath: /export
configuration:
name: minio-tenant-env
requestAutoCert: true
certConfig:
commonName: minio
organizationName:
- example.com
dnsNames:
- minio
- minio-hl
pools:
- name: pool-0
servers: 4
volumesPerServer: 1
volumeClaimTemplate:
metadata:
name: data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
storageClassName: local-path
resources:
requests:
cpu: 250m
memory: 1Gi
limits:
cpu: "1"
memory: 2Gi
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containerSecurityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
features:
bucketDNS: false
prometheusOperator: false
podManagementPolicy: Parallel
exposeServices:
minio: false
console: false
logging:
anonymous: false
json: true
quiet: false
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: minio-pdb
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/version: "2025.01.20"
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
spec:
maxUnavailable: 1
unhealthyPodEvictionPolicy: AlwaysAllow
selector:
matchLabels:
v1.min.io/tenant: minio
+46
View File
@@ -0,0 +1,46 @@
# k8s/base/managing AGENTS
Role:
- own management and operational Kubernetes base units
- model bootstrap, migration, backup, restore, maintenance, and admin workloads as declarative Kustomize bases
- keep operational workloads separate from long-running application serving workloads
Allowed:
- Job / CronJob base resources for operational tasks
- maintenance ServiceAccount / RBAC / ConfigMap / Secret reference wiring
- backup / restore / migration helper workload shapes
- admin-only service shapes when explicitly justified
Forbidden:
- long-running product application workloads
- environment-specific values that belong in `k8s/overlays/<env>`
- scripts becoming the primary source of YAML truth
- large heredoc-generated manifests as the default path
- embedding production secret values
- hiding environment differences in shell conditionals instead of overlays
- giant all-in-one jobs that mix unrelated concerns
Read first:
- `/docs/standards/infra/workload-selection.md`
- `/docs/standards/infra/db-and-migration.md`
- `/docs/standards/infra/flyway.md`
- `/docs/standards/infra/backup-restore.md`
- `/docs/standards/infra/operations-runbook-upgrade-rollback.md`
- `/docs/standards/infra/config-and-secrets.md`
- `/docs/standards/infra/security-hardening.md`
- `/docs/standards/infra/kustomize.md`
Examples:
- `/docs/examples/infra/flyway.md`
- `/docs/examples/infra/backup-restore.md`
- `/docs/examples/infra/operations-runbook-upgrade-rollback.md`
- `/docs/examples/infra/db-and-migration.md`
- `/docs/examples/infra/kustomize.md`
- `/docs/examples/infra/scripts.md`
Rules:
- management jobs are explicit operational units, not hidden app startup hooks
- migrations must stay separate from app startup
- backup and restore paths must be documented before risky stateful changes
- destructive operations require explicit opt-in and runbook backing
- operational workloads must still follow security, resource, secret, and namespace standards
+5
View File
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace
@@ -0,0 +1,80 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: migration-flyway-sql
labels:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: migration
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
data:
V1__create_users_table.sql: |
create schema if not exists auth;
create or replace function auth.set_updated_at()
returns trigger as $$
begin
new.updated_at := current_timestamp;
return new;
end;
$$ language plpgsql;
create table auth.users (
id uuid not null,
email text not null,
encoded_password text not null,
name text not null,
provider text not null,
created_at timestamp with time zone not null default current_timestamp,
updated_at timestamp with time zone not null default current_timestamp,
constraint pk_users primary key (id),
constraint uq_users__email unique (email),
constraint ck_users__provider check (provider in ('LOCAL', 'GOOGLE', 'GITHUB'))
);
create index ix_users__created_at on auth.users (created_at);
create trigger trg_users__set_updated_at
before update on auth.users
for each row
when (old.* is distinct from new.*)
execute function auth.set_updated_at();
V2__add_oauth_login_columns.sql: |
alter table auth.users alter column encoded_password drop not null;
alter table auth.users add column provider_subject text;
alter table auth.users
add constraint uq_users__provider_provider_subject unique (provider, provider_subject);
V3__add_user_provider_field_constraints.sql: |
alter table auth.users
add constraint ck_users__local_password_required
check (
(provider = 'LOCAL' and encoded_password is not null and provider_subject is null)
or (provider <> 'LOCAL')
);
alter table auth.users
add constraint ck_users__social_subject_required
check (
(provider <> 'LOCAL' and provider_subject is not null and encoded_password is null)
or (provider = 'LOCAL')
);
V4__add_keycloak_provider.sql: |
alter table auth.users drop constraint ck_users__provider;
alter table auth.users
add constraint ck_users__provider check (provider in ('LOCAL', 'KEYCLOAK', 'GOOGLE', 'GITHUB'));
V5__keycloak_only_provider.sql: |
alter table auth.users drop constraint if exists ck_users__local_password_required;
alter table auth.users drop constraint if exists ck_users__social_subject_required;
alter table auth.users drop constraint if exists ck_users__provider;
alter table auth.users drop column if exists encoded_password;
alter table auth.users alter column provider_subject set not null;
alter table auth.users
add constraint ck_users__provider check (provider = 'KEYCLOAK');
+119
View File
@@ -0,0 +1,119 @@
apiVersion: batch/v1
kind: Job
metadata:
name: migration-flyway
labels:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: migration
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
parallelism: 1
completions: 1
backoffLimit: 0
activeDeadlineSeconds: 1800
ttlSecondsAfterFinished: 86400
template:
metadata:
labels:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: migration
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
serviceAccountName: migration-flyway-sa
automountServiceAccountToken: false
restartPolicy: Never
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
initContainers:
- name: flyway-info
image: flyway/flyway:10.20.1
imagePullPolicy: IfNotPresent
command: ["/bin/sh", "-c"]
args: ["FLYWAY_USER=\"$(cat /run/secrets/db/SPRING_DATASOURCE_USERNAME)\" FLYWAY_PASSWORD=\"$(cat /run/secrets/db/SPRING_DATASOURCE_PASSWORD)\" exec /flyway/flyway info"]
env: &flywayEnv
- name: FLYWAY_URL
value: jdbc:postgresql://identity-postgres:5432/auth_server
- name: FLYWAY_LOCATIONS
value: filesystem:/flyway/sql
- name: FLYWAY_SCHEMAS
value: auth
- name: FLYWAY_DEFAULT_SCHEMA
value: auth
- name: FLYWAY_TABLE
value: flyway_schema_history
- name: FLYWAY_VALIDATE_ON_MIGRATE
value: "true"
- name: FLYWAY_BASELINE_ON_MIGRATE
value: "false"
- name: FLYWAY_OUT_OF_ORDER
value: "false"
- name: FLYWAY_MIXED
value: "false"
- name: FLYWAY_CLEAN_DISABLED
value: "true"
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
memory: 256Mi
securityContext: &flywaySC
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
volumeMounts: &flywayVM
- name: sql
mountPath: /flyway/sql
readOnly: true
- name: db-secret
mountPath: /run/secrets/db
readOnly: true
- name: tmp
mountPath: /tmp
containers:
- name: flyway-migrate
image: flyway/flyway:10.20.1
imagePullPolicy: IfNotPresent
command: ["/bin/sh", "-c"]
args: ["FLYWAY_USER=\"$(cat /run/secrets/db/SPRING_DATASOURCE_USERNAME)\" FLYWAY_PASSWORD=\"$(cat /run/secrets/db/SPRING_DATASOURCE_PASSWORD)\" exec /flyway/flyway migrate"]
env: *flywayEnv
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 512Mi
securityContext: *flywaySC
volumeMounts: *flywayVM
volumes:
- name: sql
configMap:
name: migration-flyway-sql
- name: db-secret
secret:
secretName: auth-server-db
defaultMode: 0400
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 128Mi
@@ -0,0 +1,16 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- configmap-sql.yaml
- job.yaml
labels:
- pairs:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: migration
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
@@ -0,0 +1,12 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: migration-flyway-sa
labels:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: migration
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
automountServiceAccountToken: false
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
@@ -0,0 +1,16 @@
apiVersion: v1
kind: Namespace
metadata:
name: mnt
labels:
app.kubernetes.io/name: mnt
app.kubernetes.io/instance: mnt
app.kubernetes.io/component: namespace
app.kubernetes.io/part-of: infra-platform
app.kubernetes.io/managed-by: kustomize
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: latest
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/audit-version: latest
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
+15
View File
@@ -0,0 +1,15 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: docker-registry-config
data:
REGISTRY_HTTP_ADDR: "0.0.0.0:5000"
REGISTRY_LOG_FORMATTER: "json"
REGISTRY_STORAGE: "s3"
REGISTRY_STORAGE_S3_REGION: "us-east-1"
REGISTRY_STORAGE_S3_REGIONENDPOINT: "http://minio"
REGISTRY_STORAGE_S3_BUCKET: "docker-registry"
REGISTRY_STORAGE_S3_FORCEPATHSTYLE: "true"
REGISTRY_STORAGE_S3_SKIPVERIFY: "true"
REGISTRY_STORAGE_REDIRECT_DISABLE: "true"
REGISTRY_STORAGE_DELETE_ENABLED: "true"
@@ -0,0 +1,94 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: docker-registry
spec:
replicas: 1
revisionHistoryLimit: 5
progressDeadlineSeconds: 600
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 25%
maxUnavailable: 0
selector:
matchLabels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
template:
metadata:
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
spec:
serviceAccountName: docker-registry
automountServiceAccountToken: false
terminationGracePeriodSeconds: 45
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
- name: registry
image: registry:2.8.3
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 5000
protocol: TCP
envFrom:
- configMapRef:
name: docker-registry-config
- secretRef:
name: docker-registry-minio
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
memory: 512Mi
startupProbe:
httpGet:
path: /v2/
port: http
periodSeconds: 5
failureThreshold: 12
timeoutSeconds: 3
readinessProbe:
httpGet:
path: /v2/
port: http
periodSeconds: 10
failureThreshold: 3
timeoutSeconds: 3
livenessProbe:
httpGet:
path: /v2/
port: http
periodSeconds: 30
failureThreshold: 3
timeoutSeconds: 3
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 64Mi
+19
View File
@@ -0,0 +1,19 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- configmap.yaml
- service.yaml
- deployment.yaml
labels:
- pairs:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
app.kubernetes.io/version: "2.8.3"
app.kubernetes.io/component: registry
app.kubernetes.io/part-of: platform-registry
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
+15
View File
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: docker-registry
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
ports:
- name: http
port: 5000
targetPort: http
protocol: TCP
appProtocol: http
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: docker-registry
automountServiceAccountToken: false
+6
View File
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- vault
- docker-registry
@@ -0,0 +1,115 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: oauth2-proxy
spec:
replicas: 1
revisionHistoryLimit: 5
progressDeadlineSeconds: 600
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 25%
maxUnavailable: 0
selector:
matchLabels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
template:
metadata:
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
spec:
serviceAccountName: oauth2-proxy-sa
automountServiceAccountToken: false
terminationGracePeriodSeconds: 30
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
securityContext:
runAsNonRoot: true
runAsUser: 65532
runAsGroup: 65532
fsGroup: 65532
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
- name: oauth2-proxy
image: quay.io/oauth2-proxy/oauth2-proxy:v7.15.2
imagePullPolicy: IfNotPresent
args:
- --config=/etc/oauth2-proxy/oauth2-proxy.cfg
ports:
- name: http
containerPort: 4180
protocol: TCP
- name: metrics
containerPort: 44180
protocol: TCP
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
memory: 256Mi
startupProbe:
httpGet:
path: /ping
port: http
periodSeconds: 5
failureThreshold: 12
timeoutSeconds: 3
readinessProbe:
httpGet:
path: /ready
port: http
periodSeconds: 10
failureThreshold: 3
timeoutSeconds: 3
livenessProbe:
httpGet:
path: /ping
port: http
periodSeconds: 15
failureThreshold: 3
timeoutSeconds: 3
securityContext:
runAsNonRoot: true
runAsUser: 65532
runAsGroup: 65532
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
volumeMounts:
- name: config
mountPath: /etc/oauth2-proxy
readOnly: true
- name: secrets
mountPath: /etc/oauth2-proxy-secrets
readOnly: true
- name: tmp
mountPath: /tmp
volumes:
- name: config
configMap:
name: oauth2-proxy-config
- name: secrets
secret:
secretName: oauth2-proxy-secrets
defaultMode: 0400
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 64Mi
@@ -0,0 +1,18 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- service.yaml
- deployment.yaml
labels:
- pairs:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
app.kubernetes.io/version: "7.15.2"
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
@@ -0,0 +1,20 @@
apiVersion: v1
kind: Service
metadata:
name: oauth2-proxy
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
ports:
- name: http
port: 4180
targetPort: http
protocol: TCP
appProtocol: http
- name: metrics
port: 44180
targetPort: metrics
protocol: TCP
appProtocol: http
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: oauth2-proxy-sa
automountServiceAccountToken: false
@@ -0,0 +1,12 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: vault-tokenreview-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
subjects:
- kind: ServiceAccount
name: vault
namespace: mnt
+21
View File
@@ -0,0 +1,21 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: vault-config
data:
vault.hcl: |
ui = true
disable_mlock = true
storage "file" {
path = "/vault/data"
}
listener "tcp" {
address = "0.0.0.0:8200"
cluster_address = "0.0.0.0:8201"
tls_disable = 1
}
api_addr = "http://vault:8200"
cluster_addr = "http://vault:8201"
+20
View File
@@ -0,0 +1,20 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- clusterrolebinding.yaml
- configmap.yaml
- service.yaml
- statefulset.yaml
labels:
- pairs:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: vault
app.kubernetes.io/version: "1.17.2"
app.kubernetes.io/component: secret-management
app.kubernetes.io/part-of: security-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
+20
View File
@@ -0,0 +1,20 @@
apiVersion: v1
kind: Service
metadata:
name: vault
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: vault
ports:
- name: http
port: 8200
targetPort: http
protocol: TCP
appProtocol: http
- name: cluster
port: 8201
targetPort: cluster
protocol: TCP
appProtocol: http
+4
View File
@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault
+128
View File
@@ -0,0 +1,128 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: vault
spec:
replicas: 1
serviceName: vault
podManagementPolicy: OrderedReady
updateStrategy:
type: RollingUpdate
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
selector:
matchLabels:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: vault
template:
metadata:
labels:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: vault
spec:
serviceAccountName: vault
automountServiceAccountToken: true
terminationGracePeriodSeconds: 60
securityContext:
runAsNonRoot: true
runAsUser: 100
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
containers:
- name: vault
image: hashicorp/vault:1.17.2
imagePullPolicy: IfNotPresent
command:
- vault
- server
- -config=/vault/config/vault.hcl
ports:
- name: http
containerPort: 8200
protocol: TCP
- name: cluster
containerPort: 8201
protocol: TCP
env:
- name: VAULT_ADDR
value: "http://127.0.0.1:8200"
- name: VAULT_API_ADDR
value: "http://vault:8200"
- name: SKIP_SETCAP
value: "true"
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
memory: 512Mi
startupProbe:
httpGet:
path: /v1/sys/health?standbyok=true&sealedcode=200&uninitcode=200
port: http
periodSeconds: 5
failureThreshold: 30
timeoutSeconds: 3
readinessProbe:
httpGet:
path: /v1/sys/health?standbyok=true&sealedcode=503&uninitcode=503
port: http
periodSeconds: 10
failureThreshold: 3
timeoutSeconds: 3
livenessProbe:
httpGet:
path: /v1/sys/health?standbyok=true&sealedcode=200&uninitcode=200
port: http
periodSeconds: 30
failureThreshold: 3
timeoutSeconds: 3
securityContext:
runAsNonRoot: true
runAsUser: 100
runAsGroup: 1000
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
volumeMounts:
- name: data
mountPath: /vault/data
- name: config
mountPath: /vault/config
readOnly: true
- name: tmp
mountPath: /tmp
- name: home
mountPath: /home/vault
volumes:
- name: config
configMap:
name: vault-config
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 64Mi
- name: home
emptyDir:
medium: Memory
sizeLimit: 64Mi
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes:
- ReadWriteOnce
storageClassName: local-path
volumeMode: Filesystem
resources:
requests:
storage: 5Gi
+39
View File
@@ -0,0 +1,39 @@
defaultVaultConnection:
enabled: false
controller:
# VSO 는 mnt namespace 에 설치되고 mnt 는 PSS restricted enforce 상태이므로
# chart 의 Operator Pod 도 Restricted 프로필을 만족해야 한다.
podSecurityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
manager:
resources:
limits:
cpu: 500m
memory: 128Mi
requests:
cpu: 10m
memory: 64Mi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
kubeRbacProxy:
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
# kubeRBACProxy : metrics 접근 제어
+18
View File
@@ -0,0 +1,18 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- vault-connection.yaml
- vault-auth.yaml
labels:
- pairs:
app.kubernetes.io/name: vault-secrets-operator
app.kubernetes.io/instance: vso
app.kubernetes.io/version: "0.9.0"
app.kubernetes.io/component: secret-delivery
app.kubernetes.io/part-of: security-platform
app.kubernetes.io/managed-by: kustomize
includeSelectors: false
includeTemplates: true
+5
View File
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault-secrets-operator
automountServiceAccountToken: true
+27
View File
@@ -0,0 +1,27 @@
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: vault-auth-auth-platform
spec:
vaultConnectionRef: vault-connection
method: kubernetes
mount: kubernetes
kubernetes:
role: vso-auth-platform
serviceAccount: vault-secrets-operator
audiences:
- vault
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: vault-auth-storage
spec:
vaultConnectionRef: vault-connection
method: kubernetes
mount: kubernetes
kubernetes:
role: vso-storage
serviceAccount: vault-secrets-operator
audiences:
- vault
+7
View File
@@ -0,0 +1,7 @@
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultConnection
metadata:
name: vault-connection
spec:
address: http://vault.mnt.svc.cluster.local:8200
skipTLSVerify: false
@@ -0,0 +1,19 @@
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component
resources:
- ../../base/plugins/oauth2-proxy
- oauth2-proxy-config.yaml
- oauth2-proxy-vault-secrets.yaml
- oauth2-proxy-ingress.yaml
- oauth2-proxy-middleware.yaml
- oauth2-proxy-networkpolicy.yaml
patches:
- target:
kind: Ingress
name: auth-server
patch: |-
- op: replace
path: /metadata/annotations/traefik.ingress.kubernetes.io~1router.middlewares
value: mnt-oauth2-proxy-errors@kubernetescrd,mnt-oauth2-proxy-auth@kubernetescrd,kube-system-security-headers@kubernetescrd
@@ -0,0 +1,46 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: oauth2-proxy-config
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
app.kubernetes.io/version: "7.15.2"
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
data:
oauth2-proxy.cfg: |
provider = "keycloak-oidc"
http_address = "0.0.0.0:4180"
metrics_address = "0.0.0.0:44180"
redirect_url = "http://auth.local.test/oauth2/callback"
oidc_issuer_url = "http://keycloak.local.test/realms/platform"
skip_oidc_discovery = true
login_url = "http://keycloak.local.test/realms/platform/protocol/openid-connect/auth"
redeem_url = "http://keycloak.mnt.svc.cluster.local/realms/platform/protocol/openid-connect/token"
profile_url = "http://keycloak.mnt.svc.cluster.local/realms/platform/protocol/openid-connect/userinfo"
validate_url = "http://keycloak.mnt.svc.cluster.local/realms/platform/protocol/openid-connect/userinfo"
oidc_jwks_url = "http://keycloak.mnt.svc.cluster.local/realms/platform/protocol/openid-connect/certs"
client_id = "auth-server-ingress"
client_secret_file = "/etc/oauth2-proxy-secrets/client-secret"
cookie_secret_file = "/etc/oauth2-proxy-secrets/cookie-secret"
reverse_proxy = true
upstreams = [ "static://202" ]
email_domains = [ "*" ]
scope = "openid profile email"
insecure_oidc_allow_unverified_email = true
skip_provider_button = true
cookie_secure = false
cookie_samesite = "lax"
cookie_csrf_per_request = true
cookie_refresh = "4m"
set_xauthrequest = true
set_authorization_header = true
pass_access_token = true
pass_authorization_header = true
ssl_insecure_skip_verify = false
whitelist_domains = [ "auth.local.test" ]
trusted_proxy_ips = [ "10.42.0.0/16" ]
@@ -0,0 +1,27 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: oauth2-proxy
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
app.kubernetes.io/version: "7.15.2"
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
traefik.ingress.kubernetes.io/router.middlewares: kube-system-security-headers@kubernetescrd
spec:
ingressClassName: traefik
rules:
- host: auth.local.test
http:
paths:
- path: /oauth2/
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
name: http
@@ -0,0 +1,45 @@
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: oauth2-proxy-auth
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
app.kubernetes.io/version: "7.15.2"
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
forwardAuth:
address: http://oauth2-proxy.mnt.svc.cluster.local:4180/oauth2/auth
trustForwardHeader: true
authResponseHeaders:
- Authorization
- X-Auth-Request-Access-Token
- X-Auth-Request-Email
- X-Auth-Request-Preferred-Username
- X-Auth-Request-User
- X-Forwarded-Email
- X-Forwarded-User
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: oauth2-proxy-errors
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
app.kubernetes.io/version: "7.15.2"
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
errors:
status:
- "401"
statusRewrites:
"401": 302
service:
name: oauth2-proxy
port: 4180
query: /oauth2/start?rd={url}
@@ -0,0 +1,55 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: oauth2-proxy-ingress-traefik
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 4180
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: oauth2-proxy-egress-keycloak
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
ports:
- protocol: TCP
port: 8080
@@ -0,0 +1,24 @@
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: oauth2-proxy-secrets
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
path: oauth2-proxy/forward-auth
refreshAfter: 60s
type: kv-v2
destination:
name: oauth2-proxy-secrets
create: true
labels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/component: auth-proxy
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
excludeRaw: true
templates:
client-secret:
text: '{{ if index .Secrets "client_secret" }}{{ index .Secrets "client_secret" }}{{ else }}{{ index .Secrets "client-secret" }}{{ end }}'
cookie-secret:
text: '{{ if index .Secrets "cookie_secret" }}{{ index .Secrets "cookie_secret" }}{{ else }}{{ index .Secrets "cookie-secret" }}{{ end }}'
+44
View File
@@ -0,0 +1,44 @@
# k8s/overlays AGENTS
Role:
- own environment-specific Kustomize composition
- express only the differences between environments and reusable bases
- make environment-level render, diff, apply, audit, and GitOps sync straightforward
Scope:
- `dev/`
- `staging/`
- `prod/`
Allowed:
- namespace selection
- replicas
- resource requests/limits overrides
- image tags or image references
- ingress host/TLS differences
- environment-specific patches
- environment-specific secret references
- storage class and retention differences
Forbidden:
- copying full base manifests into overlays
- redefining base resources wholesale without a clear environment-only reason
- placing production secret values in Git
- changing service ownership or workload kind without updating the base and standards
- mixing multiple environments in one overlay
Read first:
- `/docs/standards/infra/kustomize.md`
- `/docs/standards/infra/architecture-environments.md`
- `/docs/standards/infra/config-and-secrets.md`
- `/docs/standards/infra/network-ingress-tls.md`
- `/docs/standards/infra/resources-probes-availability.md`
- `/docs/standards/infra/security-hardening.md`
- `/docs/standards/infra/operations-runbook-upgrade-rollback.md`
Rules:
- overlays are environment-first by design
- keep patches small and named by target and intent
- render and diff the target environment before apply
- prod overlays must be the most conservative environment
- if an overlay starts re-declaring most of a resource, move common shape back into `k8s/base`
+19
View File
@@ -0,0 +1,19 @@
# Environment Overlays
Overlays contain environment-specific differences.
Required shape:
```text
overlays/<env>/
├── kustomization.yaml
├── app/
├── managing/
└── plugins/
```
Rules:
- Overlay packages reuse base packages.
- Overlay packages contain patches, generated config, or environment-specific resources only.
- Do not duplicate base package composition in overlays.
- A workload override belongs at `overlays/<env>/app/units/<unit>/<domain>/<workload-kind>/<workload>/`.
+27
View File
@@ -0,0 +1,27 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: auth-server
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/version: "0.1.0"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
traefik.ingress.kubernetes.io/router.middlewares: kube-system-security-headers@kubernetescrd
spec:
ingressClassName: traefik
rules:
- host: auth.local.test
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: auth-server
port:
name: http
+50
View File
@@ -0,0 +1,50 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../../base/app/identity/auth/stateless/auth-server
- ../../../base/managing/migration-flyway
- ingress.yaml
- networkpolicy.yaml
images:
- name: registry.example.com/auth-platform/auth-server
newName: registry.project.com/auth-platform/auth-server
newTag: manual-20260512071751
patches:
- target:
kind: ConfigMap
name: auth-server-config
patch: |-
- op: add
path: /data/APP_SECURITY_KEYCLOAK_ISSUER_URI
value: http://keycloak.local.test/realms/platform
- op: add
path: /data/SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUER_URI
value: http://keycloak.local.test/realms/platform
- op: add
path: /data/SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWK_SET_URI
value: http://keycloak/realms/platform/protocol/openid-connect/certs
- op: add
path: /data/SERVER_MAX_HTTP_REQUEST_HEADER_SIZE
value: 64KB
- target:
kind: ServiceAccount
name: auth-server-sa
patch: |-
- op: add
path: /imagePullSecrets
value:
- name: docker-registry-pull-credentials
- target:
kind: Job
name: migration-flyway
patch: |-
- op: add
path: /metadata/annotations
value:
argocd.argoproj.io/sync-wave: "-1"
argocd.argoproj.io/hook: PreSync
+90
View File
@@ -0,0 +1,90 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-server-ingress-traefik
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-server-egress
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
app.kubernetes.io/component: api
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
ports:
- protocol: TCP
port: 5432
- to:
- podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
ports:
- protocol: TCP
port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: migration-flyway-egress
labels:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
app.kubernetes.io/component: migration
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
ports:
- protocol: TCP
port: 5432
@@ -0,0 +1,21 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../../base/app/identity/auth/stateful/identity-postgres
- vault-secrets.yaml
- networkpolicy.yaml
patches:
- target:
kind: StatefulSet
name: identity-postgres
patch: |-
- op: replace
path: /spec/persistentVolumeClaimRetentionPolicy/whenDeleted
value: Delete
- op: replace
path: /spec/persistentVolumeClaimRetentionPolicy/whenScaled
value: Delete
@@ -0,0 +1,34 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: identity-postgres-ingress-clients
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
app.kubernetes.io/component: database
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
- podSelector:
matchLabels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
- podSelector:
matchLabels:
app.kubernetes.io/name: migration-flyway
app.kubernetes.io/instance: migration-flyway
ports:
- protocol: TCP
port: 5432
@@ -0,0 +1,63 @@
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: identity-postgres-superuser
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
type: kv-v2
path: identity-postgres/superuser
refreshAfter: 60s
destination:
name: identity-postgres-superuser
create: true
labels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/component: database
app.kubernetes.io/managed-by: vault-secrets-operator
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: keycloak-db-creds
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
type: kv-v2
path: keycloak/db
refreshAfter: 60s
destination:
name: keycloak-db
create: true
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/component: database
app.kubernetes.io/managed-by: vault-secrets-operator
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: auth-server-db-creds
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
type: kv-v2
path: auth-server/db
refreshAfter: 60s
destination:
name: auth-server-db
create: true
labels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/component: database
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
templates:
SPRING_DATASOURCE_USERNAME:
text: '{{ index .Secrets "SPRING_DATASOURCE_USERNAME" }}'
SPRING_DATASOURCE_PASSWORD:
text: '{{ index .Secrets "SPRING_DATASOURCE_PASSWORD" }}'
APP_DATASOURCE_USERNAME:
text: '{{ index .Secrets "SPRING_DATASOURCE_USERNAME" }}'
APP_DATASOURCE_PASSWORD:
text: '{{ index .Secrets "SPRING_DATASOURCE_PASSWORD" }}'
@@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- platform-realm-import.yaml
@@ -0,0 +1,62 @@
apiVersion: k8s.keycloak.org/v2beta1
kind: KeycloakRealmImport
metadata:
name: platform-realm
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/version: "26.6.1"
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
keycloakCRName: keycloak
placeholders:
AUTH_SERVER_INGRESS_CLIENT_SECRET:
secret:
name: keycloak-client-auth-server-ingress
key: client-secret
realm:
realm: platform
enabled: true
sslRequired: external
registrationAllowed: true
loginWithEmailAllowed: true
accessTokenLifespan: 300
defaultRoles:
- user
roles:
realm:
- name: user
clients:
- clientId: auth-server-ingress
name: auth-server-ingress
protocol: openid-connect
publicClient: false
clientAuthenticatorType: client-secret
secret: ${AUTH_SERVER_INGRESS_CLIENT_SECRET}
standardFlowEnabled: true
directAccessGrantsEnabled: false
serviceAccountsEnabled: false
defaultClientScopes:
- profile
- email
- roles
protocolMappers:
- name: realm roles in id token
protocol: openid-connect
protocolMapper: oidc-usermodel-realm-role-mapper
config:
user.attribute: foo
claim.name: realm_access.roles
jsonType.label: String
multivalued: "true"
access.token.claim: "true"
id.token.claim: "true"
introspection.token.claim: "true"
redirectUris:
- http://auth.local.test/oauth2/callback
webOrigins:
- http://auth.local.test
rootUrl: http://auth.local.test
baseUrl: http://auth.local.test
@@ -0,0 +1,27 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: keycloak-admin
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/version: "26.6.1"
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
traefik.ingress.kubernetes.io/router.middlewares: mnt-keycloak-security-headers@kubernetescrd
spec:
ingressClassName: traefik
rules:
- host: keycloak-admin.local.test
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: keycloak
port:
name: http
@@ -0,0 +1,48 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: keycloak-public
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/version: "26.6.1"
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
traefik.ingress.kubernetes.io/router.middlewares: mnt-keycloak-security-headers@kubernetescrd # frameDeny 제외 — admin 콘솔의 silent SSO iframe 허용
spec:
ingressClassName: traefik
rules:
- host: keycloak.local.test
http:
paths:
- path: /realms/
pathType: Prefix
backend:
service:
name: keycloak
port:
name: http
- path: /resources/
pathType: Prefix
backend:
service:
name: keycloak
port:
name: http
- path: /.well-known/
pathType: Prefix
backend:
service:
name: keycloak
port:
name: http
- path: /js/
pathType: Prefix
backend:
service:
name: keycloak
port:
name: http
@@ -0,0 +1,8 @@
apiVersion: k8s.keycloak.org/v2beta1
kind: Keycloak
metadata:
name: keycloak
spec:
hostname:
hostname: http://keycloak.local.test
admin: http://keycloak-admin.local.test
@@ -0,0 +1,18 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../../base/app/identity/keycloak/stateless/keycloak
- vault-secrets.yaml
- middleware.yaml
- ingress-public.yaml
- ingress-admin.yaml
- networkpolicy.yaml
patches:
- target:
kind: Keycloak
name: keycloak
path: keycloak-hostname-patch.yaml
+15
View File
@@ -0,0 +1,15 @@
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: keycloak-security-headers
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
headers:
browserXssFilter: true
contentTypeNosniff: true
referrerPolicy: strict-origin-when-cross-origin
@@ -0,0 +1,140 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: keycloak-peer
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: keycloak-ingress-traefik
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: keycloak-egress-postgres
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/name: identity-postgres
app.kubernetes.io/instance: identity-postgres
ports:
- protocol: TCP
port: 5432
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: keycloak-ingress-auth-server
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app.kubernetes.io/name: auth-server
app.kubernetes.io/instance: auth-server
ports:
- protocol: TCP
port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: keycloak-ingress-oauth2-proxy
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/instance: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/instance: keycloak
app.kubernetes.io/managed-by: keycloak-operator
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app.kubernetes.io/name: oauth2-proxy
app.kubernetes.io/instance: oauth2-proxy
ports:
- protocol: TCP
port: 8080
@@ -0,0 +1,90 @@
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: keycloak-bootstrap-admin
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
type: kv-v2
path: keycloak/bootstrap-admin
refreshAfter: 60s
destination:
name: keycloak-bootstrap-admin
create: true
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/managed-by: vault-secrets-operator
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: keycloak-bootstrap-admin-operator
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
type: kv-v2
path: keycloak/bootstrap-admin
refreshAfter: 60s
destination:
name: keycloak-bootstrap-admin-operator
create: true
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
excludeRaw: true
templates:
username:
text: '{{ .Secrets.KEYCLOAK_ADMIN }}'
password:
text: '{{ .Secrets.KEYCLOAK_ADMIN_PASSWORD }}'
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: keycloak-db-operator
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
type: kv-v2
path: keycloak/db
refreshAfter: 60s
destination:
name: keycloak-db-operator
create: true
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/component: database
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
excludeRaw: true
templates:
username:
text: keycloak
password:
text: '{{ .Secrets.password }}'
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: keycloak-client-auth-server-ingress
spec:
vaultAuthRef: vault-auth-auth-platform
mount: secret
type: kv-v2
path: keycloak/clients/auth-server-ingress
refreshAfter: 60s
destination:
name: keycloak-client-auth-server-ingress
create: true
labels:
app.kubernetes.io/name: keycloak
app.kubernetes.io/component: identity-provider
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
excludeRaw: true
templates:
client-secret:
text: '{{ .Secrets.client_secret }}'
+18
View File
@@ -0,0 +1,18 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../base/managing
- networkpolicy-baseline.yaml
- vault
- registry
- database
- auth
- keycloak
- storage
- keycloak-realm
components:
- ../../components/forward-auth
@@ -0,0 +1,43 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
labels:
app.kubernetes.io/name: namespace-baseline
app.kubernetes.io/instance: mnt
app.kubernetes.io/component: network-policy
app.kubernetes.io/part-of: infra-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns-egress
labels:
app.kubernetes.io/name: namespace-baseline
app.kubernetes.io/instance: mnt
app.kubernetes.io/component: network-policy
app.kubernetes.io/part-of: infra-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- https://github.com/cert-manager/cert-manager/releases/download/v1.20.2/cert-manager.yaml
@@ -0,0 +1,39 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.6.1/kubernetes/keycloaks.k8s.keycloak.org-v1.yml
- https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.6.1/kubernetes/keycloakrealmimports.k8s.keycloak.org-v1.yml
- https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.6.1/kubernetes/kubernetes.yml
- networkpolicy.yaml
patches:
- target:
kind: ClusterRoleBinding
name: keycloak-operator-clusterrole-binding
patch: |-
- op: replace
path: /subjects/0/namespace
value: mnt
- target:
kind: Deployment
name: keycloak-operator
patch: |-
- op: add
path: /spec/template/spec/securityContext
value:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
- op: add
path: /spec/template/spec/containers/0/securityContext
value:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
- op: replace
path: /spec/template/spec/containers/0/startupProbe/failureThreshold
value: 18
@@ -0,0 +1,29 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: keycloak-operator-egress-kubernetes-api
labels:
app.kubernetes.io/name: keycloak-operator
app.kubernetes.io/instance: keycloak-operator
app.kubernetes.io/component: identity-operator
app.kubernetes.io/part-of: auth-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: keycloak-operator
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 10.43.0.0/16
ports:
- protocol: TCP
port: 443
- to:
- ipBlock:
cidr: 10.208.141.0/24
ports:
- protocol: TCP
port: 6443
@@ -0,0 +1,21 @@
apiVersion: v1
kind: Service
metadata:
name: traefik-dashboard
labels:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: traefik-dev
app.kubernetes.io/component: ingress-controller
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: kustomize
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: traefik-kube-system
ports:
- name: admin
port: 8080
targetPort: traefik
protocol: TCP
appProtocol: http
@@ -0,0 +1,44 @@
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
labels:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: traefik-dev
app.kubernetes.io/component: ingress-controller
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: kustomize
spec:
valuesContent: |-
api:
dashboard: true
deployment:
replicas: 2
ingressRoute:
dashboard:
enabled: true
entryPoints:
- traefik
service:
spec:
externalTrafficPolicy: Local
ports:
web:
tls:
enabled: false
websecure:
tls:
enabled: false
ingressClass:
enabled: true
isDefaultClass: true
additionalArguments:
- "--providers.kubernetesingress.ingressclass=traefik"
- "--metrics.prometheus=true"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 256Mi
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kube-system
resources:
- dashboard-service.yaml
- helmchartconfig.yaml
- middleware.yaml
@@ -0,0 +1,16 @@
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: security-headers
labels:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: traefik-dev
app.kubernetes.io/component: ingress-controller
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: kustomize
spec:
headers:
contentTypeNosniff: true
browserXssFilter: true
referrerPolicy: strict-origin-when-cross-origin
frameDeny: true
@@ -0,0 +1,27 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: docker-registry-public
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
app.kubernetes.io/version: "2.8.3"
app.kubernetes.io/component: registry
app.kubernetes.io/part-of: platform-registry
app.kubernetes.io/managed-by: kustomize
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
traefik.ingress.kubernetes.io/router.middlewares: kube-system-security-headers@kubernetescrd,mnt-docker-registry-basic-auth@kubernetescrd
spec:
ingressClassName: traefik
rules:
- host: registry.project.com
http:
paths:
- path: /v2
pathType: Prefix
backend:
service:
name: docker-registry
port:
name: http
@@ -0,0 +1,11 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../../base/plugins/docker-registry
- vault-secrets.yaml
- middleware.yaml
- ingress-public.yaml
- networkpolicy.yaml
+14
View File
@@ -0,0 +1,14 @@
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: docker-registry-basic-auth
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
app.kubernetes.io/component: registry
app.kubernetes.io/part-of: platform-registry
app.kubernetes.io/managed-by: kustomize
spec:
basicAuth:
secret: docker-registry-basic-auth
realm: docker-registry
@@ -0,0 +1,82 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: docker-registry-ingress-intra
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
app.kubernetes.io/component: registry
app.kubernetes.io/part-of: platform-registry
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
policyTypes:
- Ingress
ingress:
- from:
- podSelector: {}
ports:
- protocol: TCP
port: 5000
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: docker-registry-ingress-traefik
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
app.kubernetes.io/component: registry
app.kubernetes.io/part-of: platform-registry
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 5000
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: docker-registry-egress-minio
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
app.kubernetes.io/component: registry
app.kubernetes.io/part-of: platform-registry
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/instance: docker-registry
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
v1.min.io/tenant: minio
ports:
- protocol: TCP
port: 80
- protocol: TCP
port: 443
- protocol: TCP
port: 9000
@@ -0,0 +1,83 @@
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: docker-registry-minio
spec:
vaultAuthRef: vault-auth-storage
mount: secret
type: kv-v2
path: docker-registry/minio
refreshAfter: 60s
destination:
name: docker-registry-minio
create: true
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/component: registry
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
excludeRaw: true
templates:
REGISTRY_STORAGE_S3_ACCESSKEY:
text: '{{ .Secrets.access_key }}'
REGISTRY_STORAGE_S3_SECRETKEY:
text: '{{ .Secrets.secret_key }}'
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: docker-registry-basic-auth
spec:
vaultAuthRef: vault-auth-storage
mount: secret
type: kv-v2
path: docker-registry/basic-auth
refreshAfter: 60s
destination:
name: docker-registry-basic-auth
create: true
overwrite: true
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/component: registry
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
excludeRaw: true
excludes:
- .*
templates:
users:
text: '{{ .Secrets.users }}'
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: docker-registry-pull-credentials
spec:
vaultAuthRef: vault-auth-storage
mount: secret
type: kv-v2
path: docker-registry/basic-auth
refreshAfter: 60s
destination:
name: docker-registry-pull-credentials
create: true
type: kubernetes.io/dockerconfigjson
labels:
app.kubernetes.io/name: docker-registry
app.kubernetes.io/component: registry
app.kubernetes.io/managed-by: vault-secrets-operator
transformation:
excludeRaw: true
templates:
.dockerconfigjson:
text: |
{
"auths": {
"registry.project.com": {
"username": "{{ .Secrets.username }}",
"password": "{{ .Secrets.password }}",
"auth": "{{ printf "%s:%s" .Secrets.username .Secrets.password | b64enc }}"
}
}
}
@@ -0,0 +1,27 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../../base/app/storage/minio/stateful/minio
- vault-secrets.yaml
- networkpolicy.yaml
patches:
- target:
group: minio.min.io
version: v2
kind: Tenant
name: minio
patch: |-
- op: replace
path: /spec/certConfig/dnsNames
value:
- minio
- minio-hl
- minio.mnt.svc.cluster.local
- "*.minio-hl.mnt.svc.cluster.local"
- op: replace
path: /spec/requestAutoCert
value: false
+140
View File
@@ -0,0 +1,140 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: minio-ingress-peer
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
v1.min.io/tenant: minio
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
v1.min.io/tenant: minio
ports:
- protocol: TCP
port: 9000
- protocol: TCP
port: 9001
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: minio-ingress-operator
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
v1.min.io/tenant: minio
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: minio-operator
podSelector:
matchLabels:
app.kubernetes.io/instance: minio-operator
app.kubernetes.io/name: operator
ports:
- protocol: TCP
port: 9000
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: minio-ingress-clients
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
v1.min.io/tenant: minio
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app.kubernetes.io/part-of: auth-platform
- podSelector:
matchLabels:
app.kubernetes.io/part-of: platform-registry
ports:
- protocol: TCP
port: 9000
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: minio-egress-peer
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
v1.min.io/tenant: minio
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
v1.min.io/tenant: minio
ports:
- protocol: TCP
port: 9000
- protocol: TCP
port: 9001
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: minio-egress-kubernetes-api
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/instance: minio
app.kubernetes.io/component: object-storage
app.kubernetes.io/part-of: storage-platform
app.kubernetes.io/managed-by: kustomize
spec:
podSelector:
matchLabels:
v1.min.io/tenant: minio
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 10.43.0.0/16
ports:
- protocol: TCP
port: 443
- to:
- ipBlock:
cidr: 10.208.141.0/24
ports:
- protocol: TCP
port: 6443
@@ -0,0 +1,17 @@
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: minio-tenant-env
spec:
vaultAuthRef: vault-auth-storage
mount: secret
type: kv-v2
path: minio/tenant-env
refreshAfter: 60s
destination:
name: minio-tenant-env
create: true
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/component: object-storage
app.kubernetes.io/managed-by: vault-secrets-operator
+17
View File
@@ -0,0 +1,17 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../../base/plugins/vault
- networkpolicy.yaml
patches:
- target:
kind: StatefulSet
name: vault
patch: |-
- op: replace
path: /spec/volumeClaimTemplates/0/spec/resources/requests/storage
value: 1Gi
+76
View File
@@ -0,0 +1,76 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: vault-ingress-clients
labels:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: vault
app.kubernetes.io/component: secret-management
app.kubernetes.io/part-of: security-platform
app.kubernetes.io/managed-by: kustomize
# -----------------------------------------------------------------------------
# Vault 정책 — 두 방향 통제
#
# (1) Ingress: VSO Controller Pod 에서만 8200 수신
# VSO 가 `vault-secrets-operator-system` ns 와 `mnt` ns 양쪽에서 접근할
# 가능성 (과거 설치 잔존 + 신규 전용 ns) 이 있어 둘 다 허용.
#
# (2) Egress: kube-apiserver 의 TokenReview 호출만 허용
# Vault k8s auth 는 클라이언트가 보낸 SA JWT 를 검증하기 위해
# `authentication.k8s.io/v1/tokenreviews` 를 호출한다. kube-apiserver 는
# host network 프로세스이므로 Pod/Namespace 라벨로 selector 를 못 쓰고
# ipBlock 이 불가피 (stock NetworkPolicy 의 한계; Cilium 이면
# `toEntities: [kube-apiserver]` 로 해결됨).
#
# 두 CIDR 이 필요한 이유:
# - 443/TCP → `kubernetes.default.svc` (Service ClusterIP). Vault 가
# 먼저 보내는 목적지. Service CIDR 에서 할당됨.
# - 6443/TCP → control-plane 노드의 실제 kube-apiserver 포트. k3s 의
# kube-proxy(iptables) 가 Service → node DNAT 를 수행하는데, 기본
# CNI (flannel + kube-router) 는 NetworkPolicy 를 veth iptables 단에서
# 평가하면서 DNAT 전/후 시점이 엇갈려 두 쪽 모두 허용해야 안전.
#
# CIDR 값의 출처:
# - 10.43.0.0/16 → k3s `service-cidr` 기본값. `/etc/rancher/k3s/config.yaml`
# 의 값과 동기화. 커스텀 service-cidr 를 쓰면 이 값을 맞춰 변경.
# - 10.208.141.0/24 → **이 overlay 의 control-plane 노드 서브넷**.
# 노드를 추가/교체하거나 staging/prod overlay 를 만들 때 실제 값으로
# 반드시 재지정. dev 단일 노드 기준으로 /24 여유를 둠.
#
# staging/prod overlay 는 이 파일을 그대로 복붙하지 말고 각자의 서브넷으로
# 교체. 재사용 구조가 필요해지면 base + per-env patch 로 분리할 것.
# -----------------------------------------------------------------------------
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: vault
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector:
matchLabels:
app.kubernetes.io/name: vault-secrets-operator
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: vault-secrets-operator-system
podSelector:
matchLabels:
app.kubernetes.io/name: vault-secrets-operator
ports:
- protocol: TCP
port: 8200
egress:
# Vault → kube-apiserver (TokenReview 용)
- to:
- ipBlock:
cidr: 10.43.0.0/16 # k3s service-cidr — kubernetes.default.svc 포함
- ipBlock:
cidr: 10.208.141.0/24 # dev control-plane 노드 서브넷 (env-specific)
ports:
- protocol: TCP
port: 443 # Service ClusterIP 경유
- protocol: TCP
port: 6443 # node DNAT 후 kube-apiserver 실포트
+7
View File
@@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: mnt
resources:
- ../../../base/plugins/vso
+43
View File
@@ -0,0 +1,43 @@
# k8s/scripts AGENTS
Role:
- own helper automation for render, diff, apply, validate, backup, restore, and CI checks
- support Kustomize and operations workflows without replacing declarative ownership
Allowed:
- bash entrypoints
- shell libraries
- CI validation scripts
- backup / restore helper scripts
- wrapper commands around `kubectl kustomize`, `kubectl diff -k`, `kubectl apply -k`
Forbidden:
- scripts becoming the primary source of YAML truth
- large heredoc-generated manifests as the default path
- embedding production secret values
- hiding environment differences in shell conditionals instead of overlays
- giant all-in-one deploy scripts that mix unrelated concerns
Read first:
- `/docs/standards/infra/scripts.md`
- `/docs/standards/infra/kustomize.md`
- `/docs/standards/infra/operations-runbook-upgrade-rollback.md`
- `/docs/standards/infra/backup-restore.md`
- `/docs/standards/infra/k3s-specific.md`
Examples:
- `/docs/examples/infra/scripts.md`
- `/docs/examples/infra/kustomize.md`
- `/docs/examples/infra/backup-restore.md`
- `/docs/examples/infra/operations-runbook-upgrade-rollback.md`
- `/docs/examples/infra/k3s-specific.md`
Rules:
- scripts are helpers, not source of truth
- prefer bash
- non-trivial scripts use functions + `main "$@"`
- use `local` for function-local variables
- ShellCheck must pass
- prefer render -> diff -> apply flow
- destructive actions require explicit opt-in
- environment/context/namespace must be explicit, not implicit
+80
View File
@@ -0,0 +1,80 @@
# Kubernetes Scripts
Imperative bootstrap, teardown, and break-glass operations for the `mnt` infra
stack. 선언적 관리(Service/Job/DB rollout 등)는 Kustomize 패키지의 몫이고,
여기 스크립트는 **phase orchestration****one-shot imperative setup**
(Vault init / unseal, Helm install) 만 담당한다.
## Layout
```
scripts/
├── bin/ # 사용자가 직접 실행하는 엔트리 포인트 (main "$@")
│ ├── bootstrap.sh # 전체 스택 부트스트랩
│ └── teardown.sh # 전체 스택 제거
├── tasks/ # bin/ 이 호출하는 단일 책임 태스크
│ ├── minio-operator-install.sh
│ ├── vault-init.sh # Vault init + unseal + k8s auth + policy/role
│ ├── vault-seed-apps.sh # 앱 시크릿 5 개 Vault KV 에 seed
│ ├── vault-setup-admin.sh
│ └── vso-install.sh # VSO Helm upgrade --install
├── lib/ # 공유 쉘 라이브러리
│ ├── common.sh # log/die, retry, confirm, kube-context 가드, finalizer 헬퍼
│ └── vault.sh # vault_exec 래퍼, 로그인/unseal 유틸
└── ci/
└── validate.sh # kustomize build + kubeconform + kube-linter + shellcheck + shfmt
```
## Entry points
```bash
# 부트스트랩 (대화형, bash history 안전)
bash k8s/scripts/bin/bootstrap.sh <dev|staging|prod>
# 제거 (정상 경로)
bash k8s/scripts/bin/teardown.sh <dev|staging|prod>
CONFIRM=yes bash k8s/scripts/bin/teardown.sh <dev|staging|prod>
# 로컬/CI 품질 게이트
bash k8s/scripts/ci/validate.sh
```
## 필수 환경 변수 (안전 가드)
쉘 rc 에 한 번 선언:
```bash
export KUBE_CONTEXT_DEV="homelab"
export KUBE_CONTEXT_STAGING="staging-cluster"
export KUBE_CONTEXT_PROD="prod-cluster"
```
- `bin/bootstrap.sh``bin/teardown.sh``ENV_NAME` 에 대응하는
`KUBE_CONTEXT_<ENV>``kubectl config current-context` 가 일치하는지
**실행 전 검증**한다. 매핑이 없으면 대화형으로 현재 context 이름 재입력을
요구한다.
- `env=prod` destructive 작업은 `ALLOW_PROD_DESTRUCTIVE=yes` 와 namespace
이름 재입력 TTY 확인이 추가로 필요하다.
## 파괴적 플래그
| 플래그 | 범위 | 효과 |
| --- | --- | --- |
| `CONFIRM=yes` | 두 엔트리 | 대화형 확인 프롬프트 자동 승인 (CI) |
| `SKIP_DIFF=yes` | bootstrap | Phase 3 의 `kubectl diff` preview 생략 (비권장) |
| `RESET_STALE_SECRETS=yes` | bootstrap | VSO-managed K8s Secret 을 선제 삭제 (Vault 값 재주입 유도) |
| `VAULT_KEYS_FILE=<path>` | bootstrap | Vault init key 저장 위치. **env=prod 는 repo 바깥 경로 필수** |
| `FORCE_FINALIZERS=yes` | teardown | Phase 6 활성 — finalizer 강제 제거 + `/finalize` API. Terminating 복구 외 금지 |
| `ALLOW_PROD_DESTRUCTIVE=yes` | teardown | env=prod teardown 허용 (namespace 재입력 추가 확인 필요) |
| `TEARDOWN_VSO_OPERATOR=yes` | teardown | **클러스터 공용** VSO Helm 릴리즈 + 전용 namespace + `vault-tokenreview-binding` 제거. 다른 env 의 Vault/VSO 가 함께 멈춤 |
| `TEARDOWN_VSO_CRDS=yes` | teardown | **클러스터 공용** VSO CRD + ClusterRole/CRB/Webhook 제거. 모든 env 의 VaultStaticSecret 인스턴스가 삭제됨 |
| `TEARDOWN_MINIO_OPERATOR=yes` | teardown | MinIO Operator Helm 릴리즈까지 제거 |
## Rules
- 서비스 단위 스크립트를 새로 만들지 않는다.
- Service / Job / Scheduler / DB rollout 은 Kustomize 패키지에서 선언.
- 스크립트는 phase orchestration 이지 Kubernetes 리소스 스펙의 출처가 아니다.
- 비트리비얼 스크립트는 `functions + main "$@"` 구조를 따른다.
- 모든 `.sh``ci/validate.sh``shellcheck -S style` + `shfmt -i 2 -bn -ci`
를 통과해야 한다.
+346
View File
@@ -0,0 +1,346 @@
#!/usr/bin/env bash
# Entry point: bootstrap the full infra stack for a given environment.
#
# Phases:
# 0. MinIO Operator Helm install (tasks/minio-operator-install.sh)
# 1. Apply namespace + PSS labels (kubectl apply -k base/managing/namespace)
# 2. (optional) Reset stale K8s Secrets (RESET_STALE_SECRETS=yes 일 때만)
# 2.5 VSO CRD 선행 설치 (VaultStaticSecret CR 가 overlay 에 포함돼 있어)
# 2.6 cert-manager 선행 설치 (ClusterIssuer / Certificate CR 가 overlay 에 포함돼 있어)
# 2.7 Keycloak Operator 선행 설치 (Keycloak / KeycloakRealmImport CR 가 overlay 에 포함돼 있어)
# 2.8 Traefik HelmChartConfig + Middleware/TLSOption (kube-system 에 배치 — root overlay 의 `namespace: mnt` 와 충돌하므로 별도 apply)
# 3. Render + diff + confirm + apply overlay (--server-side --field-manager=project-infra-bootstrap)
# 4. Wait for vault-0 Running
# 5. Vault init + unseal + KV + k8s auth + policy / role (tasks/vault-init.sh)
# 6. Seed application secrets (tasks/vault-seed-apps.sh)
# 7. Helm install VSO (tasks/vso-install.sh)
# 8. Apply VSO CRs (kubectl apply -k overlays/<env>/vso/)
# 9. MinIO docker-registry bucket/user/policy 프로비저닝 (tasks/minio-provision-registry.sh)
#
# 대화형 실행 (bash history 에 비밀번호 남지 않음):
# bash k8s/scripts/bin/bootstrap.sh dev
#
# Required context (안전 가드):
# KUBE_CONTEXT_DEV / KUBE_CONTEXT_STAGING / KUBE_CONTEXT_PROD
# 또는 KUBE_CONTEXT= 로 명시. 쉘 rc 에 선언해 두면 현재 kubectl context
# 와 일치 여부를 자동 검증한다. 매핑이 없으면 대화형으로 context 이름
# 재입력을 요구.
#
# Optional env:
# RESET_STALE_SECRETS=yes 이미 있는 VSO-managed K8s Secret 을 삭제 후 재생성
# VAULT_KEYS_FILE=<path> env=prod 필수 — repo 바깥 경로
# SKIP_DIFF=yes Phase 3 의 kubectl diff preview 생략
# CONFIRM=yes 대화형 프롬프트 자동 승인 (CI)
set -Eeuo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=../lib/common.sh
. "$SCRIPT_DIR/../lib/common.sh"
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
K8S_ROOT="$REPO_ROOT/k8s"
TASKS_DIR="$SCRIPT_DIR/../tasks"
export REPO_ROOT
NAMESPACE="mnt"
FIELD_MANAGER="project-infra-bootstrap"
# VSO 가 관리하는 K8s Secret 이름 목록 (기존 찌꺼기 감지용)
VSO_MANAGED_SECRETS=(
identity-postgres-superuser
keycloak-db
keycloak-db-operator
auth-server-db
keycloak-bootstrap-admin
keycloak-bootstrap-admin-operator
keycloak-client-auth-server-ingress
minio-tenant-env
docker-registry-minio
)
usage() {
cat >&2 <<EOF
Usage: bin/bootstrap.sh <dev|staging|prod>
Required env (권장 — 쉘 rc 에 선언):
KUBE_CONTEXT_DEV / KUBE_CONTEXT_STAGING / KUBE_CONTEXT_PROD env → kubectl context 매핑
(또는 KUBE_CONTEXT=<name> 로 한 번만 overrides)
Options:
RESET_STALE_SECRETS=yes 이미 존재하는 VSO-managed Secret 을 삭제 후 재생성
VAULT_KEYS_FILE=<path> env=prod 필수 — repo 바깥 경로
SKIP_DIFF=yes Phase 3 diff preview 생략 (비권장)
CONFIRM=yes 대화형 확인 자동 승인 (CI)
EOF
exit 1
}
precheck_namespace() {
if ! ns_exists "$NAMESPACE"; then
log "Precheck: namespace '$NAMESPACE' 없음 — 신규 생성 흐름"
return 0
fi
local phase
phase="$(ns_phase "$NAMESPACE")"
if [[ "$phase" == "Terminating" ]]; then
err "namespace '$NAMESPACE' 가 Terminating 상태입니다."
err "이전 teardown 이 완료되지 않아 새 리소스 생성이 불가능합니다."
err ""
err "복구 절차:"
err " 1) 이전 teardown 을 마저 끝내기:"
err " CONFIRM=yes bash k8s/scripts/bin/teardown.sh $ENV_NAME"
err " 2) namespace 가 완전히 사라진 걸 확인한 뒤 bootstrap 재실행:"
err " kubectl get namespace $NAMESPACE"
err " bash k8s/scripts/bin/bootstrap.sh $ENV_NAME"
die "bootstrap 중단"
fi
log "Precheck: namespace '$NAMESPACE' phase=$phase — 계속 진행"
}
phase0_minio_operator() {
log "[0/9] MinIO Operator 설치 (Tenant CRD 선행)"
bash "$TASKS_DIR/minio-operator-install.sh"
}
phase1_namespace() {
log "[1/9] Namespace + PSS 라벨 선행 apply"
kubectl apply -k "$K8S_ROOT/base/managing/namespace" \
--server-side --field-manager="$FIELD_MANAGER"
retry 5 1 kubectl get namespace "$NAMESPACE" >/dev/null
}
phase2_reset_stale_secrets() {
log "[2/9] 기존 VSO-managed K8s Secret 점검"
local stale_found=0 s
for s in "${VSO_MANAGED_SECRETS[@]}"; do
if kubectl -n "$NAMESPACE" get secret "$s" >/dev/null 2>&1; then
stale_found=$((stale_found + 1))
if [[ "${RESET_STALE_SECRETS:-}" == "yes" ]]; then
log " 삭제: $s (RESET_STALE_SECRETS=yes)"
kubectl -n "$NAMESPACE" delete secret "$s" --ignore-not-found
else
warn " $s 이미 존재 (VSO 가 덮어쓰지 않음). Vault 값 반영이 필요하면 RESET_STALE_SECRETS=yes 로 재실행하거나 수동 삭제하세요."
fi
fi
done
(( stale_found == 0 )) && log " 기존 Secret 없음"
}
phase2_5_vso_crds() {
# overlays/<env>/{database,keycloak,storage}/vault-secrets.yaml 에 VaultStaticSecret
# CR 들이 포함돼 있어, Phase 3 overlay apply 시점에 CRD 가 없으면 "resource mapping
# not found" 로 실패. Helm 차트 install 은 Phase 7 이므로 CRD 만 선행 적용.
log "[2.5/9] VSO CRD 선행 설치"
local version="${VSO_VERSION:-0.9.0}"
helm repo add hashicorp https://helm.releases.hashicorp.com >/dev/null 2>&1 || true
helm repo update hashicorp >/dev/null
helm show crds hashicorp/vault-secrets-operator --version "$version" \
| kubectl apply -f - --server-side --field-manager="$FIELD_MANAGER"
}
# cert-manager 전체 (CRD + namespace + controller + webhook) 을 overlay 밖에서
# 선행 설치. 이후 phase 3 에서 overlay 가 참조하는 ClusterIssuer / Certificate CR
# 이 등록될 수 있다. CRD established 대기를 반드시 건다 — deploy/webhook 가
# Ready 되기 전에 Certificate CR apply 시 admission webhook 이 거부함.
phase2_6_cert_manager() {
log "[2.6/9] cert-manager 선행 설치"
kubectl apply -k "$K8S_ROOT/overlays/$ENV_NAME/platform/cert-manager" \
--server-side --field-manager="$FIELD_MANAGER"
log " cert-manager CRD established 대기"
retry 30 2 kubectl wait --for=condition=Established --timeout=10s \
crd/clusterissuers.cert-manager.io \
crd/certificates.cert-manager.io \
crd/certificaterequests.cert-manager.io \
crd/orders.acme.cert-manager.io \
crd/challenges.acme.cert-manager.io
log " cert-manager Deployment Available 대기"
retry 30 5 kubectl -n cert-manager wait --for=condition=Available --timeout=10s \
deploy/cert-manager \
deploy/cert-manager-webhook \
deploy/cert-manager-cainjector
}
# Keycloak Operator (CRDs + Operator Deployment) 선행 설치. Keycloak /
# KeycloakRealmImport CR 이 phase 3 에 포함되므로 CRD 등록이 먼저 되어야 한다.
phase2_7_keycloak_operator() {
log "[2.7/9] Keycloak Operator 선행 설치"
kubectl apply -k "$K8S_ROOT/overlays/$ENV_NAME/platform/keycloak-operator" \
--server-side --field-manager="$FIELD_MANAGER"
log " Keycloak Operator CRD established 대기"
retry 30 2 kubectl wait --for=condition=Established --timeout=10s \
crd/keycloaks.k8s.keycloak.org \
crd/keycloakrealmimports.k8s.keycloak.org
log " Keycloak Operator Deployment Available 대기"
retry 60 5 kubectl -n "$NAMESPACE" wait --for=condition=Available --timeout=10s \
deploy/keycloak-operator
}
# kube-system Traefik 커스터마이징 — HelmChartConfig (K3s Helm-controller 가
# 재수렴), Middleware (https-redirect / security-headers), TLSOption (modern-tls).
# root overlay 가 `namespace: mnt` 로 전역 주입하므로 kube-system 타깃 리소스는
# 이 overlay 빌드에 포함시키지 않고 별도 apply 한다.
phase2_8_traefik() {
log "[2.8/9] Traefik HelmChartConfig + Middleware 적용 (kube-system)"
kubectl apply -k "$K8S_ROOT/overlays/$ENV_NAME/platform/traefik" \
--server-side --field-manager="$FIELD_MANAGER"
}
# render → server-side dry-run → diff → confirm → apply
#
# kubectl diff exit codes (GNU man page):
# 0 — 변경 없음
# 1 — 변경 있음 (정상)
# >1 — 실행 오류 (RBAC / API 연결 / invalid manifest 등)
# 이전 구현은 `|| true` 로 모든 비-0 을 흡수해서 에러가 apply 까지 흘러갔다.
phase3_overlay_apply() {
log "[3/9] 인프라 overlay 배포 ($OVERLAY_DIR)"
local tmpdir
tmpdir="$(mktemp -d -t project-infra-bootstrap.XXXXXX)"
trap_cleanup_path "$tmpdir"
local rendered="$tmpdir/rendered.yaml"
kustomize build "$OVERLAY_DIR" > "$rendered"
log " render 완료: $(grep -c '^kind:' "$rendered") resources"
# --- server-side dry-run: admission / RBAC / schema 를 API server 로 검증 ---
log " server-side dry-run 검증"
if ! kubectl apply -f "$rendered" --dry-run=server \
--server-side --field-manager="$FIELD_MANAGER" \
>"$tmpdir/dryrun.out" 2>&1; then
err " server-side dry-run 실패 — apply 중단"
sed 's/^/ /' "$tmpdir/dryrun.out" >&2
die "dry-run 검증 실패"
fi
# --- diff preview — 변경 있음(rc=1)은 정상, 실행 오류(rc>=2)는 die ---
if [[ "${SKIP_DIFF:-}" == "yes" ]]; then
warn " SKIP_DIFF=yes → diff preview 생략"
else
log " server-side diff preview (대용량이면 스크롤)"
local rc=0
kubectl diff -f "$rendered" --server-side --field-manager="$FIELD_MANAGER" \
>"$tmpdir/diff.out" 2>&1 || rc=$?
case "$rc" in
0) log " (변경 없음)" ;;
1)
if [[ -s "$tmpdir/diff.out" ]]; then
sed 's/^/ /' "$tmpdir/diff.out" >&2
else
log " (diff 출력 없음)"
fi
;;
*)
err " kubectl diff 실행 실패 (exit=$rc) — apply 중단"
sed 's/^/ /' "$tmpdir/diff.out" >&2
die "diff 실패"
;;
esac
fi
confirm "위 변경사항을 env=$ENV_NAME context='$(kubectl config current-context)' 에 apply 하시겠습니까?" \
|| die "사용자 취소"
kubectl apply -f "$rendered" --server-side --field-manager="$FIELD_MANAGER"
log " vault/registry/앱 워크로드 apply 완료"
}
phase4_wait_vault_running() {
# Vault readiness probe 는 initialized+unsealed 일 때만 통과하므로 초기화 *전*
# 에는 Ready 가 될 수 없음 → Running 단계까지만 기다림.
log "[4/9] vault-0 Running 대기 (120s × 재시도 3회)"
retry 3 10 kubectl -n "$NAMESPACE" wait \
--for=jsonpath='{.status.phase}'=Running pod/vault-0 --timeout=120s
}
phase5_vault_init() {
log "[5/9] Vault 초기화 / unseal / auth / policy / role"
ENV_NAME="$ENV_NAME" bash "$TASKS_DIR/vault-init.sh"
}
phase6_seed_apps() {
log "[6/9] 앱 시크릿 5 개 Vault KV 에 seed"
ENV_NAME="$ENV_NAME" bash "$TASKS_DIR/vault-seed-apps.sh"
}
phase7_vso_install() {
log "[7/9] VSO Helm upgrade --install"
ENV_NAME="$ENV_NAME" bash "$TASKS_DIR/vso-install.sh"
}
phase8_vso_crs() {
log "[8/9] VSO CR 적용 ($OVERLAY_DIR/vso/)"
kubectl apply -k "$OVERLAY_DIR/vso/" \
--server-side --field-manager="$FIELD_MANAGER"
}
# docker-registry 가 MinIO 를 S3 backend 로 쓰도록 bucket + 서비스 user +
# bucket-scoped policy 를 설정. MinIO Tenant 는 phase 8 에서 minio-tenant-env
# Secret 이 생긴 뒤 기동되므로 이 phase 는 반드시 phase 8 이후에 실행.
phase9_minio_provision_registry() {
log "[9/9] MinIO docker-registry bucket/user/policy 프로비저닝"
ENV_NAME="$ENV_NAME" bash "$TASKS_DIR/minio-provision-registry.sh"
}
summary() {
log "============================================"
log " $ENV_NAME 부트스트랩 완료"
log "============================================"
log "unseal keys + root token: ${VAULT_KEYS_FILE:-$REPO_ROOT/vault-init-keys.json} (0600)"
log " → 오프라인 / 외부 KMS 로 즉시 이동하세요 (Git 반입 금지)"
log ""
log "앱 시크릿 5 개는 Phase 6 에서 입력/seed 됨."
log "조회: vault kv get -field=password secret/<path>"
log ""
log "다음 확인:"
log " kubectl -n $NAMESPACE get pods"
log " kubectl -n $NAMESPACE get secrets"
log " kubectl -n $NAMESPACE get vaultstaticsecret"
}
main() {
ENV_NAME="${1:-}"
case "$ENV_NAME" in
dev|staging|prod) ;;
*) usage ;;
esac
OVERLAY_DIR="$K8S_ROOT/overlays/$ENV_NAME"
[[ -d "$OVERLAY_DIR" ]] || die "overlay 디렉토리 없음: $OVERLAY_DIR"
require_cmd kubectl helm jq kustomize
# 안전 가드 — 실수 클러스터 apply 방지
require_kube_context "$ENV_NAME"
log "============================================"
log " Project-Infra 부트스트랩 ($ENV_NAME)"
log " context : $(kubectl config current-context)"
log "============================================"
precheck_namespace
phase0_minio_operator
phase1_namespace
phase2_reset_stale_secrets
phase2_5_vso_crds
phase2_6_cert_manager
phase2_7_keycloak_operator
phase2_8_traefik
phase3_overlay_apply
phase4_wait_vault_running
phase5_vault_init
phase6_seed_apps
phase7_vso_install
phase8_vso_crs
phase9_minio_provision_registry
summary
}
# Google shell style — 직접 실행일 때만 main 호출 (source 된 경우 함수만 노출).
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
main "$@"
fi
+309
View File
@@ -0,0 +1,309 @@
#!/usr/bin/env bash
# Entry point: tear down the full infra stack for a given environment.
#
# 체계적 정리 — 기본 경로는 "정상 API 삭제" 만 사용한다. finalizer 강제 제거와
# /finalize API 호출 같은 파괴적 복구 경로는 별도 플래그 아래로 격리했다.
#
# Phases:
# 1. Precheck (namespace 존재 여부)
# 2. VSO CR 삭제 (controller 살아있을 때 정상 경로만)
# 3. VSO Helm uninstall
# 4. 인프라 overlay 삭제 (Vault / Registry / 앱)
# 5. namespace 삭제
# 6. (opt) FORCE_FINALIZERS=yes 일 때만 — finalizer 강제 제거 + /finalize
# 7. Cluster-scoped 리소스 정리 (ClusterRoleBinding, VSO ClusterRole/Webhook/CRD)
#
# 대화형:
# bash k8s/scripts/bin/teardown.sh dev
#
# CI (비대화):
# CONFIRM=yes bash k8s/scripts/bin/teardown.sh dev
#
# Required context (bootstrap 과 동일한 안전 가드):
# KUBE_CONTEXT_<ENV_UPPER> 또는 KUBE_CONTEXT
#
# Destructive 플래그:
# FORCE_FINALIZERS=yes
# Phase 6 실행 — PVC / CRD / 모든 namespaced 리소스 finalizer 강제 제거 및
# namespace /finalize 호출. PV 가 orphaned 되고 컨트롤러 정리 누락이 발생할
# 수 있으므로 Terminating 복구 외 목적으로는 쓰지 말 것.
#
# ALLOW_PROD_DESTRUCTIVE=yes
# env=prod teardown 을 허용. 추가로 namespace 이름 재입력 TTY 확인이 필요.
#
# TEARDOWN_VSO_OPERATOR=yes
# Phase 3 에서 VSO Helm 릴리즈와 vault-secrets-operator-system namespace 까지
# 제거. VSO operator 는 환경별이 아니라 **클러스터 공용** 이다. 다른 env 가
# 같은 클러스터에 있으면 그쪽 VSO reconciliation 이 같이 멈춘다.
# Phase 7 의 vault-tokenreview-binding (cluster-scoped 이름 고정) 도 이
# 플래그로 함께 삭제한다. VSO 를 완전히 내릴 때만 켜라. 기본은 skip.
#
# TEARDOWN_VSO_CRDS=yes
# Phase 7 에서 cluster-scoped VSO 리소스 (ClusterRole/CRB/Webhook/CRD) 를
# 제거. CRD 는 클러스터 전체 공유라 같은 클러스터에 다른 env 가 있으면
# 그쪽 인스턴스까지 날아간다. 단일-env-per-cluster 환경 또는 VSO 를 완전히
# 버리려는 의도일 때만 켜라. 기본은 skip.
#
# TEARDOWN_MINIO_OPERATOR=yes
# MinIO Operator Helm 릴리즈도 제거 (기본 유지).
set -Eeuo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=../lib/common.sh
. "$SCRIPT_DIR/../lib/common.sh"
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
K8S_ROOT="$REPO_ROOT/k8s"
NAMESPACE="mnt"
VSO_NAMESPACE="${VSO_NAMESPACE:-vault-secrets-operator-system}"
VSO_RELEASE="${VSO_RELEASE:-vault-secrets-operator}"
VSO_CRD_KINDS=(vaultstaticsecret vaultauth vaultconnection)
usage() {
cat >&2 <<EOF
Usage: bin/teardown.sh <dev|staging|prod>
Required env:
KUBE_CONTEXT_DEV / KUBE_CONTEXT_STAGING / KUBE_CONTEXT_PROD
(또는 KUBE_CONTEXT=<name>)
Options:
CONFIRM=yes 대화형 확인 자동 승인
FORCE_FINALIZERS=yes Phase 6 (finalizer 강제 + /finalize) 활성화
ALLOW_PROD_DESTRUCTIVE=yes env=prod teardown 허용 + namespace 재입력 필요
TEARDOWN_VSO_OPERATOR=yes VSO Helm 릴리즈 + 전용 namespace + vault-tokenreview-binding
모두 제거. VSO 는 클러스터 공용 플랫폼 리소스이므로
다른 env 의 reconciliation 이 함께 멈춤.
TEARDOWN_VSO_CRDS=yes VSO cluster-scoped 리소스 (ClusterRole/CRB/Webhook/CRD)
제거. 클러스터 전체에 영향.
TEARDOWN_MINIO_OPERATOR=yes MinIO Operator 도 함께 제거
EOF
exit 1
}
phase1_precheck() {
log "[1/7] Precheck"
if ns_exists "$NAMESPACE"; then
log " namespace $NAMESPACE phase=$(ns_phase "$NAMESPACE")"
else
log " namespace $NAMESPACE 없음 → 일부 phase 는 skip"
fi
}
phase2_vso_cr_delete() {
log "[2/7] VSO CR 삭제"
if kubectl get -k "$OVERLAY_DIR/vso/" >/dev/null 2>&1; then
# 정상 API 삭제만 시도. 타임아웃 나도 Phase 6 (FORCE_FINALIZERS) 에 위임.
kubectl delete -k "$OVERLAY_DIR/vso/" --ignore-not-found \
--wait=true --timeout=60s 2>&1 \
|| warn " VSO CR 삭제 타임아웃 — FORCE_FINALIZERS=yes 로 재실행하면 강제 해제"
else
log " VSO overlay 리소스 없음 → skip"
fi
}
uninstall_vso_in_ns() {
local ns="$1"
kubectl get namespace "$ns" >/dev/null 2>&1 || return 0
# pre-delete hook Job 이 stuck 이면 uninstall 이 hang → 먼저 정리
kubectl -n "$ns" get job -l app.kubernetes.io/instance="$VSO_RELEASE" -o name 2>/dev/null \
| xargs -r kubectl -n "$ns" delete --force --grace-period=0 --ignore-not-found 2>/dev/null || true
kubectl -n "$ns" delete job pdcc-vault-secrets-operator \
--force --grace-period=0 --ignore-not-found 2>/dev/null || true
if helm -n "$ns" status "$VSO_RELEASE" >/dev/null 2>&1; then
log " [$ns] helm uninstall $VSO_RELEASE"
helm -n "$ns" uninstall "$VSO_RELEASE" --wait --timeout 5m 2>&1 \
|| warn " [$ns] helm uninstall 실패 — 릴리즈 Secret 잔존물 직접 제거"
else
log " [$ns] VSO 릴리즈 없음"
fi
# uninstall 실패 / rollback 잔존 상태에서 남는 sh.helm.release.v1.* Secret 정리
kubectl -n "$ns" get secret -o name 2>/dev/null \
| grep "sh.helm.release.*${VSO_RELEASE}" \
| xargs -r kubectl -n "$ns" delete --ignore-not-found 2>/dev/null || true
}
# $NAMESPACE (mnt) 내 잔존 uninstall 은 **env-specific 레거시 cleanup** 이라
# 항상 실행한다 — 과거 VSO 가 mnt 에 설치됐던 적이 있으면 릴리즈 메타데이터가
# 남아 다음 install 을 막으므로. $VSO_NAMESPACE (공용 operator 공간) 에 대한
# uninstall 은 클러스터 공유이므로 명시적 opt-in 요구.
phase3_vso_helm_uninstall() {
log "[3/7] VSO Helm uninstall"
uninstall_vso_in_ns "$NAMESPACE"
if [[ "${TEARDOWN_VSO_OPERATOR:-}" == "yes" ]]; then
warn " TEARDOWN_VSO_OPERATOR=yes — 공용 VSO operator ($VSO_NAMESPACE) 제거"
warn " 경고: 같은 클러스터에 다른 env 의 VaultStaticSecret 이 있으면 reconciliation 이 멈춥니다."
uninstall_vso_in_ns "$VSO_NAMESPACE"
else
log " VSO operator ($VSO_NAMESPACE) 보존 (TEARDOWN_VSO_OPERATOR=yes 로 제거 가능)"
fi
}
phase4_overlay_delete() {
log "[4/7] 인프라 리소스 삭제 (kustomize overlay)"
if ns_exists "$NAMESPACE"; then
kubectl delete -k "$OVERLAY_DIR" --ignore-not-found \
--wait=true --timeout=120s 2>&1 \
|| warn " overlay 삭제 타임아웃 — 필요 시 FORCE_FINALIZERS=yes 로 Phase 6 사용"
else
log " namespace 없음 → overlay 삭제 skip"
fi
}
phase5_namespace_delete() {
log "[5/7] namespace 정상 삭제"
if ns_exists "$NAMESPACE"; then
kubectl delete namespace "$NAMESPACE" --ignore-not-found --wait=false 2>/dev/null || true
log " namespace 제거 대기 (최대 60s)"
if wait_namespace_gone "$NAMESPACE" 60; then
log " namespace $NAMESPACE 제거 완료"
else
warn " namespace $NAMESPACE 가 Terminating 60s 초과."
warn " 정상 삭제로 끝나지 않은 경우:"
warn " 1) 'kubectl get all -n $NAMESPACE' 로 남은 리소스 원인 확인"
warn " 2) controller/operator 재기동 으로 finalizer 처리 시도"
warn " 3) 그래도 막히면 FORCE_FINALIZERS=yes 로 재실행 — 단 파괴적"
fi
else
log " namespace 이미 없음"
fi
}
# FORCE_FINALIZERS=yes 일 때만 — PV orphaned / 데이터 정합성 리스크 있음.
phase6_force_finalizers() {
if [[ "${FORCE_FINALIZERS:-}" != "yes" ]]; then
log "[6/7] FORCE_FINALIZERS!=yes → skip (파괴적 경로 비활성)"
return 0
fi
log "[6/7] FORCE_FINALIZERS=yes — finalizer 강제 제거 + /finalize"
warn " 경고: PV orphan / controller 정리 누락이 발생할 수 있습니다."
warn " Terminating stuck 복구 외 목적으로 사용하지 마세요."
if ! ns_exists "$NAMESPACE"; then
log " namespace 이미 없음 → skip"
return 0
fi
strip_finalizers_in_ns "$NAMESPACE" persistentvolumeclaim
strip_finalizers_in_ns "$NAMESPACE" "${VSO_CRD_KINDS[@]}" 2>/dev/null || true
log " namespace $NAMESPACE 전체 리소스 finalizer 일괄 제거 (최후 수단)"
strip_finalizers_all_ns_resources "$NAMESPACE"
kubectl delete namespace "$NAMESPACE" --ignore-not-found --wait=false 2>/dev/null || true
if wait_namespace_gone "$NAMESPACE" 30; then
log " namespace $NAMESPACE 제거 완료 (FORCE_FINALIZERS)"
return 0
fi
warn " namespace 여전히 Terminating → /finalize API 호출"
force_finalize_namespace "$NAMESPACE" \
|| warn " /finalize 호출 실패 — 수동 확인 필요"
if wait_namespace_gone "$NAMESPACE" 30; then
log " namespace $NAMESPACE 제거 완료 (/finalize)"
else
err " namespace $NAMESPACE 여전히 존재. 'kubectl get namespace $NAMESPACE -o yaml' 로 확인 필요."
fi
}
phase7_cluster_scoped() {
log "[7/7] Cluster-scoped 리소스 정리"
# vault-tokenreview-binding 은 cluster-scoped 이름 고정이라 env 별 분리가
# 불가능하다 (kustomize base 를 env-scoped 이름으로 재설계하기 전까지).
# 기본 경로에서 삭제하면 다른 env 의 Vault k8s auth TokenReview 가 부서지므로
# TEARDOWN_VSO_OPERATOR=yes 와 함께 게이트 (플랫폼 auth 평면 전체 정리).
if [[ "${TEARDOWN_VSO_OPERATOR:-}" == "yes" ]]; then
log " vault-tokenreview-binding 제거 (TEARDOWN_VSO_OPERATOR=yes)"
kubectl delete clusterrolebinding vault-tokenreview-binding --ignore-not-found 2>&1 \
| sed 's/^/ /' || true
if ns_exists "$VSO_NAMESPACE"; then
log " VSO namespace $VSO_NAMESPACE 삭제"
kubectl delete namespace "$VSO_NAMESPACE" --ignore-not-found --wait=true --timeout=60s 2>/dev/null \
|| warn " $VSO_NAMESPACE 삭제 타임아웃 — 수동 확인 필요"
fi
else
log " vault-tokenreview-binding 보존 (TEARDOWN_VSO_OPERATOR=yes 로 제거 가능)"
fi
# VSO 의 ClusterRole / CRB / Webhook / CRD 는 클러스터 전체 공유다.
# 운영자가 의도적으로 VSO 전체를 버릴 때만 TEARDOWN_VSO_CRDS=yes 로 활성화.
if [[ "${TEARDOWN_VSO_CRDS:-}" == "yes" ]]; then
warn " TEARDOWN_VSO_CRDS=yes — cluster-scoped VSO 리소스 제거 (ClusterRole/CRB/Webhook/CRD)"
warn " 경고: 같은 클러스터의 다른 env 에서 VSO 를 쓰고 있으면 모두 영향받습니다."
local kind
for kind in clusterrole clusterrolebinding validatingwebhookconfiguration mutatingwebhookconfiguration; do
kubectl get "$kind" -o name 2>/dev/null \
| grep -E 'vault-secrets-operator' \
| xargs -r kubectl delete --ignore-not-found 2>/dev/null || true
done
kubectl get crd -o name 2>/dev/null \
| grep 'secrets.hashicorp.com' \
| xargs -r kubectl delete --ignore-not-found 2>/dev/null || true
else
log " cluster-scoped VSO 리소스 보존 (TEARDOWN_VSO_CRDS=yes 로 제거 가능)"
fi
if [[ "${TEARDOWN_MINIO_OPERATOR:-}" == "yes" ]]; then
log " TEARDOWN_MINIO_OPERATOR=yes → MinIO Operator 제거"
helm -n minio-operator uninstall minio-operator --wait --timeout 5m 2>/dev/null \
|| warn " MinIO Operator Helm uninstall 실패"
kubectl delete namespace minio-operator --ignore-not-found --wait=true --timeout=60s \
|| warn " minio-operator namespace 삭제 실패"
else
log " MinIO Operator 유지 (TEARDOWN_MINIO_OPERATOR=yes 로 제거 가능)"
fi
}
summary() {
log "============================================"
log " $ENV_NAME 삭제 완료"
log "============================================"
log "Vault unseal key 파일은 그대로 남아있습니다 (${VAULT_KEYS_FILE:-$REPO_ROOT/vault-init-keys.json})."
log "완전 초기화하려면 수동 삭제하세요."
}
main() {
ENV_NAME="${1:-}"
case "$ENV_NAME" in
dev|staging|prod) ;;
*) usage ;;
esac
OVERLAY_DIR="$K8S_ROOT/overlays/$ENV_NAME"
[[ -d "$OVERLAY_DIR" ]] || die "overlay 디렉토리 없음: $OVERLAY_DIR"
require_cmd kubectl helm jq
# 안전 가드 1 — kubectl context 가 env 와 맞는지
require_kube_context "$ENV_NAME"
# 안전 가드 2 — prod 는 추가 게이트 (namespace 재입력 포함)
require_production_gate "$ENV_NAME" "$NAMESPACE"
log "============================================"
log " Project-Infra 삭제 ($ENV_NAME)"
log " context : $(kubectl config current-context)"
log "============================================"
confirm "namespace='$NAMESPACE' 의 모든 Vault / Registry / VSO / 앱 리소스를 삭제합니다. 계속?" \
|| die "사용자 취소"
phase1_precheck
phase2_vso_cr_delete
phase3_vso_helm_uninstall
phase4_overlay_delete
phase5_namespace_delete
phase6_force_finalizers
phase7_cluster_scoped
summary
}
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
main "$@"
fi
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env bash
# Validate all kustomize overlays against schema + lint rules,
# 그리고 scripts/ 전체에 대해 shell 품질 게이트 (shellcheck + shfmt) 를 실행한다.
#
# Runs, for each environment (dev, staging, prod):
# 1. `kustomize build overlays/<env>` — structural validity
# 2. `kustomize build overlays/<env>/vso` — VSO CRD overlay (built separately
# because it requires the VSO Helm
# release to be installed first)
# 3. `kubeconform -strict -ignore-missing-schemas` — OpenAPI schema validation
# with CRD schemas fetched from the Datree catalog.
# 4. `kube-linter lint` — anti-pattern lint against
# root .kube-linter.yaml configuration.
#
# Plus (not per-overlay):
# 5. `shellcheck -S style` over k8s/scripts/**/*.sh — shell correctness + style
# 6. `shfmt -i 2 -bn -ci -d` over k8s/scripts/ — formatting diff (fail on drift)
#
# Exit codes:
# 0 — everything passed
# 1 — at least one step failed (details printed)
#
# Tools expected on $PATH:
# kustomize kubeconform kube-linter shellcheck shfmt
set -Eeuo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=../lib/common.sh
. "$SCRIPT_DIR/../lib/common.sh"
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
K8S_ROOT="$REPO_ROOT/k8s"
KUBE_LINTER_CFG="$REPO_ROOT/.kube-linter.yaml"
# Local-user tooling fallback: when tools aren't installed system-wide,
# allow users to drop them under ~/bin (CI typically installs to a PATH dir).
if [[ -d "$HOME/bin" ]]; then
export PATH="$HOME/bin:$PATH"
fi
require_cmd kustomize kubeconform kube-linter shellcheck shfmt
ENVS=(dev staging prod)
OVERLAYS_TO_BUILD=()
for env in "${ENVS[@]}"; do
root_overlay="$K8S_ROOT/overlays/$env"
vso_overlay="$K8S_ROOT/overlays/$env/vso"
if [[ -f "$root_overlay/kustomization.yaml" ]]; then
OVERLAYS_TO_BUILD+=("$root_overlay")
else
log "skip (kustomization.yaml 없음): overlays/$env"
fi
if [[ -f "$vso_overlay/kustomization.yaml" ]]; then
OVERLAYS_TO_BUILD+=("$vso_overlay")
fi
done
# Datree CRD catalog — kubeconform fetches per-CRD JSON schema on demand.
CRD_CATALOG='https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json'
WORK_DIR="$(mktemp -d -t project-infra-validate.XXXXXX)"
trap_cleanup_path "$WORK_DIR"
declare -A BUILD_STATUS SCHEMA_STATUS LINT_STATUS
for overlay in "${OVERLAYS_TO_BUILD[@]}"; do
rel="${overlay#"$REPO_ROOT/"}"
rendered="$WORK_DIR/$(echo "$rel" | tr '/' '_').yaml"
# -- 1. kustomize build ----------------------------------------------------
if kustomize build "$overlay" > "$rendered" 2>"$rendered.err"; then
BUILD_STATUS[$rel]="ok"
log "kustomize build $rel — OK ($(grep -c '^kind:' "$rendered") resources)"
else
BUILD_STATUS[$rel]="fail"
err "kustomize build $rel — FAIL"
cat "$rendered.err" >&2
continue
fi
# -- 2. kubeconform --------------------------------------------------------
if kubeconform -strict -ignore-missing-schemas \
-schema-location default \
-schema-location "$CRD_CATALOG" \
-summary \
"$rendered" >"$rendered.kconf" 2>&1; then
SCHEMA_STATUS[$rel]="ok"
log " kubeconform — OK ($(tail -1 "$rendered.kconf"))"
else
SCHEMA_STATUS[$rel]="fail"
err " kubeconform — FAIL"
cat "$rendered.kconf" >&2
fi
# -- 3. kube-linter --------------------------------------------------------
if kube-linter lint --config "$KUBE_LINTER_CFG" "$rendered" \
>"$rendered.klint" 2>&1; then
LINT_STATUS[$rel]="ok"
log " kube-linter — OK"
else
LINT_STATUS[$rel]="fail"
err " kube-linter — findings:"
sed 's/^/ /' "$rendered.klint" >&2
fi
done
# -----------------------------------------------------------------------------
# 5. shellcheck — scripts/**/*.sh 전체
# -----------------------------------------------------------------------------
SHELL_STATUS="skip"
SCRIPTS_ROOT="$K8S_ROOT/scripts"
mapfile -t SHELL_FILES < <(find "$SCRIPTS_ROOT" -type f -name '*.sh' -print | sort)
if (( ${#SHELL_FILES[@]} > 0 )); then
log "shellcheck — ${#SHELL_FILES[@]} files"
if shellcheck -S style -x "${SHELL_FILES[@]}" >"$WORK_DIR/shellcheck.out" 2>&1; then
SHELL_STATUS="ok"
log " shellcheck — OK"
else
SHELL_STATUS="fail"
err " shellcheck — findings:"
sed 's/^/ /' "$WORK_DIR/shellcheck.out" >&2
fi
fi
# -----------------------------------------------------------------------------
# 6. shfmt — 포매팅 drift 검증 (수정 없이 diff 만 출력)
# -----------------------------------------------------------------------------
FMT_STATUS="skip"
if (( ${#SHELL_FILES[@]} > 0 )); then
log "shfmt -i 2 -bn -ci -d"
if shfmt -i 2 -bn -ci -d "${SHELL_FILES[@]}" >"$WORK_DIR/shfmt.out" 2>&1; then
FMT_STATUS="ok"
log " shfmt — OK"
else
FMT_STATUS="fail"
err " shfmt — drift 감지 (로컬에서 'shfmt -i 2 -bn -ci -w k8s/scripts' 로 정렬하세요):"
sed 's/^/ /' "$WORK_DIR/shfmt.out" >&2
fi
fi
# -----------------------------------------------------------------------------
# Summary
# -----------------------------------------------------------------------------
echo >&2
log "============================================"
log " Validation summary"
log "============================================"
failed=0
for overlay in "${OVERLAYS_TO_BUILD[@]}"; do
rel="${overlay#"$REPO_ROOT/"}"
b="${BUILD_STATUS[$rel]:-skip}"
s="${SCHEMA_STATUS[$rel]:-skip}"
l="${LINT_STATUS[$rel]:-skip}"
printf ' %-40s build=%-4s schema=%-4s lint=%s\n' "$rel" "$b" "$s" "$l" >&2
[[ "$b" == "fail" || "$s" == "fail" || "$l" == "fail" ]] && failed=$((failed + 1))
done
printf ' %-40s shellcheck=%-4s shfmt=%s\n' "scripts/" "$SHELL_STATUS" "$FMT_STATUS" >&2
[[ "$SHELL_STATUS" == "fail" || "$FMT_STATUS" == "fail" ]] && failed=$((failed + 1))
if (( failed > 0 )); then
err "$failed check(s) 실패"
exit 1
fi
log "모든 check 통과"
+302
View File
@@ -0,0 +1,302 @@
#!/usr/bin/env bash
# Common shell library for Project-Infra scripts.
#
# Source this with: . "$(dirname "$0")/../lib/common.sh"
# or from tasks/: . "$SCRIPT_DIR/../lib/common.sh"
#
# Provides:
# - strict mode + safe IFS
# - log()/warn()/err()/die() to stderr with ISO 8601 + level prefix
# - trap_cleanup_path() / trap_cleanup_fn() — EXIT 시 경로 rm -rf 또는 함수 호출
# - require_cmd() / require_env() — preconditions
# - confirm() — interactive + CONFIRM=yes env-var gate
# - mask_secret() — masks sensitive values in logs
# - retry() — retry a command with linear backoff
# - require_kube_context() / require_production_gate() — env 타깃 검증 가드
# - ns_exists() / ns_phase() / strip_finalizers_in_ns() /
# strip_finalizers_all_ns_resources() / force_finalize_namespace() /
# wait_namespace_gone() — namespace teardown 복구 헬퍼
#
# All functions emit diagnostic output to stderr; stdout stays clean
# so callers can pipe subcommand output normally.
# shellcheck shell=bash
# -----------------------------------------------------------------------------
# strict mode
# -----------------------------------------------------------------------------
set -Eeuo pipefail
IFS=$'\n\t'
# -----------------------------------------------------------------------------
# logging
# -----------------------------------------------------------------------------
_ts() { date -u +"%Y-%m-%dT%H:%M:%SZ"; }
log() { printf '%s [INFO] %s\n' "$(_ts)" "$*" >&2; }
warn() { printf '%s [WARN] %s\n' "$(_ts)" "$*" >&2; }
err() { printf '%s [ERROR] %s\n' "$(_ts)" "$*" >&2; }
die() { err "$*"; exit 1; }
# -----------------------------------------------------------------------------
# cleanup registration
#
# eval 기반 문자열 cleanup 은 공통 라이브러리에 두기에 부적절하다 (셸 인젝션
# 경로가 열리기 쉽고, Google shell 가이드 기준 code smell). 대신 두 가지 구체
# 타입만 제공한다:
# - trap_cleanup_path <path> : EXIT 시 rm -rf 로 삭제할 경로
# - trap_cleanup_fn <fname> : EXIT 시 인자 없이 호출할 함수 이름
# 두 종류 다 LIFO 로 실행되고, 실패해도 전체 종료 코드는 보존된다.
# -----------------------------------------------------------------------------
_CLEANUP_PATHS=()
_CLEANUP_FNS=()
trap_cleanup_path() {
local p="$1"
[[ -n "$p" ]] || return 0
_CLEANUP_PATHS+=("$p")
}
trap_cleanup_fn() {
local fn="$1"
declare -F "$fn" >/dev/null 2>&1 \
|| { err "trap_cleanup_fn: 함수를 찾을 수 없음: $fn"; return 1; }
_CLEANUP_FNS+=("$fn")
}
_run_cleanups() {
local rc=$?
local i
for ((i = ${#_CLEANUP_FNS[@]} - 1; i >= 0; i--)); do
"${_CLEANUP_FNS[$i]}" || true
done
for ((i = ${#_CLEANUP_PATHS[@]} - 1; i >= 0; i--)); do
rm -rf -- "${_CLEANUP_PATHS[$i]}" || true
done
exit "$rc"
}
trap _run_cleanups EXIT INT TERM
# -----------------------------------------------------------------------------
# preconditions
# -----------------------------------------------------------------------------
require_cmd() {
local cmd
for cmd in "$@"; do
command -v "$cmd" >/dev/null 2>&1 || die "필수 명령어가 PATH 에 없습니다: $cmd"
done
}
require_env() {
local var
for var in "$@"; do
if [[ -z "${!var:-}" ]]; then
die "필수 환경 변수가 비어있습니다: $var"
fi
done
}
# -----------------------------------------------------------------------------
# destructive gate
# -----------------------------------------------------------------------------
# Usage: confirm "namespace 'mnt' 의 모든 리소스를 삭제합니다. 계속?"
# Returns 0 if the user said yes (interactively or via CONFIRM=yes env).
confirm() {
local prompt="$1"
if [[ "${CONFIRM:-}" == "yes" ]]; then
log "CONFIRM=yes → 자동 진행: $prompt"
return 0
fi
if [[ ! -t 0 ]]; then
die "비대화 환경에서는 CONFIRM=yes 환경 변수를 지정하세요: $prompt"
fi
local answer
read -r -p "$prompt [y/N]: " answer
[[ "$answer" == "y" || "$answer" == "Y" ]]
}
# -----------------------------------------------------------------------------
# secret masking (for logs)
# -----------------------------------------------------------------------------
# Usage: log "root token = $(mask_secret "$ROOT_TOKEN")"
mask_secret() {
local s="$1"
local n=${#s}
if (( n <= 8 )); then
printf '***'
else
printf '%s***%s' "${s:0:4}" "${s: -4}"
fi
}
# -----------------------------------------------------------------------------
# retry helper
# -----------------------------------------------------------------------------
# Usage: retry 5 2 kubectl wait --for=condition=Ready pod/vault-0 -n mnt --timeout=10s
# - $1: max attempts
# - $2: sleep seconds between attempts
# - $3..: command and arguments
retry() {
local attempts="$1"; shift
local delay="$1"; shift
local i=0
until "$@"; do
i=$((i + 1))
if (( i >= attempts )); then
err "최대 시도 횟수 ${attempts} 회 초과: $*"
return 1
fi
warn "실패 ($i/$attempts), ${delay}s 후 재시도: $*"
sleep "$delay"
done
}
# -----------------------------------------------------------------------------
# kube-context / env 타깃 검증
# -----------------------------------------------------------------------------
# require_kube_context <env_name>
#
# env → 기대 context 매핑을 다음 우선순위로 해결한다:
# 1) $KUBE_CONTEXT (명시적으로 주입된 값 — CI 에서 사용)
# 2) $KUBE_CONTEXT_<ENV_UPPER> (env 별 매핑 — 쉘 rc 에 선언하면 편함)
#
# 매칭 실패 시 현재 context 를 출력하고 사용자에게 context 이름을 직접
# 재입력받아 확인한다. 비대화 환경은 die.
#
# 이 함수를 통과하면 다음이 보장된다:
# - kubectl 이 가리키는 cluster 가 env 의 의도된 cluster
# - 사용자/CI 가 그 사실을 명시적으로 인지함 (실수 클러스터 apply 방지)
require_kube_context() {
local env_name="$1"
require_cmd kubectl
local current
current="$(kubectl config current-context 2>/dev/null || true)"
[[ -n "$current" ]] || die "kubectl current-context 가 비어있습니다. kubeconfig 를 먼저 설정하세요."
local env_upper
env_upper="$(printf '%s' "$env_name" | tr '[:lower:]' '[:upper:]')"
local mapped_var="KUBE_CONTEXT_${env_upper}"
local expected="${KUBE_CONTEXT:-${!mapped_var:-}}"
if [[ -n "$expected" ]]; then
if [[ "$current" != "$expected" ]]; then
die "kube-context 불일치: env=$env_name 기대='$expected' 현재='$current' (KUBE_CONTEXT 또는 ${mapped_var} 와 kubectl 현재 context 가 다름)"
fi
log "kube-context OK: env=$env_name context='$current'"
return 0
fi
# 매핑이 없을 때:
# - 비대화(CI) → 무조건 die. CONFIRM=yes 로도 우회 불가 (context 는
# destructive 작업의 타깃이라 명시성이 절대 원칙).
# - 대화형 TTY → 현재 context 이름 재입력으로 확인.
if [[ ! -t 0 ]]; then
die "비대화 환경에서는 KUBE_CONTEXT 또는 ${mapped_var} 가 필수입니다 (CONFIRM=yes 로 우회 불가)."
fi
warn "env=$env_name 의 기대 context 가 지정되지 않았습니다."
warn " (권장) export KUBE_CONTEXT_${env_upper}='<context-name>' 를 쉘 rc 에 선언"
warn "현재 context: $current"
local typed
read -r -p "확인을 위해 현재 context 이름을 그대로 입력하세요 ('$current'): " typed
[[ "$typed" == "$current" ]] || die "context 이름 불일치 — 중단"
}
# require_production_gate <env_name>
#
# env=prod 에서 파괴적 작업을 실행하려면 ALLOW_PROD_DESTRUCTIVE=yes 를 요구.
# 추가로 namespace 이름 재입력을 강제해서 오타 한 번으로 prod 가 날아가는 것을 막는다.
# dev/staging 은 통과.
require_production_gate() {
local env_name="$1"
local ns="$2"
[[ "$env_name" == "prod" ]] || return 0
if [[ "${ALLOW_PROD_DESTRUCTIVE:-}" != "yes" ]]; then
die "env=prod 파괴적 작업은 ALLOW_PROD_DESTRUCTIVE=yes 환경 변수가 필요합니다."
fi
if [[ ! -t 0 ]]; then
die "env=prod 는 대화형 TTY 에서만 실행 가능합니다 (namespace 재입력 확인 필요)."
fi
local typed
warn "env=prod 파괴적 작업 — namespace '$ns' 를 그대로 재입력하세요."
read -r -p "namespace: " typed
[[ "$typed" == "$ns" ]] || die "namespace 재입력 불일치 — 중단"
}
# -----------------------------------------------------------------------------
# namespace / finalizer 정리 헬퍼
# -----------------------------------------------------------------------------
# ns_phase <namespace> — namespace 의 .status.phase 를 출력. 없으면 빈 문자열.
ns_phase() {
local ns="$1"
kubectl get namespace "$ns" -o jsonpath='{.status.phase}' 2>/dev/null || true
}
# ns_exists <namespace> — 존재하면 0, 없으면 1
ns_exists() {
kubectl get namespace "$1" >/dev/null 2>&1
}
# strip_finalizers_in_ns <namespace> <kind...>
# 지정한 kind 들의 모든 인스턴스에서 metadata.finalizers 를 제거한다.
# kind 가 CRD 여도 동작 (kubectl 이 해당 API 서버에 등록되어 있기만 하면).
strip_finalizers_in_ns() {
local ns="$1"; shift
local kind obj
for kind in "$@"; do
while IFS= read -r obj; do
[[ -z "$obj" ]] && continue
kubectl -n "$ns" patch "$obj" --type=merge \
-p '{"metadata":{"finalizers":null}}' >/dev/null 2>&1 || true
log " finalizer 제거: -n $ns $obj"
done < <(kubectl -n "$ns" get "$kind" -o name 2>/dev/null || true)
done
}
# strip_finalizers_all_ns_resources <namespace>
# namespace 에 남아있는 모든 namespaced 리소스의 finalizer 를 일괄 제거.
# 최후 수단 — Terminating 에 걸린 리소스들을 떼어낼 때만 사용.
strip_finalizers_all_ns_resources() {
local ns="$1"
local kinds
# namespaced=true 리소스 종류만
kinds=$(kubectl api-resources --namespaced=true --verbs=delete -o name 2>/dev/null)
local kind
for kind in $kinds; do
while IFS= read -r obj; do
[[ -z "$obj" ]] && continue
kubectl -n "$ns" patch "$obj" --type=merge \
-p '{"metadata":{"finalizers":null}}' >/dev/null 2>&1 || true
done < <(kubectl -n "$ns" get "$kind" -o name 2>/dev/null || true)
done
}
# force_finalize_namespace <namespace>
# namespace 자체의 spec.finalizers 를 비워서 API 서버가 강제 삭제하도록 한다.
# kubectl replace --raw 로 /finalize 엔드포인트 호출.
# 전제: kubectl + jq 존재. 주의 — orphaned PV 등이 남을 수 있음.
force_finalize_namespace() {
local ns="$1"
require_cmd jq
log " namespace $ns 강제 finalize (API /finalize)"
kubectl get namespace "$ns" -o json \
| jq '.spec.finalizers = [] | .metadata.finalizers = []' \
| kubectl replace --raw "/api/v1/namespaces/${ns}/finalize" -f - >/dev/null
}
# wait_namespace_gone <namespace> <timeout_seconds>
# namespace 가 완전히 사라질 때까지 대기. timeout 초과 시 1 반환.
wait_namespace_gone() {
local ns="$1" timeout="${2:-60}" i=0
while ns_exists "$ns"; do
i=$((i + 1))
if (( i >= timeout )); then
return 1
fi
sleep 1
done
return 0
}
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env bash
# Vault helper functions. Depends on lib/common.sh being sourced first.
#
# Environment contract:
# VAULT_NAMESPACE — k8s namespace where vault-0 runs (default: mnt)
# VAULT_POD — Vault pod name (default: vault-0)
# VAULT_KEYS_FILE — path to JSON file produced by vault operator init
# (default: $REPO_ROOT/vault-init-keys.json)
# shellcheck shell=bash
# shellcheck source=./common.sh
# (common.sh must be sourced by the caller.)
: "${VAULT_NAMESPACE:=mnt}"
: "${VAULT_POD:=vault-0}"
# vault_exec <args...> — exec vault CLI inside the Vault pod.
# -i 를 붙여서 호출자의 stdin 을 컨테이너로 파이프 (vault login -no-print - 등).
vault_exec() {
kubectl exec -i -n "$VAULT_NAMESPACE" "$VAULT_POD" -- vault "$@"
}
# vault_exec_sh <script> — exec /bin/sh -c "$script" inside the Vault pod.
# Use when the vault CLI call needs shell features (heredocs, redirects, stdin).
vault_exec_sh() {
kubectl exec -i -n "$VAULT_NAMESPACE" "$VAULT_POD" -- sh -c "$1"
}
# vault_status_json — echoes the JSON output of `vault status`, or empty on error.
vault_status_json() {
kubectl exec -n "$VAULT_NAMESPACE" "$VAULT_POD" -- \
vault status -format=json 2>/dev/null || true
}
vault_is_initialized() {
local json
json="$(vault_status_json)"
[[ -n "$json" ]] && echo "$json" | jq -e '.initialized == true' >/dev/null
}
vault_is_sealed() {
local json
json="$(vault_status_json)"
[[ -n "$json" ]] && echo "$json" | jq -e '.sealed == true' >/dev/null
}
# vault_unseal_from_keyfile <keys_file>
#
# 주의: `vault operator unseal <KEY>` 형태로 argv 에 키를 넘기면 `ps` 로
# 노출된다 (container 내부 프로세스라도 보안 경계는 유지). vault CLI 는
# argv 가 없으면 stdin 에서 읽으므로 stdin 으로만 전달한다.
vault_unseal_from_keyfile() {
local keys_file="$1"
[[ -f "$keys_file" ]] || die "unseal 키 파일이 없습니다: $keys_file"
local threshold
threshold="$(jq -r '.unseal_threshold' "$keys_file")"
local i key
for ((i = 0; i < threshold; i++)); do
key="$(jq -r ".unseal_keys_b64[$i]" "$keys_file")"
vault_exec operator unseal "$key" >/dev/null
done
unset key
log "Vault unseal 완료 (threshold=${threshold})"
}
# vault_login_root_from_keyfile <keys_file>
# Logs the vault CLI inside the pod using the root token. stdout is suppressed
# so the token never reaches terminals or logs.
vault_login_root_from_keyfile() {
local keys_file="$1"
local token
token="$(jq -r '.root_token' "$keys_file")"
printf '%s' "$token" | vault_exec_sh 'vault login -no-print -' >/dev/null
}
# vault_kv_exists <path> — returns 0 if the KV v2 secret exists.
vault_kv_exists() {
local path="$1"
[[ "$path" == secret/* ]] || path="secret/${path}"
vault_exec kv get -format=json "$path" >/dev/null 2>&1
}
# vault_auth_method_enabled <mount> — returns 0 if the auth method is enabled.
vault_auth_method_enabled() {
local mount="$1"
vault_exec auth list -format=json 2>/dev/null \
| jq -e --arg m "${mount}/" '.[$m] // empty' >/dev/null
}
# vault_secrets_engine_enabled <mount> — returns 0 if the secrets engine is enabled.
vault_secrets_engine_enabled() {
local mount="$1"
vault_exec secrets list -format=json 2>/dev/null \
| jq -e --arg m "${mount}/" '.[$m] // empty' >/dev/null
}

Some files were not shown because too many files have changed in this diff Show More