# Kustomize 예시 모든 예시는 Kustomize v5 문법 기준. 렌더 검증: ```bash kubectl kustomize | kubectl apply --server-side --field-manager=ci --dry-run=server -f - ``` --- ## 좋은 예시 1: base / components / overlays 전체 구조 + 실제 base `kustomization.yaml` ```text k8s/ base/ app/units/identity/auth/ kustomization.yaml deployment.yaml service.yaml servicemonitor.yaml pdb.yaml hpa.yaml components/ with-topology-spread-zone/ kustomization.yaml patch.yaml with-pdb-tier1/ kustomization.yaml patch.yaml overlays/ prod/kr-main/ kustomization.yaml patches/ auth-resources.yaml auth-ingress-host.yaml ``` ```yaml # k8s/base/app/units/identity/auth/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - deployment.yaml - service.yaml - servicemonitor.yaml - pdb.yaml - hpa.yaml labels: - pairs: app.kubernetes.io/name: auth app.kubernetes.io/component: api app.kubernetes.io/part-of: identity-platform includeSelectors: false includeTemplates: true ``` **왜 좋은가:** - base가 환경·region을 모른다 (namespace / replicas / host / image tag 전부 없음) - `labels:` (v5) 사용, `commonLabels` 없음 → selector immutability 안전 - `includeTemplates: true`로 Pod label에는 전파되어 observability 쿼리 가능 - selector에 들어가는 label은 base의 Deployment 내부에서 명시적으로 고정 --- ## 좋은 예시 2: base Deployment (완전 apply-ready) ```yaml # k8s/base/app/units/identity/auth/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: auth labels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth app.kubernetes.io/component: api app.kubernetes.io/part-of: identity-platform app.kubernetes.io/managed-by: argocd annotations: example.com/owner-email: identity-sre@example.com spec: replicas: 2 revisionHistoryLimit: 5 strategy: type: RollingUpdate rollingUpdate: maxSurge: 25% maxUnavailable: 0 selector: matchLabels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth app.kubernetes.io/component: api template: metadata: labels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth app.kubernetes.io/component: api app.kubernetes.io/part-of: identity-platform app.kubernetes.io/managed-by: argocd annotations: prometheus.io/scrape: "true" prometheus.io/port: "8081" prometheus.io/path: "/actuator/prometheus" spec: serviceAccountName: auth automountServiceAccountToken: false securityContext: runAsNonRoot: true runAsUser: 10001 runAsGroup: 10001 fsGroup: 10001 seccompProfile: type: RuntimeDefault terminationGracePeriodSeconds: 45 topologySpreadConstraints: - maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: ScheduleAnyway labelSelector: matchLabels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth app.kubernetes.io/component: api containers: - name: auth image: registry.example.com/auth:placeholder imagePullPolicy: IfNotPresent ports: - name: http containerPort: 8080 protocol: TCP - name: management containerPort: 8081 protocol: TCP env: - name: JAVA_TOOL_OPTIONS value: "-XX:MaxRAMPercentage=75 -XX:+UseG1GC" envFrom: - configMapRef: name: auth-config - secretRef: name: auth-secrets resources: requests: cpu: 250m memory: 512Mi limits: cpu: "2" memory: 1Gi startupProbe: httpGet: path: /actuator/health/liveness port: management periodSeconds: 5 failureThreshold: 30 livenessProbe: httpGet: path: /actuator/health/liveness port: management periodSeconds: 10 timeoutSeconds: 3 failureThreshold: 3 readinessProbe: httpGet: path: /actuator/health/readiness port: management periodSeconds: 5 timeoutSeconds: 2 failureThreshold: 3 securityContext: runAsNonRoot: true runAsUser: 10001 allowPrivilegeEscalation: false readOnlyRootFilesystem: true capabilities: drop: - ALL volumeMounts: - name: tmp mountPath: /tmp - name: cache mountPath: /app/cache volumes: - name: tmp emptyDir: sizeLimit: 64Mi - name: cache emptyDir: sizeLimit: 256Mi ``` **왜 좋은가:** - image는 `:placeholder`, overlay의 `images:`가 digest로 patch → base는 버전 모름 - `revisionHistoryLimit: 5` → 대규모 cluster에서 ReplicaSet 누적 방지 - PodSecurity restricted 호환 (non-root, seccomp RuntimeDefault, capabilities drop ALL, readOnlyRootFilesystem) - startup/liveness/readiness 3종이 타이밍 분리 (startup 150s, liveness 30s, readiness 15s 윈도우) - topologySpreadConstraints로 zone별 분산 - `automountServiceAccountToken: false` (ServiceAccount token을 쓰지 않는 워크로드) --- ## 좋은 예시 3: overlay prod/kr-main — 환경 차이만 ```yaml # k8s/overlays/prod/kr-main/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: prod-identity-auth resources: - ../../../base/app/units/identity/auth components: - ../../../components/with-topology-spread-zone - ../../../components/with-pdb-tier1 labels: - pairs: example.com/environment: prod example.com/region: kr-main example.com/slo-tier: tier-1 includeSelectors: false includeTemplates: true images: - name: registry.example.com/auth digest: "sha256:f1a2b3c4d5e6f7081920aabbccddeeff00112233445566778899aabbccddeeff" replicas: - name: auth count: 6 patches: - target: kind: Deployment name: auth path: patches/auth-resources.yaml - target: kind: Ingress name: auth-public patch: |- - op: replace path: /spec/rules/0/host value: auth.example.com - op: replace path: /spec/tls/0/hosts/0 value: auth.example.com ``` ```yaml # k8s/overlays/prod/kr-main/patches/auth-resources.yaml apiVersion: apps/v1 kind: Deployment metadata: name: auth labels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth spec: selector: matchLabels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth template: metadata: labels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth spec: securityContext: runAsNonRoot: true runAsUser: 10001 runAsGroup: 10001 fsGroup: 10001 seccompProfile: type: RuntimeDefault containers: - name: auth image: registry.example.com/auth-server:1.24.0 resources: requests: cpu: 500m memory: 1Gi limits: cpu: "2" memory: 2Gi securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true capabilities: drop: ["ALL"] ``` **왜 좋은가:** - overlay 자체가 짧음 (base를 재작성하지 않음) - digest 기반 image pinning - `components:`로 zone spread + PDB tier-1을 재사용 - `labels:` 사용, `includeSelectors: false` → selector immutability 안전 - replicas override는 HPA minReplicas와 일치 (HPA base에서 `minReplicas: 6`으로 설정되어 있다고 가정) --- ## 좋은 예시 4: Kustomize Component — `with-pdb-tier1` ```yaml # k8s/components/with-pdb-tier1/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1alpha1 kind: Component resources: - pdb.yaml ``` ```yaml # k8s/components/with-pdb-tier1/pdb.yaml apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: auth labels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth app.kubernetes.io/component: api app.kubernetes.io/part-of: identity-platform app.kubernetes.io/managed-by: argocd example.com/slo-tier: tier-1 spec: minAvailable: 50% unhealthyPodEvictionPolicy: AlwaysAllow selector: matchLabels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth app.kubernetes.io/component: api ``` **왜 좋은가:** - `kind: Component`로 선언 → 여러 overlay에서 `components:` 키로 재사용 - tier-1의 PDB 정책(50% minAvailable)이 단일 파일에 고정 - 다른 tier는 별도 component (`with-pdb-tier2`, `with-pdb-tier3`) --- ## 좋은 예시 5: ConfigMap generator + hash suffix를 활용한 자동 rollout ```yaml # k8s/base/app/units/identity/auth/kustomization.yaml (with generator) apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - deployment.yaml - service.yaml configMapGenerator: - name: auth-config files: - application.yaml=config/application.yaml - logback.xml=config/logback.xml options: labels: app.kubernetes.io/name: auth app.kubernetes.io/component: config generatorOptions: disableNameSuffixHash: false ``` **왜 좋은가:** - ConfigMap 내용 변경 시 hash suffix가 바뀜 → Deployment가 새 이름을 참조 → rolling update 자동 트리거 - annotation 기반 "checksum" hack 불필요 - Secret은 generator로 만들지 않고 External Secrets로 관리 --- ## 좋은 예시 6: HPA v2 + behavior (base 리소스) ```yaml # k8s/base/app/units/identity/auth/hpa.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: auth labels: app.kubernetes.io/name: auth app.kubernetes.io/instance: auth app.kubernetes.io/component: api app.kubernetes.io/part-of: identity-platform app.kubernetes.io/managed-by: argocd spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: auth minReplicas: 2 maxReplicas: 20 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 behavior: scaleDown: stabilizationWindowSeconds: 300 policies: - type: Percent value: 25 periodSeconds: 60 scaleUp: stabilizationWindowSeconds: 0 policies: - type: Percent value: 100 periodSeconds: 30 - type: Pods value: 4 periodSeconds: 30 selectPolicy: Max ``` **왜 좋은가:** - HPA v2 behavior로 scaleDown stabilization (5분) vs scaleUp aggressive (즉시) 분리 - overlay는 `minReplicas` / `maxReplicas`만 override하고 behavior는 상속 --- ## 나쁜 예시 1: `commonLabels`로 environment 주입 → selector immutable 에러 ```yaml # k8s/overlays/prod/kustomization.yaml (BAD) apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: prod-identity-auth resources: - ../../base/app/units/identity/auth commonLabels: example.com/environment: prod ``` **문제:** `commonLabels`는 `spec.selector.matchLabels`에 자동 주입된다. 이미 live 상태인 Deployment/StatefulSet에 apply하면 `The Deployment "auth" is invalid: spec.selector: Invalid value: ...: field is immutable` 로 차단. 해결: `labels:` + `includeSelectors: false`로 교체. --- ## 나쁜 예시 2: overlay가 base를 거의 재작성 ```text k8s/base/app/units/identity/auth/deployment.yaml (150 lines) k8s/overlays/prod/deployment.yaml (140 lines, 95% identical) k8s/overlays/staging/deployment.yaml (140 lines) k8s/overlays/dev/deployment.yaml (135 lines) ``` **문제:** overlay가 base의 95%를 복붙 + 몇 줄 수정. drift 발생 시점부터 base가 의미 없어진다. 해결: overlay는 `patches:` + `images:` + `replicas:` + `labels:`만 쓰고 전체 리소스는 base에서 가져온다. --- ## 나쁜 예시 3: 운영 secret을 `secretGenerator`로 plaintext Git 커밋 ```yaml # k8s/overlays/prod/kustomization.yaml (BAD) secretGenerator: - name: auth-secrets literals: - OAUTH_CLIENT_SECRET=s3cr3t-prod-value - DB_PASSWORD=prod-db-password ``` **문제:** plaintext secret이 Git에 박힌다. 해결: External Secrets Operator + Vault / AWS Secrets Manager / Bitwarden Secrets. 또는 SealedSecrets (public key encrypted). --- ## 나쁜 예시 4: `patchesStrategicMerge` / `patchesJson6902` (deprecated) ```yaml # k8s/overlays/prod/kustomization.yaml (BAD, v5 deprecated) patchesStrategicMerge: - patches/auth-resources.yaml patchesJson6902: - target: group: apps version: v1 kind: Deployment name: auth path: patches/auth-env.yaml ``` **문제:** 두 필드는 Kustomize v5에서 deprecated (여전히 동작하지만 신규 사용 금지). 하나의 `patches:` 필드로 통합되어 strategic merge + JSON patch 양쪽을 지원하므로 혼재할 이유 없음. 해결: `patches:` 단일 키 사용. --- ## 나쁜 예시 5: base에 환경 host / domain 고정 ```yaml # k8s/base/app/units/identity/auth/ingress.yaml (BAD) apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: auth-public spec: rules: - host: auth.example.com # prod host hardcoded in base http: paths: - path: / pathType: Prefix backend: service: name: auth port: number: 8080 ``` **문제:** base가 prod를 전제한다. dev/staging overlay가 host를 교체하려고 `patches:`를 추가해야 하고, base는 더 이상 환경 중립이 아니다. 해결: base에서는 host를 placeholder (`auth.placeholder.invalid`)로 두고 overlay `patches:`에서 주입. --- ## 나쁜 예시 6: `bases:` 사용 (v2.1에서 `resources:`로 통합됨) ```yaml # k8s/overlays/prod/kustomization.yaml (BAD) apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base/app/units/identity/auth ``` **문제:** `bases:`는 v2.1에서 `resources:`에 흡수됨. 신규 코드에서 사용 금지. 해결: `resources:` 사용. --- ## 나쁜 예시 7: HPA가 있는 Deployment에 overlay `replicas:`로 고정값 주입 ```yaml # k8s/overlays/prod/kustomization.yaml (BAD — conflicts with HPA) replicas: - name: auth count: 3 ``` (한편 HPA는 `minReplicas: 6 / maxReplicas: 20`) **문제:** Kustomize가 `replicas: 3`으로 apply → HPA가 즉시 6으로 끌어올림 → 매 ArgoCD sync마다 `out-of-sync` flap. 해결: HPA 활성 리소스에서는 overlay `replicas:`를 쓰지 않고, HPA `minReplicas`를 환경별로 patch.