donghyeon.kang/project-infra
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Files
da159dcd222c8178e4cb57096dfb753363c66b65
project-infra/docs/examples/infra/workload-selection.md
T

992 lines
32 KiB
Markdown
Raw Blame History

# workload selection 예시
모든 YAML은 `kubectl apply --server-side --dry-run=server` 통과. PodSecurity `restricted` 호환.
각 예시는 namespace 하나에 그대로 붙여 넣을 수 있는 self-contained 단위.
---
## 좋은 예시 1: auth-server Deployment (tier-1 prod, 완전 동반 리소스)
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: auth
namespace: prod-identity-auth
labels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
automountServiceAccountToken: false
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth
namespace: prod-identity-auth
labels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/version: "1.24.3"
app.kubernetes.io/component: api
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
example.com/slo-tier: tier-1
spec:
replicas: 6
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 25%
maxUnavailable: 0
selector:
matchLabels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
template:
metadata:
labels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
example.com/slo-tier: tier-1
spec:
serviceAccountName: auth
automountServiceAccountToken: false
priorityClassName: tier-1-critical
terminationGracePeriodSeconds: 45
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile:
type: RuntimeDefault
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
containers:
- name: auth
image: registry.example.com/auth@sha256:f1a2b3c4d5e6f7081920aabbccddeeff00112233445566778899aabbccddeeff
imagePullPolicy: IfNotPresent
ports:
- { name: http, containerPort: 8080, protocol: TCP }
- { name: management, containerPort: 8081, protocol: TCP }
resources:
requests: { cpu: 500m, memory: 1Gi }
limits: { cpu: "2", memory: 2Gi }
startupProbe:
httpGet: { path: /actuator/health/liveness, port: management }
periodSeconds: 5
failureThreshold: 30
livenessProbe:
httpGet: { path: /actuator/health/liveness, port: management }
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
readinessProbe:
httpGet: { path: /actuator/health/readiness, port: management }
periodSeconds: 5
timeoutSeconds: 2
failureThreshold: 3
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 15"]
securityContext:
runAsNonRoot: true
runAsUser: 10001
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ "ALL" ]
volumeMounts:
- { name: tmp, mountPath: /tmp }
volumes:
- name: tmp
emptyDir: { sizeLimit: 64Mi }
---
apiVersion: v1
kind: Service
metadata:
name: auth
namespace: prod-identity-auth
labels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
ports:
- { name: http, port: 8080, targetPort: http }
- { name: management, port: 8081, targetPort: management }
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: auth
namespace: prod-identity-auth
labels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
example.com/slo-tier: tier-1
spec:
minAvailable: 50%
unhealthyPodEvictionPolicy: AlwaysAllow
selector:
matchLabels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: auth
namespace: prod-identity-auth
labels:
app.kubernetes.io/name: auth
app.kubernetes.io/instance: auth-prod
app.kubernetes.io/component: api
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: auth
minReplicas: 6
maxReplicas: 30
metrics:
- type: Resource
resource:
name: cpu
target: { type: Utilization, averageUtilization: 70 }
- type: Resource
resource:
name: memory
target: { type: Utilization, averageUtilization: 80 }
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- { type: Percent, value: 25, periodSeconds: 60 }
scaleUp:
stabilizationWindowSeconds: 0
policies:
- { type: Percent, value: 100, periodSeconds: 30 }
- { type: Pods, value: 4, periodSeconds: 30 }
selectPolicy: Max
```
**왜 좋은가:**
- stateless 장기 실행 → Deployment 정답
- PDB + HPA + topologySpread (zone+hostname) 모두 tier-1에 맞게 동반
- digest pinning, restricted PodSecurity 호환, preStop sleep으로 graceful drain
- selector에는 불변 2종만 (`name`/`instance`; version/environment 없음)
---
## 좋은 예시 2: PostgreSQL StatefulSet (operator 없는 fallback 케이스, 완전 schema)
```yaml
apiVersion: v1
kind: Service
metadata:
name: postgres-identity-headless
namespace: prod-data-postgres
labels:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
clusterIP: None
selector:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
ports:
- { name: postgres, port: 5432, targetPort: postgres }
---
apiVersion: v1
kind: Service
metadata:
name: postgres-identity
namespace: prod-data-postgres
labels:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
ports:
- { name: postgres, port: 5432, targetPort: postgres }
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-identity
namespace: prod-data-postgres
labels:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/version: "16.3"
app.kubernetes.io/component: database
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
example.com/slo-tier: tier-1
example.com/data-classification: confidential
spec:
serviceName: postgres-identity-headless
replicas: 3
podManagementPolicy: OrderedReady
updateStrategy:
type: RollingUpdate
rollingUpdate:
partition: 0
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
revisionHistoryLimit: 5
selector:
matchLabels:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
template:
metadata:
labels:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
serviceAccountName: postgres-identity
automountServiceAccountToken: false
priorityClassName: tier-1-critical
terminationGracePeriodSeconds: 120
securityContext:
runAsNonRoot: true
runAsUser: 999
runAsGroup: 999
fsGroup: 999
seccompProfile:
type: RuntimeDefault
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
containers:
- name: postgres
image: registry.example.com/postgres@sha256:aabbccddeeff00112233445566778899aabbccddeeff0011223344556677abcd
imagePullPolicy: IfNotPresent
ports:
- { name: postgres, containerPort: 5432, protocol: TCP }
env:
- name: POSTGRES_DB
value: identity
- name: POSTGRES_USER
valueFrom:
secretKeyRef: { name: postgres-identity-creds, key: username }
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef: { name: postgres-identity-creds, key: password }
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
resources:
requests: { cpu: "2", memory: 4Gi }
limits: { cpu: "4", memory: 8Gi }
startupProbe:
exec:
command: ["pg_isready", "-U", "$(POSTGRES_USER)", "-d", "$(POSTGRES_DB)"]
periodSeconds: 5
failureThreshold: 60
livenessProbe:
exec:
command: ["pg_isready", "-U", "$(POSTGRES_USER)", "-d", "$(POSTGRES_DB)"]
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
exec:
command: ["pg_isready", "-U", "$(POSTGRES_USER)", "-d", "$(POSTGRES_DB)"]
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
securityContext:
runAsNonRoot: true
runAsUser: 999
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ "ALL" ]
volumeMounts:
- { name: data, mountPath: /var/lib/postgresql/data }
- { name: tmp, mountPath: /tmp }
- { name: run, mountPath: /var/run/postgresql }
volumes:
- name: tmp
emptyDir: {}
- name: run
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: data
labels:
app.kubernetes.io/name: postgres
app.kubernetes.io/instance: postgres-identity
app.kubernetes.io/component: database
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: longhorn-replicated
resources:
requests:
storage: 200Gi
```
**왜 좋은가:**
- headless Service + clusterIP Service 양쪽 선언 (peer discovery + 일반 client)
- `persistentVolumeClaimRetentionPolicy: {whenDeleted: Retain, whenScaled: Retain}` 명시 (GA 1.27)
- `podManagementPolicy: OrderedReady` + `updateStrategy.partition: 0` (canary 시 1씩)
- zone topologySpread `DoNotSchedule`로 강제 (DB는 AZ 분산이 강건성 핵심)
- Secret 외부 참조 (External Secrets로 관리 가정)
- StorageClass `longhorn-replicated` (local-path 금지)
**실전 주의**: 1000-서비스 스케일에서는 raw StatefulSet 대신 **CloudNativePG Operator** 사용을 강력 권장. 이 예시는 operator 불가 케이스의 reference.
---
## 좋은 예시 3: fluent-bit DaemonSet (로그 shipper)
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluent-bit
namespace: prod-platform-observability
labels:
app.kubernetes.io/name: fluent-bit
app.kubernetes.io/instance: fluent-bit-prod
app.kubernetes.io/component: log-shipper
app.kubernetes.io/part-of: observability-platform
app.kubernetes.io/managed-by: argocd
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluent-bit
namespace: prod-platform-observability
labels:
app.kubernetes.io/name: fluent-bit
app.kubernetes.io/instance: fluent-bit-prod
app.kubernetes.io/version: "3.1.7"
app.kubernetes.io/component: log-shipper
app.kubernetes.io/part-of: observability-platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
spec:
selector:
matchLabels:
app.kubernetes.io/name: fluent-bit
app.kubernetes.io/instance: fluent-bit-prod
app.kubernetes.io/component: log-shipper
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
revisionHistoryLimit: 5
template:
metadata:
labels:
app.kubernetes.io/name: fluent-bit
app.kubernetes.io/instance: fluent-bit-prod
app.kubernetes.io/component: log-shipper
app.kubernetes.io/part-of: observability-platform
app.kubernetes.io/managed-by: argocd
spec:
serviceAccountName: fluent-bit
automountServiceAccountToken: true
priorityClassName: system-node-critical
hostNetwork: false
terminationGracePeriodSeconds: 30
tolerations:
- operator: Exists
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: fluent-bit
image: registry.example.com/fluent-bit@sha256:112233445566778899aabbccddeeff00112233445566778899aabbccddeeff00
imagePullPolicy: IfNotPresent
ports:
- { name: metrics, containerPort: 2020, protocol: TCP }
resources:
requests: { cpu: 100m, memory: 128Mi }
limits: { cpu: 500m, memory: 256Mi }
livenessProbe:
httpGet: { path: /, port: metrics }
periodSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet: { path: /api/v1/health, port: metrics }
periodSeconds: 5
failureThreshold: 3
securityContext:
runAsNonRoot: true
runAsUser: 10001
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ "ALL" ]
add: [ "DAC_READ_SEARCH" ]
volumeMounts:
- { name: varlog, mountPath: /var/log, readOnly: true }
- { name: varlibdockercontainers, mountPath: /var/lib/docker/containers, readOnly: true }
- { name: config, mountPath: /fluent-bit/etc }
volumes:
- name: varlog
hostPath: { path: /var/log, type: Directory }
- name: varlibdockercontainers
hostPath: { path: /var/lib/docker/containers, type: DirectoryOrCreate }
- name: config
configMap: { name: fluent-bit-config }
```
**왜 좋은가:**
- DaemonSet으로 모든 노드에 정확히 1 Pod
- `tolerations: Exists`로 control-plane taint 포함 모든 노드 커버
- `priorityClassName: system-node-critical`로 eviction 방지
- root 필요(hostPath 로그 읽기) 하지만 capabilities는 `DAC_READ_SEARCH`만 추가하고 나머지 drop
- `updateStrategy.rollingUpdate.maxUnavailable: 10%`로 대규모 클러스터 rolling 안정화
### ❌ Bad counterpart (같은 역할을 Deployment로)
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: fluent-bit
spec:
replicas: 10
```
**문제:** Deployment는 특정 노드에 Pod가 없을 수 있고, 같은 노드에 여러 Pod가 떠서 로그 중복 수집. DaemonSet만 "노드당 정확히 1"을 보장.
---
## 좋은 예시 4: flyway-migrate Job (ArgoCD PostSync hook)
```yaml
apiVersion: batch/v1
kind: Job
metadata:
name: flyway-migrate-identity-1-24-3
namespace: prod-identity-auth
labels:
app.kubernetes.io/name: flyway
app.kubernetes.io/instance: flyway-identity-1-24-3
app.kubernetes.io/version: "1.24.3"
app.kubernetes.io/component: schema-migration
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
annotations:
argocd.argoproj.io/hook: PostSync
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
spec:
backoffLimit: 2
activeDeadlineSeconds: 600
ttlSecondsAfterFinished: 86400
template:
metadata:
labels:
app.kubernetes.io/name: flyway
app.kubernetes.io/instance: flyway-identity-1-24-3
app.kubernetes.io/component: schema-migration
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
restartPolicy: Never
serviceAccountName: flyway-identity
automountServiceAccountToken: false
priorityClassName: tier-1-critical
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: flyway
image: registry.example.com/flyway@sha256:ccddeeff0011223344556677889900aabbccddeeff0011223344556677889900
imagePullPolicy: IfNotPresent
args: [ "migrate" ]
env:
- { name: FLYWAY_URL, valueFrom: { secretKeyRef: { name: flyway-identity-creds, key: url } } }
- { name: FLYWAY_USER, valueFrom: { secretKeyRef: { name: flyway-identity-creds, key: username } } }
- { name: FLYWAY_PASSWORD, valueFrom: { secretKeyRef: { name: flyway-identity-creds, key: password } } }
resources:
requests: { cpu: 200m, memory: 256Mi }
limits: { cpu: "1", memory: 512Mi }
securityContext:
runAsNonRoot: true
runAsUser: 1000
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ "ALL" ]
volumeMounts:
- { name: tmp, mountPath: /tmp }
volumes:
- name: tmp
emptyDir: { sizeLimit: 32Mi }
```
**왜 좋은가:**
- `restartPolicy: Never` + `backoffLimit: 2` → migration 실패는 debug 가능하게 노출
- `activeDeadlineSeconds: 600` → 무한 lock 방지
- `ttlSecondsAfterFinished: 86400` → 24시간 후 자동 정리 (1000-서비스 스케일 필수)
- ArgoCD `PostSync` hook으로 Deployment rollout 이후 실행
- 이름에 버전 suffix (`-1-24-3`) → 같은 이름 Job 재생성 충돌 방지
---
## 좋은 예시 5: postgres-backup CronJob (timezone + concurrencyPolicy)
```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: postgres-identity-backup
namespace: prod-data-postgres
labels:
app.kubernetes.io/name: postgres-backup
app.kubernetes.io/instance: postgres-identity-backup
app.kubernetes.io/component: backup
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
spec:
schedule: "0 */6 * * *"
timeZone: "Asia/Seoul"
concurrencyPolicy: Forbid
startingDeadlineSeconds: 600
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 5
jobTemplate:
spec:
backoffLimit: 1
activeDeadlineSeconds: 3600
template:
metadata:
labels:
app.kubernetes.io/name: postgres-backup
app.kubernetes.io/instance: postgres-identity-backup
app.kubernetes.io/component: backup
app.kubernetes.io/part-of: identity-platform
app.kubernetes.io/managed-by: argocd
spec:
restartPolicy: Never
serviceAccountName: postgres-identity-backup
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 999
runAsGroup: 999
fsGroup: 999
seccompProfile:
type: RuntimeDefault
containers:
- name: pgbackup
image: registry.example.com/pgbackup@sha256:ddeeff00112233445566778899aabbccddeeff00112233445566778899aabbcc
imagePullPolicy: IfNotPresent
env:
- { name: PGHOST, value: postgres-identity.prod-data-postgres.svc.cluster.local }
- { name: PGUSER, valueFrom: { secretKeyRef: { name: pgbackup-creds, key: username } } }
- { name: PGPASSWORD, valueFrom: { secretKeyRef: { name: pgbackup-creds, key: password } } }
- { name: S3_BUCKET, value: pg-backups-prod }
- { name: S3_PREFIX, value: identity }
resources:
requests: { cpu: 200m, memory: 256Mi }
limits: { cpu: "1", memory: 512Mi }
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ "ALL" ]
volumeMounts:
- { name: tmp, mountPath: /tmp }
volumes:
- name: tmp
emptyDir: { sizeLimit: 2Gi }
```
**왜 좋은가:**
- `timeZone: Asia/Seoul` (v1.25+) → DST 안정
- `concurrencyPolicy: Forbid` → 이전 백업이 돌고 있으면 skip
- `startingDeadlineSeconds: 600` → 노드 장애 후 미싱 누적 제한
- history limit으로 완료 Job 정리 (`successfulJobsHistoryLimit: 3`, `failedJobsHistoryLimit: 5`)
- CronJob의 jobTemplate에는 `ttlSecondsAfterFinished`를 설정하지 않음 — history limit과 중복/충돌 방지 (standalone Job에서만 사용)
**실전 주의**: CloudNativePG `ScheduledBackup` CRD를 쓰면 이 CronJob을 operator가 대체한다.
---
## 좋은 예시 6: Ingress controller Deployment + MetalLB (prod 기본)
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ingress-nginx-public
namespace: prod-platform-ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx-public-prod
app.kubernetes.io/version: "1.11.2"
app.kubernetes.io/component: controller
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: argocd
example.com/environment: prod
example.com/exposure: public
example.com/slo-tier: tier-1
spec:
replicas: 3
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx-public-prod
app.kubernetes.io/component: controller
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx-public-prod
app.kubernetes.io/component: controller
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: argocd
spec:
serviceAccountName: ingress-nginx
automountServiceAccountToken: true
priorityClassName: system-cluster-critical
terminationGracePeriodSeconds: 300
securityContext:
runAsNonRoot: true
runAsUser: 101
fsGroup: 101
seccompProfile: { type: RuntimeDefault }
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx-public-prod
app.kubernetes.io/component: controller
containers:
- name: controller
image: registry.example.com/ingress-nginx@sha256:eeff00112233445566778899aabbccddeeff00112233445566778899aabbccdd
args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-public
- --election-id=ingress-nginx-public-leader
- --controller-class=k8s.io/ingress-nginx-public
- --ingress-class=nginx-public
- --configmap=$(POD_NAMESPACE)/ingress-nginx-public
env:
- { name: POD_NAMESPACE, valueFrom: { fieldRef: { fieldPath: metadata.namespace } } }
- { name: POD_NAME, valueFrom: { fieldRef: { fieldPath: metadata.name } } }
ports:
- { name: http, containerPort: 80, protocol: TCP }
- { name: https, containerPort: 443, protocol: TCP }
- { name: metrics, containerPort: 10254, protocol: TCP }
resources:
requests: { cpu: 500m, memory: 512Mi }
limits: { cpu: "2", memory: 1Gi }
livenessProbe:
httpGet: { path: /healthz, port: 10254, scheme: HTTP }
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet: { path: /healthz, port: 10254, scheme: HTTP }
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
lifecycle:
preStop:
exec:
command: ["/wait-shutdown"]
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 101
capabilities:
drop: [ "ALL" ]
add: [ "NET_BIND_SERVICE" ]
volumeMounts:
- { name: tmp, mountPath: /tmp }
- { name: nginx-etc, mountPath: /etc/nginx }
- { name: nginx-cache, mountPath: /var/cache/nginx }
volumes:
- name: tmp
emptyDir: {}
- name: nginx-etc
emptyDir: {}
- name: nginx-cache
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx-public
namespace: prod-platform-ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx-public-prod
app.kubernetes.io/component: controller
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: argocd
annotations:
metallb.universe.tf/address-pool: prod-public-pool
metallb.universe.tf/allow-shared-ip: "ingress-nginx-public"
spec:
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx-public-prod
app.kubernetes.io/component: controller
ports:
- { name: http, port: 80, targetPort: http, protocol: TCP }
- { name: https, port: 443, targetPort: https, protocol: TCP }
```
**왜 좋은가:**
- Deployment + LoadBalancer (MetalLB L2/BGP) → HPA 가능, 노드 수 ≠ replica 수
- `externalTrafficPolicy: Local` → source IP 보존 + 노드 horn-in 회피
- `priorityClassName: system-cluster-critical`
- `NET_BIND_SERVICE` capability만 추가 (80/443 바인딩), 나머지 drop
- public / internal IngressClass 분리 가능 (별도 Deployment)
**DaemonSet 선택이 맞는 케이스**: bare-metal + 외부 LB 없음 + 모든 edge 노드가 고정 IP로 80/443 직접 노출. 이 경우 `hostNetwork: true` + DaemonSet + `tolerations`로 edge 노드만 label selector.
---
## 나쁜 예시 1: auth-server를 StatefulSet으로
```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: auth-server
spec:
serviceName: auth-server
replicas: 3
selector:
matchLabels: { app: auth-server }
template:
metadata: { labels: { app: auth-server } }
spec:
containers:
- name: auth
image: registry.example.com/auth:1.24.3
```
**문제:** stable identity/storage 요구가 없는 stateless 앱에 StatefulSet. rolling update가 OrderedReady로 느려지고, replica 증설 시 `auth-server-2`, `auth-server-3` 이름이 의미 없이 고정된다. 해결: Deployment + HPA.
---
## 나쁜 예시 2: postgres를 Deployment로
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
spec:
replicas: 1
template:
spec:
containers:
- name: postgres
image: postgres:16
volumeMounts:
- { name: data, mountPath: /var/lib/postgresql/data }
volumes:
- name: data
persistentVolumeClaim: { claimName: postgres-pvc }
```
**문제:** replica=1 Deployment + 단일 PVC는 rolling update 시 잠깐 새 Pod가 뜨면서 같은 PVC에 두 Pod가 붙으려다 RWO 충돌. StatefulSet이면 `OrderedReady`로 이전 Pod 완전히 내려간 다음 새 Pod가 뜬다. 해결: StatefulSet + volumeClaimTemplates + podManagementPolicy: OrderedReady.
---
## 나쁜 예시 3: fluent-bit을 Deployment로 배포
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: fluent-bit
spec:
replicas: 10
template:
spec:
containers:
- name: fluent-bit
image: fluent/fluent-bit:latest
volumeMounts:
- { name: varlog, mountPath: /var/log }
volumes:
- name: varlog
hostPath: { path: /var/log }
```
**문제:** Deployment는 Pod 배치를 스케줄러에 맡김 → 어떤 노드에는 0 Pod(로그 유실), 어떤 노드에는 2 Pod(중복 수집). 노드 수가 바뀌면 수동으로 replicas 조정. 해결: DaemonSet.
---
## 나쁜 예시 4: flyway를 앱 startup에 포함
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth
spec:
template:
spec:
initContainers:
- name: flyway-migrate
image: registry.example.com/flyway:1.24.3
args: ["migrate"]
containers:
- name: auth
image: registry.example.com/auth:1.24.3
```
**문제:** Deployment scale-up 때마다 모든 새 Pod가 migration 시도 → DB lock 경합. migration 실패가 앱 부팅 실패로 섞여 debug 불가. rollback 시 downgrade migration 제어 불가. 해결: 독립 Job + ArgoCD PostSync hook.
---
## 나쁜 예시 5: CronJob에 `concurrencyPolicy`와 `startingDeadlineSeconds` 누락
```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: backup
spec:
schedule: "*/5 * * * *"
jobTemplate:
spec:
template:
spec:
restartPolicy: OnFailure
containers:
- name: backup
image: registry.example.com/backup:1.0.0
```
**문제:**
1. `concurrencyPolicy` 미지정 → 기본 `Allow` → 이전 backup이 오래 걸리면 중복 실행, PVC lock 충돌
2. `startingDeadlineSeconds` 미지정 → 노드 장애 후 수십 개 missed Job이 한꺼번에 생성
3. history limit 미지정 → 완료 Job이 무한 누적
해결: 모든 prod CronJob에 `concurrencyPolicy: Forbid` + `startingDeadlineSeconds: <short>` + history limit.
---
## 나쁜 예시 6: StatefulSet에 `persistentVolumeClaimRetentionPolicy` 미지정 (prod)
```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: vault
spec:
serviceName: vault
replicas: 3
# persistentVolumeClaimRetentionPolicy missing
volumeClaimTemplates:
- metadata: { name: data }
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: longhorn-replicated
resources:
requests: { storage: 20Gi }
```
**문제:** 명시가 없으면 기본값 (`{whenDeleted: Retain, whenScaled: Retain}`)이 적용되어 "동작은 맞지만" 의도가 코드에 드러나지 않는다. 팀원이 `{Delete, Delete}`인지 추측. 1000-서비스 스케일에서는 모든 StatefulSet이 이 필드를 **명시**해야 정책이 audit 가능. 해결: 항상 명시.
Reference in New Issue View Git Blame Copy Permalink