# config / secrets 예시

모든 YAML은 `kubectl apply --dry-run=server -f -` 기준 clean. VSO는 Helm chart `hashicorp/vault-secrets-operator`로 `vault-secrets-operator` namespace에 설치되어 있고, Vault는 `vault` namespace(`https://vault.vault.svc:8200`)에서 기동 중이며, Kubernetes auth method(`auth/kubernetes`)가 활성화되어 있다고 가정한다.

---

## 좋은 예시 1: 비기밀 ConfigMap (hash-suffixed by Kustomize)

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: auth-server-config
  namespace: auth-prod
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/part-of: identity-platform
data:
  application.yaml: |
    server:
      port: 8080
      shutdown: graceful
    management:
      endpoints:
        web:
          base-path: /actuator
          exposure:
            include: health,info,prometheus
      server:
        port: 9090
    spring:
      main:
        banner-mode: off
      datasource:
        hikari:
          maximum-pool-size: 20
          connection-timeout: 5000
    logging:
      level:
        root: INFO
        com.example.auth: INFO
```

**왜 좋은가:**

- 비밀값은 하나도 없다(username/password/url 제외). Hikari pool size, log level, actuator 경로 같은 operational config만.
- Kustomize `configMapGenerator`로 hash suffix를 붙이면 Deployment가 자동 rollout.

---

## 좋은 예시 2: VSO 전체 스택 (VaultConnection + VaultAuth + VaultStaticSecret + VaultDynamicSecret + VaultPKISecret)

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: auth-prod
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: v1.29
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: auth-server
  namespace: auth-prod
automountServiceAccountToken: true
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultConnection
metadata:
  name: vault
  namespace: auth-prod
spec:
  address: https://vault.vault.svc:8200
  skipTLSVerify: false
  caCertSecretRef: vault-ca-bundle
  headers: {}
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
  name: auth-server
  namespace: auth-prod
spec:
  vaultConnectionRef: vault
  method: kubernetes
  mount: kubernetes
  kubernetes:
    role: auth-server
    serviceAccount: auth-server
    audiences:
      - vault
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: auth-server-oidc-client
  namespace: auth-prod
spec:
  vaultAuthRef: auth-server
  mount: kv
  type: kv-v2
  path: identity/auth-server/prod/oidc
  refreshAfter: 1h
  destination:
    name: auth-server-oidc-client
    create: true
    type: Opaque
  rolloutRestartTargets:
    - kind: Deployment
      name: auth-server
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  name: auth-server-db
  namespace: auth-prod
spec:
  vaultAuthRef: auth-server
  mount: database
  path: creds/auth-server-role
  destination:
    name: auth-server-db
    create: true
    type: Opaque
    transformation:
      templates:
        DB_URL:
          text: 'jdbc:postgresql://identity-postgres.data-prod.svc:5432/auth?user={{ .Secrets.username }}&password={{ .Secrets.password }}&sslmode=require'
        DB_USERNAME:
          text: '{{ .Secrets.username }}'
        DB_PASSWORD:
          text: '{{ .Secrets.password }}'
  rolloutRestartTargets:
    - kind: Deployment
      name: auth-server
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultPKISecret
metadata:
  name: auth-server-internal-tls
  namespace: auth-prod
spec:
  vaultAuthRef: auth-server
  mount: pki_int
  role: auth-server
  commonName: auth-server.auth-prod.svc
  altNames:
    - auth-server.auth-prod.svc.cluster.local
    - auth-server
  ttl: 24h
  destination:
    name: auth-server-internal-tls
    create: true
    type: kubernetes.io/tls
  rolloutRestartTargets:
    - kind: Deployment
      name: auth-server
```

**왜 좋은가:**

- Vault가 source of truth. 모든 비밀이 `kv/identity/auth-server/prod/*` 또는 database/PKI engine에서 발급.
- VSO가 결과물을 표준 Kubernetes Secret(`Opaque`, `kubernetes.io/tls`)으로 materialize.
- Dynamic DB credential은 Postgres role에서 TTL 기반 자동 발급/폐기. Rotation 시 `rolloutRestartTargets`로 Deployment rolling restart.
- PKI Secret은 `kubernetes.io/tls` 타입 → Traefik/앱 TLS에 그대로 소비 가능.

---

## 좋은 예시 3: VSO Secret을 소비하는 Deployment (envFrom + volume 혼합)

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-server
  namespace: auth-prod
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/part-of: identity-platform
    app.kubernetes.io/version: 1.42.0
    app.kubernetes.io/managed-by: argocd
spec:
  replicas: 6
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app.kubernetes.io/name: auth-server
      app.kubernetes.io/instance: auth-server
  template:
    metadata:
      labels:
        app.kubernetes.io/name: auth-server
        app.kubernetes.io/instance: auth-server
        app.kubernetes.io/part-of: identity-platform
        app.kubernetes.io/version: 1.42.0
    spec:
      serviceAccountName: auth-server
      automountServiceAccountToken: true
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: auth-server
          image: registry.example.com/identity/auth-server@sha256:8f3c0a8c6b3a2a7a0f1b2c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f6071
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
            - name: metrics
              containerPort: 9090
          envFrom:
            - configMapRef:
                name: auth-server-config
            - secretRef:
                name: auth-server-db
            - secretRef:
                name: auth-server-oidc-client
          volumeMounts:
            - name: internal-tls
              mountPath: /var/run/secrets/tls
              readOnly: true
            - name: appconfig
              mountPath: /workspace/config
              readOnly: true
            - name: tmp
              mountPath: /tmp
          resources:
            requests:
              cpu: 200m
              memory: 512Mi
            limits:
              cpu: "2"
              memory: 1Gi
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: http
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: http
          securityContext:
            runAsNonRoot: true
            runAsUser: 10001
            runAsGroup: 10001
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
            seccompProfile:
              type: RuntimeDefault
        - name: metrics-exporter
          image: registry.example.com/platform/jmx-exporter@sha256:1111111111111111111111111111111111111111111111111111111111111111
          ports:
            - name: jmx-metrics
              containerPort: 9091
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              cpu: 200m
              memory: 128Mi
          securityContext:
            runAsNonRoot: true
            runAsUser: 10001
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
            seccompProfile:
              type: RuntimeDefault
      volumes:
        - name: internal-tls
          secret:
            secretName: auth-server-internal-tls
            defaultMode: 0400
        - name: appconfig
          configMap:
            name: auth-server-config
        - name: tmp
          emptyDir:
            medium: Memory
            sizeLimit: 64Mi
      imagePullSecrets:
        - name: registry-example-com
```

**왜 좋은가:**

- VSO가 생성한 `auth-server-db`, `auth-server-oidc-client`를 envFrom으로 소비. 앱 코드는 `DB_USERNAME`, `DB_PASSWORD`, `OIDC_CLIENT_SECRET` 환경변수를 읽기만 함.
- TLS private key는 volume(`/var/run/secrets/tls`, mode 0400)으로만 마운트. env 노출 없음.
- `metrics-exporter` sidecar에는 **어떤 secret도 envFrom/volumeMount로 전달하지 않는다**. Scope 최소화.
- image는 digest pin, `imagePullPolicy: IfNotPresent`.

---

## 나쁜 예시 1: plain Secret manifest + ConfigMap에 비밀 혼재

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: auth-server-db
  namespace: auth-prod
type: Opaque
stringData:
  username: prod-admin
  password: S3cur3P@ssw0rd!
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: auth-server-config
  namespace: auth-prod
data:
  application.yaml: |
    spring:
      datasource:
        url: jdbc:postgresql://prod-db:5432/auth
        username: prod-admin
        password: S3cur3P@ssw0rd!
```

**문제:**

- 운영 비밀이 Git에 평문으로 커밋된다. base64/stringData 여부와 무관.
- ConfigMap에 password가 들어가 있음 → RBAC `configmaps:get` 권한을 가진 모든 SA가 읽을 수 있음.
- secret source가 두 곳에 있어 회전 불가능.
- VSO/ESO/SealedSecrets 어느 경로에도 부합하지 않음.

---

## 좋은 예시 4: ImagePullSecret을 VSO로 Vault에서 sync

```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: registry-example-com
  namespace: auth-prod
spec:
  vaultAuthRef: auth-server
  mount: kv
  type: kv-v2
  path: platform/registry/example-com
  refreshAfter: 24h
  destination:
    name: registry-example-com
    create: true
    type: kubernetes.io/dockerconfigjson
    transformation:
      templates:
        .dockerconfigjson:
          text: |
            {
              "auths": {
                "registry.example.com": {
                  "username": "{{ .Secrets.username }}",
                  "password": "{{ .Secrets.password }}",
                  "auth": "{{ printf "%s:%s" .Secrets.username .Secrets.password | b64enc }}"
                }
              }
            }
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: auth-server
  namespace: auth-prod
automountServiceAccountToken: true
imagePullSecrets:
  - name: registry-example-com
```

**왜 좋은가:**

- registry credential도 Vault가 SoT. 하드코딩 없음.
- VSO가 `kubernetes.io/dockerconfigjson` 타입 Secret을 생성. kubelet이 바로 인식.
- SA에 묶여 있어 Deployment마다 imagePullSecrets 반복 선언 불필요.

---

## 좋은 예시 5: cert-manager + VSO 비교 — Ingress TLS는 cert-manager, internal mTLS는 VSO PKI

cert-manager가 외부 공인 도메인용 `kubernetes.io/tls` Secret을 발급하고, VSO `VaultPKISecret`은 internal service mesh mTLS용 단기 인증서를 발급한다. 두 경로 모두 최종 형태는 `kubernetes.io/tls` Secret으로 동일하므로 앱은 secret name만 구분한다.

```yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: auth-example-com
  namespace: auth-prod
spec:
  secretName: auth-example-com-tls
  issuerRef:
    kind: ClusterIssuer
    name: letsencrypt-prod
  dnsNames:
    - auth.example.com
  duration: 2160h
  renewBefore: 360h
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultPKISecret
metadata:
  name: auth-server-internal-tls
  namespace: auth-prod
spec:
  vaultAuthRef: auth-server
  mount: pki_int
  role: auth-server
  commonName: auth-server.auth-prod.svc
  ttl: 24h
  destination:
    name: auth-server-internal-tls
    create: true
    type: kubernetes.io/tls
  rolloutRestartTargets:
    - kind: Deployment
      name: auth-server
```

**왜 좋은가:**

- 외부 ACME 인증서는 공인 CA(Let's Encrypt), 내부는 조직 CA(Vault PKI)로 분리.
- 둘 다 같은 Secret 타입이라 Traefik/앱이 동일하게 소비 가능.
- VSO PKI는 24h TTL로 짧게 회전 → lateral movement window 최소화.

---

## 좋은 예시 6: Vault Agent Injector가 K8s Secret 없이 파일로 템플릿 렌더링

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: legacy-report-generator
  namespace: reports-prod
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: report-generator
  template:
    metadata:
      labels:
        app.kubernetes.io/name: report-generator
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: "report-generator"
        vault.hashicorp.com/agent-inject-secret-report.conf: "kv/data/reports/smtp"
        vault.hashicorp.com/agent-inject-template-report.conf: |
          {{- with secret "kv/data/reports/smtp" -}}
          [smtp]
          host = {{ .Data.data.host }}
          port = {{ .Data.data.port }}
          username = {{ .Data.data.username }}
          password = {{ .Data.data.password }}
          {{- end }}
        vault.hashicorp.com/secret-volume-path-report.conf: "/vault/secrets"
        vault.hashicorp.com/agent-inject-containers: "report-generator"
        vault.hashicorp.com/agent-run-as-user: "10001"
        vault.hashicorp.com/agent-run-as-group: "10001"
    spec:
      serviceAccountName: report-generator
      automountServiceAccountToken: true
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: report-generator
          image: registry.example.com/reports/generator@sha256:2222222222222222222222222222222222222222222222222222222222222222
          ports:
            - name: http
              containerPort: 8080
          volumeMounts:
            - name: tmp
              mountPath: /tmp
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              cpu: 500m
              memory: 512Mi
          securityContext:
            runAsNonRoot: true
            runAsUser: 10001
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
            seccompProfile:
              type: RuntimeDefault
      volumes:
        - name: tmp
          emptyDir:
            medium: Memory
            sizeLimit: 32Mi
```

**왜 좋은가:**

- K8s Secret object가 **생성되지 않는다**. RBAC audit이 Secret API 호출 없이 Vault audit log로 대체된다.
- Vault Agent sidecar가 tmpfs에 템플릿 렌더링 → 앱은 파일만 읽음.
- legacy 앱이 INI/TOML 포맷 설정 파일을 요구할 때 적합.

**VSO vs Vault Agent Injector:**

| 항목 | VSO | Vault Agent Injector |
|---|---|---|
| 결과 | K8s Secret | Pod tmpfs 파일 |
| K8s API 노출 | Secret object 존재 | 없음 |
| 소비 방식 | envFrom/volume | file read |
| 회전 시 | `rolloutRestartTargets` | Agent re-render(인메모리) |
| 복잡도 | 낮음(CRD만) | 높음(sidecar/init) |
| 권장 | **운영 기본** | 템플릿/legacy 앱 |

---

## 나쁜 예시 2: Vault Injector annotation을 모든 컨테이너에 적용 + env 렌더링

```yaml
metadata:
  annotations:
    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/role: "auth-server"
    vault.hashicorp.com/agent-inject-secret-db: "kv/data/auth-server/db"
    vault.hashicorp.com/agent-inject-template-db: |
      {{- with secret "kv/data/auth-server/db" -}}
      export DB_USERNAME={{ .Data.data.username }}
      export DB_PASSWORD={{ .Data.data.password }}
      {{- end }}
```

**문제:**

- `agent-inject-containers` 미지정 → sidecar(metrics, proxy) 포함 모든 컨테이너의 `/vault/secrets`가 보임.
- `export DB_PASSWORD=...`를 `source`로 읽는 launcher 스크립트 → process env로 비밀이 흘러 `/proc/<pid>/environ` 노출.
- dynamic lease renew를 활용하지 못하고, 회전 시 rollout trigger 없음.

---

## 좋은 예시 7: SealedSecret (VSO 미도입 환경/bootstrap)

```yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: vault-bootstrap-token
  namespace: vault
spec:
  encryptedData:
    token: AgCd9sK... (public key로 암호화된 blob)
  template:
    metadata:
      name: vault-bootstrap-token
      namespace: vault
    type: Opaque
```

**왜 좋은가:**

- Git 커밋 가능(public key로 암호화, cluster controller만 복호화).
- VSO 자체를 기동하기 위한 bootstrap credential(Vault root token, unseal key 대신 KMS auto-unseal 권장)에 적합.
- SealedSecrets controller가 `Secret`을 namespace에 materialize.

**주의:**

- 운영에서 **VSO가 기동되면 SealedSecrets 경로는 최소화**. 이중 source of truth 방지.
- Key 회전은 controller의 sealing key rotation 절차 준수.

---

## 좋은 예시 8: EncryptionConfiguration for Secret at-rest (API Server 레벨)

```yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - kms:
          apiVersion: v2
          name: platform-kms-v2
          endpoint: unix:///var/run/kmsplugin/socket.sock
          timeout: 3s
      - aescbc:
          keys:
            - name: fallback-2026-q1
              secret: <kms-fallback-key-base64>
      - identity: {}
```

**K3s 활성화:**

```yaml
# /etc/rancher/k3s/config.yaml
secrets-encryption: true
kube-apiserver-arg:
  - "encryption-provider-config=/etc/rancher/k3s/encryption-config.yaml"
  - "audit-policy-file=/etc/rancher/k3s/audit-policy.yaml"
  - "audit-log-path=/var/log/k3s-audit.log"
```

**왜 좋은가:**

- KMS v2 provider가 primary → envelope encryption, 키는 KMS 외부에 존재.
- `aescbc`는 fallback. `identity`는 마지막(평문), 기존 Secret을 재암호화하기 전 decryption용.
- K3s config.yaml에 `secrets-encryption: true`로 선언. 서버 재시작 후 `kubectl get secrets -A -o json | kubectl replace -f -`로 기존 Secret 재암호화.

---

## 나쁜 예시 3: Kustomize secretGenerator로 운영 비밀 literal

```yaml
# overlays/prod/kustomization.yaml
secretGenerator:
  - name: auth-server-db
    literals:
      - username=prod-admin
      - password=S3cur3P@ssw0rd!
```

**문제:**

- 운영 비밀이 Git에 literal 평문 저장.
- Kustomize hash suffix는 비밀 보호가 아님.
- 회전 시 매번 Git 커밋 필요(감사/리뷰 시 비밀 노출).
- 운영은 VSO/ESO/SealedSecrets 경로로만 비밀을 배포해야 한다.