project-infra/docs/examples/infra/flyway.md

Flyway 예시

전 예시는 kubectl apply -f 가능한 완성 매니페스트다. 1000+ 서비스 규모에서 복사/수정해 쓸 수 있도록 full manifest로 구성했다.

---

좋은 예시 1: 완전한 Flyway Job (Helm hook 패턴)

(1) ConfigMap — migration SQL

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: auth-flyway-sql
  namespace: auth-prod
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/component: db-migration
    app.kubernetes.io/part-of: auth-platform
    app.kubernetes.io/managed-by: Helm
data:
  V1__init_auth_schema.sql: |
    CREATE TABLE IF NOT EXISTS users (
        id           bigserial PRIMARY KEY,
        email        text NOT NULL,
        display_name text,
        created_at   timestamptz NOT NULL DEFAULT now()
    );
    CREATE UNIQUE INDEX IF NOT EXISTS idx_users_email ON users(lower(email));
  V2__add_refresh_tokens.sql: |
    CREATE TABLE IF NOT EXISTS refresh_tokens (
        id         bigserial PRIMARY KEY,
        user_id    bigint NOT NULL REFERENCES users(id) ON DELETE CASCADE,
        token_hash bytea NOT NULL,
        expires_at timestamptz NOT NULL,
        created_at timestamptz NOT NULL DEFAULT now()
    );
    CREATE INDEX IF NOT EXISTS idx_refresh_tokens_user_id ON refresh_tokens(user_id);
  V3__add_last_login_column.sql: |
    ALTER TABLE users ADD COLUMN IF NOT EXISTS last_login_at timestamptz;
  V4__create_idx_last_login_concurrently.sql: |
    -- flyway:executeInTransaction=false
    -- Long-running DDL. Schedule in low-traffic window.
    CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_users_last_login
        ON users(last_login_at);
  R__refresh_active_users_view.sql: |
    CREATE OR REPLACE VIEW active_users AS
    SELECT id, email, display_name, last_login_at
      FROM users
     WHERE last_login_at > now() - interval '30 days';

(2) Vault Secrets Operator — DB password

---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: auth-pg-app
  namespace: auth-prod
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/component: db-migration
spec:
  type: kv-v2
  mount: kv
  path: auth-prod/postgres/app
  destination:
    name: auth-pg-app
    create: true
    type: Opaque
  refreshAfter: 1h
  vaultAuthRef: vault-auth-auth-prod

(3) Flyway Job — pre-upgrade / pre-install

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: auth-flyway
  namespace: auth-prod
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/component: db-migration
---
apiVersion: batch/v1
kind: Job
metadata:
  name: auth-flyway-migrate
  namespace: auth-prod
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/instance: auth-server-prod
    app.kubernetes.io/component: db-migration
    app.kubernetes.io/part-of: auth-platform
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "2026.04.16"
  annotations:
    "helm.sh/hook": "pre-upgrade,pre-install"
    "helm.sh/hook-weight": "-10"
    "helm.sh/hook-delete-policy": "before-hook-creation,hook-succeeded"
spec:
  parallelism: 1
  completions: 1
  backoffLimit: 0
  activeDeadlineSeconds: 1800
  ttlSecondsAfterFinished: 86400
  template:
    metadata:
      labels:
        app.kubernetes.io/name: auth-server
        app.kubernetes.io/component: db-migration
    spec:
      serviceAccountName: auth-flyway
      restartPolicy: Never
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
        seccompProfile: {type: RuntimeDefault}
      initContainers:
        - name: flyway-info
          image: flyway/flyway@sha256:7d9f7c4e2a1b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d
          imagePullPolicy: IfNotPresent
          args: ["info"]
          env: &flywayEnv
            - {name: FLYWAY_URL,                 value: "jdbc:postgresql://auth-pg-rw.data-prod.svc:5432/auth?sslmode=require"}
            - {name: FLYWAY_USER,                value: "auth_app"}
            - {name: FLYWAY_LOCATIONS,           value: "filesystem:/flyway/sql"}
            - {name: FLYWAY_SCHEMAS,             value: "auth_server"}
            - {name: FLYWAY_DEFAULT_SCHEMA,      value: "auth_server"}
            - {name: FLYWAY_TABLE,               value: "flyway_schema_history"}
            - {name: FLYWAY_VALIDATE_ON_MIGRATE, value: "true"}
            - {name: FLYWAY_BASELINE_ON_MIGRATE, value: "false"}
            - {name: FLYWAY_OUT_OF_ORDER,        value: "false"}
            - {name: FLYWAY_MIXED,               value: "false"}
            - {name: FLYWAY_CLEAN_DISABLED,      value: "true"}
            - name: FLYWAY_PASSWORD
              valueFrom: {secretKeyRef: {name: auth-pg-app, key: password}}
          resources:
            requests: {cpu: "50m",  memory: "128Mi"}
            limits:   {cpu: "500m", memory: "512Mi"}
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities: {drop: ["ALL"]}
          volumeMounts:
            - {name: sql, mountPath: /flyway/sql, readOnly: true}
            - {name: tmp, mountPath: /tmp}
        - name: flyway-validate
          image: flyway/flyway@sha256:7d9f7c4e2a1b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d
          imagePullPolicy: IfNotPresent
          args: ["validate"]
          env: *flywayEnv
          resources:
            requests: {cpu: "50m",  memory: "128Mi"}
            limits:   {cpu: "500m", memory: "512Mi"}
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities: {drop: ["ALL"]}
          volumeMounts:
            - {name: sql, mountPath: /flyway/sql, readOnly: true}
            - {name: tmp, mountPath: /tmp}
      containers:
        - name: flyway-migrate
          image: flyway/flyway@sha256:7d9f7c4e2a1b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d
          imagePullPolicy: IfNotPresent
          args: ["-X", "migrate"]
          env: *flywayEnv
          resources:
            requests: {cpu: "100m", memory: "256Mi"}
            limits:   {cpu: "1",    memory: "1Gi"}
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities: {drop: ["ALL"]}
          volumeMounts:
            - {name: sql, mountPath: /flyway/sql, readOnly: true}
            - {name: tmp, mountPath: /tmp}
      volumes:
        - name: sql
          configMap: {name: auth-flyway-sql}
        - name: tmp
          emptyDir: {}

왜 좋은가:

• Helm hook으로 app Deployment보다 먼저 실행 (pre-upgrade,pre-install, weight -10)
• before-hook-creation,hook-succeeded 삭제 정책으로 과거 Job 정리
• initContainer로 info + validate를 먼저 실행해 실패를 앞당김
• 메인 container에서 migrate (advisory lock 덕분에 같은 Job이 중복 실행돼도 직렬화됨)
• FLYWAY_CLEAN_DISABLED=true (production 필수)
• FLYWAY_BASELINE_ON_MIGRATE=false, FLYWAY_OUT_OF_ORDER=false
• digest pinning, restricted PSA, anchor/alias로 env 중복 제거
• parallelism: 1, backoffLimit: 0, activeDeadlineSeconds: 1800

---

좋은 예시 2: Argo CD sync-wave 패턴

---
apiVersion: batch/v1
kind: Job
metadata:
  name: auth-flyway-migrate
  namespace: auth-prod
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/component: db-migration
    app.kubernetes.io/managed-by: argocd
  annotations:
    argocd.argoproj.io/sync-wave: "-1"
    argocd.argoproj.io/hook: Sync
    argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
spec:
  parallelism: 1
  completions: 1
  backoffLimit: 0
  activeDeadlineSeconds: 1800
  ttlSecondsAfterFinished: 86400
  template:
    spec:
      restartPolicy: Never
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      containers:
        # (containers 세부는 예시 1과 동일; 요지만 재현)
        - name: flyway
          image: flyway/flyway@sha256:7d9f7c4e2a1b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d
          args: ["-X", "migrate"]
          resources:
            requests: { cpu: 100m, memory: 256Mi }
            limits:   { memory: 1Gi }
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-server
  namespace: auth-prod
  annotations:
    argocd.argoproj.io/sync-wave: "0"
  labels:
    app.kubernetes.io/name: auth-server
    app.kubernetes.io/instance: auth-server-prod
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: auth-server
      app.kubernetes.io/instance: auth-server-prod
  template:
    metadata:
      labels:
        app.kubernetes.io/name: auth-server
        app.kubernetes.io/instance: auth-server-prod
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: auth-server
          image: registry.example.com/identity/auth-server:1.24.0
          resources:
            requests: { cpu: 500m, memory: 1Gi }
            limits:   { memory: 1536Mi }
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]

왜 좋은가:

• Argo CD가 wave -1 → 0 순서로 sync
• Helm hook과 혼용하지 않음
• BeforeHookCreation 정책으로 이전 Job 정리 후 새 Job 실행

---

좋은 예시 3: non-transactional DDL 전용 migration

V4__create_idx_last_login_concurrently.sql:

-- flyway:executeInTransaction=false
-- CREATE INDEX CONCURRENTLY는 Postgres에서 트랜잭션 내 실행 불가.
-- Flyway 8.2+ directive로 파일 단위 트랜잭션 비활성화.
-- Runtime estimate: 약 15분 (50M rows 기준).
-- Deploy window: 주간 트래픽 저점 (예: 화요일 03:00 UTC)
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_users_last_login
    ON users(last_login_at);

왜 좋은가:

• 파일 단독으로 분리 (다른 statement 없음)
• 주석에 runtime / window 명시
• IF NOT EXISTS로 재실행 안전성 (CREATE INDEX CONCURRENTLY 실패 시 INVALID 인덱스가 남을 수 있음 — 별도 cleanup 필요)

❌ 나쁜 예시 1: 트랜잭션 내 CREATE INDEX CONCURRENTLY

-- V4__.sql (executeInTransaction directive 없음)
CREATE INDEX CONCURRENTLY idx_users_last_login ON users(last_login_at);

문제:

• Flyway가 자동으로 트랜잭션을 열어 실행 → ERROR: CREATE INDEX CONCURRENTLY cannot run inside a transaction block
• 해결: -- flyway:executeInTransaction=false directive

---

좋은 예시 4: history table schema를 명시적으로 분리

env:

- {name: FLYWAY_CREATE_SCHEMAS,   value: "false"}
- {name: FLYWAY_INIT_SQL,         value: "CREATE SCHEMA IF NOT EXISTS auth_server; CREATE SCHEMA IF NOT EXISTS flyway_history"}
- {name: FLYWAY_DEFAULT_SCHEMA,   value: "flyway_history"}
- {name: FLYWAY_SCHEMAS,          value: "flyway_history,auth_server"}
- {name: FLYWAY_TABLE,            value: "flyway_schema_history"}

왜 좋은가:

• history table은 flyway_history.flyway_schema_history
• migration 대상 schema는 auth_server
• createSchemas=false 조건 하에서 initSql로 schema 사전 생성

---

좋은 예시 5: 운영 절차 (Helm + on-demand CNPG backup 연계)

# 1. pending migration 확인 (로컬)
docker run --rm -v $PWD/sql:/flyway/sql:ro \
  -e FLYWAY_URL=jdbc:postgresql://stage.../auth \
  -e FLYWAY_USER=auth_app -e FLYWAY_PASSWORD=... \
  flyway/flyway@sha256:... info

# 2. PR review + migration 영향 분석

# 3. 운영 배포 직전 on-demand backup
cat <<EOF | kubectl apply -f -
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: auth-pg-pre-2026-04-16
  namespace: data-prod
spec:
  cluster: {name: auth-pg}
  method: barmanObjectStore
EOF

# 4. Helm upgrade (pre-upgrade hook이 Flyway Job 실행)
helm upgrade auth-server ./charts/auth-server \
  --namespace auth-prod \
  --values values/prod.yaml \
  --atomic --timeout 20m

# 5. Flyway Job 로그 확인
kubectl -n auth-prod logs job/auth-flyway-migrate --all-containers

# 6. Deployment rollout 확인
kubectl -n auth-prod rollout status deploy/auth-server --timeout=10m

# 7. flyway info 재실행 (post-check)

왜 좋은가:

• migration 직전 on-demand backup으로 PITR 지점 확보
• helm upgrade --atomic으로 실패 시 자동 롤백
• hook이 hook-succeeded 정책으로 정리됨

---

좋은 예시 6: expand → migrate → contract (여러 릴리즈)

Release 1 (V120~V121) — Expand

-- V120__add_email_canonical_nullable.sql
-- flyway:executeInTransaction=false
ALTER TABLE users ADD COLUMN email_canonical text;
CREATE INDEX CONCURRENTLY idx_users_email_canonical ON users(email_canonical);

-- V121__backfill_email_canonical.sql
-- Flyway 기본 트랜잭션 모드 — 소규모 테이블용. 대용량은 별도 배치 Job.
UPDATE users
   SET email_canonical = lower(trim(email))
 WHERE email_canonical IS NULL
   AND email IS NOT NULL;

앱: 쓰기 시 두 컬럼 채움. 읽기는 아직 email.

Release 2 (V122) — Migrate

-- V122__add_email_canonical_constraints.sql
ALTER TABLE users ALTER COLUMN email_canonical SET NOT NULL;
ALTER TABLE users ADD CONSTRAINT users_email_canonical_unique UNIQUE (email_canonical);

앱: 읽기/쓰기 모두 email_canonical. 기존 email도 fallback 유지.

Release 3 (V130) — Contract

-- V130__drop_legacy_email_column.sql
ALTER TABLE users DROP COLUMN email;

왜 좋은가:

• N-1 ↔ N 동시 배포 허용
• 각 릴리즈가 독립 롤백 가능 (V130 제외 모두 non-destructive)
• expand와 contract가 같은 릴리즈에 섞이지 않음

---

좋은 예시 7: repeatable migration은 정의성 오브젝트에만

sql/
  V120__add_email_canonical_nullable.sql
  V121__backfill_email_canonical.sql
  V122__add_email_canonical_constraints.sql
  V130__drop_legacy_email_column.sql
  R__refresh_active_users_view.sql
  R__user_signup_function.sql

왜 좋은가:

• 핵심 schema change는 versioned
• view / function만 repeatable — 체크섬 변경 시 재적용

❌ 나쁜 예시 2: 순서 중요한 schema change를 R__로

R__create_users_table.sql     ← 잘못. 순서 보장 없음
R__add_refresh_tokens.sql

문제:

• repeatable은 ordering 보장 없음 — 의존성 있는 change에 부적합

---

나쁜 예시 3: app startup에 migration 숨김

# application.properties
spring.flyway.enabled=true
spring.flyway.baseline-on-migrate=true
spring.flyway.out-of-order=true

문제:

• replicas=3이면 Pod 3개가 동시 migrate 시도 (advisory lock이 직렬화는 하지만 기동 latency 증가)
• app rollout 실패와 migration 실패가 섞임
• 신규 Pod 기동마다 validate 수행 → 오차 탐지 시점이 흐려짐
• baseline-on-migrate=true + out-of-order=true 조합은 migration history 신뢰도 저하

---

나쁜 예시 4: validate 실패 후 바로 repair

flyway validate || flyway repair
flyway migrate

문제:

• history 문제를 원인 분석 없이 덮음
• repair를 정상 운영 흐름처럼 사용 — 탐지력 저하

---

나쁜 예시 5: 적용된 migration 파일 수정

V42__add_refresh_token_column.sql
  # 처음엔 빈 migration
  # prod apply 후 컬럼 타입을 나중에 editor로 수정

문제:

• checksum mismatch → validate 실패
• 환경 간 재현성 깨짐
• 대응은 "새 V__ migration으로 교정"

---

나쁜 예시 6: U__ undo migration 작성

V120__add_column.sql
U120__drop_column.sql     ← OSS Flyway는 실행 불가

문제:

• flyway undo는 Teams/Enterprise 전용
• OSS 환경에서는 U__ 파일이 실행되지 않아 오해 유발
• rollback은 forward-only + PITR로

---

나쁜 예시 7: parallelism 누락 + 재시도 무한

spec:
  # parallelism, backoffLimit, activeDeadlineSeconds 모두 누락
  template:
    spec:
      restartPolicy: OnFailure   # 무한 재시도 유발

문제:

• backoffLimit 기본 6 + restartPolicy: OnFailure → 실패 시 지수 backoff로 계속 재시도
• activeDeadlineSeconds 없음 → hang된 migration이 영원히 살아있음
• advisory lock이 걸린 실패 Job이 새 Job을 블록

---

나쁜 예시 8: Helm hook + Argo CD hook 혼용

annotations:
  "helm.sh/hook": "pre-upgrade"
  "helm.sh/hook-weight": "-10"
  argocd.argoproj.io/sync-wave: "-1"
  argocd.argoproj.io/hook: Sync

문제:

• Argo CD가 Helm chart를 렌더링할 때 Helm hook annotation을 일반 리소스의 annotation으로 해석
• 결과적으로 Flyway Job이 일반 리소스로 취급되거나, 두 시스템이 서로 다른 시점에 Job을 만들어 race 발생
• 하나의 배포 도구에 맞춰 한쪽만 사용할 것