chore: initialize from backend template 0a6dd0e

2026-08-13 20:31:02 +09:00
commit e64e701fe5
3223 changed files with 388401 additions and 0 deletions
@@ -0,0 +1,96 @@
+---
+title: Runbook — MIGRATION_FAILED (DB 마이그레이션 실패)
+category: INTERNAL
+error_codes: [MIGRATION_FAILED]
+severity: P1
+owner: oncall
+last_updated: 2026-07-29
+status: active
+---
+
+# Runbook: MIGRATION_FAILED (`runbook://migration/failed`)
+
+## Symptoms
+
+- Container exits with code 70 (migration failure exit)
+- Structured log with `error.code=MIGRATION_FAILED`, `startup.phase=migration`
+- App refuses to start (fail-fast)
+- JPA capability adapter refuses activation because its
+  `capability_schema_registry.lifecycle_state` is not `ACTIVE`
+
+## Diagnosis
+
+1. Stop rollout and keep the failed revision out of readiness. Do not route traffic to a partially
+   migrated instance.
+2. Identify the exact stream from `src/config/jpa/readiness-cards.yaml`. Each stream has an
+   independent `location` and `history-table`; do not infer ownership from a broad
+   `classpath:db/migration` scan.
+3. From a privileged migration session, capture the stream state before changing anything:
+
+   ```sql
+   select installed_rank, version, description, success
+   from <owned_history_table>
+   order by installed_rank;
+
+   select capability_id, installation_origin, core_epoch, feature_revision, lifecycle_state
+   from capability_schema_registry
+   where capability_id = '<card-id>';
+   ```
+
+4. Check whether any owned relation was created without a successful history entry. Compare only
+   against the owned tables in the reviewed migration; do not drop unrelated relations.
+5. Classify the failure:
+   - lock/statement timeout: remove the blocker or reduce rollout concurrency, then rerun;
+   - SQL/data precondition: create a new forward migration that makes the precondition explicit;
+   - checksum mismatch: compare the deployed artifact with the already applied script before
+     considering repair;
+   - connection/TLS failure: fix transport or credentials without changing Flyway history.
+
+## Action
+
+1. Prefer forward recovery. Fix the environmental blocker or add a new immutable migration, then
+   rerun the same owned stream with its exact history table.
+2. For an optional stream that never installed successfully, keep the capability marker absent and
+   the runtime adapter disabled until migration succeeds.
+3. After a successful migration, validate:
+   - the history contains only successful expected versions;
+   - `core_epoch` and `feature_revision` match the readiness registry;
+   - the marker is `INSTALLED_INACTIVE`;
+   - owned objects and constraints exist.
+4. Change the marker to `ACTIVE` only after the compatible application revision is deployed and its
+   readiness check succeeds. Disabling or rolling back application code changes the marker to
+   `INSTALLED_INACTIVE`; it does not drop history or owned data.
+5. Re-run the candidate evidence task before promoting:
+
+   ```bash
+   cd src
+   ./gradlew :adapter:outbound:persistence-jpa:verifyJpaCandidateEvidence --console=plain
+   ```
+
+6. Record the failed revision, stream/history table, root cause, recovery migration, elapsed time
+   and verification artifact in the incident.
+
+Do not:
+
+- edit an already applied migration;
+- delete or rewrite Flyway history to make validation green;
+- run `flyway repair` before checksum provenance is proven and reviewed;
+- use `clean`, destructive rollback, or schema-wide restore as the first response;
+- mark a capability `ACTIVE` before its migration and adapter readiness succeed.
+
+If commit outcome was indeterminate during the failure, reconcile by the application
+`OperationId`/idempotency reference before retrying business work. Never blind-retry a commit whose
+result is unknown.
+
+## Escalation
+
+- P1 immediate: the required application revision cannot become ready.
+- Escalate to the database owner before Flyway history repair, destructive DDL, point-in-time
+  recovery, or primary failover.
+- R3 restore/PITR and failover rehearsal requires a target-like backup topology; local
+  Testcontainers evidence is not a substitute.
+
+---
+
+This runbook is forward-only. The reviewed migration artifact and the per-card evidence manifest
+are the audit sources.