fix(migration): 실패한 동시 인덱스 마이그레이션 복구와 검사기 설명을 실제 동작에 맞춘다

문제 (재리뷰 round7 영속성 RR7-03-01~05)
- 실패한 동시 인덱스 마이그레이션의 복구를 가이드·검사기·학습 문서가
  "DROP INDEX 하나"로 안내한다. Flyway는 트랜잭션 밖에서 실패한 버전의
  이력 행(success = false)을 남기고, 다음 배포의 validate가 그 행 때문에
  다시 멈춘다. runbook에는 이 경우가 없고 이력 수정은 금지다.
  docs/jpa/runbooks.md "An invalid index exists"도 같은 이유로 재적용에서
  멈춘다.
- 검사기 Javadoc·학습 문서가 "섞인 파일은 부분 적용되지 않는다"고 쓴다.
  가이드가 요구하는 executeInTransaction=false를 붙이면 Flyway가 문장마다
  커밋해 앞 문장이 남는다.
- 가이드·검사기 Javadoc·실패 메시지·학습 문서가 "Flyway는 REINDEX …
  CONCURRENTLY를 트랜잭션 안에서 돌린다"고 쓴다. REINDEX SCHEMA|DATABASE
  CONCURRENTLY는 Flyway가 스스로 트랜잭션 밖에서 돌린다.
- LockWaitMigrationInspector는 .sql.conf가 executeInTransaction=true면
  SET LOCAL 뒤의 VACUUM FULL·REINDEX SCHEMA를 통과시킨다. Javadoc은
  "늘 거부"라고 쓴다.
- ledger 잠금 대기 IT가 동시 인덱스 파일을 "이 테스트에 예외로 적으라"고
  안내하지만 예외를 적을 곳이 없다. 같은 원인으로 ledger의 다른 IT 두 개
  (GrpcOperationLedgerPostgresIT, JpaGrpcOperationLedgerContractIT)도
  모든 파일을 한 문자열로 실행해, 동시 인덱스 파일이 생기면 "cannot be
  executed within a pipeline"으로 실패한다.

고른 방법
- 복구(RR7-03-03): 소유자 결정(2026-11-06)대로 migration-failed.md에
  "Recover a failed concurrent index migration" 절을 둔다. 진행 중 빌드
  확인 → 무효 인덱스 DROP INDEX CONCURRENTLY → 원인 정리 → 그 버전의
  실패 행만 DELETE → 재배포. "Do not"에 이 한 가지 예외를 적고
  flyway repair는 계속 금지한다. SQL 세 블록에 runbook-sql 표시를 달고
  postgresqlConcurrentIndexMigrationIntegrationTest가 runbook에서 읽어
  실 PostgreSQL에서 돌린다(outbox runbook과 같은 방식). 가이드·검사기
  Javadoc·메시지·학습 문서·jpa/runbooks.md는 그 절을 가리킨다.
- 섞인 파일(RR7-03-04)·REINDEX(RR7-03-02): 문장을 고치고, 같은 IT와
  잠금 대기 IT에 사실을 고정한다(섞인 파일 두 경우, REINDEX 다섯 형태).
- 검사기(RR7-03-05): 트랜잭션 안에서 돌 수 없는 잠금 문장은 .sql.conf와
  상관없이 거부한다(리뷰 권장안). 버린 안: Javadoc만 좁히기 — 그 파일은
  PostgreSQL이 배포 때 25001로 거부하므로 검사기가 먼저 알려 주는 쪽이
  맞다. 잠그지 않는 문장(plain VACUUM 등)은 이 검사기의 범위가 아니라
  그대로 둔다.
- ledger(RR7-03-01): IT에 RUN_OUTSIDE_A_TRANSACTION(파일 → 이유) 목록을
  두고 잠금 대기 케이스에서 뺀다. 목록의 파일은 같은 IT가 동시 인덱스
  문장 하나·executeInTransaction=false인지 ConcurrentIndexMigration
  Inspector로 본다. optional 빌드는 testkit에 의존할 수 없어
  sharedMigrationFileSources가 그 파일과 MigrationResource를 복사한다
  (VersionedMigrationFiles와 같은 방식). ledger IT 셋은 마이그레이션을
  파일마다 따로 실행한다(Flyway처럼). 버린 안: 안내만 "다룰 수 없다"로
  바꾸기 — 다음 동시 인덱스 파일이 생길 때 같은 질문이 다시 남는다.

위험·한계
- 검사기 변경으로 서비스의 자기 스트림 검사가 새로 실패할 수 있다.
  걸리는 파일은 PostgreSQL이 배포 때 거부하는 파일뿐이라 배포된 적이
  없다(업그레이드 안내 4.1에 적음).
- runbook의 DELETE는 운영자가 version을 옮겨 적는다. NOT success라
  성공한 행은 지우지 못하지만, 다른 버전의 실패 행을 잘못 적으면 그 행이
  지워진다. 한 행만 지워야 하고 0행이면 다시 분류하라고 적었다.
- ledger 복사 파일 두 개도 JDK만 써야 한다(각 Javadoc에 적음).
- Spotless 검사는 같은 worktree에서 spotlessApply가 변경 없이 끝난 직후라
  UP-TO-DATE로 처리됐다.

검증 (desktop, 공유 잠금, 로그 ~/r8-f3-logs/)
RED — 기준 73d39212 + 이 커밋의 테스트(+ 기준 문서 주장을 단언하는 임시
메서드, + 임시 ledger V004):
- persistence-jpa test --tests LockWaitMigrationInspectorTest: 80건 중
  2건 실패, 검사기가 빈 목록(통과)을 돌려줌 (red-unit.log)
- postgresqlMigrationLockWaitIntegrationTest: 47건 중 4건 실패 — 옛 주장
  대로면 거부여야 할 REINDEX SCHEMA|DATABASE CONCURRENTLY가 applied,
  conf=true VACUUM FULL·REINDEX SCHEMA를 PostgreSQL은 25001로 거부하는데
  검사기는 통과 (red2-lockwait.log)
- postgresqlConcurrentIndexMigrationIntegrationTest: 9건 중 3건 실패 —
  옛 절차(DROP INDEX) 뒤 재배포가 "Detected failed migration to version
  2", conf를 붙인 섞인 파일이 행 3001·무효 인덱스·실패 이력을 남김,
  runbook 블록이 없어 복구 테스트 준비 실패(RED로 세지 않음)
  (red2-concurrent.log)
- ledger integrationTest(V004 동시 인덱스): 6건 중 4건 실패 — 잠금 대기
  IT "name it as an exception here", 다른 IT 둘 "cannot be executed within
  a pipeline" (red-ledger.log)
GREEN — 이 커밋의 트리 b49369e7:
- :adapter:outbound:persistence-jpa:check rc=0 (test 666건 실패 0),
  :testQualityCheck rc=0
- postgresqlMigrationLockWaitIntegrationTest rc=0 (42건 실패 0)
- postgresqlConcurrentIndexMigrationIntegrationTest rc=0 (7건 실패 0)
- -p optional-platforms :grpc:grpc-operation-ledger-jpa:check
  integrationTest testQualityCheck rc=0 (IT 3+2+42건 실패 0)
- :app-bootstrap:documentContractTest rc=0 (66건 실패 0)
- 두 빌드의 verifyStrictLaneStages --dry-run rc=0
- ledger 탐침(임시 V004): 목록에 없으면 잠금 대기 케이스만 실패하고
  메시지가 RUN_OUTSIDE_A_TRANSACTION을 가리킴(48건 중 1건), 목록에 넣으면
  rc=0, 목록에 넣고 SET LOCAL을 섞으면 목록 검사가 "mixes a concurrent
  index statement"로 거부
- python3 scripts/check-work-docs.py --links docs: 깨진 링크 0

Refs: docs/reviews/2026-11-03-rereview/round7/03-persistence.md
This commit is contained in:
DongHyeonka
2026-11-07 01:14:09 +09:00
parent 73d39212f5
commit 6ec0c4f7fa
18 changed files with 587 additions and 95 deletions
@@ -265,6 +265,14 @@ strictTestLanes {
}
}
// The recovery of a failed concurrent index migration (re-review 2026-11-03 round 7, RR7-03-03) is
// SQL the migration runbook gives an operator; the lane reads it out of the runbook the same way.
tasks.named('postgresqlConcurrentIndexMigrationIntegrationTest', Test) {
def migrationFailedRunbook = new File(rootProject.projectDir.parentFile, 'docs/runbooks/migration-failed.md')
inputs.file(migrationFailedRunbook).withPathSensitivity(PathSensitivity.RELATIVE).withPropertyName('migrationFailedRunbook')
systemProperty 'migration.failed.runbook', migrationFailedRunbook.absolutePath
}
tasks.named('postgresqlSecurityBaselineIntegrationTest') {
// Cross-leaf task edge: see the handoff. verifyCleanArchitectureDependencies inspects
// configurations, not the task graph, so this edge is invisible to it.
@@ -7,7 +7,8 @@ import java.nio.file.Path;
import java.util.Map;
/**
* The SQL an outbox runbook gives an operator, read out of the runbook itself.
* The SQL a runbook gives an operator, read out of the runbook itself: the outbox runbooks, and the
* migration runbook's recovery of a failed concurrent index migration.
*
* <p>A runbook block is marked {@code <!-- runbook-sql: id -->} and followed by a {@code sql}
* fence. The lanes that run these blocks against real PostgreSQL read them from the document rather
@@ -48,7 +49,7 @@ final class OutboxRunbookSql {
String marker = "<!-- runbook-sql: " + id + " -->";
int at = runbook.indexOf(marker);
if (at < 0) {
throw new IllegalStateException("the outbox runbook has no SQL block marked " + id);
throw new IllegalStateException("the runbook has no SQL block marked " + id);
}
int open = runbook.indexOf("```sql\n", at);
int close = runbook.indexOf("\n```", open + 1);
@@ -14,6 +14,7 @@ import java.sql.SQLException;
import java.time.Duration;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
@@ -24,6 +25,8 @@ import java.util.concurrent.TimeoutException;
import javax.sql.DataSource;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
@@ -53,6 +56,14 @@ import org.springframework.jdbc.datasource.DelegatingDataSource;
* way every Flyway this template constructs itself does ({@link FlywaySchemaPolicy#configure()}). A
* deploy still waiting after {@link #FINISHES_WITHIN} is cancelled, which leaves the invalid index
* an interrupted build leaves in production too.
*
* <p>Three more cases pin what happens when such a deploy fails (re-review 2026-11-03 round 7,
* RR7-03-03 and RR7-03-04). A concurrent build next to another statement is refused before any of
* it runs when nothing configures the file, and is left half applied when {@code
* executeInTransaction=false} does: Flyway commits each statement on its own. A failed build leaves
* an invalid index and a failed history row, and the deploy after it is refused until the row is
* gone: the recovery case runs the SQL of {@code docs/runbooks/migration-failed.md} ("A concurrent
* index migration failed"), read out of the runbook itself, and deploys again.
*/
class PostgreSqlConcurrentIndexMigrationIntegrationTest {
@@ -64,6 +75,23 @@ class PostgreSqlConcurrentIndexMigrationIntegrationTest {
INSERT INTO probe SELECT g, 'note ' || g FROM generate_series(1, 1000) AS g;
""";
/** A second row with row 1's note, so that a unique index on the note cannot be built. */
private static final String DUPLICATE_NOTE =
"INSERT INTO probe (id, note) VALUES (2001, 'note 1');\n";
private static final String UNIQUE_NOTE_INDEX =
"CREATE UNIQUE INDEX CONCURRENTLY ix_probe_note_u ON probe (note)";
/** A concurrent build after another statement, the file the guide says not to write. */
private static final String MIXED =
"INSERT INTO probe (id, note) VALUES (3001, 'partial');\n" + UNIQUE_NOTE_INDEX + ";\n";
private static final String FLYWAY_REFUSES_A_MIXED_MIGRATION =
"Detected both transactional and non-transactional statements";
/** The system property naming {@code docs/runbooks/migration-failed.md} for this lane. */
private static final String MIGRATION_FAILED_RUNBOOK = "migration.failed.runbook";
private static PostgreSqlReadinessSupport postgres;
@BeforeAll
@@ -126,6 +154,103 @@ class PostgreSqlConcurrentIndexMigrationIntegrationTest {
}
}
@Test
@DisplayName("설정 파일 없이 동시 인덱스 문장과 다른 문장을 섞은 마이그레이션은 Flyway가 아무것도 실행하기 전에 거부한다")
void aMixedMigrationWithoutConfigurationIsRefusedBeforeAnyOfItRuns(@TempDir Path stream)
throws Exception {
try (HikariDataSource database = postgres.createDatabase(databaseName())) {
// given
Files.writeString(stream.resolve("V1__probe.sql"), FIRST_RELEASE + DUPLICATE_NOTE);
migrateTheWayTheTemplateDoes(database, stream);
Files.writeString(stream.resolve("V2__mixed.sql"), MIXED);
// when
Throwable thrown = catchThrowable(() -> migrateTheWayTheTemplateDoes(database, stream));
// then
assertThat(
List.of(
String.valueOf(thrown).contains(FLYWAY_REFUSES_A_MIXED_MIGRATION),
aftermath(database)))
.containsExactly(
true, "row 3001 0, indexes [ix_probe_id true, probe_pkey true], history [1 true]");
}
}
@Test
@DisplayName(
"executeInTransaction=false를 붙여 섞은 마이그레이션은 앞 문장을 커밋한 채 동시 빌드에서 실패해 부분 적용, 무효 인덱스, 실패 이력을 남긴다")
void aMixedMigrationMarkedOutsideATransactionIsLeftHalfApplied(@TempDir Path stream)
throws Exception {
try (HikariDataSource database = postgres.createDatabase(databaseName())) {
// given
Files.writeString(stream.resolve("V1__probe.sql"), FIRST_RELEASE + DUPLICATE_NOTE);
migrateTheWayTheTemplateDoes(database, stream);
Files.writeString(stream.resolve("V2__mixed.sql"), MIXED);
Files.writeString(stream.resolve("V2__mixed.sql.conf"), "executeInTransaction=false\n");
// when
Throwable thrown = catchThrowable(() -> migrateTheWayTheTemplateDoes(database, stream));
// then
assertThat(List.of(thrown != null, aftermath(database)))
.containsExactly(
true,
"row 3001 1, indexes [ix_probe_id true, ix_probe_note_u false, probe_pkey true],"
+ " history [1 true, 2 false]");
}
}
@Test
@DisplayName(
"실패한 동시 인덱스 마이그레이션은 runbook 절차(빌드 확인, 무효 인덱스 삭제, 원인 정리, 그 버전의 실패 이력 행 삭제) 뒤 다시 배포하면 적용된다")
void aFailedConcurrentIndexMigrationIsRecoveredByTheRunbookProcedure(@TempDir Path stream)
throws Exception {
OutboxRunbookSql runbook = OutboxRunbookSql.fromSystemProperty(MIGRATION_FAILED_RUNBOOK);
try (HikariDataSource database = postgres.createDatabase(databaseName())) {
// given
Files.writeString(stream.resolve("V1__probe.sql"), FIRST_RELEASE + DUPLICATE_NOTE);
migrateTheWayTheTemplateDoes(database, stream);
String checksumOfV1 = checksumOf(database, "1");
writeConcurrentMigration(stream, UNIQUE_NOTE_INDEX);
Throwable failed = catchThrowable(() -> migrateTheWayTheTemplateDoes(database, stream));
String afterFailure = aftermath(database);
JdbcTemplate jdbc = new JdbcTemplate(database);
// when
List<Map<String, Object>> buildsRunning =
jdbc.queryForList(runbook.block("failed-concurrent-index-builds"));
jdbc.execute(
runbook.block(
"failed-concurrent-index-drop", Map.of("invalid_index", "ix_probe_note_u")));
jdbc.execute("DELETE FROM probe WHERE id = 2001");
jdbc.execute(
runbook.block(
"failed-concurrent-index-history-row",
Map.of("owned_history_table", "flyway_schema_history", "version", "2")));
Throwable again = catchThrowable(() -> migrateTheWayTheTemplateDoes(database, stream));
// then
assertThat(
List.of(
failed != null,
afterFailure,
buildsRunning.size(),
String.valueOf(again),
aftermath(database),
checksumOf(database, "1")))
.containsExactly(
true,
"row 3001 0, indexes [ix_probe_id true, ix_probe_note_u false, probe_pkey true],"
+ " history [1 true, 2 false]",
0,
"null",
"row 3001 0, indexes [ix_probe_id true, ix_probe_note_u true, probe_pkey true],"
+ " history [1 true, 2 true]",
checksumOfV1);
}
}
/** V2: the one concurrent statement, with the companion configuration the guide asks for. */
private static void writeConcurrentMigration(Path stream, String statement) throws IOException {
Files.writeString(stream.resolve("V2__concurrent_index.sql"), statement + ";\n");
@@ -222,6 +347,37 @@ class PostgreSqlConcurrentIndexMigrationIntegrationTest {
return "version " + version + ", indexes valid " + valid;
}
/**
* What a deploy left behind: whether the row the mixed V2 inserts first is there, every index on
* the table with whether it is valid, and every history row with whether it succeeded.
*/
private static String aftermath(DataSource database) {
JdbcTemplate jdbc = new JdbcTemplate(database);
Integer row = jdbc.queryForObject("SELECT count(*) FROM probe WHERE id = 3001", Integer.class);
List<String> indexes =
jdbc.queryForList(
"SELECT c.relname || ' ' || i.indisvalid FROM pg_index i"
+ " JOIN pg_class c ON c.oid = i.indexrelid"
+ " WHERE i.indrelid = 'probe'::regclass ORDER BY c.relname COLLATE \"C\"",
String.class);
List<String> history =
jdbc.queryForList(
"SELECT coalesce(version, '-') || ' ' || success FROM flyway_schema_history"
+ " ORDER BY installed_rank",
String.class);
return "row 3001 " + row + ", indexes " + indexes + ", history " + history;
}
/** The checksum the history holds for a successful version. */
private static String checksumOf(DataSource database, String version) {
return String.valueOf(
new JdbcTemplate(database)
.queryForObject(
"SELECT checksum FROM flyway_schema_history WHERE version = ? AND success",
Integer.class,
version));
}
private static String databaseName() {
return "concurrent_" + UUID.randomUUID().toString().replace("-", "").toLowerCase(Locale.ROOT);
}
@@ -53,6 +53,13 @@ import org.springframework.jdbc.core.JdbcTemplate;
* LockWaitMigrationInspectorTest}, which pins Flyway's own patterns in the pull request's {@code
* ci} (round 6, RR6-03-03).
*
* <p>Two more cases pin what the documents say about the edges of that list (round 7, RR7-03-02 and
* RR7-03-05): Flyway runs {@code REINDEX INDEX|TABLE ... CONCURRENTLY} in the transaction, where
* PostgreSQL refuses it, but {@code REINDEX SCHEMA|DATABASE ... CONCURRENTLY} outside one on its
* own; and a {@code VACUUM FULL} or {@code REINDEX SCHEMA} that a {@code .sql.conf} keeps in the
* transaction ({@code executeInTransaction=true}) is refused by PostgreSQL at deploy, and by the
* inspector before it.
*
* <p>The migrations run under the template's Flyway settings ({@code FlywaySchemaPolicy}), whose
* PostgreSQL lock is held for the session rather than in an open transaction (RR6-03-01): the bound
* is shown under the lock the deploy actually takes.
@@ -231,15 +238,26 @@ class PostgreSqlMigrationLockWaitIntegrationTest {
}
}
@Test
@ParameterizedTest
@CsvSource(
delimiter = '|',
value = {
"REINDEX TABLE CONCURRENTLY probe | refused 25001",
"REINDEX INDEX CONCURRENTLY probe_pkey | refused 25001",
"REINDEX (CONCURRENTLY) TABLE probe | refused 25001",
"REINDEX SCHEMA CONCURRENTLY public | applied",
"REINDEX DATABASE CONCURRENTLY <database> | applied"
})
@DisplayName(
"Flyway는 REINDEX TABLE CONCURRENTLY를 트랜잭션 안에서 돌려 PostgreSQL이 거부하므로 executeInTransaction=false가 필요하다")
void flywayRunsReindexConcurrentlyInATransactionWherePostgresqlRefusesIt(@TempDir Path stream)
throws Exception {
try (HikariDataSource database = postgres.createDatabase(databaseName())) {
"Flyway는 REINDEX INDEX·TABLE … CONCURRENTLY를 트랜잭션 안에서 돌려 PostgreSQL이 거부하고, REINDEX SCHEMA·DATABASE … CONCURRENTLY는 스스로 트랜잭션 밖에서 돌린다")
void flywayRunsOnlyReindexIndexOrTableConcurrentlyInATransactionWherePostgresqlRefusesIt(
String statement, String expected, @TempDir Path stream) throws Exception {
String name = databaseName();
try (HikariDataSource database = postgres.createDatabase(name)) {
// given
new JdbcTemplate(database).execute("CREATE TABLE probe (id INT PRIMARY KEY)");
Files.writeString(stream.resolve("V1__reindex.sql"), "REINDEX TABLE CONCURRENTLY probe;\n");
Files.writeString(
stream.resolve("V1__reindex.sql"), statement.replace("<database>", name) + ";\n");
Flyway flyway =
FlywaySchemaPolicy.configure()
.dataSource(database)
@@ -250,9 +268,47 @@ class PostgreSqlMigrationLockWaitIntegrationTest {
// when
Throwable thrown = catchThrowable(flyway::migrate);
String outcome =
thrown == null ? "applied" : "refused " + new Attempt(thrown, Duration.ZERO).sqlState();
// then
assertThat(new Attempt(thrown, Duration.ZERO).sqlState()).isEqualTo(ACTIVE_SQL_TRANSACTION);
assertThat(outcome).as("%s (%s)", statement, thrown).isEqualTo(expected);
}
}
@ParameterizedTest
@ValueSource(strings = {"VACUUM FULL probe", "REINDEX SCHEMA public"})
@DisplayName(
"executeInTransaction=true로 트랜잭션 안에 둔 VACUUM FULL·REINDEX SCHEMA는 PostgreSQL이 배포 때 거부하고 검사기도 거부한다")
void aLockingStatementAConfigurationKeepsInATransactionIsRefusedByPostgresqlAndTheInspector(
String statement, @TempDir Path stream) throws Exception {
try (HikariDataSource database = postgres.createDatabase(databaseName())) {
// given
new JdbcTemplate(database).execute("CREATE TABLE probe (id INT PRIMARY KEY)");
Path script =
Files.writeString(
stream.resolve("V1__probe.sql"),
"SET LOCAL lock_timeout = '5s';\n" + statement + ";\n");
Files.writeString(stream.resolve("V1__probe.sql.conf"), "executeInTransaction=true\n");
Flyway flyway =
FlywaySchemaPolicy.configure()
.dataSource(database)
.locations("filesystem:" + stream.toAbsolutePath())
.baselineOnMigrate(true)
.baselineVersion("0")
.load();
// when
Throwable thrown = catchThrowable(flyway::migrate);
List<String> named = new LockWaitMigrationInspector().unboundedLockingStatements(script);
// then
assertThat(
List.of(
String.valueOf(new Attempt(thrown, Duration.ZERO).sqlState()),
named.toString().contains("cannot run in a transaction")))
.as("%s: Flyway %s, inspector %s", statement, thrown, named)
.containsExactly(ACTIVE_SQL_TRANSACTION, true);
}
}
@@ -381,6 +381,29 @@ class LockWaitMigrationInspectorTest {
assertThat(unbounded).isEmpty();
}
@ParameterizedTest
@ValueSource(
strings = {
"SET LOCAL lock_timeout = '5s';\nVACUUM FULL outbox_event;",
"SET LOCAL lock_timeout = '5s';\nREINDEX SCHEMA public;"
})
@DisplayName("executeInTransaction=true 설정이 있어도 트랜잭션 안에서 돌 수 없는 잠금 문장은 마이그레이션에 두지 말라는 이유로 거부된다")
void aLockingStatementThatCannotRunInATransactionIsRefusedWhateverTheConfigurationSays(
String sql, @TempDir Path stream) throws IOException {
// given
Path script = Files.writeString(stream.resolve("V99__change.sql"), sql);
Files.writeString(stream.resolve("V99__change.sql.conf"), "executeInTransaction=true\n");
// when
List<String> unbounded = inspector.unboundedLockingStatements(script);
// then
assertThat(unbounded)
.singleElement(STRING)
.contains("cannot run in a transaction")
.contains("executeInTransaction=true");
}
/** Every static {@link Pattern} the class declares, by field name. */
private static Map<String, String> staticPatterns(Class<?> type) throws IllegalAccessException {
Map<String, String> patterns = new TreeMap<>();
@@ -10,19 +10,35 @@ import java.util.regex.Pattern;
*
* <p>PostgreSQL refuses {@code CREATE INDEX CONCURRENTLY}, {@code DROP INDEX CONCURRENTLY} and
* {@code REINDEX ... CONCURRENTLY} inside a transaction block. Flyway 11 runs a script outside its
* transaction on its own when it holds {@code CREATE} or {@code DROP INDEX CONCURRENTLY}, but not
* {@code REINDEX ... CONCURRENTLY}: that one Flyway runs in the transaction, and PostgreSQL refuses
* it at deploy. So {@link #validate} wants {@code executeInTransaction=false} beside every
* concurrent statement, whether Flyway would detect it or not. Next to any other statement, one
* Flyway runs outside a transaction makes Flyway refuse the whole script before it runs any of it
* (its default {@code mixed=false}); {@link #requireIsolatedStatement} wants the statement alone so
* that a failed build is recovered by dropping one invalid index.
* transaction on its own when it holds {@code CREATE} or {@code DROP INDEX CONCURRENTLY}, or {@code
* REINDEX SCHEMA|DATABASE} with or without {@code CONCURRENTLY}. It runs {@code REINDEX INDEX|TABLE
* ... CONCURRENTLY} — the option-list form {@code REINDEX (CONCURRENTLY) TABLE} included — in the
* transaction, and PostgreSQL refuses it at deploy ({@code
* postgresqlMigrationLockWaitIntegrationTest} shows the five forms; re-review 2026-11-03 round 7,
* RR7-03-02). So {@link #validate} wants {@code executeInTransaction=false} beside every concurrent
* statement, whether Flyway would detect it or not.
*
* <p>{@link #requireIsolatedStatement} wants the statement alone in its migration. Next to another
* statement it is refused by Flyway before any of the script runs when nothing configures the file
* (Flyway's default {@code mixed=false}); with the {@code executeInTransaction=false} that {@link
* #validate} asks for, Flyway runs and commits each statement on its own, and a failed build leaves
* the statements before it applied (round 7, RR7-03-04). Alone, a failed build leaves an invalid
* index and its version's failed history row, which Flyway refuses the next deploy over until the
* row is gone: {@code docs/runbooks/migration-failed.md} ("Recover a failed concurrent index
* migration") drops the index and deletes that one row ({@code
* postgresqlConcurrentIndexMigrationIntegrationTest} runs it; round 7, RR7-03-03).
*
* <p>Outside a transaction, {@code CREATE INDEX CONCURRENTLY} and {@code REINDEX ... CONCURRENTLY}
* still wait for every transaction older than themselves. They finish because the template's Flyway
* holds its PostgreSQL lock for the session ({@code FlywaySchemaPolicy.configure()}): under
* Flyway's default the lock's own open transaction is one of those, and the deploy never ends
* (re-review 2026-11-03 round 6, RR6-03-01 and RR6-03-05).
*
* <p>The gRPC ledger's lane uses it too. It lives in the optional build, which cannot depend on
* this testkit, so {@code grpc-operation-ledger-jpa} compiles this file and {@link
* MigrationResource} from here into its integration-test sources at build time (its {@code
* build.gradle}, {@code sharedMigrationFileSources}; round 7, RR7-03-01). Hence the rule for both
* files: the JDK only, nothing else from this testkit.
*/
public final class ConcurrentIndexMigrationInspector {
@@ -53,8 +69,8 @@ public final class ConcurrentIndexMigrationInspector {
throw new IllegalStateException(
migration.name()
+ " must set executeInTransaction=false: PostgreSQL refuses a concurrent index"
+ " statement inside a transaction block, and Flyway runs REINDEX ... CONCURRENTLY in"
+ " one unless the migration says otherwise");
+ " statement inside a transaction block, and Flyway runs REINDEX INDEX|TABLE ..."
+ " CONCURRENTLY in one unless the migration says otherwise");
}
/** Whether the script contains a statement PostgreSQL forbids inside a transaction. */
@@ -68,9 +84,12 @@ public final class ConcurrentIndexMigrationInspector {
/**
* Fails when a concurrent-index migration also contains other statements.
*
* <p>A concurrent index build can fail and leave an invalid index behind. Recovering from that is
* a single {@code DROP INDEX} when the migration did nothing else, and a manual reconstruction of
* partial state when it did.
* <p>A concurrent index build can fail and leave an invalid index and its version's failed
* history row behind. When the migration did nothing else, recovering is the runbook's few
* statements: drop the index, delete that one failed row, deploy again ({@code
* docs/runbooks/migration-failed.md}, "Recover a failed concurrent index migration"). When it did
* more, Flyway ran the file outside a transaction and has committed the statements before the
* build, and that partial state has to be reconstructed by hand.
*/
public void requireIsolatedStatement(MigrationResource migration) {
Objects.requireNonNull(migration, "migration");
@@ -88,9 +107,11 @@ public final class ConcurrentIndexMigrationInspector {
if (statements > 1L) {
throw new IllegalStateException(
migration.name()
+ " mixes a concurrent index statement with other statements; a failed concurrent"
+ " build must be recoverable by dropping one invalid index, which requires the"
+ " migration to contain nothing else");
+ " mixes a concurrent index statement with other statements; outside a"
+ " transaction Flyway commits each of them on its own, so a failed build leaves the"
+ " others applied. Alone, a failed build is recovered by dropping its invalid index"
+ " and deleting its version's failed history row (docs/runbooks/migration-failed.md,"
+ " \"Recover a failed concurrent index migration\")");
}
}
}
@@ -61,7 +61,9 @@ import java.util.regex.Pattern;
* that holds any other statement is refused at deploy; with {@code mixed=true} it runs, unbounded.
* A locking statement that cannot run in a transaction itself ({@code VACUUM FULL}, a
* non-concurrent {@code REINDEX SCHEMA}) can never be bounded and is refused with the advice to
* keep it out of migrations (re-review 2026-11-03 round 4, RR4-02-01). The check assumes Flyway's
* keep it out of migrations (re-review 2026-11-03 round 4, RR4-02-01) — whatever the {@code
* .sql.conf} says: {@code executeInTransaction=true} keeps the {@code SET LOCAL} in force, but only
* for PostgreSQL to refuse the statement at deploy (round 7, RR7-03-05). The check assumes Flyway's
* defaults, as the template runs it: {@code executeInTransaction=true} and {@code mixed=false} for
* the whole run.
*/
@@ -147,11 +149,11 @@ public final class LockWaitMigrationInspector {
* The statements Flyway runs outside a transaction on PostgreSQL 12 and later, as Flyway 11's
* {@code PostgreSQLParser#detectCanExecuteInTransaction} lists them: the statement's leading
* words outside parentheses, lower case and one space apart, match one of these in full. {@code
* ALTER TYPE ... ADD VALUE} is on Flyway's list only below PostgreSQL 12. {@code REINDEX TABLE
* CONCURRENTLY} is not on it: Flyway runs it in the transaction and PostgreSQL refuses it there.
* {@code LockWaitMigrationInspectorTest} pins Flyway's own patterns, read from {@code
* PostgreSQLParser}, so a Flyway upgrade that adds a statement or changes one fails the pull
* request's {@code ci} until this copy follows. {@code
* ALTER TYPE ... ADD VALUE} is on Flyway's list only below PostgreSQL 12. {@code REINDEX
* INDEX|TABLE CONCURRENTLY} is not on it: Flyway runs it in the transaction and PostgreSQL
* refuses it there. {@code LockWaitMigrationInspectorTest} pins Flyway's own patterns, read from
* {@code PostgreSQLParser}, so a Flyway upgrade that adds a statement or changes one fails the
* pull request's {@code ci} until this copy follows. {@code
* postgresqlMigrationLockWaitIntegrationTest} compares this copy with Flyway on real PostgreSQL
* for the statements it lists, and only those (re-review 2026-11-03 round 6, RR6-03-03).
*/
@@ -254,14 +256,15 @@ public final class LockWaitMigrationInspector {
for (Pattern pattern : LOCKING) {
Matcher statement = pattern.matcher(text);
while (statement.find()) {
boolean cannotRunInATransaction = within(statement.start(), outside);
Map.Entry<Integer, Boolean> setting = bounds.lowerEntry(statement.start());
boolean bounded = setting != null && setting.getValue();
boolean bounded = !cannotRunInATransaction && setting != null && setting.getValue();
if (!bounded && !createdEarlier(statement, createdAt)) {
String named = statement.group().trim();
found.add(
new Located(
statement.start(),
inTransaction
inTransaction && !cannotRunInATransaction
? named
: named + " (" + whyUnbounded(statement.start(), outside, configured) + ")"));
}
@@ -274,14 +277,17 @@ public final class LockWaitMigrationInspector {
}
/**
* Why a locking statement at {@code at} has no bound in a script that runs outside a transaction.
* Why a locking statement at {@code at} has no bound: it cannot run in a transaction itself, or
* the script runs outside one.
*/
private static String whyUnbounded(int at, List<Statement> outside, Configured configured) {
for (Statement statement : outside) {
if (statement.start() <= at && at < statement.end()) {
return "cannot run in a transaction, so no SET LOCAL can bound its wait: keep it out of"
+ " migrations and run it by hand as an operations step, outside deploys";
}
if (within(at, outside)) {
return "cannot run in a transaction, so no SET LOCAL can bound its wait"
+ (configured == Configured.IN_A_TRANSACTION
? " (executeInTransaction=true only makes PostgreSQL refuse it at deploy)"
: "")
+ ": keep it out of migrations and run it by hand as an operations step, outside"
+ " deploys";
}
if (configured == Configured.OUTSIDE_A_TRANSACTION) {
return "executeInTransaction=false: SET LOCAL does nothing outside a transaction; move it to"
@@ -294,6 +300,16 @@ public final class LockWaitMigrationInspector {
+ " in a migration of its own";
}
/** Whether the position {@code at} is inside one of the statements Flyway runs outside one. */
private static boolean within(int at, List<Statement> outside) {
for (Statement statement : outside) {
if (statement.start() <= at && at < statement.end()) {
return true;
}
}
return false;
}
/**
* The statements Flyway runs outside a transaction, found the way Flyway finds them: each
* statement's words outside parentheses, up to {@link #FLYWAY_DETECTION_WORDS} words in all,
@@ -5,6 +5,9 @@ import java.util.Objects;
/**
* One migration script as the inspector sees it (design §32).
*
* <p>The gRPC ledger's lane in the optional build compiles this file with {@link
* ConcurrentIndexMigrationInspector} (see there), so it keeps to the JDK only.
*
* @param name the versioned script name, e.g. {@code V42__order_index.sql}
* @param sql the script body
*/