3423 lines
128 KiB
Markdown
3423 lines
128 KiB
Markdown
# Fileserver Platform Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Spring 기반 Backend Skeleton에 로컬 파일시스템·PVC·제한형 NFS를 대상으로 안전한 streaming upload, 상태 기반 publish, HTTP Range 다운로드, MVC·WebFlux, Nginx 위임, tus 1.0을 제공하는 운영 가능한 Fileserver 플랫폼을 구현한다.
|
|
|
|
**Architecture:** `fileserver-core-api`는 저장소 구현과 Spring 타입이 새지 않는 ID·상태·Port를 정의하고, `fileserver-application`이 metadata와 content store를 조정한다. 로컬 저장소는 staging과 immutable content를 분리하고, 관계형 metadata DB의 version·lease·READY 상태가 공개 가능 여부를 결정한다. HTTP adapter, 검사, Nginx, 재개 업로드는 별도 모듈로 분리한다.
|
|
|
|
**Tech Stack:** Java 21, Gradle Kotlin DSL, Spring MVC, Spring WebFlux, Spring Data JPA, Flyway, Reactor, Micrometer, OpenTelemetry, JUnit 5, AssertJ, ArchUnit, Testcontainers, Toxiproxy, Awaitility, BlockHound, Nginx.
|
|
|
|
## Global Constraints
|
|
|
|
- 공개 API에는 `Path`, 실제 파일명, mount 경로를 노출하지 않는다.
|
|
- 공개 식별자는 opaque `FileId`와 `UploadId`다.
|
|
- metadata store가 상태와 공개 가능 여부의 authoritative source다.
|
|
- READY가 아닌 파일은 direct와 Nginx 경로 모두에서 다운로드할 수 없다.
|
|
- 로컬 staging·content·quarantine은 동일 `FileStore`에 둔다.
|
|
- create-only가 기본이며 overwrite에는 `If-Match` 또는 metadata version이 필요하다.
|
|
- 서버 계산 SHA-256과 actual size를 저장한다.
|
|
- client filename과 `Content-Type`은 비신뢰 metadata다.
|
|
- Spring MVC streaming은 bounded 전용 executor를 사용한다.
|
|
- Spring WebFlux event loop에서 filesystem, JDBC, scanner blocking call을 실행하지 않는다.
|
|
- multi-instance upload는 DB writer lease와 optimistic version을 사용한다.
|
|
- NFS lock을 단독 정합성 근거로 사용하지 않는다.
|
|
- timeout 후 write는 blind retry하지 않고 ambiguous completion을 표현한다.
|
|
- tus 1.0은 Stable 모듈, HTTPbis draft-12는 Experimental 모듈이다.
|
|
- arbitrary path, symlink follow, hard link 생성, recursive delete는 구현하지 않는다.
|
|
- 실제 file ID, filename, path, checksum 원문을 metric label에 기록하지 않는다.
|
|
- 모든 작업은 실패 테스트 작성 → 실패 확인 → 최소 구현 → 통과 확인 → 커밋 순서로 진행한다.
|
|
- 각 작업은 독립 검토가 가능한 하나의 커밋으로 종료한다.
|
|
|
|
---
|
|
|
|
## 1. 확정 파일 구조
|
|
|
|
```text
|
|
backend-skeleton/
|
|
├── settings.gradle.kts
|
|
├── build.gradle.kts
|
|
├── build-logic/
|
|
│ └── src/main/kotlin/fileserver-library-conventions.gradle.kts
|
|
├── modules/fileserver/
|
|
│ ├── fileserver-core-api/
|
|
│ ├── fileserver-application/
|
|
│ ├── fileserver-metadata-jpa/
|
|
│ ├── fileserver-storage-local/
|
|
│ ├── fileserver-verification/
|
|
│ ├── fileserver-mvc/
|
|
│ ├── fileserver-webflux/
|
|
│ ├── fileserver-nginx/
|
|
│ ├── fileserver-admin/
|
|
│ ├── fileserver-tus/
|
|
│ ├── fileserver-resumable-httpbis-draft12/
|
|
│ ├── fileserver-spring-boot-starter/
|
|
│ └── fileserver-testkit/
|
|
├── infra/fileserver/
|
|
│ ├── nginx/
|
|
│ ├── nfs/
|
|
│ └── kubernetes/
|
|
├── docs/fileserver/
|
|
│ ├── support-matrix.md
|
|
│ ├── http-contract.md
|
|
│ ├── storage-certification.md
|
|
│ ├── security.md
|
|
│ ├── operations.md
|
|
│ └── upgrade-guide.md
|
|
└── docs/superpowers/specs/2026-08-07-fileserver-platform-design.md
|
|
```
|
|
|
|
## 2. 핵심 패키지
|
|
|
|
```text
|
|
io.backend.skeleton.fileserver.api
|
|
io.backend.skeleton.fileserver.api.content
|
|
io.backend.skeleton.fileserver.api.error
|
|
io.backend.skeleton.fileserver.api.metadata
|
|
io.backend.skeleton.fileserver.api.security
|
|
io.backend.skeleton.fileserver.api.transfer
|
|
io.backend.skeleton.fileserver.application
|
|
io.backend.skeleton.fileserver.jpa
|
|
io.backend.skeleton.fileserver.local
|
|
io.backend.skeleton.fileserver.verification
|
|
io.backend.skeleton.fileserver.mvc
|
|
io.backend.skeleton.fileserver.webflux
|
|
io.backend.skeleton.fileserver.nginx
|
|
io.backend.skeleton.fileserver.admin
|
|
io.backend.skeleton.fileserver.tus
|
|
io.backend.skeleton.fileserver.httpbisdraft12
|
|
io.backend.skeleton.fileserver.autoconfigure
|
|
io.backend.skeleton.fileserver.testkit
|
|
```
|
|
|
|
---
|
|
|
|
### Task 1: Gradle 멀티모듈과 공통 품질 규칙 구성
|
|
|
|
**Files:**
|
|
- Modify: `settings.gradle.kts`
|
|
- Create: `build-logic/src/main/kotlin/fileserver-library-conventions.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-core-api/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-application/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-storage-local/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-verification/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-mvc/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-webflux/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-nginx/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-admin/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-tus/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-resumable-httpbis-draft12/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/build.gradle.kts`
|
|
- Create: `modules/fileserver/fileserver-testkit/build.gradle.kts`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/ModuleSmokeTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces all Gradle project paths used by later tasks.
|
|
- `fileserver-core-api` must have no Spring MVC, WebFlux, JPA, NIO filesystem implementation dependency.
|
|
- Java toolchain is 21.
|
|
|
|
- [ ] **Step 1: Write the failing core module smoke test**
|
|
|
|
```java
|
|
package io.backend.skeleton.fileserver.api;
|
|
|
|
import org.junit.jupiter.api.Test;
|
|
|
|
import static org.assertj.core.api.Assertions.assertThat;
|
|
|
|
class ModuleSmokeTest {
|
|
@Test
|
|
void coreApiModuleLoads() {
|
|
assertThat(ModuleSmokeTest.class.getPackageName())
|
|
.isEqualTo("io.backend.skeleton.fileserver.api");
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Register module paths and verify the build fails before module build files exist**
|
|
|
|
Add to `settings.gradle.kts`:
|
|
|
|
```kotlin
|
|
include(
|
|
":modules:fileserver:fileserver-core-api",
|
|
":modules:fileserver:fileserver-application",
|
|
":modules:fileserver:fileserver-metadata-jpa",
|
|
":modules:fileserver:fileserver-storage-local",
|
|
":modules:fileserver:fileserver-verification",
|
|
":modules:fileserver:fileserver-mvc",
|
|
":modules:fileserver:fileserver-webflux",
|
|
":modules:fileserver:fileserver-nginx",
|
|
":modules:fileserver:fileserver-admin",
|
|
":modules:fileserver:fileserver-tus",
|
|
":modules:fileserver:fileserver-resumable-httpbis-draft12",
|
|
":modules:fileserver:fileserver-spring-boot-starter",
|
|
":modules:fileserver:fileserver-testkit"
|
|
)
|
|
```
|
|
|
|
Run:
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test
|
|
```
|
|
|
|
Expected: FAIL because the registered module build files do not exist.
|
|
|
|
- [ ] **Step 3: Add the convention plugin and module dependency boundaries**
|
|
|
|
Create `fileserver-library-conventions.gradle.kts`:
|
|
|
|
```kotlin
|
|
plugins {
|
|
`java-library`
|
|
id("java-test-fixtures")
|
|
}
|
|
|
|
java {
|
|
toolchain {
|
|
languageVersion.set(JavaLanguageVersion.of(21))
|
|
}
|
|
}
|
|
|
|
tasks.withType<Test>().configureEach {
|
|
useJUnitPlatform()
|
|
failFast = false
|
|
}
|
|
|
|
dependencies {
|
|
"testImplementation"(platform("org.junit:junit-bom:5.12.2"))
|
|
"testImplementation"("org.junit.jupiter:junit-jupiter")
|
|
"testImplementation"("org.assertj:assertj-core:3.27.3")
|
|
}
|
|
```
|
|
|
|
Apply it to every Fileserver module. Add only these directed dependencies:
|
|
|
|
```text
|
|
application → core-api
|
|
metadata-jpa → core-api
|
|
storage-local → core-api
|
|
verification → core-api
|
|
mvc → application, core-api
|
|
webflux → application, core-api
|
|
nginx → application, core-api
|
|
admin → application, core-api
|
|
tus → application, core-api
|
|
httpbis-draft12 → application, core-api
|
|
starter → all runtime modules
|
|
testkit → core-api, application
|
|
```
|
|
|
|
- [ ] **Step 4: Run module tests and dependency report**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
:modules:fileserver:fileserver-core-api:dependencies
|
|
```
|
|
|
|
Expected: PASS; dependency report contains no Spring MVC, WebFlux, Hibernate, or `java.nio.file.Path`-specific adapter library.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add settings.gradle.kts build-logic modules/fileserver
|
|
git commit -m "build: add fileserver module boundaries"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: 식별자, 상태, 범위 값 객체 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/FileId.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/UploadId.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/ContentKey.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/StorageNamespace.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/FileState.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/ByteRange.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/FileStateMachine.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/DefaultFileStateMachine.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/FileStateMachineTest.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/ValueObjectTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces `FileId`, `UploadId`, `ContentKey`, `StorageNamespace`, `FileState`, `ByteRange`.
|
|
- Later persistence and HTTP tasks use these exact types.
|
|
|
|
- [ ] **Step 1: Write failing value object and transition tests**
|
|
|
|
```java
|
|
class FileStateMachineTest {
|
|
private final FileStateMachine stateMachine = new DefaultFileStateMachine();
|
|
|
|
@Test
|
|
void allowsUploadedToVerifying() {
|
|
assertThat(stateMachine.canTransition(FileState.UPLOADED, FileState.VERIFYING))
|
|
.isTrue();
|
|
}
|
|
|
|
@Test
|
|
void rejectsCreatedToReady() {
|
|
assertThatThrownBy(() ->
|
|
stateMachine.requireTransition(FileState.CREATED, FileState.READY))
|
|
.isInstanceOf(IllegalStateException.class)
|
|
.hasMessageContaining("CREATED -> READY");
|
|
}
|
|
}
|
|
```
|
|
|
|
```java
|
|
class ValueObjectTest {
|
|
@Test
|
|
void rejectsInvalidContentKey() {
|
|
assertThatThrownBy(() -> new ContentKey("../../etc/passwd"))
|
|
.isInstanceOf(IllegalArgumentException.class);
|
|
}
|
|
|
|
@Test
|
|
void calculatesInclusiveRangeLength() {
|
|
assertThat(new ByteRange(10, 19).length()).isEqualTo(10);
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
--tests '*FileStateMachineTest' --tests '*ValueObjectTest'
|
|
```
|
|
|
|
Expected: FAIL because the types do not exist.
|
|
|
|
- [ ] **Step 3: Implement exact state transitions and validation**
|
|
|
|
```java
|
|
public final class DefaultFileStateMachine implements FileStateMachine {
|
|
private static final Map<FileState, Set<FileState>> ALLOWED = Map.ofEntries(
|
|
Map.entry(FileState.CREATED, Set.of(FileState.UPLOADING)),
|
|
Map.entry(FileState.UPLOADING, Set.of(
|
|
FileState.UPLOADED, FileState.FAILED, FileState.EXPIRED, FileState.DELETING)),
|
|
Map.entry(FileState.UPLOADED, Set.of(
|
|
FileState.VERIFYING, FileState.FAILED, FileState.DELETING)),
|
|
Map.entry(FileState.VERIFYING, Set.of(
|
|
FileState.READY, FileState.QUARANTINED, FileState.REJECTED, FileState.FAILED)),
|
|
Map.entry(FileState.QUARANTINED, Set.of(
|
|
FileState.VERIFYING, FileState.READY, FileState.REJECTED, FileState.DELETING)),
|
|
Map.entry(FileState.READY, Set.of(FileState.DELETING)),
|
|
Map.entry(FileState.REJECTED, Set.of(FileState.DELETING)),
|
|
Map.entry(FileState.FAILED, Set.of(
|
|
FileState.UPLOADING, FileState.VERIFYING, FileState.DELETING, FileState.EXPIRED)),
|
|
Map.entry(FileState.DELETING, Set.of(FileState.DELETED, FileState.FAILED)),
|
|
Map.entry(FileState.EXPIRED, Set.of(FileState.DELETING)),
|
|
Map.entry(FileState.DELETED, Set.of())
|
|
);
|
|
|
|
@Override
|
|
public boolean canTransition(FileState current, FileState target) {
|
|
return ALLOWED.getOrDefault(current, Set.of()).contains(target);
|
|
}
|
|
|
|
@Override
|
|
public void requireTransition(FileState current, FileState target) {
|
|
if (!canTransition(current, target)) {
|
|
throw new IllegalStateException("illegal file transition: " + current + " -> " + target);
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Implement ID records with non-null validation and `ContentKey`/namespace regex exactly as the design document.
|
|
|
|
- [ ] **Step 4: Run the module tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api
|
|
git commit -m "feat: add fileserver core value objects and state machine"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: 안정된 오류 모델과 failure context 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileserverException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileserverFailureContext.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/UploadOffsetMismatchException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/AmbiguousCompletionException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileNotReadyException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/StorageFullException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/IntegrityMismatchException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileNotFoundException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileAlreadyExistsException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/InvalidPathException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/PathOutsideNamespaceException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileAccessDeniedException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/QuotaExceededException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileTooLargeException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/UnsupportedMediaTypeException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/UploadExpiredException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/AtomicPublishUnsupportedException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/TransferTimeoutException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/PartialWriteException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/StorageUnavailableException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/ConcurrentFileModificationException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/MalwareDetectedException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/RangeNotSatisfiableException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/TransferAdmissionRejectedException.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileserverErrorCode.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/error/FileserverExceptionTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces `FileserverException#context()` and stable `FileserverErrorCode` values.
|
|
- HTTP adapters map these errors without inspecting storage-driver exceptions.
|
|
|
|
- [ ] **Step 1: Write a failing ambiguous execution test**
|
|
|
|
```java
|
|
class FileserverExceptionTest {
|
|
@Test
|
|
void ambiguousCompletionCarriesReconciliationFlag() {
|
|
AmbiguousCompletionException exception = new AmbiguousCompletionException(
|
|
"publish result is unknown",
|
|
FileserverFailureContext.forUpload(
|
|
FileserverErrorCode.AMBIGUOUS_COMPLETION,
|
|
new UploadId(UUID.randomUUID()),
|
|
false,
|
|
true,
|
|
true
|
|
)
|
|
);
|
|
|
|
assertThat(exception.context().ambiguous()).isTrue();
|
|
assertThat(exception.context().reconciliationRequired()).isTrue();
|
|
assertThat(exception.context().retryable()).isFalse();
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the test to verify it fails**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
--tests '*FileserverExceptionTest'
|
|
```
|
|
|
|
Expected: FAIL because the exception hierarchy does not exist.
|
|
|
|
- [ ] **Step 3: Implement the hierarchy and context**
|
|
|
|
```java
|
|
public abstract class FileserverException extends RuntimeException {
|
|
private final FileserverFailureContext context;
|
|
|
|
protected FileserverException(String message, FileserverFailureContext context) {
|
|
super(message);
|
|
this.context = Objects.requireNonNull(context, "context");
|
|
}
|
|
|
|
public final FileserverFailureContext context() {
|
|
return context;
|
|
}
|
|
}
|
|
```
|
|
|
|
```java
|
|
public record FileserverFailureContext(
|
|
FileserverErrorCode code,
|
|
boolean retryable,
|
|
boolean ambiguous,
|
|
boolean reconciliationRequired,
|
|
Optional<FileId> fileId,
|
|
Optional<UploadId> uploadId,
|
|
OptionalLong expectedOffset,
|
|
OptionalLong currentOffset,
|
|
Optional<FileState> currentState
|
|
) {}
|
|
```
|
|
|
|
Add all design error codes, including `FILE_NOT_FOUND`, `FILE_NOT_READY`, `FILE_TOO_LARGE`, `QUOTA_EXCEEDED`, `STORAGE_FULL`, `UPLOAD_OFFSET_MISMATCH`, `INTEGRITY_MISMATCH`, `CONCURRENT_MODIFICATION`, `STORAGE_UNAVAILABLE`, and `AMBIGUOUS_COMPLETION`.
|
|
|
|
- [ ] **Step 4: Run error tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
--tests '*FileserverExceptionTest'
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error \
|
|
modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/error
|
|
git commit -m "feat: define fileserver failure semantics"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Content Store capability와 blocking·async Port 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/ContentStoreCapabilities.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/BlockingContentStore.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/AsyncContentStore.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/UploadHandle.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/CreateContentCommand.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/FinalizeContentCommand.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/AppendResult.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/StoredContent.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/ContentMetadata.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/DeletePrecondition.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/DeleteResult.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/content/ContentStoreApiArchitectureTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces the exact storage SPI consumed by application and implemented by local storage.
|
|
- No public signature may include `Path`, `Resource`, `DataBuffer`, `Flux`, or provider SDK types.
|
|
|
|
- [ ] **Step 1: Write a failing architecture test**
|
|
|
|
```java
|
|
class ContentStoreApiArchitectureTest {
|
|
@Test
|
|
void publicContentApiDoesNotExposeFrameworkOrFilesystemTypes() {
|
|
Set<String> forbidden = Set.of(
|
|
"java.nio.file.Path",
|
|
"org.springframework.core.io.Resource",
|
|
"org.springframework.core.io.buffer.DataBuffer",
|
|
"reactor.core.publisher.Flux"
|
|
);
|
|
|
|
for (Method method : BlockingContentStore.class.getMethods()) {
|
|
assertThat(method.getReturnType().getName()).isNotIn(forbidden);
|
|
assertThat(Arrays.stream(method.getParameterTypes()).map(Class::getName))
|
|
.doesNotContainAnyElementsOf(forbidden);
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the test to verify it fails**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
--tests '*ContentStoreApiArchitectureTest'
|
|
```
|
|
|
|
Expected: FAIL because the interfaces do not exist.
|
|
|
|
- [ ] **Step 3: Implement the blocking and async contracts**
|
|
|
|
Use these signatures exactly:
|
|
|
|
```java
|
|
public interface BlockingContentStore {
|
|
UploadHandle createUpload(CreateContentCommand command);
|
|
AppendResult append(UploadHandle handle, long expectedOffset,
|
|
ReadableByteChannel source, long contentLength);
|
|
StoredContent finalizeUpload(UploadHandle handle, FinalizeContentCommand command);
|
|
ContentMetadata stat(ContentKey key);
|
|
ReadableByteChannel openRead(ContentKey key, ByteRange range);
|
|
DeleteResult delete(ContentKey key, DeletePrecondition precondition);
|
|
ContentStoreCapabilities capabilities();
|
|
}
|
|
```
|
|
|
|
```java
|
|
public interface AsyncContentStore {
|
|
CompletionStage<UploadHandle> createUpload(CreateContentCommand command);
|
|
CompletionStage<AppendResult> append(
|
|
UploadHandle handle, long expectedOffset, Flow.Publisher<ByteBuffer> content);
|
|
CompletionStage<StoredContent> finalizeUpload(
|
|
UploadHandle handle, FinalizeContentCommand command);
|
|
CompletionStage<ContentMetadata> stat(ContentKey key);
|
|
Flow.Publisher<ByteBuffer> openRead(ContentKey key, ByteRange range);
|
|
CompletionStage<DeleteResult> delete(
|
|
ContentKey key, DeletePrecondition precondition);
|
|
ContentStoreCapabilities capabilities();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run API and architecture tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test
|
|
```
|
|
|
|
Expected: PASS; `jdeps` or ArchUnit output confirms no forbidden adapter dependency.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api
|
|
git commit -m "feat: define content store ports"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Metadata Store, upload session, lease, quota Port 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/FileRecord.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/FileRecordDraft.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/FileRecordMutation.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/FileDescriptor.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/FileRecoveryQuery.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/FileMetadataStore.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/UploadSession.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/UploadSessionDraft.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/UploadSessionStore.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/WriterLease.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/QuotaReservation.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/metadata/FileQuotaService.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/metadata/MetadataPortContractTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces optimistic transition and writer lease signatures used by Tasks 6, 12, 15, and 24.
|
|
- Offset commit always requires a lease token and expected offset.
|
|
|
|
- [ ] **Step 1: Write failing port signature tests**
|
|
|
|
```java
|
|
class MetadataPortContractTest {
|
|
@Test
|
|
void offsetCommitRequiresLeaseAndExpectedOffset() throws Exception {
|
|
Method method = UploadSessionStore.class.getMethod(
|
|
"commitOffset",
|
|
UploadId.class,
|
|
WriterLease.class,
|
|
long.class,
|
|
long.class
|
|
);
|
|
|
|
assertThat(method.getReturnType()).isEqualTo(UploadSession.class);
|
|
}
|
|
|
|
@Test
|
|
void fileTransitionRequiresExpectedVersionAndState() throws Exception {
|
|
Method method = FileMetadataStore.class.getMethod(
|
|
"transition",
|
|
FileId.class,
|
|
long.class,
|
|
FileState.class,
|
|
FileState.class,
|
|
FileRecordMutation.class
|
|
);
|
|
|
|
assertThat(method).isNotNull();
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
--tests '*MetadataPortContractTest'
|
|
```
|
|
|
|
Expected: FAIL because the port types do not exist.
|
|
|
|
- [ ] **Step 3: Implement metadata records and exact methods**
|
|
|
|
```java
|
|
public interface FileMetadataStore {
|
|
FileRecord insert(FileRecordDraft draft);
|
|
Optional<FileRecord> find(FileId fileId);
|
|
FileRecord transition(
|
|
FileId fileId,
|
|
long expectedVersion,
|
|
FileState expectedState,
|
|
FileState targetState,
|
|
FileRecordMutation mutation
|
|
);
|
|
FileRecord markDeleting(FileId fileId, long expectedVersion);
|
|
List<FileRecord> findRecoverable(FileRecoveryQuery query);
|
|
}
|
|
```
|
|
|
|
```java
|
|
public interface UploadSessionStore {
|
|
UploadSession create(UploadSessionDraft draft);
|
|
Optional<UploadSession> find(UploadId uploadId);
|
|
WriterLease acquireLease(
|
|
UploadId uploadId,
|
|
String owner,
|
|
Instant now,
|
|
Duration leaseDuration,
|
|
long expectedVersion
|
|
);
|
|
UploadSession commitOffset(
|
|
UploadId uploadId,
|
|
WriterLease lease,
|
|
long expectedOffset,
|
|
long committedOffset
|
|
);
|
|
void releaseLease(UploadId uploadId, WriterLease lease);
|
|
List<UploadSession> findExpired(Instant cutoff, int limit);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run the core API tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api
|
|
git commit -m "feat: define fileserver metadata and lease ports"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Flyway metadata schema와 JPA entity 구성
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/resources/db/migration/fileserver/V1__create_fileserver_metadata.sql`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/entity/FileEntity.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/entity/UploadSessionEntity.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/entity/VerificationResultEntity.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/entity/QuotaReservationEntity.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/entity/CleanupItemEntity.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/repository/JpaFileRepository.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/repository/JpaUploadSessionRepository.java`
|
|
- Test: `modules/fileserver/fileserver-metadata-jpa/src/test/java/io/backend/skeleton/fileserver/jpa/FileserverMigrationTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes `FileState`, IDs, and metadata records from Tasks 2 and 5.
|
|
- Produces database tables and JPA repositories used by Task 7.
|
|
|
|
- [ ] **Step 1: Write a failing migration test**
|
|
|
|
```java
|
|
@Testcontainers
|
|
class FileserverMigrationTest {
|
|
@Container
|
|
static final PostgreSQLContainer<?> POSTGRES =
|
|
new PostgreSQLContainer<>("postgres:17-alpine");
|
|
|
|
@Test
|
|
void createsFileserverTablesAndVersionColumns() throws Exception {
|
|
Flyway.configure()
|
|
.dataSource(POSTGRES.getJdbcUrl(), POSTGRES.getUsername(), POSTGRES.getPassword())
|
|
.locations("classpath:db/migration/fileserver")
|
|
.load()
|
|
.migrate();
|
|
|
|
try (Connection connection = DriverManager.getConnection(
|
|
POSTGRES.getJdbcUrl(), POSTGRES.getUsername(), POSTGRES.getPassword())) {
|
|
assertThat(columnExists(connection, "fs_file", "version")).isTrue();
|
|
assertThat(columnExists(connection, "fs_upload_session", "lease_until")).isTrue();
|
|
assertThat(columnExists(connection, "fs_quota_reservation", "reserved_bytes")).isTrue();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the migration test to verify it fails**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-metadata-jpa:test \
|
|
--tests '*FileserverMigrationTest'
|
|
```
|
|
|
|
Expected: FAIL because the migration does not exist.
|
|
|
|
- [ ] **Step 3: Create the schema and entity mappings**
|
|
|
|
Use the following core DDL shape:
|
|
|
|
```sql
|
|
create table fs_file (
|
|
file_id uuid primary key,
|
|
namespace varchar(63) not null,
|
|
state varchar(32) not null,
|
|
content_key varchar(200),
|
|
original_name varchar(255) not null,
|
|
claimed_media_type varchar(255),
|
|
verified_media_type varchar(255),
|
|
expected_size bigint,
|
|
actual_size bigint,
|
|
sha256 char(64),
|
|
strong_etag varchar(80),
|
|
published_at timestamptz,
|
|
last_error_code varchar(64),
|
|
version bigint not null default 0,
|
|
created_at timestamptz not null,
|
|
updated_at timestamptz not null,
|
|
constraint ck_fs_file_size check (actual_size is null or actual_size >= 0)
|
|
);
|
|
|
|
create table fs_upload_session (
|
|
upload_id uuid primary key,
|
|
file_id uuid not null references fs_file(file_id),
|
|
protocol varchar(32) not null,
|
|
expected_length bigint,
|
|
committed_offset bigint not null default 0,
|
|
expires_at timestamptz not null,
|
|
lease_owner varchar(128),
|
|
lease_token uuid,
|
|
lease_until timestamptz,
|
|
version bigint not null default 0,
|
|
created_at timestamptz not null,
|
|
updated_at timestamptz not null,
|
|
constraint ck_fs_upload_offset check (committed_offset >= 0)
|
|
);
|
|
```
|
|
|
|
Add the verification, quota, and cleanup tables from the design with indexes on state, expiry, lease, and cleanup schedule. Map optimistic version with `@Version`.
|
|
|
|
- [ ] **Step 4: Run migration and JPA schema validation**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-metadata-jpa:test \
|
|
--tests '*FileserverMigrationTest'
|
|
```
|
|
|
|
Expected: PASS; Hibernate schema validation reports no mismatch.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-metadata-jpa
|
|
git commit -m "feat: add fileserver metadata schema"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: JPA Metadata Store와 optimistic transition 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/JpaFileMetadataStore.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/JpaUploadSessionStore.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/JpaFileQuotaService.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/FileEntityMapper.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/repository/FileTransitionRepository.java`
|
|
- Create: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/repository/UploadLeaseRepository.java`
|
|
- Test: `modules/fileserver/fileserver-metadata-jpa/src/test/java/io/backend/skeleton/fileserver/jpa/JpaFileMetadataStoreTest.java`
|
|
- Test: `modules/fileserver/fileserver-metadata-jpa/src/test/java/io/backend/skeleton/fileserver/jpa/JpaUploadSessionStoreTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes metadata ports from Task 5 and schema from Task 6.
|
|
- Produces transactional implementations used by the application layer.
|
|
|
|
- [ ] **Step 1: Write failing concurrent transition and lease tests**
|
|
|
|
```java
|
|
@Test
|
|
void onlyOneReadyTransitionWinsForTheSameVersion() {
|
|
FileRecord record = fixture.insertVerifyingFile();
|
|
|
|
CompletableFuture<FileRecord> first = async(() -> store.transition(
|
|
record.fileId(), record.version(), FileState.VERIFYING, FileState.READY,
|
|
FileRecordMutation.publish(fixture.contentKey(), 10, fixture.sha256(), fixture.etag())));
|
|
CompletableFuture<FileRecord> second = async(() -> store.transition(
|
|
record.fileId(), record.version(), FileState.VERIFYING, FileState.READY,
|
|
FileRecordMutation.publish(fixture.contentKey(), 10, fixture.sha256(), fixture.etag())));
|
|
|
|
assertThat(successCount(first, second)).isEqualTo(1);
|
|
assertThat(concurrentModificationCount(first, second)).isEqualTo(1);
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void onlyOneWriterLeaseIsValid() {
|
|
UploadSession session = fixture.insertActiveUpload();
|
|
Instant now = Instant.parse("2026-08-07T10:00:00Z");
|
|
|
|
WriterLease first = store.acquireLease(
|
|
session.uploadId(), "node-a", now, Duration.ofSeconds(30), session.version());
|
|
|
|
assertThatThrownBy(() -> store.acquireLease(
|
|
session.uploadId(), "node-b", now.plusSeconds(1), Duration.ofSeconds(30), session.version()))
|
|
.isInstanceOf(ConcurrentFileModificationException.class);
|
|
assertThat(first.owner()).isEqualTo("node-a");
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-metadata-jpa:test \
|
|
--tests '*JpaFileMetadataStoreTest' --tests '*JpaUploadSessionStoreTest'
|
|
```
|
|
|
|
Expected: FAIL because store implementations do not exist.
|
|
|
|
- [ ] **Step 3: Implement conditional update repositories**
|
|
|
|
Use an update query that includes both state and version:
|
|
|
|
```java
|
|
@Modifying
|
|
@Query("""
|
|
update FileEntity f
|
|
set f.state = :targetState,
|
|
f.contentKey = :contentKey,
|
|
f.actualSize = :actualSize,
|
|
f.sha256 = :sha256,
|
|
f.strongEtag = :strongEtag,
|
|
f.publishedAt = :publishedAt,
|
|
f.version = f.version + 1,
|
|
f.updatedAt = :updatedAt
|
|
where f.fileId = :fileId
|
|
and f.state = :expectedState
|
|
and f.version = :expectedVersion
|
|
""")
|
|
int transition(...);
|
|
```
|
|
|
|
Lease acquisition must update only when `lease_until is null or lease_until < now` and the expected version matches. `commitOffset` must require matching `lease_token`, current offset, and unexpired lease.
|
|
|
|
- [ ] **Step 4: Run all JPA tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-metadata-jpa:test
|
|
```
|
|
|
|
Expected: PASS; repeated concurrency runs produce one winner only.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-metadata-jpa
|
|
git commit -m "feat: implement fileserver metadata stores"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 8: 원본 파일명 sanitization과 path 정책 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/OriginalFilenamePolicy.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/SanitizedFilename.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalStorageLayout.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/PhysicalPathResolver.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/DefaultPhysicalPathResolver.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/security/OriginalFilenamePolicyTest.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/PhysicalPathResolverTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces sanitized display names and package-private physical path resolution.
|
|
- No controller may call `PhysicalPathResolver` directly.
|
|
|
|
- [ ] **Step 1: Write failing malicious filename and root escape tests**
|
|
|
|
```java
|
|
class OriginalFilenamePolicyTest {
|
|
private final OriginalFilenamePolicy policy = new OriginalFilenamePolicy(255);
|
|
|
|
@Test
|
|
void removesPathAndHeaderInjectionCharacters() {
|
|
SanitizedFilename result = policy.sanitize("../report\r\nX-Test: yes.pdf");
|
|
|
|
assertThat(result.value()).doesNotContain("..", "/", "\\", "\r", "\n");
|
|
assertThat(result.value()).endsWith(".pdf");
|
|
}
|
|
|
|
@Test
|
|
void replacesWindowsReservedName() {
|
|
assertThat(policy.sanitize("CON").value()).isEqualTo("_CON");
|
|
}
|
|
}
|
|
```
|
|
|
|
```java
|
|
class PhysicalPathResolverTest {
|
|
@TempDir Path root;
|
|
|
|
@Test
|
|
void generatedContentPathAlwaysStaysBelowContentRoot() {
|
|
DefaultPhysicalPathResolver resolver = new DefaultPhysicalPathResolver(root);
|
|
Path result = resolver.contentPath(new ContentKey("ab/cd/0123456789abcdef"));
|
|
|
|
assertThat(result.normalize()).startsWith(root.resolve("content").normalize());
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
:modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*OriginalFilenamePolicyTest' --tests '*PhysicalPathResolverTest'
|
|
```
|
|
|
|
Expected: FAIL because policy and resolver do not exist.
|
|
|
|
- [ ] **Step 3: Implement sanitization and server-generated layout**
|
|
|
|
`OriginalFilenamePolicy` must:
|
|
|
|
```text
|
|
strip path separators and NUL
|
|
replace control and bidi override characters
|
|
remove CR/LF and quote injection
|
|
trim trailing dot and space
|
|
prefix Windows reserved names with `_`
|
|
truncate by UTF-8 byte length, preserving the final extension when possible
|
|
return `file` when the normalized name becomes empty
|
|
```
|
|
|
|
`DefaultPhysicalPathResolver` must only accept validated IDs and construct:
|
|
|
|
```text
|
|
staging/<first-two>/<next-two>/<upload-id>.part
|
|
content/<first-two>/<next-two>/<content-key>.bin
|
|
quarantine/<first-two>/<next-two>/<content-key>.bin
|
|
```
|
|
|
|
- [ ] **Step 4: Run filename and path tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
:modules:fileserver:fileserver-storage-local:test
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api modules/fileserver/fileserver-storage-local
|
|
git commit -m "feat: enforce fileserver filename and path policy"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 9: Local staging 생성과 `CREATE_NEW` 경쟁 제어 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalBlockingContentStore.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalStorageProperties.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/SafeFileChannelFactory.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalUploadHandle.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/LocalCreateUploadTest.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/LocalCreateUploadConcurrencyTest.java`
|
|
|
|
**Interfaces:**
|
|
- Implements `BlockingContentStore#createUpload` from Task 4.
|
|
- Produces `LocalUploadHandle` used by append and finalize tasks.
|
|
|
|
- [ ] **Step 1: Write failing create-only and concurrent-create tests**
|
|
|
|
```java
|
|
@Test
|
|
void createsStagingFileWithZeroLengthAndNoOriginalName() {
|
|
UploadHandle handle = store.createUpload(commandFor("../../secret.pdf"));
|
|
|
|
Path staging = testSupport.pathOf(handle);
|
|
assertThat(staging).exists().isEmptyFile();
|
|
assertThat(staging.getFileName().toString()).doesNotContain("secret.pdf");
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void exactlyOneConcurrentCreateWinsForSameUploadId() {
|
|
CreateContentCommand command = fixture.commandWithFixedUploadId();
|
|
|
|
List<Throwable> failures = runConcurrently(2, () -> store.createUpload(command));
|
|
|
|
assertThat(failures).hasSize(1);
|
|
assertThat(failures.getFirst()).isInstanceOf(FileAlreadyExistsException.class);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*LocalCreateUploadTest' --tests '*LocalCreateUploadConcurrencyTest'
|
|
```
|
|
|
|
Expected: FAIL because local store is not implemented.
|
|
|
|
- [ ] **Step 3: Implement safe staging creation**
|
|
|
|
Open the staging file with:
|
|
|
|
```java
|
|
Set<OpenOption> options = Set.of(
|
|
StandardOpenOption.CREATE_NEW,
|
|
StandardOpenOption.WRITE,
|
|
LinkOption.NOFOLLOW_LINKS
|
|
);
|
|
```
|
|
|
|
Create parent directories from server-generated components only. Before and after open, verify that no parent is a symbolic link. Set owner-only permissions on POSIX providers. Convert `FileAlreadyExistsException`, `AccessDeniedException`, and `FileSystemException` into stable Fileserver errors.
|
|
|
|
- [ ] **Step 4: Run local storage creation tests repeatedly**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*LocalCreateUpload*' --rerun-tasks
|
|
```
|
|
|
|
Expected: PASS for 20 repeated runs; exactly one concurrent create succeeds.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-storage-local
|
|
git commit -m "feat: create safe local upload staging files"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 10: Storage capability probe와 startup gate 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalStorageCapabilityProbe.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalStorageProbeResult.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/content/PublishMode.java`
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/src/main/java/io/backend/skeleton/fileserver/autoconfigure/FileserverStartupValidator.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/LocalStorageCapabilityProbeTest.java`
|
|
- Test: `modules/fileserver/fileserver-spring-boot-starter/src/test/java/io/backend/skeleton/fileserver/autoconfigure/FileserverStartupValidatorTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces runtime `ContentStoreCapabilities` and selected `PublishMode`.
|
|
- Later finalize logic must consume this result instead of assuming atomic move.
|
|
|
|
- [ ] **Step 1: Write failing same-FileStore and required-atomic tests**
|
|
|
|
```java
|
|
@Test
|
|
void reportsAtomicCreateAndSameFileStore() {
|
|
LocalStorageProbeResult result = probe.run();
|
|
|
|
assertThat(result.atomicCreate()).isTrue();
|
|
assertThat(result.sameFileStore()).isTrue();
|
|
assertThat(result.symlinkNoFollow()).isTrue();
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void requiredAtomicModeRejectsUnsupportedStorage() {
|
|
LocalStorageProbeResult result = fixture.resultWithAtomicMove(false);
|
|
|
|
assertThatThrownBy(() -> validator.validate(
|
|
PublishMode.ATOMIC_MOVE_REQUIRED, result))
|
|
.isInstanceOf(IllegalStateException.class)
|
|
.hasMessageContaining("atomic move");
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
:modules:fileserver:fileserver-spring-boot-starter:test \
|
|
--tests '*LocalStorageCapabilityProbeTest' \
|
|
--tests '*FileserverStartupValidatorTest'
|
|
```
|
|
|
|
Expected: FAIL because probe and validator do not exist.
|
|
|
|
- [ ] **Step 3: Implement real filesystem probes**
|
|
|
|
The probe must create files below `${root}/probe` and verify:
|
|
|
|
```text
|
|
writable root
|
|
concurrent CREATE_NEW
|
|
staging/content/quarantine FileStore equality
|
|
ATOMIC_MOVE
|
|
replace semantics
|
|
NOFOLLOW_LINKS
|
|
open-delete behavior
|
|
capacity access
|
|
```
|
|
|
|
Delete all probe artifacts in `finally`. In `ATOMIC_MOVE_PREFERRED`, return `METADATA_POINTER` as fallback when atomic move is unavailable. In `ATOMIC_MOVE_REQUIRED`, fail startup.
|
|
|
|
- [ ] **Step 4: Run probe tests and a local integration probe**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
:modules:fileserver:fileserver-spring-boot-starter:test
|
|
```
|
|
|
|
Expected: PASS; probe directory is empty after completion.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-storage-local \
|
|
modules/fileserver/fileserver-core-api \
|
|
modules/fileserver/fileserver-spring-boot-starter
|
|
git commit -m "feat: probe fileserver storage capabilities"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 11: Streaming append, size 제한, SHA-256 계산 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalAppendEngine.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/StreamingDigest.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/TransferBufferPool.java`
|
|
- Modify: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalBlockingContentStore.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/LocalAppendEngineTest.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/LocalAppendMemoryTest.java`
|
|
|
|
**Interfaces:**
|
|
- Implements `BlockingContentStore#append`.
|
|
- Produces `AppendResult(committedOffset, appendedBytes, sha256Snapshot)`.
|
|
- Uses 128 KiB default buffer and never allocates proportional to file size.
|
|
|
|
- [ ] **Step 1: Write failing offset, digest, and bounded-buffer tests**
|
|
|
|
```java
|
|
@Test
|
|
void appendsAtExpectedOffsetAndCalculatesDigest() throws Exception {
|
|
UploadHandle handle = fixture.emptyUpload();
|
|
byte[] payload = "fileserver".getBytes(StandardCharsets.UTF_8);
|
|
|
|
AppendResult result = store.append(
|
|
handle, 0, Channels.newChannel(new ByteArrayInputStream(payload)), payload.length);
|
|
|
|
assertThat(result.committedOffset()).isEqualTo(payload.length);
|
|
assertThat(result.appendedBytes()).isEqualTo(payload.length);
|
|
assertThat(result.sha256()).isEqualTo(sha256Hex(payload));
|
|
}
|
|
|
|
@Test
|
|
void rejectsOffsetMismatchWithoutWriting() throws Exception {
|
|
UploadHandle handle = fixture.uploadContaining("abc");
|
|
|
|
assertThatThrownBy(() -> store.append(
|
|
handle, 2, Channels.newChannel(new ByteArrayInputStream("d".getBytes())), 1))
|
|
.isInstanceOf(UploadOffsetMismatchException.class);
|
|
|
|
assertThat(fixture.readBytes(handle)).isEqualTo("abc".getBytes());
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void maxObservedBufferDoesNotGrowWithPayload() throws Exception {
|
|
fixture.appendGeneratedBytes(256L * 1024 * 1024);
|
|
assertThat(bufferPool.maxBorrowedBytes()).isLessThanOrEqualTo(128 * 1024);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*LocalAppendEngineTest' --tests '*LocalAppendMemoryTest'
|
|
```
|
|
|
|
Expected: FAIL because append engine and digest tracking do not exist.
|
|
|
|
- [ ] **Step 3: Implement sequential channel append**
|
|
|
|
```java
|
|
public AppendResult append(
|
|
Path staging,
|
|
long expectedOffset,
|
|
ReadableByteChannel source,
|
|
long contentLength,
|
|
long maximumFileSize
|
|
) {
|
|
try (FileChannel target = FileChannel.open(
|
|
staging, StandardOpenOption.WRITE, LinkOption.NOFOLLOW_LINKS)) {
|
|
long actualOffset = target.size();
|
|
if (actualOffset != expectedOffset) {
|
|
throw UploadOffsetMismatchException.of(expectedOffset, actualOffset);
|
|
}
|
|
target.position(expectedOffset);
|
|
return copyAndDigest(target, source, contentLength, maximumFileSize);
|
|
}
|
|
}
|
|
```
|
|
|
|
`copyAndDigest` must:
|
|
|
|
```text
|
|
borrow one bounded buffer
|
|
update SHA-256 for every written byte
|
|
stop immediately when maximumFileSize would be exceeded
|
|
verify fixed contentLength when non-negative
|
|
return only after bytes are written to the channel
|
|
release the buffer in finally
|
|
```
|
|
|
|
- [ ] **Step 4: Run append tests and inspect heap allocation**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*LocalAppend*'
|
|
```
|
|
|
|
Expected: PASS; 256 MiB test uses at most the configured transfer buffer plus test harness overhead.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-storage-local
|
|
git commit -m "feat: stream local file appends with sha256"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 12: Quota reservation과 transfer admission control 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/quota/QuotaScope.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/quota/TransferAdmissionController.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/quota/DefaultTransferAdmissionController.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/quota/TransferPermit.java`
|
|
- Modify: `modules/fileserver/fileserver-metadata-jpa/src/main/java/io/backend/skeleton/fileserver/jpa/JpaFileQuotaService.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/quota/TransferAdmissionControllerTest.java`
|
|
- Test: `modules/fileserver/fileserver-metadata-jpa/src/test/java/io/backend/skeleton/fileserver/jpa/JpaFileQuotaServiceTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes `FileQuotaService` from Task 5.
|
|
- Produces `TransferPermit` required before create or append.
|
|
- Default standard profile: 100 MiB file, 16 instance uploads, 4 scope uploads, soft 70%, hard 85%.
|
|
|
|
- [ ] **Step 1: Write failing quota and concurrency tests**
|
|
|
|
```java
|
|
@Test
|
|
void rejectsWhenScopeConcurrencyIsExhausted() {
|
|
TransferPermit first = controller.acquire(scope("tenant-a"), 10);
|
|
TransferPermit second = controller.acquire(scope("tenant-a"), 10);
|
|
TransferPermit third = controller.acquire(scope("tenant-a"), 10);
|
|
TransferPermit fourth = controller.acquire(scope("tenant-a"), 10);
|
|
|
|
assertThatThrownBy(() -> controller.acquire(scope("tenant-a"), 10))
|
|
.isInstanceOf(QuotaExceededException.class);
|
|
|
|
Stream.of(first, second, third, fourth).forEach(TransferPermit::close);
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void reservationCommitUsesActualBytesAndReleasesRemainder() {
|
|
QuotaReservation reservation = quota.reserve(scope, 1000, Duration.ofHours(1));
|
|
quota.commit(reservation, 600);
|
|
|
|
assertThat(fixture.committedBytes(scope)).isEqualTo(600);
|
|
assertThat(fixture.reservedBytes(scope)).isZero();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
:modules:fileserver:fileserver-metadata-jpa:test \
|
|
--tests '*TransferAdmissionControllerTest' --tests '*JpaFileQuotaServiceTest'
|
|
```
|
|
|
|
Expected: FAIL because admission control is not implemented.
|
|
|
|
- [ ] **Step 3: Implement reservation and bounded permits**
|
|
|
|
Use DB conditional updates for quota bytes and JVM semaphores for per-instance transfer concurrency. A create request with unknown length reserves the configured initial chunk; append extends the reservation before writing additional bytes. On cancellation or failure, release the reservation in `finally` or cleanup recovery.
|
|
|
|
```java
|
|
public interface TransferAdmissionController {
|
|
TransferPermit acquireUpload(QuotaScope scope, long requestedBytes);
|
|
TransferPermit acquireDirectDownload(QuotaScope scope);
|
|
}
|
|
```
|
|
|
|
A hard storage high-water condition maps to `StorageFullException`; scope limit maps to `QuotaExceededException`; temporary permit exhaustion maps to `TransferAdmissionRejectedException` with `retryable=true`.
|
|
|
|
- [ ] **Step 4: Run quota and concurrency tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
:modules:fileserver:fileserver-metadata-jpa:test
|
|
```
|
|
|
|
Expected: PASS; no permit or reservation remains after test cleanup.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-application \
|
|
modules/fileserver/fileserver-metadata-jpa
|
|
git commit -m "feat: enforce fileserver quota and transfer admission"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 13: Atomic move와 metadata pointer publish 전략 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/ContentPublisher.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/AtomicMoveContentPublisher.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/MetadataPointerContentPublisher.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/PublishResult.java`
|
|
- Modify: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalBlockingContentStore.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/AtomicMoveContentPublisherTest.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/MetadataPointerContentPublisherTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes `PublishMode` and probe results from Task 10.
|
|
- Implements `BlockingContentStore#finalizeUpload`.
|
|
- Produces immutable `StoredContent` and never exposes a partial final target.
|
|
|
|
- [ ] **Step 1: Write failing publish strategy tests**
|
|
|
|
```java
|
|
@Test
|
|
void atomicPublisherMovesStagingToCreateOnlyTarget() throws Exception {
|
|
LocalUploadHandle handle = fixture.uploadContaining("ready");
|
|
|
|
PublishResult result = publisher.publish(handle, fixture.finalizeCommand());
|
|
|
|
assertThat(result.contentPath()).exists();
|
|
assertThat(handle.stagingPath()).doesNotExist();
|
|
assertThat(Files.readString(result.contentPath())).isEqualTo("ready");
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void pointerPublisherKeepsImmutableObjectAndReturnsNewContentKey() throws Exception {
|
|
LocalUploadHandle handle = fixture.uploadContaining("ready");
|
|
|
|
PublishResult result = pointerPublisher.publish(handle, fixture.finalizeCommand());
|
|
|
|
assertThat(result.contentKey()).isNotNull();
|
|
assertThat(result.contentPath()).exists();
|
|
assertThat(result.atomicMoveUsed()).isFalse();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*ContentPublisherTest'
|
|
```
|
|
|
|
Expected: FAIL because publishers do not exist.
|
|
|
|
- [ ] **Step 3: Implement publish strategies**
|
|
|
|
`AtomicMoveContentPublisher` must use `ATOMIC_MOVE` and omit `REPLACE_EXISTING` for create-only. `MetadataPointerContentPublisher` must complete an immutable physical object under a fresh `ContentKey`; public visibility remains false until the application commits metadata READY.
|
|
|
|
Both implementations must:
|
|
|
|
```text
|
|
verify expected length
|
|
verify SHA-256
|
|
optionally force the channel according to durability profile
|
|
stat the final object
|
|
return actual size and content key
|
|
map uncertain filesystem results to AmbiguousCompletionException
|
|
```
|
|
|
|
- [ ] **Step 4: Run publish tests including process-visible observer checks**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*ContentPublisherTest' --rerun-tasks
|
|
```
|
|
|
|
Expected: PASS; observers see no partial final target.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-storage-local
|
|
git commit -m "feat: publish files with atomic or pointer strategy"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 14: Finalize orchestration과 READY invariant 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/FileVerificationService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/FinalizeUploadService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/DefaultFinalizeUploadService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/FinalizeUploadRequest.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/FileView.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/FinalizeUploadServiceTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes metadata stores, content store, state machine, quota service.
|
|
- Consumes the `FileVerificationService` Port created in this Task; Task 16 provides its production coordinator implementation. Tests use a deterministic ACCEPT stub.
|
|
- Produces READY or non-public VERIFYING/REJECTED results.
|
|
|
|
- [ ] **Step 1: Write failing READY and checksum mismatch tests**
|
|
|
|
```java
|
|
@Test
|
|
void publishesAndTransitionsToReadyOnlyAfterPhysicalVerification() {
|
|
FileView result = service.finalizeUpload(
|
|
fixture.uploadedSession(),
|
|
new FinalizeUploadRequest(Optional.of(fixture.sha256()), false),
|
|
fixture.context());
|
|
|
|
assertThat(result.state()).isEqualTo(FileState.READY);
|
|
assertThat(fixture.metadata(result.fileId()).contentKey()).isPresent();
|
|
assertThat(fixture.contentExists(result.fileId())).isTrue();
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void digestMismatchNeverTransitionsToReady() {
|
|
assertThatThrownBy(() -> service.finalizeUpload(
|
|
fixture.uploadedSession(),
|
|
new FinalizeUploadRequest(Optional.of("0".repeat(64)), false),
|
|
fixture.context()))
|
|
.isInstanceOf(IntegrityMismatchException.class);
|
|
|
|
assertThat(fixture.fileState()).isEqualTo(FileState.REJECTED);
|
|
assertThat(fixture.publicDownloadAvailable()).isFalse();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the test to verify it fails**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*FinalizeUploadServiceTest'
|
|
```
|
|
|
|
Expected: FAIL because finalize service does not exist.
|
|
|
|
- [ ] **Step 3: Implement the finalize sequence**
|
|
|
|
Implement this exact order:
|
|
|
|
```text
|
|
load upload and file
|
|
validate expected length
|
|
transition UPLOADING → UPLOADED when final append is complete
|
|
compare client digest if supplied
|
|
transition UPLOADED → VERIFYING
|
|
run verifier coordinator
|
|
on ACCEPT call contentStore.finalizeUpload
|
|
stat published object
|
|
transition VERIFYING → READY with content key, size, digest, etag, publishedAt
|
|
commit quota with actual bytes
|
|
release writer lease
|
|
```
|
|
|
|
On REJECT, transition to REJECTED and enqueue cleanup. On QUARANTINE, transition to QUARANTINED. Do not return READY when metadata transition fails after physical publish; enqueue reconciliation and throw `AmbiguousCompletionException`.
|
|
|
|
- [ ] **Step 4: Run finalize tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*FinalizeUploadServiceTest'
|
|
```
|
|
|
|
Expected: PASS; every READY fixture has readable content and matching size/digest.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-application
|
|
git commit -m "feat: finalize uploads with ready invariants"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 15: Ambiguous completion과 파일 reconciliation 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/recovery/FileReconciliationService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/recovery/DefaultFileReconciliationService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/recovery/ReconciliationResult.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/recovery/ReconciliationStatus.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/recovery/RecoveryQueue.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/recovery/FileReconciliationServiceTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes content `stat`, metadata version/state, expected size/digest.
|
|
- Produces `CONFIRMED_SUCCESS`, `CONFIRMED_NOT_APPLIED`, `RECOVERABLE_PARTIAL`, `QUARANTINE_REQUIRED`, or `UNRESOLVED`.
|
|
|
|
- [ ] **Step 1: Write failing ambiguous publish recovery tests**
|
|
|
|
```java
|
|
@Test
|
|
void confirmsSuccessWhenPhysicalObjectAndMetadataMatch() {
|
|
fixture.preparePhysicalObjectAndVerifyingMetadata();
|
|
|
|
ReconciliationResult result = service.reconcile(fixture.fileId());
|
|
|
|
assertThat(result.status()).isEqualTo(ReconciliationStatus.CONFIRMED_SUCCESS);
|
|
assertThat(fixture.fileState()).isEqualTo(FileState.READY);
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void neverGuessesReadyWhenDigestCannotBeVerified() {
|
|
fixture.prepareUnknownPhysicalObject();
|
|
|
|
ReconciliationResult result = service.reconcile(fixture.fileId());
|
|
|
|
assertThat(result.status()).isEqualTo(ReconciliationStatus.UNRESOLVED);
|
|
assertThat(fixture.fileState()).isNotEqualTo(FileState.READY);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*FileReconciliationServiceTest'
|
|
```
|
|
|
|
Expected: FAIL because reconciliation is absent.
|
|
|
|
- [ ] **Step 3: Implement deterministic reconciliation**
|
|
|
|
Use the following decision rules:
|
|
|
|
```text
|
|
metadata READY + physical size/digest match → CONFIRMED_SUCCESS
|
|
metadata pre-publish + no physical target → CONFIRMED_NOT_APPLIED
|
|
staging exists + known committed offset → RECOVERABLE_PARTIAL
|
|
physical exists + expected key/size/digest match + version unchanged → transition READY
|
|
physical exists but key/size/digest differ → QUARANTINE_REQUIRED
|
|
insufficient evidence → UNRESOLVED
|
|
```
|
|
|
|
Never perform blind write retry from this service. Store recovery attempts and reason codes in the cleanup/recovery queue.
|
|
|
|
- [ ] **Step 4: Run recovery tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*FileReconciliationServiceTest'
|
|
```
|
|
|
|
Expected: PASS; no unresolved case changes the file to READY.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-application
|
|
git commit -m "feat: reconcile ambiguous fileserver operations"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 16: Verification pipeline과 quarantine 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/FileVerifier.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/VerificationRequest.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/VerificationResult.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/VerificationVerdict.java`
|
|
- Create: `modules/fileserver/fileserver-verification/src/main/java/io/backend/skeleton/fileserver/verification/VerificationCoordinator.java`
|
|
- Create: `modules/fileserver/fileserver-verification/src/main/java/io/backend/skeleton/fileserver/verification/Sha256Verifier.java`
|
|
- Create: `modules/fileserver/fileserver-verification/src/main/java/io/backend/skeleton/fileserver/verification/MediaTypeVerifier.java`
|
|
- Create: `modules/fileserver/fileserver-verification/src/main/java/io/backend/skeleton/fileserver/verification/VerificationPolicyCombiner.java`
|
|
- Test: `modules/fileserver/fileserver-verification/src/test/java/io/backend/skeleton/fileserver/verification/VerificationCoordinatorTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces `VerificationCoordinator#verify(VerificationRequest)` consumed by Task 14.
|
|
- Verifiers return only safe metadata and stable reason codes.
|
|
|
|
- [ ] **Step 1: Write failing accept, quarantine, and retry tests**
|
|
|
|
```java
|
|
@Test
|
|
void rejectDominatesAccept() {
|
|
VerificationCoordinator coordinator = coordinator(
|
|
verifier("digest", VerificationVerdict.ACCEPT),
|
|
verifier("malware", VerificationVerdict.REJECT));
|
|
|
|
VerificationResult result = coordinator.verify(fixture.request()).toCompletableFuture().join();
|
|
|
|
assertThat(result.verdict()).isEqualTo(VerificationVerdict.REJECT);
|
|
assertThat(result.code()).isEqualTo("MALWARE_REJECTED");
|
|
}
|
|
|
|
@Test
|
|
void scannerTimeoutDoesNotBecomeAccept() {
|
|
VerificationCoordinator coordinator = coordinator(timeoutVerifier("scanner"));
|
|
|
|
VerificationResult result = coordinator.verify(fixture.request()).toCompletableFuture().join();
|
|
|
|
assertThat(result.verdict()).isEqualTo(VerificationVerdict.RETRY);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-verification:test \
|
|
--tests '*VerificationCoordinatorTest'
|
|
```
|
|
|
|
Expected: FAIL because verification types do not exist.
|
|
|
|
- [ ] **Step 3: Implement ordered verification and policy combination**
|
|
|
|
Run verifiers in this order:
|
|
|
|
```text
|
|
length
|
|
sha256
|
|
filename policy
|
|
media-type detection
|
|
signature/parser
|
|
optional malware scanner
|
|
optional CDR
|
|
```
|
|
|
|
Combination precedence is `REJECT > QUARANTINE > RETRY > ACCEPT`. Apply per-verifier timeout and record started/completed timestamps through the metadata adapter. Never log content samples or scanner raw payloads.
|
|
|
|
- [ ] **Step 4: Run verification tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-verification:test
|
|
```
|
|
|
|
Expected: PASS; timeout, reject, quarantine, and accept paths are deterministic.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api modules/fileserver/fileserver-verification
|
|
git commit -m "feat: add fileserver verification pipeline"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 17: Authorization hook과 upload application service 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/FileAccessPolicy.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/FileOperation.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/FileAccessSubject.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/security/RequestContext.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/UploadProtocol.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/UploadApplicationService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/DefaultUploadApplicationService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/CreateUploadRequest.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/UploadSessionView.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/AppendUploadResult.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/UploadApplicationServiceTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes metadata, content store, quota, state machine, filename policy, access policy.
|
|
- Produces create, append, status, cancel methods used by HTTP adapters.
|
|
|
|
- [ ] **Step 1: Write failing authorization, create, append, cancel tests**
|
|
|
|
```java
|
|
@Test
|
|
void authorizationRunsBeforeQuotaAndStorageMutation() {
|
|
accessPolicy.deny(FileOperation.CREATE);
|
|
|
|
assertThatThrownBy(() -> service.create(fixture.createRequest(), fixture.context()))
|
|
.isInstanceOf(FileAccessDeniedException.class);
|
|
|
|
assertThat(fixture.fileRecordCount()).isZero();
|
|
assertThat(fixture.stagingFileCount()).isZero();
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void createAppendAndCancelMaintainStateAndOffset() throws Exception {
|
|
UploadSessionView created = service.create(fixture.createRequest(), fixture.context());
|
|
AppendUploadResult appended = service.append(
|
|
created.uploadId(), 0, fixture.channel("abc"), 3, fixture.context());
|
|
service.cancel(created.uploadId(), fixture.context());
|
|
|
|
assertThat(appended.committedOffset()).isEqualTo(3);
|
|
assertThat(fixture.fileState(created.fileId())).isEqualTo(FileState.DELETING);
|
|
assertThat(fixture.publicDownloadAvailable(created.fileId())).isFalse();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*UploadApplicationServiceTest'
|
|
```
|
|
|
|
Expected: FAIL because upload orchestration is absent.
|
|
|
|
- [ ] **Step 3: Implement create, append, status, cancel**
|
|
|
|
Create sequence:
|
|
|
|
```text
|
|
authorize CREATE
|
|
sanitize original filename
|
|
validate expected length
|
|
acquire admission permit
|
|
reserve quota
|
|
insert CREATED file
|
|
insert upload session
|
|
create staging
|
|
transition CREATED → UPLOADING
|
|
return offset 0 and expiry
|
|
```
|
|
|
|
Append sequence:
|
|
|
|
```text
|
|
authorize APPEND
|
|
load non-expired session
|
|
acquire writer lease
|
|
validate metadata offset and physical length
|
|
extend quota reservation if needed
|
|
stream append
|
|
commit offset with lease token
|
|
release lease and transfer permit
|
|
```
|
|
|
|
Cancel sequence transitions to DELETING first, then queues cleanup. It does not synchronously remove large content from the request thread.
|
|
|
|
- [ ] **Step 4: Run upload application tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*UploadApplicationServiceTest'
|
|
```
|
|
|
|
Expected: PASS; authorization denial creates no side effect and offset commits are monotonic.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api modules/fileserver/fileserver-application
|
|
git commit -m "feat: implement fileserver upload application flow"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 18: HTTP Range, validator, header contract core 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/HttpRangeResolver.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/DefaultHttpRangeResolver.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/RangeBudget.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/ResolvedRanges.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/ConditionalRequestEvaluator.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/DownloadDecision.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/transfer/ContentDispositionFactory.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/transfer/HttpRangeResolverTest.java`
|
|
- Test: `modules/fileserver/fileserver-core-api/src/test/java/io/backend/skeleton/fileserver/api/transfer/ConditionalRequestEvaluatorTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces a framework-neutral `DownloadDecision` used by MVC, WebFlux, and Nginx.
|
|
- Default public budget is one range; optional multi-range budget is eight merged ranges.
|
|
|
|
- [ ] **Step 1: Write failing Range and conditional tests**
|
|
|
|
```java
|
|
@ParameterizedTest
|
|
@CsvSource({
|
|
"bytes=0-9,0,9",
|
|
"bytes=90-,90,99",
|
|
"bytes=-10,90,99"
|
|
})
|
|
void resolvesSingleRanges(String header, long start, long end) {
|
|
ResolvedRanges result = resolver.resolve(header, 100, RangeBudget.single());
|
|
assertThat(result.ranges()).containsExactly(new ByteRange(start, end));
|
|
}
|
|
|
|
@Test
|
|
void unsatisfiableRangeCarriesRepresentationLength() {
|
|
assertThatThrownBy(() -> resolver.resolve("bytes=100-200", 100, RangeBudget.single()))
|
|
.isInstanceOf(RangeNotSatisfiableException.class)
|
|
.extracting("representationLength")
|
|
.isEqualTo(100L);
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void mismatchedIfRangeFallsBackToFullResponse() {
|
|
DownloadDecision result = evaluator.evaluate(fixture.requestWithIfRange("\"old\""),
|
|
fixture.representation("\"new\"", 100));
|
|
|
|
assertThat(result.status()).isEqualTo(200);
|
|
assertThat(result.ranges()).isEmpty();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
--tests '*HttpRangeResolverTest' --tests '*ConditionalRequestEvaluatorTest'
|
|
```
|
|
|
|
Expected: FAIL because HTTP contract utilities do not exist.
|
|
|
|
- [ ] **Step 3: Implement parsing and decision order**
|
|
|
|
Implement:
|
|
|
|
```text
|
|
If-Match / If-Unmodified-Since
|
|
If-None-Match / If-Modified-Since
|
|
Range syntax and budget
|
|
If-Range
|
|
200 / 206 / 304 / 412 / 416
|
|
```
|
|
|
|
Merge overlapping ranges only when multi-range is enabled. Reject more than eight ranges or a total requested byte count above the configured budget. `ContentDispositionFactory` must emit sanitized ASCII `filename` and UTF-8 `filename*` without CR/LF.
|
|
|
|
- [ ] **Step 4: Run all transfer contract tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-core-api:test \
|
|
--tests '*transfer*'
|
|
```
|
|
|
|
Expected: PASS for first, middle, suffix, open-ended, empty, invalid, conditional, and If-Range cases.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api
|
|
git commit -m "feat: implement fileserver HTTP range contract"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 19: Spring MVC raw·multipart upload adapter 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/FileUploadController.java`
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/RawUploadRequestMapper.java`
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/MultipartUploadRequestMapper.java`
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/MvcTransferExecutorConfiguration.java`
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/BatchUploadResponse.java`
|
|
- Test: `modules/fileserver/fileserver-mvc/src/test/java/io/backend/skeleton/fileserver/mvc/FileUploadControllerTest.java`
|
|
- Test: `modules/fileserver/fileserver-mvc/src/test/java/io/backend/skeleton/fileserver/mvc/MvcUploadExecutorSaturationTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes `UploadApplicationService` and `FinalizeUploadService`.
|
|
- Implements `POST /v1/files`, `POST /v1/files:raw`, `POST /v1/files:batch`.
|
|
|
|
- [ ] **Step 1: Write failing MVC endpoint tests**
|
|
|
|
```java
|
|
@Test
|
|
void rawUploadStreamsWithoutCallingReadAllBytes() throws Exception {
|
|
mockMvc.perform(post("/v1/files:raw")
|
|
.contentType(MediaType.APPLICATION_OCTET_STREAM)
|
|
.header("X-Filename", "report.bin")
|
|
.content("abc"))
|
|
.andExpect(status().isCreated())
|
|
.andExpect(header().exists("Location"))
|
|
.andExpect(jsonPath("$.state").value("READY"));
|
|
|
|
verify(uploadService).append(any(), eq(0L), any(ReadableByteChannel.class), eq(3L), any());
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void batchReturnsPerPartResultsAndIsExplicitlyNonAtomic() throws Exception {
|
|
mockMvc.perform(multipart("/v1/files:batch")
|
|
.file(new MockMultipartFile("files", "a.txt", "text/plain", "a".getBytes()))
|
|
.file(new MockMultipartFile("files", "b.txt", "text/plain", "b".getBytes())))
|
|
.andExpect(status().isOk())
|
|
.andExpect(jsonPath("$.results.length()").value(2));
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run MVC tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-mvc:test \
|
|
--tests '*FileUploadControllerTest' --tests '*MvcUploadExecutorSaturationTest'
|
|
```
|
|
|
|
Expected: FAIL because the controller and executor are absent.
|
|
|
|
- [ ] **Step 3: Implement controllers with bounded streaming executor**
|
|
|
|
Use `ServletInputStream` through `Channels.newChannel`. Do not call `getBytes()` on `MultipartFile`. Submit blocking transfer work to a `ThreadPoolTaskExecutor` configured with core 8, max 32, queue 64. Convert rejection to retryable `429` or `503` with `Retry-After`.
|
|
|
|
Batch behavior:
|
|
|
|
```text
|
|
maximum 16 parts
|
|
one independent upload per part
|
|
successes are retained when another part fails
|
|
return 200 with ordered result array
|
|
never expose container temp path
|
|
```
|
|
|
|
- [ ] **Step 4: Run MVC upload and saturation tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-mvc:test
|
|
```
|
|
|
|
Expected: PASS; saturation does not create unbounded threads or queues.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-mvc
|
|
git commit -m "feat: add MVC streaming upload endpoints"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 20: Spring MVC GET·HEAD·Range download adapter 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/DownloadApplicationService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/DefaultDownloadApplicationService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/DownloadDescriptor.java`
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/FileDownloadController.java`
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/MvcDownloadResponseWriter.java`
|
|
- Test: `modules/fileserver/fileserver-mvc/src/test/java/io/backend/skeleton/fileserver/mvc/FileDownloadControllerContractTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes authorization, metadata, `HttpRangeResolver`, conditional evaluator, content store.
|
|
- Produces identical headers for GET and HEAD and exact `200/206/304/412/416` behavior.
|
|
|
|
- [ ] **Step 1: Write failing GET, HEAD, Range, and READY-gate tests**
|
|
|
|
```java
|
|
@Test
|
|
void headMatchesGetHeadersWithoutBody() throws Exception {
|
|
MvcResult get = mockMvc.perform(get(contentUrl()).header("Authorization", token()))
|
|
.andExpect(status().isOk())
|
|
.andReturn();
|
|
|
|
MvcResult head = mockMvc.perform(head(contentUrl()).header("Authorization", token()))
|
|
.andExpect(status().isOk())
|
|
.andExpect(content().bytes(new byte[0]))
|
|
.andReturn();
|
|
|
|
assertThat(head.getResponse().getHeader("ETag"))
|
|
.isEqualTo(get.getResponse().getHeader("ETag"));
|
|
assertThat(head.getResponse().getHeader("Content-Length"))
|
|
.isEqualTo(get.getResponse().getHeader("Content-Length"));
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void returnsPartialContentForSingleRange() throws Exception {
|
|
mockMvc.perform(get(contentUrl())
|
|
.header("Authorization", token())
|
|
.header("Range", "bytes=2-4"))
|
|
.andExpect(status().isPartialContent())
|
|
.andExpect(header().string("Content-Range", "bytes 2-4/10"))
|
|
.andExpect(content().bytes(new byte[]{2, 3, 4}));
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void nonReadyFileIsNeverOpened() throws Exception {
|
|
fixture.fileInState(FileState.VERIFYING);
|
|
|
|
mockMvc.perform(get(contentUrl()).header("Authorization", token()))
|
|
.andExpect(status().isConflict());
|
|
|
|
verify(contentStore, never()).openRead(any(), any());
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run MVC download tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-mvc:test \
|
|
--tests '*FileDownloadControllerContractTest'
|
|
```
|
|
|
|
Expected: FAIL because download service and controller do not exist.
|
|
|
|
- [ ] **Step 3: Implement application decision and MVC writer**
|
|
|
|
`DefaultDownloadApplicationService` must authorize before opening content, require READY, evaluate validators and Range, then return a descriptor with status, headers, content key, and normalized ranges. `MvcDownloadResponseWriter` uses a `StreamingResponseBody` or repeatable file resource; it must not use `InputStreamResource` for Range.
|
|
|
|
Add headers:
|
|
|
|
```text
|
|
ETag
|
|
Last-Modified
|
|
Accept-Ranges
|
|
Content-Type
|
|
Content-Disposition
|
|
Cache-Control
|
|
Content-Length or Content-Range
|
|
```
|
|
|
|
For `416`, include `Content-Range: bytes */<size>`.
|
|
|
|
- [ ] **Step 4: Run full MVC HTTP contract tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-mvc:test
|
|
```
|
|
|
|
Expected: PASS for full, HEAD, first, middle, suffix, unsatisfiable, ETag, If-Range, and non-READY cases.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-application modules/fileserver/fileserver-mvc
|
|
git commit -m "feat: add MVC fileserver download contract"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 21: Spring WebFlux raw·multipart upload adapter 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/ReactiveUploadApplicationService.java`
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/FileUploadHandler.java`
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/PartEventUploadReader.java`
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/DataBufferByteBufferPublisher.java`
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/FileserverIoScheduler.java`
|
|
- Test: `modules/fileserver/fileserver-webflux/src/test/java/io/backend/skeleton/fileserver/webflux/FileUploadHandlerTest.java`
|
|
- Test: `modules/fileserver/fileserver-webflux/src/test/java/io/backend/skeleton/fileserver/webflux/DataBufferReleaseTest.java`
|
|
- Test: `modules/fileserver/fileserver-webflux/src/test/java/io/backend/skeleton/fileserver/webflux/WebFluxBlockingCallTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes `AsyncContentStore` when available or adapts the blocking application service on a dedicated bounded scheduler.
|
|
- Every received pooled `DataBuffer` is forwarded or released exactly once.
|
|
|
|
- [ ] **Step 1: Write failing upload, cancellation, and buffer-release tests**
|
|
|
|
```java
|
|
@Test
|
|
void rawUploadConsumesFluxWithoutJoiningWholeBody() {
|
|
webTestClient.post()
|
|
.uri("/v1/files:raw")
|
|
.contentType(MediaType.APPLICATION_OCTET_STREAM)
|
|
.header("X-Filename", "large.bin")
|
|
.body(Flux.just(buffer("abc"), buffer("def")), DataBuffer.class)
|
|
.exchange()
|
|
.expectStatus().isCreated()
|
|
.expectBody()
|
|
.jsonPath("$.state").isEqualTo("READY");
|
|
|
|
assertThat(testBufferFactory.joinInvocationCount()).isZero();
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void cancellationReleasesAllObservedBuffers() {
|
|
StepVerifier.create(handler.consume(fixture.cancellableBuffers()))
|
|
.thenCancel()
|
|
.verify();
|
|
|
|
assertThat(fixture.allocatedBufferCount()).isEqualTo(fixture.releasedBufferCount());
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run WebFlux tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-webflux:test \
|
|
--tests '*FileUploadHandlerTest' --tests '*DataBufferReleaseTest' \
|
|
--tests '*WebFluxBlockingCallTest'
|
|
```
|
|
|
|
Expected: FAIL because handlers and buffer adapters do not exist.
|
|
|
|
- [ ] **Step 3: Implement streaming adapters and dedicated scheduler**
|
|
|
|
`PartEventUploadReader` must process windowed multipart events sequentially and enforce part count and byte limits. Use `DataBufferUtils.release(buffer)` in every discard, error, and cancellation path. For a blocking local store, schedule filesystem work on a fixed bounded scheduler named `fileserver-io`; never use the Reactor Netty event loop.
|
|
|
|
```java
|
|
public final class FileserverIoScheduler implements AutoCloseable {
|
|
private final Scheduler scheduler;
|
|
|
|
public FileserverIoScheduler(int workers, int queueCapacity) {
|
|
this.scheduler = Schedulers.newBoundedElastic(
|
|
workers, queueCapacity, "fileserver-io", 60, false);
|
|
}
|
|
|
|
public Scheduler scheduler() {
|
|
return scheduler;
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run WebFlux tests with leak detection and BlockHound**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-webflux:test
|
|
```
|
|
|
|
Expected: PASS; no unreleased buffers and no blocking call on event-loop threads.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-webflux
|
|
git commit -m "feat: add WebFlux streaming upload adapter"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 22: Spring WebFlux download와 zero-copy capability 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/FileDownloadHandler.java`
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/ReactiveDownloadResponseWriter.java`
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/ZeroCopyEligibility.java`
|
|
- Test: `modules/fileserver/fileserver-webflux/src/test/java/io/backend/skeleton/fileserver/webflux/FileDownloadHandlerContractTest.java`
|
|
- Test: `modules/fileserver/fileserver-webflux/src/test/java/io/backend/skeleton/fileserver/webflux/SlowClientBackpressureTest.java`
|
|
|
|
**Interfaces:**
|
|
- Reuses the exact `DownloadDecision` from Task 18.
|
|
- Produces HTTP parity with Task 20.
|
|
|
|
- [ ] **Step 1: Write failing parity and backpressure tests**
|
|
|
|
```java
|
|
@Test
|
|
void rangeHeadersMatchMvcContract() {
|
|
webTestClient.get()
|
|
.uri(contentUrl())
|
|
.header("Authorization", token())
|
|
.header("Range", "bytes=2-4")
|
|
.exchange()
|
|
.expectStatus().isEqualTo(206)
|
|
.expectHeader().valueEquals("Content-Range", "bytes 2-4/10")
|
|
.expectBody().isEqualTo(new byte[]{2, 3, 4});
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void slowSubscriberDoesNotExceedInFlightBufferLimit() {
|
|
StepVerifier.withVirtualTime(() -> fixture.slowDownload())
|
|
.thenAwait(Duration.ofSeconds(10))
|
|
.thenCancel()
|
|
.verify();
|
|
|
|
assertThat(fixture.maxInFlightBuffers()).isLessThanOrEqualTo(8);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-webflux:test \
|
|
--tests '*FileDownloadHandlerContractTest' --tests '*SlowClientBackpressureTest'
|
|
```
|
|
|
|
Expected: FAIL because download handler is absent.
|
|
|
|
- [ ] **Step 3: Implement reactive write and optional zero-copy**
|
|
|
|
For async stores, map `Flow.Publisher<ByteBuffer>` to `Flux<DataBuffer>` with bounded demand. For local files, use zero-copy only when the response implementation supports it, no body transformation is required, and TLS/runtime constraints allow it. Zero-copy remains an optimization and does not alter the public contract.
|
|
|
|
- [ ] **Step 4: Run WebFlux download contract tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-webflux:test
|
|
```
|
|
|
|
Expected: PASS; MVC and WebFlux golden HTTP snapshots are equal for shared scenarios.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-webflux
|
|
git commit -m "feat: add WebFlux fileserver downloads"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 23: Nginx `X-Accel-Redirect` 위임 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-nginx/src/main/java/io/backend/skeleton/fileserver/nginx/NginxInternalUriMapper.java`
|
|
- Create: `modules/fileserver/fileserver-nginx/src/main/java/io/backend/skeleton/fileserver/nginx/DefaultNginxInternalUriMapper.java`
|
|
- Create: `modules/fileserver/fileserver-nginx/src/main/java/io/backend/skeleton/fileserver/nginx/NginxDownloadStrategy.java`
|
|
- Create: `modules/fileserver/fileserver-nginx/src/main/java/io/backend/skeleton/fileserver/nginx/NginxDelegationProperties.java`
|
|
- Create: `infra/fileserver/nginx/nginx.conf`
|
|
- Test: `modules/fileserver/fileserver-nginx/src/test/java/io/backend/skeleton/fileserver/nginx/NginxInternalUriMapperTest.java`
|
|
- Test: `modules/fileserver/fileserver-nginx/src/test/java/io/backend/skeleton/fileserver/nginx/NginxDownloadIntegrationTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes an authorized READY `DownloadDescriptor`.
|
|
- Produces a validated relative internal URI, never an absolute physical path.
|
|
- Default threshold is 16 MiB.
|
|
|
|
- [ ] **Step 1: Write failing URI mapping and internal-path tests**
|
|
|
|
```java
|
|
@Test
|
|
void mapsValidatedContentKeyWithoutExposingAbsolutePath() {
|
|
String internalUri = mapper.map(new ContentKey("ab/cd/0123456789abcdef"));
|
|
|
|
assertThat(internalUri).isEqualTo("/__files/ab/cd/0123456789abcdef.bin");
|
|
assertThat(internalUri).doesNotContain("/var/lib", "..", "\");
|
|
}
|
|
|
|
@Test
|
|
void rejectsMalformedContentKeyEvenWhenCalledInternally() {
|
|
assertThatThrownBy(() -> mapper.mapUnchecked("../../etc/passwd"))
|
|
.isInstanceOf(InvalidPathException.class);
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void directAccessToInternalLocationIsRejected() {
|
|
nginxClient.get("/__files/ab/cd/0123456789abcdef.bin")
|
|
.expectStatus(404);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run unit and integration tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-nginx:test \
|
|
--tests '*NginxInternalUriMapperTest' --tests '*NginxDownloadIntegrationTest'
|
|
```
|
|
|
|
Expected: FAIL because URI mapper and Nginx configuration do not exist.
|
|
|
|
- [ ] **Step 3: Implement safe relative mapping and Nginx internal location**
|
|
|
|
`DefaultNginxInternalUriMapper` accepts only a validated `ContentKey`, rebuilds the shard components, and returns a URI below `/__files/`. Configure Nginx:
|
|
|
|
```nginx
|
|
location /__files/ {
|
|
internal;
|
|
alias /srv/files/content/;
|
|
sendfile on;
|
|
sendfile_max_chunk 2m;
|
|
add_header X-Content-Type-Options nosniff always;
|
|
}
|
|
```
|
|
|
|
The application response includes `X-Accel-Redirect` only after authorization and READY gate. Ensure the header is consumed by Nginx and not copied to the client. The resulting URI path after `/__files/` must map exactly to the local content layout.
|
|
|
|
- [ ] **Step 4: Run direct-vs-Nginx HTTP parity tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-nginx:test
|
|
```
|
|
|
|
Expected: PASS for full GET, HEAD, Range, ETag, Content-Disposition, private cache headers, and external internal-location rejection.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-nginx infra/fileserver/nginx
|
|
git commit -m "feat: delegate large downloads to nginx"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 24: Delete, copy, move, cleanup lifecycle 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/FileLifecycleService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/DefaultFileLifecycleService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/cleanup/CleanupService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/cleanup/DefaultCleanupService.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/cleanup/CleanupItem.java`
|
|
- Modify: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/LocalBlockingContentStore.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/FileLifecycleServiceTest.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/cleanup/CleanupServiceTest.java`
|
|
|
|
**Interfaces:**
|
|
- Implements logical delete first, bounded asynchronous physical cleanup.
|
|
- Public move changes logical namespace metadata only.
|
|
- Copy defaults to create-only target.
|
|
|
|
- [ ] **Step 1: Write failing delete and cleanup-race tests**
|
|
|
|
```java
|
|
@Test
|
|
void logicalDeleteBlocksDownloadBeforePhysicalDeleteCompletes() {
|
|
fixture.readyFileWithSlowPhysicalDelete();
|
|
|
|
service.delete(fixture.fileId(), fixture.version(), fixture.context());
|
|
|
|
assertThat(fixture.fileState()).isEqualTo(FileState.DELETING);
|
|
assertThat(fixture.publicDownloadAvailable()).isFalse();
|
|
assertThat(fixture.physicalObjectExists()).isTrue();
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void cleanupDoesNotDeleteContentOwnedByAnActiveLease() {
|
|
fixture.cleanupItemForActiveUpload();
|
|
|
|
CleanupBatchResult result = cleanup.runBatch(100, 1L << 30);
|
|
|
|
assertThat(result.skippedActiveLease()).isEqualTo(1);
|
|
assertThat(fixture.physicalObjectExists()).isTrue();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*FileLifecycleServiceTest' --tests '*CleanupServiceTest'
|
|
```
|
|
|
|
Expected: FAIL because lifecycle services do not exist.
|
|
|
|
- [ ] **Step 3: Implement lifecycle operations**
|
|
|
|
Delete:
|
|
|
|
```text
|
|
authorize DELETE
|
|
validate If-Match/version
|
|
transition to DELETING
|
|
enqueue cleanup
|
|
return 202 or 204
|
|
worker deletes physical content
|
|
release quota
|
|
transition to DELETED
|
|
```
|
|
|
|
Copy creates a new FileRecord and physical target; partial target is queued for cleanup on failure. Move changes logical namespace metadata without moving immutable physical content. Cleanup verifies state, version, lease, and content key before deleting.
|
|
|
|
- [ ] **Step 4: Run lifecycle tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
--tests '*FileLifecycleServiceTest' --tests '*CleanupServiceTest'
|
|
```
|
|
|
|
Expected: PASS; active content is never deleted and logical delete blocks reads immediately.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-application modules/fileserver/fileserver-storage-local
|
|
git commit -m "feat: implement fileserver lifecycle and cleanup"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 25: 별도 Admin Plane 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-admin/src/main/java/io/backend/skeleton/fileserver/admin/FileserverAdminController.java`
|
|
- Create: `modules/fileserver/fileserver-admin/src/main/java/io/backend/skeleton/fileserver/admin/StorageHealthView.java`
|
|
- Create: `modules/fileserver/fileserver-admin/src/main/java/io/backend/skeleton/fileserver/admin/OrphanAdminService.java`
|
|
- Create: `modules/fileserver/fileserver-admin/src/main/java/io/backend/skeleton/fileserver/admin/AdminAuditService.java`
|
|
- Test: `modules/fileserver/fileserver-admin/src/test/java/io/backend/skeleton/fileserver/admin/FileserverAdminControllerTest.java`
|
|
- Test: `modules/fileserver/fileserver-admin/src/test/java/io/backend/skeleton/fileserver/admin/OrphanAdminServiceTest.java`
|
|
|
|
**Interfaces:**
|
|
- Exposes management-only health, capabilities, orphan dry-run/apply, reverify, force-delete, incomplete upload cleanup.
|
|
- Never returns physical root, filename, raw scanner data, or signed tokens.
|
|
|
|
- [ ] **Step 1: Write failing management-isolation and dry-run tests**
|
|
|
|
```java
|
|
@Test
|
|
void publicApplicationPortDoesNotExposeAdminEndpoints() {
|
|
publicWebClient.get().uri("/internal/fileserver/capabilities")
|
|
.exchange()
|
|
.expectStatus().isNotFound();
|
|
}
|
|
|
|
@Test
|
|
void orphanReconcileDefaultsToDryRun() {
|
|
managementWebClient.post().uri("/internal/fileserver/orphans:reconcile")
|
|
.bodyValue(Map.of("limit", 100))
|
|
.exchange()
|
|
.expectStatus().isOk()
|
|
.expectBody()
|
|
.jsonPath("$.dryRun").isEqualTo(true);
|
|
|
|
assertThat(fixture.deletedObjectCount()).isZero();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run admin tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-admin:test \
|
|
--tests '*FileserverAdminControllerTest' --tests '*OrphanAdminServiceTest'
|
|
```
|
|
|
|
Expected: FAIL because the admin module is not implemented.
|
|
|
|
- [ ] **Step 3: Implement management-only endpoints and audit**
|
|
|
|
Implement endpoints from the design. `force-delete` requires an explicit reason and a second authorization predicate. Orphan apply requests require `dryRun=false`, expected object fingerprint, and bounded byte budget. Audit records operation, reason code, actor fingerprint, result, and trace ID without path or filename.
|
|
|
|
- [ ] **Step 4: Run admin isolation and behavior tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-admin:test
|
|
```
|
|
|
|
Expected: PASS; admin routes exist only on the management context and all mutating actions emit audit records.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-admin
|
|
git commit -m "feat: add isolated fileserver admin plane"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 26: 다중 인스턴스 writer lease와 NFS ambiguity 처리 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/concurrency/WriterLeaseCoordinator.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/concurrency/DefaultWriterLeaseCoordinator.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/concurrency/LeaseHeartbeat.java`
|
|
- Create: `modules/fileserver/fileserver-storage-local/src/main/java/io/backend/skeleton/fileserver/local/AmbiguousFilesystemOperationDetector.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/concurrency/MultiInstanceWriterLeaseTest.java`
|
|
- Test: `modules/fileserver/fileserver-storage-local/src/test/java/io/backend/skeleton/fileserver/local/AmbiguousFilesystemOperationDetectorTest.java`
|
|
|
|
**Interfaces:**
|
|
- Builds on DB lease methods from Task 7.
|
|
- A writer whose lease token expired or changed may not commit offset or READY state.
|
|
- Filesystem timeout with possible server-side completion becomes `AmbiguousCompletionException`.
|
|
|
|
- [ ] **Step 1: Write failing two-node and expired-writer tests**
|
|
|
|
```java
|
|
@Test
|
|
void onlyOneNodeCanAppendTheSameUpload() {
|
|
UploadId uploadId = fixture.activeUpload();
|
|
|
|
CompletableFuture<AppendUploadResult> nodeA = node("a").append(uploadId, 0, "abc");
|
|
CompletableFuture<AppendUploadResult> nodeB = node("b").append(uploadId, 0, "xyz");
|
|
|
|
assertThat(successCount(nodeA, nodeB)).isEqualTo(1);
|
|
assertThat(conflictCount(nodeA, nodeB)).isEqualTo(1);
|
|
assertThat(fixture.committedOffset(uploadId)).isEqualTo(3);
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void pausedWriterCannotCommitAfterLeaseTakeover() {
|
|
WriterLease stale = coordinator.acquire(fixture.uploadId(), "node-a");
|
|
clock.advance(Duration.ofMinutes(1));
|
|
WriterLease current = coordinator.acquire(fixture.uploadId(), "node-b");
|
|
|
|
assertThatThrownBy(() -> coordinator.commitOffset(stale, 0, 3))
|
|
.isInstanceOf(ConcurrentFileModificationException.class);
|
|
assertThat(current.owner()).isEqualTo("node-b");
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
:modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*MultiInstanceWriterLeaseTest' \
|
|
--tests '*AmbiguousFilesystemOperationDetectorTest'
|
|
```
|
|
|
|
Expected: FAIL because coordinator and ambiguity classification are absent.
|
|
|
|
- [ ] **Step 3: Implement lease heartbeat and ambiguity classification**
|
|
|
|
Heartbeat renews at one third of the lease duration. Every commit validates upload ID, owner, token, expiry, expected offset, and metadata version. Do not use `FileLock` as a correctness dependency.
|
|
|
|
Classify NFS-style outcomes:
|
|
|
|
```text
|
|
request definitely not sent → retryable failure
|
|
server explicitly rejected → definite failure
|
|
response lost after possible rename/write → ambiguous completion
|
|
stale handle with physical evidence available → reconciliation required
|
|
```
|
|
|
|
- [ ] **Step 4: Run multi-instance tests with repeated scheduling jitter**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
:modules:fileserver:fileserver-storage-local:test \
|
|
--tests '*MultiInstanceWriterLeaseTest' \
|
|
--tests '*AmbiguousFilesystemOperationDetectorTest' --rerun-tasks
|
|
```
|
|
|
|
Expected: PASS; no run commits bytes from a stale lease.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-application modules/fileserver/fileserver-storage-local
|
|
git commit -m "feat: enforce multi-instance fileserver leases"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 27: tus 1.0 Stable 모듈 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-tus/src/main/java/io/backend/skeleton/fileserver/tus/TusController.java`
|
|
- Create: `modules/fileserver/fileserver-tus/src/main/java/io/backend/skeleton/fileserver/tus/TusRequestParser.java`
|
|
- Create: `modules/fileserver/fileserver-tus/src/main/java/io/backend/skeleton/fileserver/tus/TusResponseHeaders.java`
|
|
- Create: `modules/fileserver/fileserver-tus/src/main/java/io/backend/skeleton/fileserver/tus/TusProperties.java`
|
|
- Create: `modules/fileserver/fileserver-tus/src/main/java/io/backend/skeleton/fileserver/tus/TusChecksumVerifier.java`
|
|
- Test: `modules/fileserver/fileserver-tus/src/test/java/io/backend/skeleton/fileserver/tus/TusProtocolContractTest.java`
|
|
- Test: `modules/fileserver/fileserver-tus/src/test/java/io/backend/skeleton/fileserver/tus/TusOffsetConcurrencyTest.java`
|
|
|
|
**Interfaces:**
|
|
- Consumes `UploadApplicationService` create/status/append/cancel.
|
|
- Supports creation, HEAD, PATCH, checksum, expiration, termination.
|
|
- Concatenation is Beta and feature-flagged.
|
|
|
|
- [ ] **Step 1: Write failing tus creation, HEAD, PATCH, mismatch tests**
|
|
|
|
```java
|
|
@Test
|
|
void createsAndAppendsTusUpload() {
|
|
String location = client.post("/v1/uploads")
|
|
.header("Tus-Resumable", "1.0.0")
|
|
.header("Upload-Length", "6")
|
|
.expectStatus(201)
|
|
.returnHeader("Location");
|
|
|
|
client.patch(location)
|
|
.header("Tus-Resumable", "1.0.0")
|
|
.header("Upload-Offset", "0")
|
|
.contentType("application/offset+octet-stream")
|
|
.body("abc")
|
|
.expectStatus(204)
|
|
.expectHeader("Upload-Offset", "3");
|
|
|
|
client.head(location)
|
|
.header("Tus-Resumable", "1.0.0")
|
|
.expectStatus(204)
|
|
.expectHeader("Upload-Offset", "3");
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void mismatchedOffsetReturns409WithoutMutation() {
|
|
fixture.uploadAtOffset(3);
|
|
|
|
client.patch(fixture.location())
|
|
.header("Tus-Resumable", "1.0.0")
|
|
.header("Upload-Offset", "1")
|
|
.contentType("application/offset+octet-stream")
|
|
.body("x")
|
|
.expectStatus(409);
|
|
|
|
assertThat(fixture.offset()).isEqualTo(3);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tus tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-tus:test \
|
|
--tests '*TusProtocolContractTest' --tests '*TusOffsetConcurrencyTest'
|
|
```
|
|
|
|
Expected: FAIL because tus endpoints do not exist.
|
|
|
|
- [ ] **Step 3: Implement tus 1.0 protocol mapping**
|
|
|
|
Implement:
|
|
|
|
```text
|
|
POST creation with Location
|
|
HEAD with Upload-Offset and Upload-Length
|
|
PATCH application/offset+octet-stream
|
|
409 on offset mismatch without body mutation
|
|
Upload-Checksum validation
|
|
Upload-Expires
|
|
DELETE termination
|
|
Tus-Resumable validation on every protocol request
|
|
```
|
|
|
|
Use one writer lease per upload. Return `410` after expiration and release quota on termination. Concatenation uses independent part resources and verifies each part before final combine.
|
|
|
|
- [ ] **Step 4: Run tus protocol suite**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-tus:test
|
|
```
|
|
|
|
Expected: PASS for create, append, resume after restart, checksum, expiry, termination, and concurrent offset conflict.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-tus
|
|
git commit -m "feat: add tus 1.0 resumable uploads"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 28: HTTPbis resumable upload draft-12 Experimental 모듈 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-resumable-httpbis-draft12/src/main/java/io/backend/skeleton/fileserver/httpbisdraft12/Draft12UploadController.java`
|
|
- Create: `modules/fileserver/fileserver-resumable-httpbis-draft12/src/main/java/io/backend/skeleton/fileserver/httpbisdraft12/Draft12Headers.java`
|
|
- Create: `modules/fileserver/fileserver-resumable-httpbis-draft12/src/main/java/io/backend/skeleton/fileserver/httpbisdraft12/Draft12ProblemDetails.java`
|
|
- Create: `modules/fileserver/fileserver-resumable-httpbis-draft12/src/main/java/io/backend/skeleton/fileserver/httpbisdraft12/Draft12Properties.java`
|
|
- Test: `modules/fileserver/fileserver-resumable-httpbis-draft12/src/test/java/io/backend/skeleton/fileserver/httpbisdraft12/Draft12ProtocolTest.java`
|
|
- Test: `modules/fileserver/fileserver-resumable-httpbis-draft12/src/test/java/io/backend/skeleton/fileserver/httpbisdraft12/DraftIsolationTest.java`
|
|
|
|
**Interfaces:**
|
|
- Reuses application upload services but has a distinct endpoint namespace and media types.
|
|
- Module is disabled by default and its package, properties, and docs include `draft12`.
|
|
|
|
- [ ] **Step 1: Write failing draft protocol and isolation tests**
|
|
|
|
```java
|
|
@Test
|
|
void disabledDraftDoesNotRegisterEndpoints() {
|
|
contextRunner.withPropertyValues("backend.fileserver.httpbis-draft12.enabled=false")
|
|
.run(context -> assertThat(context).doesNotHaveBean(Draft12UploadController.class));
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void offsetMismatchReturnsDraftProblemDetail() {
|
|
fixture.uploadAtOffset(10);
|
|
|
|
client.patch(fixture.draftLocation())
|
|
.header("Upload-Offset", "5")
|
|
.contentType("application/partial-upload")
|
|
.body("abc")
|
|
.expectStatus(409)
|
|
.expectJsonPath("$.expectedOffset", 10)
|
|
.expectJsonPath("$.providedOffset", 5);
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-resumable-httpbis-draft12:test
|
|
```
|
|
|
|
Expected: FAIL because the Experimental module is absent.
|
|
|
|
- [ ] **Step 3: Implement draft-12 behind an explicit feature flag**
|
|
|
|
Implement only the researched draft-12 contract: `Upload-Offset`, `Upload-Complete`, `application/partial-upload`, offset mismatch problem detail, and runtime capability for 104 interim response. Do not share controller paths or DTOs with tus. Add an `ExperimentalApi` marker annotation and runtime warning on enablement.
|
|
|
|
- [ ] **Step 4: Run isolation and protocol tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-resumable-httpbis-draft12:test
|
|
```
|
|
|
|
Expected: PASS; disabled mode registers no endpoints and Stable modules have no dependency on draft types.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-resumable-httpbis-draft12
|
|
git commit -m "feat: add experimental HTTP resumable draft12"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 29: HTTP Problem Detail과 보안 hardening 통합 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-mvc/src/main/java/io/backend/skeleton/fileserver/mvc/FileserverMvcExceptionHandler.java`
|
|
- Create: `modules/fileserver/fileserver-webflux/src/main/java/io/backend/skeleton/fileserver/webflux/FileserverWebFluxExceptionHandler.java`
|
|
- Create: `modules/fileserver/fileserver-core-api/src/main/java/io/backend/skeleton/fileserver/api/error/FileserverProblem.java`
|
|
- Create: `modules/fileserver/fileserver-verification/src/main/java/io/backend/skeleton/fileserver/verification/ScriptableContentPolicy.java`
|
|
- Test: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/security/PathTraversalSecurityTest.java`
|
|
- Test: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/security/SymlinkRaceSecurityTest.java`
|
|
- Test: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/security/FilenameInjectionSecurityTest.java`
|
|
- Test: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/security/RangeBombSecurityTest.java`
|
|
- Test: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/security/ScriptableContentSecurityTest.java`
|
|
|
|
**Interfaces:**
|
|
- Maps the same core failure context to MVC and WebFlux `application/problem+json`.
|
|
- Security tests run against both adapters.
|
|
|
|
- [ ] **Step 1: Write failing problem-detail and attack tests**
|
|
|
|
```java
|
|
@Test
|
|
void offsetMismatchProblemDoesNotExposePath() {
|
|
ProblemResponse response = client.patchOffsetMismatch();
|
|
|
|
assertThat(response.status()).isEqualTo(409);
|
|
assertThat(response.json("code")).isEqualTo("UPLOAD_OFFSET_MISMATCH");
|
|
assertThat(response.body()).doesNotContain("/var/lib", "staging", "java.nio.file");
|
|
}
|
|
```
|
|
|
|
```java
|
|
@ParameterizedTest
|
|
@ValueSource(strings = {"../x", "%2e%2e%2fx", "/etc/passwd", "C:\\Windows\\system.ini"})
|
|
void rejectsPathShapedInputs(String input) {
|
|
client.uploadWithFilename(input).expectNoStorageEscape();
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void excessiveRangesAreRejectedBeforeContentOpen() {
|
|
client.getWithRange("bytes=0-0,2-2,4-4,6-6,8-8,10-10,12-12,14-14,16-16")
|
|
.expectClientError();
|
|
assertThat(fixture.contentOpenCount()).isZero();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run security tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-testkit:test \
|
|
--tests '*security*'
|
|
```
|
|
|
|
Expected: FAIL because unified error mapping and all guards are not connected.
|
|
|
|
- [ ] **Step 3: Implement error mapping and hardening**
|
|
|
|
Map every `FileserverErrorCode` to the design status code and emit:
|
|
|
|
```json
|
|
{
|
|
"type": "urn:fileserver:problem:<code>",
|
|
"title": "stable title",
|
|
"status": 409,
|
|
"code": "UPLOAD_OFFSET_MISMATCH",
|
|
"retryable": true,
|
|
"traceId": "..."
|
|
}
|
|
```
|
|
|
|
Add `X-Content-Type-Options: nosniff`; default scriptable content to attachment; enforce range budget before content open; ensure symlink checks occur at open time, not only at path construction.
|
|
|
|
- [ ] **Step 4: Run MVC, WebFlux, and security suites**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-mvc:test \
|
|
:modules:fileserver:fileserver-webflux:test \
|
|
:modules:fileserver:fileserver-testkit:test \
|
|
--tests '*security*' --tests '*ExceptionHandler*'
|
|
```
|
|
|
|
Expected: PASS; MVC and WebFlux problem JSON is equivalent and contains no sensitive path data.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-core-api \
|
|
modules/fileserver/fileserver-mvc \
|
|
modules/fileserver/fileserver-webflux \
|
|
modules/fileserver/fileserver-verification \
|
|
modules/fileserver/fileserver-testkit
|
|
git commit -m "feat: harden fileserver HTTP and error handling"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 30: Metric, trace, audit와 민감정보 차단 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/observability/FileserverMetrics.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/observability/FileserverTracing.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/observability/SafeFileFingerprint.java`
|
|
- Create: `modules/fileserver/fileserver-application/src/main/java/io/backend/skeleton/fileserver/application/observability/FileserverAuditEvent.java`
|
|
- Test: `modules/fileserver/fileserver-application/src/test/java/io/backend/skeleton/fileserver/application/observability/FileserverObservabilityTest.java`
|
|
- Test: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/security/SensitiveTelemetryLeakTest.java`
|
|
|
|
**Interfaces:**
|
|
- Produces metric names and spans defined in the design.
|
|
- High-cardinality IDs and raw metadata are prohibited.
|
|
|
|
- [ ] **Step 1: Write failing metric and leak tests**
|
|
|
|
```java
|
|
@Test
|
|
void uploadMetricUsesBoundedTags() {
|
|
metrics.recordUpload(
|
|
UploadProtocol.RAW,
|
|
"LOCAL",
|
|
"READY",
|
|
SizeBucket.MEDIUM,
|
|
Duration.ofMillis(10),
|
|
1024);
|
|
|
|
Meter meter = registry.find("fileserver.upload.duration").meter();
|
|
assertThat(meter.getId().getTags())
|
|
.extracting(Tag::getKey)
|
|
.containsExactlyInAnyOrder("protocol", "storage", "result", "size_bucket");
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void telemetryNeverContainsFilenamePathOrRawIds() {
|
|
fixture.runUpload("private-name.pdf", "/var/lib/backend/files", fixture.fileId());
|
|
|
|
assertThat(fixture.allTelemetryText())
|
|
.doesNotContain("private-name.pdf", "/var/lib/backend/files", fixture.fileId().toString());
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run observability tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
:modules:fileserver:fileserver-testkit:test \
|
|
--tests '*FileserverObservabilityTest' --tests '*SensitiveTelemetryLeakTest'
|
|
```
|
|
|
|
Expected: FAIL because instrumentation is absent.
|
|
|
|
- [ ] **Step 3: Implement bounded metrics, spans, and audit**
|
|
|
|
Add timers/counters for upload, download, active transfer, interruption, offset mismatch, checksum, verification queue, temp/orphan, quota, cleanup, delegation, and access denial. Add spans named exactly as the design. When correlation is required, use a keyed HMAC fingerprint; never emit the raw file ID or checksum.
|
|
|
|
- [ ] **Step 4: Run observability and sensitive-log tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-application:test \
|
|
:modules:fileserver:fileserver-testkit:test \
|
|
--tests '*Observability*' --tests '*SensitiveTelemetryLeakTest'
|
|
```
|
|
|
|
Expected: PASS; all tags belong to the approved bounded vocabulary.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-application modules/fileserver/fileserver-testkit
|
|
git commit -m "feat: add safe fileserver observability"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 31: Spring Boot properties와 auto-configuration 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/src/main/java/io/backend/skeleton/fileserver/autoconfigure/FileserverProperties.java`
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/src/main/java/io/backend/skeleton/fileserver/autoconfigure/FileserverAutoConfiguration.java`
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/src/main/java/io/backend/skeleton/fileserver/autoconfigure/FileserverMvcAutoConfiguration.java`
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/src/main/java/io/backend/skeleton/fileserver/autoconfigure/FileserverWebFluxAutoConfiguration.java`
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/src/main/java/io/backend/skeleton/fileserver/autoconfigure/FileserverNginxAutoConfiguration.java`
|
|
- Create: `modules/fileserver/fileserver-spring-boot-starter/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`
|
|
- Test: `modules/fileserver/fileserver-spring-boot-starter/src/test/java/io/backend/skeleton/fileserver/autoconfigure/FileserverAutoConfigurationTest.java`
|
|
- Test: `modules/fileserver/fileserver-spring-boot-starter/src/test/java/io/backend/skeleton/fileserver/autoconfigure/FileserverPropertiesValidationTest.java`
|
|
|
|
**Interfaces:**
|
|
- Binds the exact `backend.fileserver.*` property tree from the design.
|
|
- Creates MVC or WebFlux adapters only when their runtime is present.
|
|
- Production startup must fail without a real `FileAccessPolicy`.
|
|
|
|
- [ ] **Step 1: Write failing default-binding and invalid-startup tests**
|
|
|
|
```java
|
|
@Test
|
|
void bindsStandardProfileDefaults() {
|
|
contextRunner.withPropertyValues(
|
|
"backend.fileserver.enabled=true",
|
|
"backend.fileserver.storage.root=" + tempDir)
|
|
.withUserConfiguration(TestAccessPolicyConfiguration.class)
|
|
.run(context -> {
|
|
FileserverProperties properties = context.getBean(FileserverProperties.class);
|
|
assertThat(properties.upload().maxFileSize()).isEqualTo(DataSize.ofMegabytes(100));
|
|
assertThat(properties.storage().bufferSize()).isEqualTo(DataSize.ofKilobytes(128));
|
|
assertThat(properties.upload().maxParts()).isEqualTo(16);
|
|
});
|
|
}
|
|
```
|
|
|
|
```java
|
|
@Test
|
|
void productionRejectsNoOpAuthorizationPolicy() {
|
|
contextRunner.withPropertyValues(
|
|
"spring.profiles.active=prod",
|
|
"backend.fileserver.enabled=true",
|
|
"backend.fileserver.storage.root=" + tempDir)
|
|
.run(context -> assertThat(context).hasFailed());
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run starter tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-spring-boot-starter:test \
|
|
--tests '*FileserverAutoConfigurationTest' \
|
|
--tests '*FileserverPropertiesValidationTest'
|
|
```
|
|
|
|
Expected: FAIL because properties and auto-configurations do not exist.
|
|
|
|
- [ ] **Step 3: Implement typed properties and conditional beans**
|
|
|
|
Bind these groups exactly:
|
|
|
|
```text
|
|
storage
|
|
upload
|
|
download
|
|
nginx
|
|
verification
|
|
quota
|
|
cleanup
|
|
tus
|
|
httpbis-draft12
|
|
mvc.executor
|
|
webflux
|
|
```
|
|
|
|
Validate:
|
|
|
|
```text
|
|
root is absolute and outside configured webroot/config roots
|
|
maxRequestSize >= maxFileSize
|
|
soft limit < hard limit
|
|
maxRanges between 1 and 8
|
|
ATOMIC_MOVE_REQUIRED matches probe
|
|
scanner-required has a verifier bean
|
|
nginx enabled has token service and internal prefix
|
|
tus and draft endpoints do not collide
|
|
```
|
|
|
|
Use `@ConditionalOnWebApplication` and `@ConditionalOnClass` so MVC and WebFlux adapters do not appear together accidentally unless an explicit dual-adapter test application requests both.
|
|
|
|
- [ ] **Step 4: Run starter context tests**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-spring-boot-starter:test
|
|
```
|
|
|
|
Expected: PASS; invalid property combinations fail during context startup with stable validation messages.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-spring-boot-starter
|
|
git commit -m "feat: add fileserver Spring Boot starter"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 32: Filesystem, HTTP, fault, performance Testkit 구현
|
|
|
|
**Files:**
|
|
- Create: `modules/fileserver/fileserver-testkit/src/main/java/io/backend/skeleton/fileserver/testkit/ContentStoreContract.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/main/java/io/backend/skeleton/fileserver/testkit/HttpDownloadContract.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/main/java/io/backend/skeleton/fileserver/testkit/CrashPoint.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/main/java/io/backend/skeleton/fileserver/testkit/ProcessCrashHarness.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/main/java/io/backend/skeleton/fileserver/testkit/NfsTestEnvironment.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/main/java/io/backend/skeleton/fileserver/testkit/PvcCertificationDescriptor.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/LocalContentStoreContractTest.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/CrashRecoveryMatrixTest.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/LargeFileBoundedMemoryTest.java`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/NfsAmbiguityIntegrationTest.java`
|
|
- Create: `infra/fileserver/nfs/compose.yml`
|
|
- Create: `infra/fileserver/kubernetes/pvc-certification-job.yaml`
|
|
|
|
**Interfaces:**
|
|
- Produces reusable contracts for future Object Storage adapters.
|
|
- Provides crash points before/after append, publish, and metadata commit.
|
|
- Certification descriptors identify Kubernetes, CSI, StorageClass, access mode, backend, and mount options.
|
|
|
|
- [ ] **Step 1: Write failing contract and crash-matrix tests**
|
|
|
|
```java
|
|
abstract class ContentStoreContract {
|
|
protected abstract BlockingContentStore store();
|
|
|
|
@Test
|
|
void createAppendFinalizeStatReadDeleteRoundTrip() throws Exception {
|
|
UploadHandle handle = store().createUpload(fixture.createCommand());
|
|
store().append(handle, 0, fixture.channel("abcdef"), 6);
|
|
StoredContent content = store().finalizeUpload(handle, fixture.finalizeCommand());
|
|
|
|
assertThat(store().stat(content.contentKey()).size()).isEqualTo(6);
|
|
assertThat(fixture.read(store().openRead(content.contentKey(), new ByteRange(1, 3))))
|
|
.isEqualTo("bcd");
|
|
assertThat(store().delete(content.contentKey(), DeletePrecondition.none()).deleted())
|
|
.isTrue();
|
|
}
|
|
}
|
|
```
|
|
|
|
```java
|
|
@ParameterizedTest
|
|
@EnumSource(CrashPoint.class)
|
|
void readyInvariantSurvivesEveryCrashPoint(CrashPoint crashPoint) {
|
|
harness.runUploadAndKillAt(crashPoint);
|
|
harness.restartAndReconcile();
|
|
|
|
assertThat(harness.readyFiles())
|
|
.allSatisfy(file -> {
|
|
assertThat(file.physicalContentExists()).isTrue();
|
|
assertThat(file.digestMatches()).isTrue();
|
|
});
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run testkit tests to verify they fail**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-testkit:test \
|
|
--tests '*ContentStoreContract*' --tests '*CrashRecoveryMatrixTest'
|
|
```
|
|
|
|
Expected: FAIL because the testkit contracts and harness do not exist.
|
|
|
|
- [ ] **Step 3: Implement reusable certification harnesses**
|
|
|
|
Implement contract scenarios for:
|
|
|
|
```text
|
|
create-only race
|
|
append offset
|
|
range read
|
|
checksum
|
|
finalize
|
|
logical and physical delete
|
|
symlink no-follow
|
|
disk full
|
|
permission denied
|
|
process kill at every crash point
|
|
slow client
|
|
network interruption
|
|
NFS rename ambiguity
|
|
large-file bounded heap and direct memory
|
|
```
|
|
|
|
The NFS environment must support server restart and a network cut. The PVC job writes a machine-readable result containing the full certification tuple and probe results.
|
|
|
|
- [ ] **Step 4: Run local, NFS, and large-file suites**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-testkit:test
|
|
```
|
|
|
|
Expected: PASS for local tests; NFS tests are tagged and run when `FILESERVER_NFS_TESTS=true`. Large-file test confirms heap does not scale with file size.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add modules/fileserver/fileserver-testkit infra/fileserver/nfs infra/fileserver/kubernetes
|
|
git commit -m "test: add fileserver certification harness"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 33: CI matrix, 지원 문서, 운영 Runbook, release gate 연결
|
|
|
|
**Files:**
|
|
- Create: `.github/workflows/fileserver-pr.yml`
|
|
- Create: `.github/workflows/fileserver-nightly.yml`
|
|
- Create: `.github/workflows/fileserver-release.yml`
|
|
- Create: `docs/fileserver/support-matrix.md`
|
|
- Create: `docs/fileserver/http-contract.md`
|
|
- Create: `docs/fileserver/storage-certification.md`
|
|
- Create: `docs/fileserver/security.md`
|
|
- Create: `docs/fileserver/operations.md`
|
|
- Create: `docs/fileserver/upgrade-guide.md`
|
|
- Create: `modules/fileserver/fileserver-testkit/src/test/java/io/backend/skeleton/fileserver/testkit/DocumentationCoverageTest.java`
|
|
|
|
**Interfaces:**
|
|
- Connects every support claim to a CI job or certification artifact.
|
|
- Documents Stable, Beta, Limited, Compatibility, and Experimental levels.
|
|
|
|
- [ ] **Step 1: Write a failing documentation coverage test**
|
|
|
|
```java
|
|
class DocumentationCoverageTest {
|
|
@Test
|
|
void everyRuntimeProfileHasAReferencedCiJob() throws Exception {
|
|
SupportMatrix matrix = SupportMatrix.load(Path.of("docs/fileserver/support-matrix.md"));
|
|
WorkflowIndex workflows = WorkflowIndex.load(Path.of(".github/workflows"));
|
|
|
|
assertThat(matrix.requiredProfiles())
|
|
.allMatch(profile -> workflows.containsJob(profile.ciJob()));
|
|
}
|
|
|
|
@Test
|
|
void everyPublicEndpointAppearsInHttpContract() throws Exception {
|
|
Set<String> endpoints = EndpointScanner.scanPublicFileserverEndpoints();
|
|
String contract = Files.readString(Path.of("docs/fileserver/http-contract.md"));
|
|
|
|
assertThat(endpoints).allMatch(contract::contains);
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run the coverage test to verify it fails**
|
|
|
|
```bash
|
|
./gradlew :modules:fileserver:fileserver-testkit:test \
|
|
--tests '*DocumentationCoverageTest'
|
|
```
|
|
|
|
Expected: FAIL because workflows and docs do not exist.
|
|
|
|
- [ ] **Step 3: Add workflows and complete operational documentation**
|
|
|
|
PR workflow runs:
|
|
|
|
```text
|
|
unit and architecture tests
|
|
local ext4 contract
|
|
MVC Tomcat contract
|
|
WebFlux Reactor Netty contract
|
|
security suite
|
|
bounded-memory regression
|
|
```
|
|
|
|
Nightly runs:
|
|
|
|
```text
|
|
XFS
|
|
NFSv4.1 and server restart
|
|
Windows NTFS compatibility
|
|
large-file performance
|
|
slow client
|
|
process-kill matrix
|
|
```
|
|
|
|
Release runs:
|
|
|
|
```text
|
|
Spring Framework 6.2 and 7.0 compatible lines
|
|
Nginx stable
|
|
PVC RWO certification
|
|
optional PVC RWX certification
|
|
multi-instance lease
|
|
fault injection
|
|
sensitive telemetry scan
|
|
support matrix diff
|
|
```
|
|
|
|
`operations.md` must include storage-full, orphan growth, verification backlog, NFS ambiguity, PVC remount, Nginx delegation failure, and cleanup backlog runbooks with exact metric names and recovery commands.
|
|
|
|
- [ ] **Step 4: Run documentation coverage and full release verification**
|
|
|
|
```bash
|
|
./gradlew clean test
|
|
./gradlew :modules:fileserver:fileserver-testkit:test \
|
|
--tests '*DocumentationCoverageTest'
|
|
```
|
|
|
|
Expected: PASS; every support claim maps to a concrete workflow job and every public endpoint is documented.
|
|
|
|
- [ ] **Step 5: Commit**
|
|
|
|
```bash
|
|
git add .github/workflows docs/fileserver modules/fileserver/fileserver-testkit
|
|
git commit -m "docs: connect fileserver support claims to CI"
|
|
```
|
|
|
|
---
|
|
|
|
## 3. 작업 간 의존 순서
|
|
|
|
```text
|
|
Task 1
|
|
├─ Task 2
|
|
│ ├─ Task 3
|
|
│ ├─ Task 4
|
|
│ └─ Task 5
|
|
│ └─ Task 6
|
|
│ └─ Task 7
|
|
├─ Task 8
|
|
│ └─ Task 9
|
|
│ ├─ Task 10
|
|
│ └─ Task 11
|
|
├─ Task 12
|
|
├─ Task 13
|
|
│ └─ Task 14
|
|
│ └─ Task 15
|
|
├─ Task 16
|
|
│ └─ Task 14 integration
|
|
├─ Task 17
|
|
├─ Task 18
|
|
│ ├─ Task 20
|
|
│ ├─ Task 22
|
|
│ └─ Task 23
|
|
├─ Task 19
|
|
├─ Task 21
|
|
├─ Task 24
|
|
│ └─ Task 25
|
|
├─ Task 26
|
|
│ ├─ Task 27
|
|
│ └─ Task 28
|
|
├─ Task 29
|
|
├─ Task 30
|
|
├─ Task 31
|
|
├─ Task 32
|
|
└─ Task 33
|
|
```
|
|
|
|
권장 직렬 실행 순서는 Task 1부터 Task 33까지다. 병렬 실행은 다음 묶음에서만 허용한다.
|
|
|
|
```text
|
|
Task 16 verification ↔ Task 18 HTTP contract
|
|
Task 19 MVC upload ↔ Task 21 WebFlux upload
|
|
Task 20 MVC download ↔ Task 22 WebFlux download
|
|
Task 27 tus ↔ Task 28 draft12, 단 Task 26 완료 후
|
|
Task 29 security ↔ Task 30 observability, 공통 API가 안정된 후
|
|
```
|
|
|
|
---
|
|
|
|
## 4. 단계별 Release 기준
|
|
|
|
### Milestone A — Core Alpha
|
|
|
|
완료 작업:
|
|
|
|
```text
|
|
Task 1~15
|
|
```
|
|
|
|
Gate:
|
|
|
|
- core module dependency boundary 통과
|
|
- metadata migration·optimistic locking 통과
|
|
- local create·append·digest·publish contract 통과
|
|
- READY invariant와 ambiguous reconciliation 통과
|
|
- 100 MiB upload에서 bounded memory 확인
|
|
|
|
### Milestone B — HTTP Beta
|
|
|
|
완료 작업:
|
|
|
|
```text
|
|
Task 16~22, Task 29
|
|
```
|
|
|
|
Gate:
|
|
|
|
- raw·multipart upload
|
|
- GET·HEAD·single Range
|
|
- conditional request
|
|
- MVC·WebFlux parity
|
|
- DataBuffer leak 0
|
|
- path·symlink·filename·range security suite 통과
|
|
|
|
### Milestone C — Distributed RC
|
|
|
|
완료 작업:
|
|
|
|
```text
|
|
Task 23~26, Task 30~32
|
|
```
|
|
|
|
Gate:
|
|
|
|
- Nginx parity
|
|
- logical delete와 cleanup
|
|
- admin isolation
|
|
- two-node writer lease
|
|
- PVC RWO certification
|
|
- process-kill matrix
|
|
- sensitive telemetry scan
|
|
|
|
### Milestone D — Extended Release
|
|
|
|
완료 작업:
|
|
|
|
```text
|
|
Task 27~28, Task 33
|
|
```
|
|
|
|
Gate:
|
|
|
|
- tus 1.0 protocol suite
|
|
- draft12 isolation
|
|
- NFS limited profile fault tests
|
|
- support matrix와 CI mapping
|
|
- operations runbook review
|
|
|
|
---
|
|
|
|
## 5. 구현자가 임의로 변경하면 안 되는 결정
|
|
|
|
- `ContentStore`에 `Path` 또는 provider SDK 타입을 추가하지 않는다.
|
|
- public endpoint에 path query parameter를 추가하지 않는다.
|
|
- state 변경을 JPA entity setter로 우회하지 않는다.
|
|
- READY gate를 controller마다 복제하지 않고 application service에서 강제한다.
|
|
- create-only 기본을 overwrite 기본으로 바꾸지 않는다.
|
|
- atomic move 지원을 설정값만으로 가정하지 않는다.
|
|
- `Files.exists` 후 create하는 TOCTOU 패턴을 사용하지 않는다.
|
|
- WebFlux body를 `DataBufferUtils.join`으로 전체 적재하지 않는다.
|
|
- MVC에서 `MultipartFile#getBytes()`를 사용하지 않는다.
|
|
- filename 또는 client MIME을 physical key·보안 verdict로 사용하지 않는다.
|
|
- scanner timeout을 ACCEPT로 변환하지 않는다.
|
|
- multi-instance 정확성을 `FileLock` 또는 NFS lock에 맡기지 않는다.
|
|
- Nginx internal URI에 physical path를 넣지 않는다.
|
|
- tus와 HTTPbis draft DTO·endpoint를 공유하지 않는다.
|
|
- cleanup이 version·lease 확인 없이 삭제하지 않는다.
|
|
- `AmbiguousCompletionException`을 일반 retryable exception으로 낮추지 않는다.
|
|
|
|
---
|
|
|
|
## 6. 계획 자체 검증 체크리스트
|
|
|
|
- [ ] 설계서의 포함 범위가 최소 하나의 Task에 매핑된다.
|
|
- [ ] 설계서의 비지원 범위를 구현하는 Task가 없다.
|
|
- [ ] Task 1~33 번호가 연속적이다.
|
|
- [ ] 모든 Task에 Files, Interfaces, 실패 테스트, 실패 확인, 구현, 통과 확인, commit이 있다.
|
|
- [ ] later Task가 사용하는 공개 타입은 earlier Task에서 정의된다.
|
|
- [ ] MVC·WebFlux·Nginx가 동일한 `DownloadDecision`을 사용한다.
|
|
- [ ] READY transition은 physical stat·digest 검증 뒤에만 실행된다.
|
|
- [ ] multi-instance append는 lease token과 expected offset을 요구한다.
|
|
- [ ] tus Stable과 draft Experimental이 분리돼 있다.
|
|
- [ ] security suite가 traversal, symlink, filename, Range, scriptable content를 포함한다.
|
|
- [ ] CI와 support matrix가 자동 coverage test로 연결된다.
|
|
- [ ] 문서에 미확정 표식, 빈 구현 지시, 무정의 type이 없다.
|
|
|
|
---
|
|
|
|
## 7. 실행 인계
|
|
|
|
계획 실행 시 권장 방식은 `superpowers:subagent-driven-development`다. 각 Task마다 새 작업자를 사용하고 다음 두 단계 review를 적용한다.
|
|
|
|
```text
|
|
1. 요구사항·설계 일치 review
|
|
2. 코드 품질·테스트 evidence review
|
|
```
|
|
|
|
동일 세션에서 실행할 경우 `superpowers:executing-plans`를 사용하고 Milestone A, B, C, D마다 전체 test·diff·문서 gate를 확인한다.
|