donghyeon.kang/tech-log-backend
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

chore: record pre-existing uncommitted repository state

Browse Source 
Snapshot of the in-flight state that already existed, identically, in both
this worktree and the main checkout before this session began: the initial
HTTP Client platform implementation (previously untracked), the redis-lab
removal, and the JPA / object-storage / notification integration work.

Kept separate from this session's HTTP Client review response, which lands
in the following commit, so the two bodies of work stay reviewable apart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
2026-08-11 16:48:43 +09:00
co-authored by Claude Opus 5
parent 1a3b560678
commit 5f10b791d3
1857 changed files with 130925 additions and 72491 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/fileserver/configuration.md
+176
View File
@@ -0,0 +1,176 @@
# Fileserver configuration
Every key below lives under `app.fileserver-platform` (environment form
`APP_FILESERVER_PLATFORM_*`). That namespace is the HTTP platform's alone: `app.fileserver.*`
belongs to the R2 tabular publication capability and `app.file-export.*` to the R1 CSV export, and
the three are deliberately separate so switching one on cannot switch on another.
While `app.fileserver-platform.enabled` is false none of these keys is bound at all — the
auto-configuration that binds them is not processed — so a malformed value in a block nobody
enabled cannot fail a startup. Once enabled, binding is strict: an unknown key under the prefix is
refused rather than ignored. The defaults are the conservative ones: the
capability is off, the admin plane is off, background reclamation is off, and there is no permissive
authorization fallback. Turning the capability on is a deliberate act, and so is every surface it
exposes.
## Minimum to start
```yaml
ca-skeleton:
fileserver:
enabled: true
instance-id: ${HOSTNAME} # writer-lease owner; must be unique per node
storage:
root: /var/lib/backend/files # absolute, outside any webroot or config dir
security:
access-policy: role-based # or supply your own FileAccessPolicy bean
observability:
fingerprint-key: ${FILESERVER_FINGERPRINT_KEY}
```
Startup fails, rather than degrading, when any of these is missing or unsafe:
| Condition | Why it is fatal |
| --- | --- |
| `security.access-policy` left at `required` with no `FileAccessPolicy` bean | a file capability that authorizes by default is worse than one that refuses to start |
| `observability.fingerprint-key` unset while metrics are on | an unkeyed digest of an enumerable identifier is reversible |
| the storage root fails a mandatory capability probe | a volume that cannot create atomically, keep staging and content on one FileStore, or refuse symlinks is unsafe, not degraded |
| `storage.publish-mode: atomic-move-required` on a volume where the probe could not prove an atomic move | the configured guarantee cannot be delivered |
| `security.access-policy: unenforced` under a `prod` profile | a value that was convenient in development must not survive promotion |
## Authorization — `security`
| Key | Default | Meaning |
| --- | --- | --- |
| `access-policy` | `required` | `required` (supply your own bean), `role-based`, or `unenforced` |
| `read-roles` | `ROLE_FILE_READ` | grants metadata read and download |
| `write-roles` | `ROLE_FILE_WRITE` | grants create, append, finalize, delete, copy, move |
| `admin-roles` | `ROLE_FILE_ADMIN` | grants reverify and force-delete, and gates `/internal/fileserver/**` at the servlet chain |
There is no anonymous-read switch. Every Fileserver route is authenticated by the servlet chain
before any application policy is consulted, so such a setting could only ever have described a
permission the transport had already refused — a configuration that reads as if it grants access
and does not.
The three tiers do not inherit. An admin role cannot delete through the data plane, and a write role
cannot reach the management plane — a role model where "can delete" implied "can force-delete" would
make the audited plane reachable through the unaudited one.
`unenforced` authorizes everything and exists so a developer can exercise upload and download before
deciding on a role model. It is refused under a production profile.
## Storage — `storage`
| Key | Default | Meaning |
| --- | --- | --- |
| `root` | `/var/lib/backend/files` | absolute path; the only place a path exists |
| `publish-mode` | `atomic-move-preferred` | `atomic-move-required`, `atomic-move-preferred`, `metadata-pointer` |
| `buffer-size` | `128KB` | bounds every transfer allocation; memory never scales with file size |
| `forbidden-root-ancestors` | `/app,/etc,/usr/share/nginx/html` | roots the storage root must not live under (webroot, config dirs) |
`root` must be absolute. A relative root resolves against the process working directory, which is
one path in a container and another in a test, so it is refused at binding time.
Three former keys are gone, pinned as constants instead: staging and content share one FileStore,
symbolic links are never followed, and the object and its directory are synced before READY. Each
is an invariant the atomic publish and the namespace boundary are built on — a deployment that
could switch one off would be running a different capability under the same name and the same
tests.
The storage provider has no selector either. There is exactly one implementation, and a `type` key
with one legal value is a promise of pluggability that nothing keeps.
## Upload, download, transfer
| Key | Default | Meaning |
| --- | --- | --- |
| `upload.max-file-size` | `100MB` | hard ceiling; also drives `spring.servlet.multipart.max-file-size` |
| `upload.max-request-size` | `110MB` | request envelope; must be at least `max-file-size` |
| `upload.initial-reservation` | `8MB` | quota reserved when the length is unknown |
| `upload.ttl` | `1h` | how long a resumable upload stays claimable |
| `upload.reservation-ttl` | `24h` | how long an unsettled quota reservation survives |
| `upload.lease-duration` | `30s` | writer lease; renewed at one third of this |
| `upload.max-parts` | `16` | multipart part ceiling |
| `upload.require-content-length` | `false` | refuse chunked raw uploads |
| `download.cache-control` | `private, no-store` | emitted on every content response |
| `download.inline-allowed` | `false` | scriptable content is always an attachment regardless |
| `download.max-ranges` | `1` | multi-range responses are opt-in |
| `download.max-range-bytes` | `100MB` | total bytes one ranged response may cover |
| `download.zero-copy-enabled` | `true` | hand large plaintext responses to the kernel |
| `download.zero-copy-minimum-bytes` | `16MB` | below this the syscall setup costs more than it saves |
| `transfer.core-size` / `max-size` / `queue-capacity` | `8` / `32` / `64` | blocking transfer pool bounds |
| `transfer.await-seconds` | `300` | how long a transfer may occupy a pool thread |
Zero copy changes no header and no status. When storage declines it — an unreadable region, an
unsupported backend — the response is streamed instead and is byte-identical.
## Verification — `verification`
| Key | Default | Meaning |
| --- | --- | --- |
| `timeout` | `5s` | per-verifier ceiling |
| `require-media-type-verdict` | `false` | refuse a file whose type could not be determined |
| `inline-safe-profile` | `false` | accept scriptable content instead of quarantining it |
Set `inline-safe-profile: true` only when downloads are never served inline from a trusted origin.
## Quota and admission — `quota`
| Key | Default | Meaning |
| --- | --- | --- |
| `instance-upload-permits` | `16` | concurrent uploads this node admits |
| `scope-upload-permits` | `4` | concurrent uploads one namespace admits |
| `direct-download-permits` | `64` | concurrent non-delegated downloads |
| `soft-high-water` | `0.70` | storage fraction at which pressure is reported |
| `hard-high-water` | `0.85` | storage fraction at which uploads are refused |
When the storage fraction cannot be read, admission treats it as unknown and does not apply the
high-water rule — a synthetic `0` would silently disable the guard, and a synthetic `1` would take
the capability down over a failed syscall.
## Background reclamation — `cleanup`
| Key | Default | Meaning |
| --- | --- | --- |
| `enabled` | `false` | run the cleanup worker on this node |
| `interval` | `60s` | fixed delay between batches, not fixed rate |
| `max-items` | `100` | items one batch may claim |
| `max-bytes` | `1GB` | bytes one batch may reclaim |
| `retry-backoff` | `5m` | how long a failed item waits before it is due again |
The worker deletes physical objects, so it is off until a deployment decides otherwise. A node
without it still queues cleanup items; another node or an operator reclaims them. An item that fails
eight times is abandoned rather than retried forever — it stays visible to an operator, parked
rather than discarded.
## Management plane — `admin`
| Key | Default | Meaning |
| --- | --- | --- |
| `enabled` | `false` | expose `/internal/fileserver/**` |
| `orphan-minimum-age` | `1h` | how long an unreferenced object must exist before a scan may name it |
Publishing content and committing its record are two steps. Anything younger than
`orphan-minimum-age` is assumed to be mid-commit rather than abandoned; shortening this makes
concurrent uploads look like orphans.
## Front-proxy delegation — `nginx`
| Key | Default | Meaning |
| --- | --- | --- |
| `enabled` | `false` | emit `X-Accel-Redirect` instead of a body |
| `internal-prefix` | `/__files/` | must be an `internal` location resolving to the content root |
| `object-suffix` | `.bin` | layout suffix the proxy appends |
| `minimum-size` | `16MB` | below this the application serves the transfer itself |
Delegation is decided only after authorization and the READY gate, so an internal redirect can only
ever name content the caller was already allowed to read.
## Protocols — `tus`
| Key | Default | Meaning |
| --- | --- | --- |
| `enabled` | `false` | expose the tus 1.0 endpoints |
The HTTPbis resumable-upload draft-12 surface is experimental and documented in
[support-matrix.md](support-matrix.md).
docs/fileserver/design-deviations.md
+213
View File
@@ -0,0 +1,213 @@
# Fileserver — deviations from the design
The design specification and the implementation plan are frozen documents. Where implementation
found them under-specified or self-contradicting, the resolution is recorded here rather than by
editing the specification, and every entry names the test that pins the decision.
## Resolved inconsistencies in the state machine and contracts
### 1. `CREATED → FAILED` has no edge in the transition table
A create that fails after the record exists must end in `FAILED`, but the table has no direct edge.
The record therefore walks `CREATED → UPLOADING → FAILED`, which is also the honest reading: the
upload had been admitted before it failed.
Pinned by `UploadApplicationServiceTest` (application-core).
### 2. `ContentKey`'s alphabet admits a leading separator
The design's key pattern `[a-z0-9/_-]{16,200}` matches `/etc/passwd/...`. Rejecting an absolute path
at the value type would change a design-fixed contract, so the stricter shape check lives in
`PhysicalPathResolver`, per §12.2 rule 1 — the only place that turns an identifier into a path.
Pinned by `PhysicalPathResolverTest`.
### 3. `VERIFYING → DELETING` has no edge
Deleting a file that is mid-verification has no legal transition. The lifecycle service refuses it
with `409 FILE_NOT_READY` rather than inventing an edge, which matches the allowed-state list the
JPA `markDeleting` statement already enforced.
Pinned by `FileLifecycleServiceTest`.
### 4. `If-Match` is specified as an ETag but the lifecycle was designed around the row version
The HTTP contract sends an entity tag; the metadata store guards on a numeric version. The service
takes `Optional<String> expectedEtag` and compares against the record's strong validator, so the
precondition a client sends is the precondition that is checked.
Pinned by `FileLifecycleServiceTest`.
### 5. The filename policy left `:` intact
`C:\Windows\system.ini` sanitized to `C:Windowssystem.ini` — a drive-qualified name surviving into
display text and headers. `:` joined the structural strip set.
Pinned by `FileserverHardeningContractTest` and `AmbiguousFilesystemOperationDetectorTest`.
## Additions the design implies but does not specify
### 6. `fs_recovery_item`
§10.2 lists five core tables and none of them can hold the recovery queue, yet §29.3 requires one:
reconciliation reports files whose bytes and metadata disagree, and holding that list in memory
would lose exactly the cases a restart interrupted. Added in
`V2__fileserver_recovery_and_staging_cleanup.sql` with one open item per file, so repeated sweeps
update a worklist rather than accumulating a log.
Pinned by `PostgreSqlFileserverReclamationIntegrationTest`.
### 7. `fs_cleanup_item.upload_id`
A staging object is addressed by upload, not by file. Without this column a queued staging cleanup
could name only already-published content, so a cancelled or expired upload left bytes nothing could
find. Added in the same migration, with a check constraint that an item names exactly one target.
Pinned by `PostgreSqlFileserverReclamationIntegrationTest`.
### 8. `ContentReferenceLedger` and `StagingUploadLocator`
The orphan scan must ask whether a record still claims a physical object, and reconciliation must
map a file back to the upload that last staged it. Neither question is answerable through the
design's `FileMetadataStore` or `UploadSessionStore` as written. Rather than widen those
design-fixed interfaces, both are narrow single-method ports.
Pinned by `PostgreSqlFileserverReclamationIntegrationTest` and `LocalOrphanScanAdapterTest`.
## Interpretations
### 9. Quota settlement is FIFO within a scope
Nothing links a reservation row to the upload that took it, and the design deliberately reclaims
stragglers by TTL and the `STALE_QUOTA_RESERVATION` cleanup type rather than threading a reservation
id through the upload session. `QuotaCommitGateway` therefore settles the oldest live reservation in
the file's namespace.
Which row closes does not change any quota decision: enforcement sums reserved and committed bytes
per scope and never reads an individual row. Concurrent uploads of different sizes can leave the
reserved total transiently high or low, and it converges as each settles. Durable usage with no live
reservation behind it — an upload that outlived its TTL — is still recorded, because a ledger that
silently under-counts is worse than one that is briefly imprecise.
Pinned by `PostgreSqlFileserverReclamationIntegrationTest`.
### 10. Zero copy is a channel transfer, not a file handoff
Task 22 asks for zero copy on local files; §19 forbids a `Path` leaving the storage adapter, and §5
of the plan forbids adding `Path` to the content store. WebFlux's zero-copy API takes a `Path`, so
that route is closed.
The servlet path takes the other one: `ZeroCopyDownloadGateway` receives a `WritableByteChannel` from
the transport and the storage adapter performs `FileChannel.transferTo` into it. That is a genuine
kernel-level transfer with no filesystem concept leaving storage. The reactive path continues to
stream with bounded demand.
Zero copy is an optimization with no observable difference: when storage declines, the response is
streamed and is byte-identical.
Pinned by `LocalStorageGatewayContractTest` and `ZeroCopyEligibilityTest`.
### 11. The Fileserver JPA stores are gated on the capability switch
The metadata store, session store, quota service, queues, ledger, and staging locator carry
`@ConditionalOnProperty(app.fileserver-platform.enabled)` even though the rest of
`adapter:outbound:persistence-jpa` is unconditional.
Without the gate, every composition root that includes the persistence module built these beans —
including `sample-portfolio`, which has no Fileserver — and each of them needs collaborators only
the Fileserver configuration provides. That is the same rule the design states for the transport
surface, applied to persistence: no surface appears merely because the dependency is present.
`FileStateMachine` is bound alongside them, in `FileserverStorageConfiguration`. It had no
production binding at all before, which made the metadata store unconstructible in any
component-scanned context.
Pinned by `SampleApplicationContextTest` (the capability off) and
`FileserverRuntimeAssemblyTest` (the capability on).
### 12. Transaction boundaries are owned by the application services, and are deliberately narrow
The design does not say where a transaction begins. The repository does:
`adapter:outbound:persistence-jpa` forbids a repository adapter from owning a `@Transactional`
boundary, and `application-core` owns them through `TransactionPort`. The Fileserver follows that
rule — every `Jpa*` store here declares no `@Transactional` of its own.
What is specific to this capability is how narrow the boundaries are. A boundary covers a contiguous
run of metadata writes and **stops before every storage call**, because a filesystem operation
inside a database transaction would hold a connection for the length of a byte transfer. The upload
path therefore has three boundaries, not one: acquire the lease, transfer the bytes, commit the
offset.
Where several stores must agree, they share one boundary:
| Unit | Why it is one boundary |
| --- | --- |
| reserve quota + insert record + create session | a reservation that outlived a failed insert holds capacity for a file that never existed |
| READY transition + quota commit | a finished file whose reservation was never converted holds capacity until the reservation expires |
| `markDeleting` + enqueue cleanup | a file that stopped being reachable with nothing queued to reclaim it is never collected |
| content delete settlement: reclaim + retire record + close queue item | half of it leaves the item to be retried against content that no longer exists |
What this cannot make atomic is the storage/metadata seam itself — no database boundary could. That
seam is exactly what the ambiguous-completion path and the reconciler exist for, and the one
hand-written compensation that remains (staging creation failing after the records committed) is
there for the same reason.
Pinned by `FileserverRoundTripContractTest` against real PostgreSQL; the application tests use
`DirectTransactions`, which runs a boundary inline and counts it.
## Not implemented
### `AsyncContentStore`, `CapacityAwareContentStore`, `CopyCapableContentStore`, `DelegatedDownloadStore`
Four optional content-store SPIs are declared in `application-core` with no implementation. Each is
an extension point for a backend this template does not ship:
- `AsyncContentStore` — for a backend whose native client is non-blocking. The local platform is
blocking, and the reactive transport bridges to it on a dedicated I/O scheduler.
- `CapacityAwareContentStore` — capacity is reported through `StorageHealthPort` and
`StorageUsageProbe`, which the local platform implements.
- `CopyCapableContentStore` — server-side copy is delivered by `CopyContentGateway`; the local
platform has no cheaper primitive than a streamed copy.
- `DelegatedDownloadStore` — delegation is delivered at the transport boundary by the nginx
`X-Accel-Redirect` strategy, which needs no store participation.
`ContentStoreCapabilities` reports what the running store actually supports, so no unimplemented SPI
is advertised as available.
## Known deviation from the repository's application-layer contract
### 5. Fileserver application services are not `CommandUseCase` / `QueryUseCase`
`src/application-core/CLAUDE.md` requires every inbound port implementation to extend
`CommandUseCase` or `QueryUseCase` and to carry `@UseCaseCapability`, which declares its transaction
mode, idempotency and repository access. The Fileserver instead exposes multi-method services —
`UploadApplicationService`, `DownloadApplicationService`, `FileLifecycleService`,
`FileserverAdminService` and their `Default*` implementations.
This is a real deviation, not an oversight, and it is unenforced: the ArchUnit rules
`inbound_port_implementations_end_with_use_case` and
`inbound_port_implementations_declare_capability` only match types that implement `UseCase`, so a
service that never does is silently exempt. The capability contract that every other feature in
this repository declares is therefore absent here.
Two things follow from it. The transaction mode of each operation is expressed only by which
`TransactionPort` method the body happens to call, rather than declared and checked. And the
application layer holds transport policy it would not hold if each operation were a use case with
its own command: HTTP status codes on `FileserverErrorCode`, `Range` and conditional-request
parsing in `api.transfer`, and `Content-Disposition` construction.
The status mapping in particular is a deliberate trade rather than an accident. It lives in
`application-core` so the servlet transport, the reactive transport and the Nginx delegation path
cannot answer the same failure with three different statuses. Moving it to the transport layer
resolves the layering complaint and reintroduces exactly that drift, which is why this is an
architecture decision rather than a cleanup.
**Status: open, deliberately unresolved in this change set.** Closing it means roughly thirty
command/query use cases, a decision about where the shared status vocabulary lives, and a change to
the ArchUnit rules so a service that bypasses the contract fails the build instead of being exempt
from it. That belongs in its own ADR with its own review, and doing it inside a correctness patch
would mix a large mechanical refactor into changes that need to be readable.
Nothing here is pinned by a test, because the deviation is the absence of a constraint. The next
step is the ADR, not another test.
docs/fileserver/http-contract.md
+105
View File
@@ -0,0 +1,105 @@
# Fileserver HTTP contract
Every public endpoint is listed here. `FileserverDocumentationCoverageTest` scans the controllers
and fails if one is missing, so this file cannot silently fall behind the code.
## Public endpoints
| Method | Path | Success | Notes |
|---|---|---|---|
| POST | `/v1/files` | `201` READY, `202` VERIFYING | multipart single upload |
| POST | `/v1/files:raw` | `201`, `202` | the whole request body is the file |
| POST | `/v1/files:batch` | `200` | ordered per-part results; explicitly non-atomic |
| GET | `/v1/files/{fileId}` | `200` | public metadata; never a content key or path |
| GET | `/v1/files/{fileId}/content` | `200`, `206`, `304` | download |
| HEAD | `/v1/files/{fileId}/content` | `200`, `304` | identical headers, no body |
| DELETE | `/v1/files/{fileId}` | `202`, `204` | logical delete first |
| POST | `/v1/files/{fileId}:copy` | `202` | create-only target |
| POST | `/v1/files/{fileId}:move` | `200` | logical namespace change only |
| OPTIONS | `/v1/uploads` | `204` | tus capability discovery |
| POST | `/v1/uploads` | `201` | tus creation |
| HEAD | `/v1/uploads/{uploadId}` | `204` | tus offset |
| PATCH | `/v1/uploads/{uploadId}` | `204` | tus append |
| DELETE | `/v1/uploads/{uploadId}` | `204` | tus termination |
| POST | `/v1/experimental/draft12/uploads` | `201` | Experimental; off by default |
| PATCH | `/v1/experimental/draft12/uploads/{uploadId}` | `204` | Experimental; off by default |
## Management endpoints
Reachable only where both `app.fileserver-platform.enabled=true` and
`app.fileserver-platform.admin.enabled=true`, gated at the servlet chain on
`app.fileserver-platform.security.admin-roles`, and intended for a management
port rather than the public one.
| Method | Path |
|---|---|
| GET | `/internal/fileserver/storage-health` |
| GET | `/internal/fileserver/capabilities` |
| GET | `/internal/fileserver/orphans` |
| POST | `/internal/fileserver/orphans:reconcile` |
| POST | `/internal/fileserver/files/{fileId}:reverify` |
| POST | `/internal/fileserver/files/{fileId}:force-delete` |
| GET | `/internal/fileserver/uploads/incomplete` |
| POST | `/internal/fileserver/uploads:cleanup` |
## Status codes
| Status | Condition |
|---:|---|
| `200` | metadata, full GET, batch result, move |
| `201` | file or upload created |
| `202` | verification or physical cleanup deferred |
| `204` | append, cancel, bodyless update |
| `206` | satisfiable Range |
| `304` | validator matched on GET or HEAD |
| `400` | malformed header or header combination |
| `401` | unauthenticated |
| `403` / `404` | denied, or hidden under the existence-hiding profile |
| `409` | state, offset, or lease conflict |
| `410` | expired upload resource |
| `411` | `require-content-length` profile with no length |
| `412` | precondition failed |
| `413` | size or quota policy violation |
| `415` | upload media type not accepted |
| `416` | unsatisfiable Range; carries the real length |
| `422` | digest, signature, or scanner rejection |
| `429` | transfer admission or rate limit |
| `503` | storage or scanner unavailable |
| `504` | downstream timeout |
| `507` | out of storage capacity |
## Failure body
Every failure answers `application/problem+json` with a stable code and its URN:
```json
{
"type": "urn:fileserver:problem:upload-offset-mismatch",
"title": "Upload offset mismatch",
"status": 409,
"code": "UPLOAD_OFFSET_MISMATCH",
"retryable": true,
"ambiguous": false,
"reconciliationRequired": false,
"traceId": "..."
}
```
The server-side exception message never appears. `ambiguous` is the field a client must read before
retrying: an ambiguous failure may already have taken effect.
## Header contract
| Header | Contract |
|---|---|
| `Content-Type` | client value is a claim; the verified type is stored separately |
| `Content-Disposition` | `attachment` by default; scriptable types are never inline |
| `Accept-Ranges` | `bytes` |
| `Range` | single range by default; multi-range only under an explicit budget |
| `Content-Range` | actual range on `206`; the unsatisfied form on `416` |
| `ETag` | strong validator derived from the SHA-256 |
| `Last-Modified` | metadata publication instant, never a filesystem timestamp |
| `Cache-Control` | `private, no-store` by default |
| `X-Content-Type-Options` | always `nosniff` on a download |
| `Retry-After` | on retryable `409`, `429`, `503`, and `504` |
| `X-Accel-Redirect` | internal only; never forwarded to a client |
docs/fileserver/operations.md
+128
View File
@@ -0,0 +1,128 @@
# Fileserver runbooks
Each runbook names the exact metric that fires it and the exact command that resolves it. A runbook
whose trigger is "someone noticed" is not actionable, so every one below starts from a signal.
## Storage full
**Signal** — `fileserver.quota{result="rejected"}` rising, or `507` responses appearing.
Storage capacity is exhausted or the high-water guard tripped. Uploads are rejected before any bytes
are written, so nothing is corrupt; the system is refusing work it cannot complete.
```bash
curl -s $ADMIN/internal/fileserver/storage-health | jq '.usedFraction, .usableBytes'
curl -s -X POST "$ADMIN/internal/fileserver/uploads:cleanup?maxItems=500&maxBytes=10737418240"
curl -s "$ADMIN/internal/fileserver/orphans?limit=200" | jq '[.[].sizeBytes] | add'
```
Drain the cleanup backlog first — it reclaims space the system already knows is dead. Only then
consider an orphan reconcile, and start with a dry run.
## Orphan growth
**Signal** — `fileserver.cleanup{result="skipped"}` climbing, or the orphan scan returning more
objects each run.
Physical objects exist with no metadata record pointing at them. This is not immediately dangerous —
nothing serves them — but it consumes capacity indefinitely.
```bash
# Always look first. A reconcile without dryRun=false is a plan, not an action.
curl -s -X POST "$ADMIN/internal/fileserver/orphans:reconcile" \
-H 'content-type: application/json' -d '{"limit":100}' | jq '.candidates'
# Apply only the fingerprints you were just shown.
curl -s -X POST "$ADMIN/internal/fileserver/orphans:reconcile" \
-H 'content-type: application/json' \
-d '{"dryRun":false,"limit":100,"maxBytes":1073741824,
"expectedFingerprints":["<from the dry run>"],"reasonCode":"ORPHAN_GROWTH_RUNBOOK"}'
```
Echoing the fingerprints is the safety property: an object that changed between the scan and the
apply is skipped rather than deleted.
## Verification backlog
**Signal** — `fileserver.verification.queue{age_bucket="old"}` non-zero, or files sitting in
VERIFYING.
A verifier is slow or unavailable. Files stay non-public, which is the correct failure direction: a
`RETRY` verdict never becomes an `ACCEPT`.
```bash
curl -s $ADMIN/internal/fileserver/capabilities | jq '.storageType'
# Once the verifier is healthy, quarantined files can be re-examined individually.
curl -s -X POST "$ADMIN/internal/fileserver/files/$FILE_ID:reverify"
```
Do not clear the backlog by disabling verification. A file that reached READY without an accepting
verdict cannot be distinguished later from one that was verified.
## NFS ambiguity
**Signal** — problem documents carrying `"ambiguous": true`, or
`fileserver.transfer.interruption{reason="stale_handle"}`.
An operation's outcome could not be determined: the response was lost after the write or rename may
have landed. These are never retried automatically.
```bash
# The recovery queue holds the files awaiting a decision.
curl -s "$ADMIN/internal/fileserver/uploads/incomplete?limit=100" | jq
```
Reconciliation compares the physical size and digest against the record and only confirms READY when
all four of key, size, digest, and version agree. Anything short of that is reported, never guessed.
## PVC remount
**Signal** — startup failure naming "atomic move", "same file store", or "not writable".
The volume was remounted somewhere the probe can no longer prove a required capability. The
application refuses traffic rather than serving from storage it cannot publish to atomically.
```bash
kubectl apply -f infra/fileserver/kubernetes/pvc-certification-job.yaml
kubectl logs job/fileserver-pvc-certification
```
Compare the printed tuple with the certified one in `docs/fileserver/storage-certification.md`. A
mismatch in CSI driver, StorageClass, access mode, or mount options is the cause; the certification
does not carry across it.
## Nginx delegation failure
**Signal** — `fileserver.download.delegation{delegated="true"}` with client-visible `404`s.
The internal location is misconfigured, so the proxy cannot resolve the redirect it was handed.
```bash
# The internal prefix must resolve to the content root and must be marked `internal`.
grep -A5 '__files' infra/fileserver/nginx/nginx.conf
curl -s $ADMIN/internal/fileserver/capabilities | jq '.capabilities.delegatedDownload'
```
Turning delegation off is a safe immediate mitigation: the application serves the transfer itself,
slower but correct.
```bash
app.fileserver-platform.nginx.enabled=false
```
## Cleanup backlog
**Signal** — `fileserver.cleanup{result="deferred"}` rising, or reclaimed bytes flat while deletes
continue.
Items are being deferred faster than they drain. The usual cause is an active writer lease still
holding staging objects, which is correct behaviour, not a fault.
```bash
curl -s "$ADMIN/internal/fileserver/uploads/incomplete?limit=100" \
| jq '[.[] | select(.leaseUntil != null)] | length'
curl -s -X POST "$ADMIN/internal/fileserver/uploads:cleanup?maxItems=500&maxBytes=10737418240"
```
If the deferrals are all `ACTIVE_WRITER_LEASE`, the backlog resolves itself as those uploads expire.
Never delete staging content to clear a backlog: an upload that is mid-flight will corrupt.
docs/fileserver/security.md
+74
View File
@@ -0,0 +1,74 @@
# Fileserver security model
## The rule everything else follows
Uploaded content is attacker-controlled. Every guard below exists because some part of the request
— the filename, the declared media type, the range, the offset — is a value the caller chose.
## Path safety
A client value never becomes a path. The physical key is server-generated, and `ContentKey`'s
character class excludes `.` entirely, so no traversal or extension-shaped segment survives
validation. `DefaultPhysicalPathResolver` is the only place an identifier becomes a `Path`, and it
normalizes and re-checks containment after construction rather than trusting the input.
Symlink refusal happens at open time, not only at construction. A parent directory can be replaced
between the two, so a check that ran only at path-building time would be a race, not a guard.
## Filename handling
`OriginalFilenamePolicy` strips path separators, NUL, quoting characters, and the colon — the last
because on Windows it opens both a drive reference and an NTFS alternate data stream, so a name that
keeps it is still path-shaped after the slashes are gone. Control characters and bidirectional
overrides are removed, dot runs collapsed, reserved device names guarded, and the result is bounded
in UTF-8 bytes.
The sanitized name is display data. It is never used to build a key, and it reaches a header only
through `ContentDispositionFactory`, which restricts the ASCII form and percent-encodes the UTF-8
form.
## Content type
The client's `Content-Type` is stored as a claim. The verified type comes from the verification
pipeline, and only the verified type is served. A claimed type that contradicts the content is
quarantined rather than corrected.
Scriptable types are never served inline, whatever the caller asked for: serving stored HTML or SVG
inline from an upload origin is a stored cross-site scripting primitive. Every download also carries
`X-Content-Type-Options: nosniff`.
## Verification precedence
`REJECT > QUARANTINE > RETRY > ACCEPT`. A verifier that times out or throws is `RETRY`, never a
silent pass, and an empty verifier chain answers `RETRY` rather than accepting. A file becomes
publicly readable only after an `ACCEPT`.
## Range safety
The range budget is enforced before content is opened, so a request naming many ranges is rejected
without amplifying into storage work. An unsatisfiable range answers `416` with the real length and
opens nothing.
## Authorization
Every public operation calls the injected `FileAccessPolicy` before any quota reservation or storage
mutation, so a denial leaves no record, no reservation, and no staging object. Startup refuses to
run a production profile with an allow-all policy.
## Delegation
`X-Accel-Redirect` is emitted only after authorization and the READY gate, and only for a full,
unconditional response. The internal prefix must be an `internal` Nginx location; the front proxy
also strips any client-supplied delegation header so a caller cannot name an internal object.
## Telemetry
No metric label, span attribute, or audit record carries a file id, upload id, filename, path, or
user id. Where correlation is needed the value is a keyed HMAC fingerprint — keyed because the
identifier space is enumerable and an unkeyed digest of it is reversible by brute force.
## Ambiguous failures
A failure whose operation may already have taken effect is reported as ambiguous and is never
retryable. On a network filesystem a lost response is indistinguishable from a rejection at the
socket level, so anything not provably safe is treated as ambiguous and sent to reconciliation.
docs/fileserver/storage-certification.md
+66
View File
@@ -0,0 +1,66 @@
# Storage certification
## Why a certification is per-volume
Atomic rename, same-file-store guarantees, and symlink refusal are properties of a specific
filesystem behind a specific mount — not of "Kubernetes" or "a PVC". Change the CSI driver, the
StorageClass, the access mode, the backend, or the mount options and any of them can differ. A
certification that does not name all five is not transferable.
## What is certified
| Property | Why it matters |
|---|---|
| Same file store for staging and content | A rename across stores is a copy, so publication stops being atomic. |
| Atomic rename | The publish path's default strategy. |
| Atomic create (`O_EXCL`) | Makes a publish create-only rather than a silent overwrite. |
| Symlink refusal | Stops a replaced parent from redirecting a write outside the root. |
| Ranged read | The download contract depends on it. |
## Running the certification
```bash
kubectl apply -f infra/fileserver/kubernetes/pvc-certification-job.yaml
kubectl logs job/fileserver-pvc-certification
```
The job writes a machine-readable result to the claim itself, carrying the full tuple:
```json
{
"kubernetesVersion": "...",
"csiDriver": "...",
"storageClass": "...",
"accessMode": "ReadWriteOnce",
"backend": "ext2/ext3",
"mountOptions": "rw,relatime",
"atomicMove": true,
"sameFileStore": true,
"atomicCreate": true
}
```
The job fails closed: a volume whose staging and content areas are on different stores is not
certified, because its publish would silently degrade to a copy.
## Network filesystems
```bash
docker compose -f infra/fileserver/nfs/compose.yml up -d
FILESERVER_NFS_TESTS=true ./gradlew :adapter:outbound:fileserver:test
```
The mount is `hard`, deliberately. A `soft` mount converts a slow server into a short write, which
is exactly the corruption this design refuses to accept.
## Startup enforcement
`FileserverStartupValidator` re-runs the probe at boot and refuses to accept traffic when a required
capability is missing — `ATOMIC_MOVE_REQUIRED` on a filesystem that cannot prove an atomic move
fails closed rather than degrading silently.
## Adding a new store
Extend `ContentStoreContract` and pass it. A prose claim of compatibility is not accepted; the
contract is executable precisely so a future object-storage adapter has to demonstrate the same
offset, digest, and create-only behaviour the local store does.
docs/fileserver/support-matrix.md
+83
View File
@@ -0,0 +1,83 @@
# Fileserver support matrix
A support level here is a claim about evidence, not about intent. Every row names the CI job that
produces that evidence; `FileserverDocumentationCoverageTest` fails the build if a row names a job
that does not exist, so a level can never outlive the test that justified it.
## Levels
| Level | What it means |
|---|---|
| Stable | Certified on every pull request. Contract changes are breaking changes. |
| Beta | Certified nightly. The contract may still change with a deprecation notice. |
| Limited | Certified on the release gate only, under stated constraints. |
| Compatibility | Accepted but not optimized; known caveats are listed inline. |
| Experimental | Off by default, unratified upstream, may change without notice. |
## Runtime profiles
| Profile | Level | CI job |
|---|---|---|
| Local filesystem (ext4) content store | Stable | `fileserver-local-ext4-contract` |
| Spring MVC transport (raw, multipart, batch, download) | Stable | `fileserver-http-contract` |
| Spring WebFlux transport | Experimental | `fileserver-http-contract` |
| Path, filename, range, and problem-detail hardening | Stable | `fileserver-security-suite` |
| Bounded-memory transfer | Stable | `fileserver-bounded-memory` |
| Application and architecture invariants | Stable | `fileserver-unit-and-architecture` |
| Runtime assembly (the capability starts with the flag on) | Stable | `fileserver-unit-and-architecture` |
| tus 1.0 resumable uploads | Stable | `fileserver-http-contract` |
| Crash-recovery matrix | Beta | `fileserver-process-kill-matrix` |
| NFSv4 ambiguity handling | Beta | `fileserver-nfs-ambiguity` |
| Large-file and slow-client performance | Beta | `fileserver-large-file-performance` |
| Multi-instance writer lease | Beta | `fileserver-multi-instance-lease` |
| Kubernetes ReadWriteOnce PVC | Limited | `fileserver-pvc-certification` (manifest checks in CI; cluster run is operator-driven) |
| Nginx `X-Accel-Redirect` delegation | Limited | `fileserver-http-contract` |
| Telemetry sensitive-data suppression | Stable | `fileserver-sensitive-telemetry-scan` |
| Documentation and support-claim coverage | Stable | `fileserver-documentation-gate` |
| Full release verification | Stable | `fileserver-full-verification` |
| HTTP resumable uploads draft-12 | Experimental | `fileserver-http-contract` |
### Why WebFlux is Experimental, not Stable
The reactive router, handlers and readers are now wired: `FileserverReactiveConfiguration`
contributes the scheduler, the handlers and a `RouterFunction` bean under
`@ConditionalOnWebApplication(type = REACTIVE)` plus the platform master switch. Previously nothing
built them at all, so "Stable" described the source tree rather than a running server.
It stays `Experimental` because the shipped composition cannot select it. `adapter:inbound:web`
also puts `DispatcherServlet` on the classpath — deliberately, so adding `spring-webflux` does not
drag a second embedded server onto the runtime — and Boot's application-type deduction therefore
resolves SERVLET. A fork that removes the servlet stack and adds a reactive server gets working
routes without editing any Fileserver code; the shipped template does not exercise that path.
Raising it to Stable requires a contract job that drives the routes over a running reactive server
rather than through direct construction.
## Explicitly not claimed
These have no job, and therefore no claim:
- An automated Kubernetes cluster result. `fileserver-pvc-certification` validates the manifest on
every release and applies it only when a release cluster is configured; without one it warns and
records that nothing was certified. The cluster tuple is produced by an operator and read from
[storage-certification.md](storage-certification.md).
- Kubernetes ReadWriteMany PVC. Concurrent writers across nodes are not certified.
- Windows NTFS as a production storage root. The filename policy strips the characters NTFS
reserves, but no job certifies the publish path there.
- Object storage as a content store. The contract exists (`ContentStoreContract`) but no adapter
implements it yet.
- Server-side malware scanning. The verification pipeline has the port and the verdict precedence;
no scanner is shipped.
## Where the rest is written down
- [configuration.md](configuration.md) — every `app.fileserver-platform.*` key, its default, and the
conditions that fail startup rather than degrade.
- [design-deviations.md](design-deviations.md) — where the implementation departs from the frozen
design, why, and the test that pins each decision.
- [http-contract.md](http-contract.md) — the wire contract.
- [security.md](security.md) — the threat model and what enforces each control.
- [operations.md](operations.md) — runbooks, each starting from a metric.
- [storage-certification.md](storage-certification.md) — how a volume is certified.
- [upgrade-guide.md](upgrade-guide.md) — what changes between versions.
docs/fileserver/upgrade-guide.md
+82
View File
@@ -0,0 +1,82 @@
# Fileserver upgrade guide
## Enabling the capability
The Fileserver ships off. Nothing is registered — no endpoint, no thread pool, no metric — until it
is enabled explicitly.
```yaml
ca-skeleton:
fileserver:
enabled: true
instance-id: ${HOSTNAME}
default-namespace: default
observability:
fingerprint-key: ${FILESERVER_FINGERPRINT_KEY}
```
`instance-id` must be unique per instance: it is the writer-lease owner, and two nodes sharing one
would both believe they hold the same lease.
`fingerprint-key` is required and has no default. Startup fails without it rather than falling back
to an unkeyed digest, which would be reversible for an enumerable identifier space.
## Optional surfaces
Each is a separate switch, and each defaults to off:
```yaml
ca-skeleton:
fileserver:
admin:
enabled: false # management plane; intended for a management port
tus:
enabled: false # tus 1.0 Stable
httpbis-draft12:
enabled: false # Experimental; unratified, may change without notice
nginx:
enabled: false # front-proxy delegation; needs a validated internal location
```
## Database schema
The metadata schema is installed as a capability migration and starts inactive:
```
V1__create_fileserver_metadata.sql → capability_schema_registry: jpa-fileserver-metadata-v1
```
Activate it deliberately. Enabling the capability without an activated schema fails at startup
rather than at the first upload.
## Choosing a publish mode
| Mode | When |
|---|---|
| `atomic-move-preferred` | Default. Uses an atomic rename when the probe proves one, else a metadata pointer. |
| `atomic-move-required` | Fail closed. Refuses to start on storage that cannot prove an atomic move. |
| `metadata-pointer` | For storage without atomic rename; publication is the metadata commit. |
Pick `atomic-move-required` when the storage is certified and you want a misconfiguration to surface
at boot rather than at publish time.
## Behaviour that will surprise you
- **A delete answers `202`, not `204`, when content still exists.** The file is already unreadable;
the physical reclaim is deferred. Treating `202` as a failure will produce spurious retries.
- **A batch upload answers `200` even when parts failed.** The batch is explicitly non-atomic, and a
single status could not report a partial outcome honestly. Read `results[].problem`.
- **An ambiguous failure must not be retried.** Check `"ambiguous": true` in the problem document.
- **`If-Match` takes the strong ETag, not a version number.** A client can only assert about the
representation it was actually served.
- **Inline rendering is refused for scriptable types** even when the caller asks for it.
## Verifying an upgrade
```bash
cd src
./gradlew verifyCleanArchitectureDependencies --console=plain
./gradlew :application-core:check :adapter:inbound:web:check \
:adapter:outbound:fileserver:check --console=plain
./gradlew :app-bootstrap:test --tests '*Fileserver*' --console=plain
```