chore: initialize from backend template 0a6dd0e
This commit is contained in:
@@ -0,0 +1,882 @@
|
||||
# Fileserver R2 Control Plane and Provider Selection Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||
> `superpowers:subagent-driven-development` to implement this plan task-by-task. Steps use checkbox
|
||||
> (`- [ ]`) syntax for tracking. Repository policy is `human-only`: do not stage, commit, amend, or
|
||||
> push.
|
||||
|
||||
**Goal:** Add an explicit provider-neutral Fileserver R2 control plane and qualify
|
||||
`local-persistent` as the first provider without making local filesystem the production default.
|
||||
|
||||
**Architecture:** `application-core` keeps the existing `FilePublicationPort` and gains only one
|
||||
provider-neutral achieved-durability value. The fileserver leaf compiles `app.fileserver`
|
||||
destination/provider settings into an exact registry, routes requests through one port bean, and
|
||||
coordinates versioned operation, manifest, and reference records. A strict
|
||||
`local-persistent` provider attests its root before use and advances the durable publication state
|
||||
machine in forced, recoverable steps.
|
||||
|
||||
**Tech Stack:** Java 21, Spring Boot 4 configuration properties/autoconfiguration, JDK NIO/POSIX,
|
||||
JUnit 5, AssertJ, ApplicationContextRunner, Gradle quality gates.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add the provider-neutral achieved durability
|
||||
|
||||
**Files:**
|
||||
- Modify:
|
||||
`src/application-core/src/main/java/dev/caskeleton/application/filepublication/FilePublishReceipt.java`
|
||||
- Modify:
|
||||
`src/application-core/src/test/java/dev/caskeleton/application/filepublication/FilePublicationContractTest.java`
|
||||
|
||||
- [x] **Step 1: Write the failing contract test**
|
||||
|
||||
Add a test that constructs a receipt with the new achieved value and proves no provider or path type
|
||||
is introduced:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void receiptCanReportFileAndDirectorySyncWithoutExposingAProviderType() {
|
||||
FilePublishReceipt receipt =
|
||||
receiptWith(DurabilityGuarantee.FILE_AND_DIRECTORY_SYNC);
|
||||
|
||||
assertThat(receipt.durabilityGuarantee())
|
||||
.isEqualTo(DurabilityGuarantee.FILE_AND_DIRECTORY_SYNC);
|
||||
assertThat(FilePublishReceipt.class.getDeclaredFields())
|
||||
.allSatisfy(field -> assertThat(field.getType().getName())
|
||||
.doesNotContain("java.nio.file", "fileserver", "sftp"));
|
||||
}
|
||||
```
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :application-core:test --tests '*FilePublicationContractTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because `FILE_AND_DIRECTORY_SYNC` does not exist.
|
||||
|
||||
- [x] **Step 3: Implement the minimum contract change**
|
||||
|
||||
Add only this enum member:
|
||||
|
||||
```java
|
||||
public enum DurabilityGuarantee {
|
||||
PROCESS_LOCAL_SYNC,
|
||||
FILE_AND_DIRECTORY_SYNC,
|
||||
PROVIDER_ACK_ONLY
|
||||
}
|
||||
```
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Compile exact destination/provider settings with no local fallback
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverR2Settings.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/CompiledFileDestination.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompiler.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompilerTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing exact-binding tests**
|
||||
|
||||
Cover:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void enabledSettingsRequireAnExplicitDestinationAndProvider() {
|
||||
assertThatThrownBy(() -> FileserverBindingCompiler.compile(enabled(Map.of(), Map.of())))
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessageContaining("destination");
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsUnknownOrUnimplementedProviderTypes() {
|
||||
assertThatThrownBy(() -> compile("shared-mounted"))
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessageContaining("local-persistent");
|
||||
}
|
||||
|
||||
@Test
|
||||
void compilesOnlyAnExactLocalPersistentBinding() {
|
||||
Map<FileDestinationId, CompiledFileDestination> result =
|
||||
FileserverBindingCompiler.compile(validSettings());
|
||||
|
||||
assertThat(result).containsOnlyKeys(new FileDestinationId("local-export"));
|
||||
assertThat(result.get(new FileDestinationId("local-export")).providerId())
|
||||
.isEqualTo("local-primary");
|
||||
}
|
||||
```
|
||||
|
||||
Also reject blank IDs, unknown `provider-ref`, duplicate normalized IDs, non-absolute root, enabled
|
||||
`auto-create`, unsupported publication/durability values, and non-positive row/byte bounds.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverBindingCompilerTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because the settings/compiler do not exist.
|
||||
|
||||
- [x] **Step 3: Implement typed settings**
|
||||
|
||||
Use one public configuration-properties record:
|
||||
|
||||
```java
|
||||
@ConfigurationProperties(prefix = "app.fileserver")
|
||||
public record FileserverR2Settings(
|
||||
boolean enabled,
|
||||
Map<String, DestinationSettings> destinations,
|
||||
Map<String, ProviderSettings> providers) {
|
||||
|
||||
public record DestinationSettings(
|
||||
String providerRef,
|
||||
String requiredPublication,
|
||||
String requiredDurability,
|
||||
long maximumRows,
|
||||
long maximumEncodedBytes) {}
|
||||
|
||||
public record ProviderSettings(
|
||||
String type,
|
||||
String rootDirectory,
|
||||
boolean autoCreate,
|
||||
boolean strictPathSecurity,
|
||||
String expectedFileStoreName,
|
||||
String expectedFileStoreType,
|
||||
String mountSentinelName,
|
||||
String mountSentinelSha256,
|
||||
String expectedOwner,
|
||||
String maximumRootMode) {}
|
||||
}
|
||||
```
|
||||
|
||||
The compiler accepts exactly:
|
||||
|
||||
```text
|
||||
type=local-persistent
|
||||
required-publication=unique-atomic-create
|
||||
required-durability=file-and-directory-sync
|
||||
auto-create=false
|
||||
strict-path-security=true
|
||||
```
|
||||
|
||||
`CompiledFileDestination` contains validated application destination ID, provider ID, absolute
|
||||
root, limits, root attestation inputs, and no Spring type.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Attest a pre-provisioned persistent root
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRootEvidence.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRootAttestor.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRootAttestorTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing attestation tests**
|
||||
|
||||
Create a real POSIX temporary root and sentinel. Test successful evidence and each fail-closed
|
||||
condition:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void attestsOwnerModeStoreSentinelSecureDirectoryAndSyncPrimitives() {
|
||||
CompiledFileDestination destination = destinationFor(attestedRoot());
|
||||
|
||||
LocalPersistentRootEvidence evidence =
|
||||
new LocalPersistentRootAttestor().attest(destination);
|
||||
|
||||
assertThat(evidence.root()).isEqualTo(root.toRealPath());
|
||||
assertThat(evidence.secureDirectoryStream()).isTrue();
|
||||
assertThat(evidence.directorySync()).isTrue();
|
||||
assertThat(evidence.exclusiveHardLink()).isTrue();
|
||||
}
|
||||
```
|
||||
|
||||
Separate tests reject:
|
||||
|
||||
- relative or missing root;
|
||||
- symlink root/ancestor;
|
||||
- owner mismatch;
|
||||
- group/world-writable root;
|
||||
- FileStore name/type mismatch;
|
||||
- missing, symlinked, non-regular, or digest-mismatched sentinel;
|
||||
- staging/data/control on a different FileStore;
|
||||
- unavailable `SecureDirectoryStream`, hard-link, or directory-force probe.
|
||||
|
||||
Probe collaborators may be package-private injectable functions so negative paths do not depend on
|
||||
the host filesystem lacking a feature.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentRootAttestorTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because attestation types do not exist.
|
||||
|
||||
- [x] **Step 3: Implement strict attestation**
|
||||
|
||||
The attestor must:
|
||||
|
||||
```text
|
||||
reject before creating anything when root/sentinel/owner/mode/store mismatch
|
||||
capture root real path, file key, FileStore name/type, sentinel digest
|
||||
create private .ca-fileserver, data, staging, operations, manifests, references, probe directories
|
||||
set newly-created directories to 0700
|
||||
force each created parent directory
|
||||
open a SecureDirectoryStream on root
|
||||
run unique exclusive-create + force + hard-link + directory-force probe
|
||||
delete probe artifacts and force the probe directory
|
||||
return immutable evidence used for pre/post identity checks
|
||||
```
|
||||
|
||||
Do not silently downgrade to R1.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS on the supported Linux/POSIX lane.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add strict reference, journal-v2, manifest, and reference records
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/R2PublishedReferenceCodec.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/DurablePublicationRecord.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/PrivateFileManifest.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/PublishedReferenceRecord.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverControlRecordCodec.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverControlRecordCodecTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing codec tests**
|
||||
|
||||
Test:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void referenceRoundTripRejectsForgeryUnknownRouteAndTruncation() {
|
||||
PublishedFileReference reference = codec.encode("routea1", fixedFileId());
|
||||
|
||||
assertThat(codec.decode(reference, Set.of("routea1")).fileId()).isEqualTo(fixedFileId());
|
||||
assertThatThrownBy(() -> codec.decode(tamper(reference), Set.of("routea1")))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
assertThatThrownBy(() -> codec.decode(reference, Set.of("routeb2")))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
```
|
||||
|
||||
For all three records prove:
|
||||
|
||||
- canonical encode/decode round trip;
|
||||
- maximum encoded length;
|
||||
- exact schema version;
|
||||
- state and revision invariants;
|
||||
- single-segment internal locators;
|
||||
- lowercase SHA-256 fields;
|
||||
- no absolute path, raw row/cell, credential, URI, or control character;
|
||||
- newer schema and duplicate/unknown fields fail closed.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverControlRecordCodecTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because R2 records/codecs do not exist.
|
||||
|
||||
- [x] **Step 3: Implement bounded canonical records**
|
||||
|
||||
Use a strict flat canonical JSON codec owned by this leaf. The record state is:
|
||||
|
||||
```java
|
||||
enum State {
|
||||
WRITING,
|
||||
SEALED,
|
||||
DATA_PUBLISHED,
|
||||
MANIFEST_PUBLISHED,
|
||||
REFERENCE_PUBLISHED,
|
||||
PUBLISHED,
|
||||
QUARANTINED
|
||||
}
|
||||
```
|
||||
|
||||
`R2PublishedReferenceCodec` uses:
|
||||
|
||||
```text
|
||||
fsr1.<route-token>.<32-lower-hex-file-id>.<first-12-hex-of-sha256(prefix)>
|
||||
```
|
||||
|
||||
The check digits detect corruption only and are not authentication.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Persist forced control records and operation locks
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlane.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlaneTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing control-plane tests**
|
||||
|
||||
Test direct lookup and forced revision handling:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void storesAndDirectlyLoadsOperationManifestAndReferenceRecords() {
|
||||
controlPlane.storeOperation(writingRecord());
|
||||
controlPlane.storeManifest(manifest());
|
||||
controlPlane.storeReference(referenceRecord());
|
||||
|
||||
assertThat(controlPlane.findOperation(OPERATION_ID)).contains(writingRecord());
|
||||
assertThat(controlPlane.findManifest(FILE_ID)).contains(manifest());
|
||||
assertThat(controlPlane.findReference(FILE_ID)).contains(referenceRecord());
|
||||
}
|
||||
```
|
||||
|
||||
Also prove:
|
||||
|
||||
- lower/equal incompatible state revision is rejected;
|
||||
- request fingerprint mismatch is conflict;
|
||||
- temp file is force-written before atomic replace;
|
||||
- target parent is forced after replace;
|
||||
- shard creation forces its parent;
|
||||
- symlink shard/record is rejected with `NOFOLLOW_LINKS`;
|
||||
- reads, temporary creation, stat, and delete use attested directory-relative names through
|
||||
`SecureDirectoryStream`; operations without a portable secure hard-link/flagged atomic-replace
|
||||
overload remain limited to the private-owner root and require pre/post identity checks;
|
||||
- same operation is serialized by JVM stripe plus OS `FileLock`;
|
||||
- record corruption is never treated as absent.
|
||||
|
||||
Use a package-private fault-point callback to observe/throw at:
|
||||
|
||||
```text
|
||||
TEMP_FORCED
|
||||
RECORD_REPLACED
|
||||
PARENT_FORCED
|
||||
```
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentControlPlaneTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because the control plane does not exist.
|
||||
|
||||
- [x] **Step 3: Implement durable storage**
|
||||
|
||||
All writes follow:
|
||||
|
||||
```text
|
||||
CREATE_NEW sibling temp
|
||||
write all bytes
|
||||
FileChannel.force(true)
|
||||
ATOMIC_MOVE + REPLACE_EXISTING for the control record only
|
||||
force parent directory
|
||||
read-back and verify identity/revision/digest
|
||||
```
|
||||
|
||||
Payload publication must never use overwrite-capable move. Control record replacement is safe only
|
||||
under the operation lock and monotonically increasing `stateRevision`.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Implement the local-persistent R2 provider and deterministic recovery
|
||||
|
||||
**Files:**
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/CompiledFileDestination.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompiler.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlane.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPublicationJournalCodec.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FilePublicationProvider.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPublicationProvider.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPayloadOperations.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FilePublicationCanonicalDigests.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRecoveryVerifier.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompilerTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlaneTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPayloadOperationsTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPublicationProviderTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPublicationRecoveryTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing publication-order tests**
|
||||
|
||||
First add failing compiler/control-plane assertions for:
|
||||
|
||||
```text
|
||||
deterministic route token = "r" + first 31 lowercase hex of canonical policy digest
|
||||
same startup allowlist route-token collision -> startup failure
|
||||
length-prefixed effective policy/schema/format digest stability
|
||||
same secure operation lookup -> typed canonical v1 or v2
|
||||
v1 is read-only; malformed UTF-8/non-canonical/newer schema is indeterminate, never absent
|
||||
control fault context identifies record kind, identity,
|
||||
applicable operation state/revision, and force boundary
|
||||
```
|
||||
|
||||
Then use a deterministic file ID/clock and a fault recorder. Prove exact order:
|
||||
|
||||
```text
|
||||
J_WRITING
|
||||
STAGE_FORCED
|
||||
J_SEALED
|
||||
DATA_LINKED
|
||||
DATA_DIRECTORY_FORCED
|
||||
J_DATA_PUBLISHED
|
||||
MANIFEST_FORCED
|
||||
J_MANIFEST_PUBLISHED
|
||||
REFERENCE_FORCED
|
||||
J_REFERENCE_PUBLISHED
|
||||
J_PUBLISHED
|
||||
```
|
||||
|
||||
Verify the receipt has an opaque `fsr1` reference,
|
||||
`UNIQUE_ATOMIC_CREATE`, and `FILE_AND_DIRECTORY_SYNC`.
|
||||
|
||||
Also test producer once, streaming bounds, formula mitigation, target collision no overwrite,
|
||||
root-identity change indeterminate, and manifest/reference locator non-disclosure. The stored
|
||||
`internalLocator` is the generated filename only; its data shard is derived from the first two
|
||||
hex characters of `fileId`.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverBindingCompilerTest' \
|
||||
--tests '*LocalPersistentControlPlaneTest' \
|
||||
--tests '*LocalPersistentPublicationProviderTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation/test failure because the compiled identity, typed compatibility lookup,
|
||||
contextual fault seam, payload operations, and provider do not exist.
|
||||
|
||||
- [x] **Step 3: Implement prerequisites and minimal R2 publication**
|
||||
|
||||
Compile one restart-stable destination identity without adding a config key:
|
||||
|
||||
```text
|
||||
effectivePolicyDigest = SHA-256(length-prefixed canonical descriptor fields)
|
||||
routeToken = "r" + first 31 lowercase hex of effectivePolicyDigest
|
||||
```
|
||||
|
||||
The canonical descriptor includes destination/provider IDs, limits, required guarantees, and the
|
||||
format/encoder revision. The schema and format policy use the same length-prefixed digest helper.
|
||||
Reject route-token collisions across the compiled startup allowlist. Keep digest/token derivation
|
||||
on the production SHA-256 path only. Exercise the otherwise impractical collision branch through
|
||||
the same package-private pure route-registry check used by production, using two different test
|
||||
digests whose first 31 hex characters collide; expose no digest/token runtime override.
|
||||
|
||||
Extend `LocalPersistentControlPlane` with one secure relative typed operation lookup. It returns
|
||||
schema-v1 only through strict UTF-8 plus canonical v1 re-encode byte equality and never writes v1;
|
||||
schema-v2 remains the only write format. Enrich its package-private fault callback with record kind,
|
||||
identity, operation state/revision, and force boundary so Task 8 can stop at an exact record force.
|
||||
|
||||
The provider:
|
||||
|
||||
```text
|
||||
validates destination and request before producer invocation
|
||||
acquires operation lock
|
||||
loads operation by direct ID
|
||||
allocates fileId/name before WRITING
|
||||
streams with existing StreamingCsvEncoder
|
||||
forces stage and stores SEALED
|
||||
exclusive hard-links data and forces data directory
|
||||
publishes private manifest
|
||||
publishes reference index
|
||||
stores terminal receipt snapshot
|
||||
returns only after terminal journal parent force/read-back
|
||||
```
|
||||
|
||||
`LocalPersistentPayloadOperations` owns restrictive staging/data shard creation, secure relative
|
||||
stage create/write/force, stable no-follow artifact inspection/digest, exact stage deletion,
|
||||
exclusive no-replace hard-link, standalone recovery-time data-shard directory force, and
|
||||
attested-root-relative R1 artifact inspection. Absolute hard-link/directory-force calls are allowed
|
||||
only inside the attested private-owner boundary with file/root/directory identity checks. An
|
||||
existing matching data artifact discovered from `SEALED` must have its shard directory forced
|
||||
again before the journal may advance; it is never republished through a collision path. A
|
||||
root-level R1 artifact is restored only after bounded SDS-relative no-follow inspection matches the
|
||||
terminal R1 journal.
|
||||
|
||||
Before and after the hard-link commit, compare root real path, file key, FileStore, and sentinel
|
||||
digest to `LocalPersistentRootEvidence`.
|
||||
|
||||
- [x] **Step 4: Write failing recovery matrix tests**
|
||||
|
||||
For every non-terminal state construct matching/missing artifacts and retry with a producer that
|
||||
throws if called. Expected:
|
||||
|
||||
```text
|
||||
SEALED + stage -> resume data publish
|
||||
SEALED + matching data -> resume manifest
|
||||
DATA_PUBLISHED -> resume manifest
|
||||
MANIFEST_PUBLISHED -> resume reference
|
||||
REFERENCE_PUBLISHED -> finish terminal journal
|
||||
PUBLISHED + all matching -> restore exact receipt
|
||||
non-terminal data/manifest/reference mismatch -> QUARANTINED / integrity failure
|
||||
PUBLISHED artifact/metadata/receipt mismatch -> preserve all terminal evidence; integrity / indeterminate
|
||||
required artifact missing -> fail-closed indeterminate / quarantine, never success
|
||||
fingerprint mismatch -> CONFLICT
|
||||
root identity mismatch -> PUBLISH_INDETERMINATE
|
||||
WRITING producer/stage failure -> exact cleanup + unsealed QUARANTINED
|
||||
retry with existing WRITING -> producer is not invoked; indeterminate / quarantine
|
||||
retry of unsealed QUARANTINED -> producer is not invoked
|
||||
```
|
||||
|
||||
`LocalPersistentRecoveryVerifier` must cross-check the operation, incoming request, stable data
|
||||
digest, canonical manifest/reference digests, all locators/counts/timestamps, and guarantees.
|
||||
Because operation schema v2 does not carry a standalone format-policy snapshot, it must require an
|
||||
exact current compiled effective-policy revision/digest match before using the current
|
||||
format-policy digest; it must fail closed instead of guessing across an encoder-policy change.
|
||||
Current configured byte/row limits apply to a new attempt. Recovery inspection is bounded by the
|
||||
already frozen operation byte size (with overflow-safe equality), so a later lower configuration
|
||||
limit does not reinterpret a sealed artifact. If both stage and data exist, their stable file keys
|
||||
must match before exact stage deletion; equal bytes alone are insufficient.
|
||||
Restore a terminal receipt only when it equals the full receipt reconstructed from the verified
|
||||
manifest/reference; checking only operation ID/count/SHA is insufficient. Reuse a verified
|
||||
immutable manifest/reference `publishedAt` after a crash instead of generating a conflicting time.
|
||||
`QUARANTINED` journal transitions are limited to non-terminal operations. A mismatch discovered
|
||||
from `PUBLISHED` must not replace the terminal journal or delete/overwrite data, manifest, or
|
||||
reference records; return typed integrity/indeterminate and preserve all terminal evidence. A
|
||||
separate immutable quarantine incident record is outside this increment.
|
||||
|
||||
- [x] **Step 5: Write failing R1 compatibility tests**
|
||||
|
||||
Pre-provision an existing R1 root so it passes every R2 root attestation condition, then configure
|
||||
that same root as the R2 destination. Place a valid journal schema-v1 terminal record at the shared
|
||||
hashed operation path and a matching root-level R1 artifact.
|
||||
The R2 reader may restore its original `PROCESS_LOCAL_SYNC` receipt, but must not create an R2
|
||||
manifest/reference, change its guarantee, or rewrite the record as schema v2. Newer/corrupt R1
|
||||
records remain indeterminate. Also prove malformed UTF-8 and a decodable but non-canonical v1
|
||||
encoding fail, and that simultaneous R1/R2 bean activation is not required for migration.
|
||||
|
||||
- [x] **Step 6: Verify compatibility RED, then implement read-only compatibility**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentPublicationRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected before implementation: the R1 restoration assertion fails. Reuse the existing schema-v1
|
||||
model/codec behind an added strict UTF-8 and canonical re-encode equality guard, only as a read-only
|
||||
compatibility reader; do not add schema-v1 write paths or an unconfigured second root.
|
||||
|
||||
- [x] **Step 7: Verify recovery RED, then implement recovery**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentPublicationRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected before recovery implementation: failures at each resume assertion. Implement only the
|
||||
matrix and verifier rules above. When producer or staging fails after `J_WRITING`, preserve the
|
||||
original exception, attach cleanup/control failures as suppressed, exact-delete the partial stage,
|
||||
and store unsealed `QUARANTINED` evidence so retry cannot replay the producer. A retry that finds
|
||||
`WRITING` after a process crash also must not invoke the producer. Then rerun. Expected: PASS.
|
||||
|
||||
- [x] **Step 8: Verify provider GREEN**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentPublicationProviderTest' \
|
||||
--tests '*LocalPersistentPublicationRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Add one routing port bean and reject ambiguous R1/R2 activation
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/RoutingFilePublicationAdapter.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverR2Config.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverActivationValidator.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverR2ConfigTest.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportConfig.java`
|
||||
- Rename:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportProperties.java`
|
||||
to
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportSettings.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FilePublicationConfigTest.java`
|
||||
- Modify:
|
||||
`src/app-bootstrap/build.gradle`
|
||||
- Modify:
|
||||
`src/config/architecture/modules.json`
|
||||
- Modify:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/OptionalAdapterBeanGatingTest.java`
|
||||
- Modify:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/DisabledAdapterArchitectureTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing composition/routing tests**
|
||||
|
||||
Prove:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void disabledR2CreatesNoPortOrFilesystemSideEffect() {}
|
||||
|
||||
@Test
|
||||
void enabledR2CreatesExactlyOneRoutingPortForExplicitBindings() {}
|
||||
|
||||
@Test
|
||||
void requestForUnknownDestinationFailsBeforeProducerInvocation() {}
|
||||
|
||||
@Test
|
||||
void enablingLegacyR1AndR2TogetherFailsStartup() {}
|
||||
|
||||
@Test
|
||||
void configuredButUnimplementedSharedOrSftpProviderFailsStartup() {}
|
||||
```
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverR2ConfigTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation/test failure because R2 composition does not exist.
|
||||
|
||||
Execution note: the production composition skeleton had already been introduced before the
|
||||
delegated test task returned, so a standalone RED Gradle run was no longer reproducible without
|
||||
reverting work. The tests still exposed the missing method-level conditional gate through the
|
||||
bootstrap architecture check; that failure was observed and fixed before GREEN.
|
||||
|
||||
- [x] **Step 3: Implement exact routing composition**
|
||||
|
||||
`RoutingFilePublicationAdapter` contains an immutable
|
||||
`Map<FileDestinationId, FilePublicationProvider>` and delegates only after exact lookup.
|
||||
`FileserverR2Config`:
|
||||
|
||||
- is conditional on `app.fileserver.enabled=true`;
|
||||
- enables `FileserverR2Settings`;
|
||||
- compiles and attests every configured binding at startup;
|
||||
- creates one provider instance per provider ID;
|
||||
- creates exactly one `FilePublicationPort`;
|
||||
- rejects `ca-skeleton.fileserver.enabled=true` in the same environment before either R1 root
|
||||
creation or R2 attestation, independently of Spring bean creation order;
|
||||
- rejects different provider IDs that resolve to the same normalized root;
|
||||
- never creates directories/connections when disabled.
|
||||
|
||||
The same package-private activation validator runs first in both R1 bean factories and the R2
|
||||
routing factory; conditional precedence is not an acceptable substitute for an ambiguity failure.
|
||||
Use strict configuration-properties binding (`ignoreUnknownFields = false`). Wire the fileserver
|
||||
leaf into `app-bootstrap` through the architecture registry and Gradle dependency in this task so
|
||||
the runtime composition is real, while keeping all local provider/control types private to the
|
||||
leaf. Rename the legacy configuration-properties type to the repository-required `*Settings`
|
||||
suffix before exposing this leaf to bootstrap naming checks.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Add process-crash qualification, docs, and full gates
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverCrashScenarioMain.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentCrashRecoveryTest.java`
|
||||
- Modify: `src/adapter/outbound/fileserver/README.md`
|
||||
- Modify: `src/adapter/outbound/fileserver/CLAUDE.md`
|
||||
- Modify:
|
||||
`docs/superpowers/specs/2026-07-26-fileserver-production-capability-design.md`
|
||||
- Modify:
|
||||
`docs/superpowers/specs/2026-07-28-fileserver-r2-control-plane-provider-selection-design.md`
|
||||
- Modify:
|
||||
`docs/superpowers/plans/2026-07-28-fileserver-r2-control-plane-provider-selection.md`
|
||||
- Modify: `docs/registries/env-keys.yaml`
|
||||
|
||||
- [x] **Step 1: Write the failing forked-process crash test**
|
||||
|
||||
Launch a new JVM with the test runtime classpath. The helper receives a fault point and calls
|
||||
`Runtime.getRuntime().halt(91)` immediately after that point. Cover:
|
||||
|
||||
```text
|
||||
J_WRITING
|
||||
STAGE_FORCED
|
||||
J_SEALED
|
||||
DATA_LINKED
|
||||
DATA_DIRECTORY_FORCED
|
||||
MANIFEST_FORCED
|
||||
MANIFEST_DIRECTORY_FORCED
|
||||
REFERENCE_FORCED
|
||||
REFERENCE_DIRECTORY_FORCED
|
||||
TERMINAL_JOURNAL_FORCED
|
||||
TERMINAL_JOURNAL_DIRECTORY_FORCED
|
||||
```
|
||||
|
||||
Restart in a second JVM/process and assert exact receipt restoration or a documented typed
|
||||
indeterminate/quarantine outcome, never producer replay or partial final bytes.
|
||||
|
||||
Also run a forked cross-process operation-lock proof using the same attested root and operation ID:
|
||||
process A acquires and reports the OS lock, process B uses a bounded non-blocking/timed attempt and
|
||||
must not enter the critical section while A is alive, then must acquire after A releases or is
|
||||
forcibly terminated. This proof must exercise the OS `FileLock`; the same-JVM stripe test is not a
|
||||
substitute and every wait requires a timeout.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentCrashRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: failure until every fault point is injectable and recoverable.
|
||||
|
||||
Execution note: the contextual control-plane and payload fault seams introduced in Task 6 already
|
||||
covered all eleven boundaries. The first complete forked-process run therefore passed without a
|
||||
new production hook; no implementation was reverted merely to manufacture a RED result.
|
||||
|
||||
- [x] **Step 3: Implement only missing fault hooks/recovery transitions**
|
||||
|
||||
Fault hooks remain package-private test collaborators. No runtime setting or production bean may
|
||||
allow arbitrary process termination.
|
||||
|
||||
- [x] **Step 4: Verify focused and module checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :application-core:check :adapter:outbound:fileserver:check --console=plain
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [x] **Step 5: Update readiness documentation**
|
||||
|
||||
Record:
|
||||
|
||||
- provider-neutral control plane and exact selector implemented;
|
||||
- `local-persistent` is the only qualified R2 provider;
|
||||
- `FILE_AND_DIRECTORY_SYNC` does not claim physical device power-loss protection;
|
||||
- `shared-mounted`, SFTP, reaper/retention/quota/observability remain unimplemented;
|
||||
- R1 compatibility artifacts are never auto-promoted.
|
||||
|
||||
Register the exact local provider environment keys from the design (`ROOT`, expected FileStore
|
||||
name/type, sentinel digest, expected owner) with restart-only policy and conditional
|
||||
`app.fileserver.enabled` validation. Do not add SFTP/NFS keys before those providers exist.
|
||||
|
||||
- [x] **Step 6: Run full repository gates**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew check --console=plain
|
||||
./gradlew \
|
||||
:application-core:verifyDependencyLocks \
|
||||
:adapter:outbound:fileserver:verifyDependencyLocks \
|
||||
:app-bootstrap:verifyDependencyLocks \
|
||||
:sample-portfolio:verifyDependencyLocks \
|
||||
verifyCleanArchitectureDependencies \
|
||||
verifyPublicPathSnapshot \
|
||||
verifyEnvKeys --console=plain
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: all commands PASS.
|
||||
|
||||
- [x] **Step 7: Request final independent review**
|
||||
|
||||
Review against:
|
||||
|
||||
- the R2 design spec;
|
||||
- HARD-STOP rules;
|
||||
- provider fallback/activation ambiguity;
|
||||
- path/symlink/mount identity;
|
||||
- crash ordering and recovery;
|
||||
- receipt guarantee truthfulness;
|
||||
- R1 compatibility and no unrelated adapter dependency.
|
||||
|
||||
Fix every Critical/Important issue and rerun the affected focused test plus full gates.
|
||||
Reference in New Issue
Block a user