tech-log-backend/docs/superpowers/specs/2026-07-26-production-capability-platform-design.md

# Production Capability Platform Design

- Date: 2026-07-26
- Status: Proposed for implementation approval
- Scope: architecture and staged implementation design only
- Baseline: Java 21, Spring Boot 4.0.0, Gradle multi-module Clean Architecture template

## 1. Executive decision

This repository should evolve from a collection of integration seams into an **opt-in production
capability platform**.

The target is not to enable Redis, Kafka, MongoDB, sessions, CDC, and every transport in every
application. The target is:

1. a developer selects a capability and provider by typed configuration;
2. the composition root validates the selected topology and guarantees at startup;
3. application code depends only on semantic, framework-free ports;
4. the selected adapter supplies a real client, bounded defaults, health, metrics, failure
   semantics, and reusable contract tests;
5. unused capabilities create no connection, background worker, schema, or implicit runtime
   behavior;
6. advanced strategies remain available without pretending that one strategy is correct for every
   domain.

The default template remains light. Production capability packs are **available by default but
inactive by default**. A capability is not called production-ready merely because a class or client
seam exists.

### Primary decisions

- Keep `domain-core`, `application-core`, and `shared-contract` free of Spring, Redis, Kafka,
  persistence, transport, and observability SDK types.
- Put use-case-owned semantic ports and application policies in `application-core`; put
  skeleton-wide transport/operational contracts in `shared-contract`; put provider selection and
  Spring composition in `app-bootstrap`.
- Initially preserve the registered 19-leaf topology. Expand the existing technology leaves in
  cohesive packages and split a leaf only when inbound/outbound direction or independent lifecycle
  requires it.
- Treat `adapter:outbound:cache-redis` as the first Redis technology capability provider, but never
  reuse cache's fail-open behavior for sessions, idempotency, locks, or strict rate limits.
- Add a future `adapter:inbound:messaging-kafka` leaf before implementing Kafka consumers. A
  consumer is a driving adapter and does not belong in the existing outbound producer leaf.
- Keep an outbox append operation in the same source-of-truth datastore transaction as the business
  write. Make the **dispatch mechanism** selectable: polling or CDC.
- Define end-to-end messaging as at-least-once delivery plus idempotent consumers/inbox. Do not
  advertise generic exactly-once delivery across a database and broker.
- Separate evictable cache data from correctness-sensitive Redis data at the Redis deployment or
  cluster level, not only by key prefix or database number.
- Provide capability-level safe operations and versioned Lua scripts. Do not expose a general
  `RedisTemplate`, Kafka producer, HTTP client, or cloud SDK to use cases.

## 2. Scope and non-goals

This design covers:

- Redis cache, rate limiting, sessions, idempotency, locks, and reusable atomic operations;
- polling and CDC outbox, Kafka producer/consumer, inbox, delivery semantics, and query/read models;
- file server, object storage, HTTP client, notification, JPA, MongoDB, web, GraphQL, gRPC, and
  WebSocket production baselines;
- provider selection, typed settings, health, observability, Gradle ownership, and test strategy;
- a phased path from the current skeleton to an operational baseline.

This design intentionally does not:

- select one infrastructure topology for every future product;
- invent domain-specific throughput or latency numbers;
- claim benchmark improvements without a real workload and environment;
- make every optional dependency active in the default application;
- promise cross-store atomicity, generic exactly-once processing, or strong consistency from a
  Redis lock;
- place example business concepts in production modules;
- create a universal repository, universal query DSL, or a raw infrastructure facade in
  `application-core`.

## 3. Evidence-based current state

The repository already has stronger boundaries than a typical starter, but many adapters stop at an
extension seam.

| Capability | Current evidence | Current operational gap |
| --- | --- | --- |
| Redis cache | `CacheStore` exposes only `get/put(String)` and `RedisClient` is project-supplied | No Redis SDK, TTL, delete, CAS, bulk operations, serialization policy, real health, or integration test |
| Rate limit | `RateLimitAlgorithm` has only `FIXED_WINDOW`; `FixedWindowRateLimiter` is in-process | Not sliding-window as previously assumed; not multi-node; key map has no removal policy |
| Session | `SecurityConfig` fixes JWT, CSRF disabled, and `STATELESS` | No Redis session repository, stateful security profile, rotation, shared logout, or multi-pod contract |
| Idempotency | Framework-free executor/port plus JPA implementation | Port documentation is DB-specific; no owner token; execution lease and replay TTL are conflated; Redis provider absent |
| Lock | Framework-free lock port plus local/JDBC `LockRegistry` providers | Multi-node means JDBC only; no owner token, renewal, lease-lost signal, fencing, or Redis provider |
| Outbox | JPA/PostgreSQL polling relay with claim, retry, FIFO, and metrics | PostgreSQL `SKIP LOCKED` and mutable status row are coupled to polling; no CDC mode or immutable CDC envelope |
| Kafka | `KafkaSender` is a project-supplied seam and the module has no Kafka SDK | No broker acknowledgement, producer security/tuning, consumers, inbox, retries, DLT, schema contract, or rebalance handling |
| HTTP client | Connect/read timeout, a `globalCallTimeout` label, retry, circuit breaker, buffered size bound, and diagnostics exist | The global value neither bounds nor cancels an active call; no explicit bulkhead, per-client pool controls, SSRF policy, TLS/mTLS profile, redirect policy, or OTel-owned propagation |
| Object storage | Actual filesystem and synchronous S3/MinIO adapters exist | Entire object is `byte[]`; unsafe local-oriented defaults; no multipart, presigned operation, checksum contract, encryption, lifecycle, or orphan cleanup |
| File server | CSV is built and written with JDK filesystem APIs | Whole export is buffered; overwrite is non-atomic; no fsync/rename protocol, quota, retention, symlink defense, or shared-filesystem semantics |
| Notification | Route/fan-out/fail-open framework plus Google/Slack client seams | No real provider SDK, durable delivery, template/versioning, preference, dedupe, receipt, fallback, or provider rate control |
| MongoDB | Opt-in Spring Data Mongo configuration | No production document/port contract kit, concern policy, index/migration contract, replica-set transaction test, or change-stream checkpoint |
| GraphQL | Minimal schema/controller and error resolver | No depth/complexity policy, persisted queries, DataLoader baseline, field authorization, production schema checks, or subscription policy |
| gRPC | Netty server lifecycle, health, reflection, and error mapping | No feature proto build convention, TLS/mTLS, auth, deadline enforcement, retry contract, message limits, or stream backpressure baseline |
| WebSocket | In-process domain-event to STOMP simple-broker bridge | Not cluster-safe or durable; no broker relay profile, destination auth, bounded queues, reconnect/resume, or explicit drop policy |
| Observability | Structured logging, Micrometer, OTel bridge, actuator, and selected metrics exist | Coverage is uneven; several registry entries are not wired; manual `traceparent` propagation competes with real instrumentation |
| Bootstrap composition | The default `app-bootstrap` graph omits file server, object storage, MongoDB, GraphQL, gRPC, and WebSocket leaves | Their source presence is not a runtime guarantee; blindly adding all dependencies would also activate unsafe defaults |
| Settings contract | YAML, environment-key registry, typed settings, and conditional beans are not consistently end-to-end aligned | Notification provider keys diverge, and JWT exposes JWKS/clock-skew settings while the implementation uses issuer discovery and a fixed skew |

Important correctness findings:

- `IdempotencyStorePort.complete(scope)` and `discard(scope)` cannot distinguish an expired original
  owner from a new owner. A stale caller can overwrite or delete a reclaimed record.
- The multi-instance startup validator checks bean names, not capability types or guarantees. Its
  test accepts plain `Object` beans, so it can report safety without a working distributed
  implementation.
- The current outbox row is mutated through `PENDING/IN_FLIGHT/PUBLISHED/...`. Debezium's standard
  Outbox Event Router expects the outbox event source to behave as an insert-only queue, so CDC
  cannot be attached to the current mutable design without a schema/behavior split.
- The current authenticated rate-limit key lacks a route or policy dimension, while the
  unauthenticated key includes the route. This makes policy isolation inconsistent.
- Blanket `catch (Exception)` cache fail-open behavior can hide codec or programming defects as
  ordinary cache misses.
- Merely composing every existing leaf is unsafe: GraphQL and WebSocket lack a uniform module
  enable gate, gRPC defaults include plaintext/reflection behavior, and filesystem object storage
  can activate through a missing-property default.
- The HTTP decorator order currently lets the circuit breaker observe a complete retry bundle,
  while its documentation says each attempt is counted. The declared global call timeout also
  does not actively cancel an already-running attempt.

## 4. Alternatives considered

### A. One large infrastructure starter in `app-bootstrap`

This is rejected. It would make classpath presence activate too much auto-configuration, blur
provider ownership, and turn the composition root into an infrastructure implementation module.

### B. A new Gradle leaf for every capability-provider pair

Examples would be `cache-redis`, `lock-redis`, `session-redis`, `idempotency-redis`, and
`rate-limit-redis`. This has the cleanest physical isolation but creates module and connection
configuration duplication immediately. It also changes the exact 19-leaf registry before the
semantic contracts are stable.

This becomes appropriate only when a provider has an independent release cadence, security
boundary, deployment lifecycle, or dependency graph.

### C. Semantic ports plus existing technology leaves

This is the selected first-stage design.

- `application-core` owns semantic ports and framework-free orchestration.
- existing adapter leaves own actual SDKs and provider-specific behavior;
- `app-bootstrap` selects exactly one provider per required capability;
- provider packages use separate failure policies even when they share a client library;
- module splits are made only for direction or lifecycle reasons.

The known compromise is the `cache-redis` module name. Its short-term responsibility becomes Redis
technology capabilities, while a rename to `adapter:outbound:redis` is deferred to a separately
approved module-registry migration.

### D. Extract an external platform BOM/starter repository now

This is deferred. Extraction before the contracts and test kits are proven would freeze immature
APIs and make local architecture verification harder. A later extraction may publish provider
artifacts and a BOM after at least two real consumers validate the contracts.

## 5. Target architecture

```mermaid
flowchart LR
    WEB[Web / GraphQL / gRPC / WebSocket] --> APP[application-core use cases]
    KIN[Future inbound Kafka adapter] --> APP
    APP --> PORTS[Semantic outbound ports]
    PORTS --> JPA[JPA/PostgreSQL providers]
    PORTS --> MONGO[Mongo providers]
    PORTS --> REDIS[Redis capability providers]
    PORTS --> MSG[Messaging producer providers]
    PORTS --> IO[HTTP / notification / storage / files]
    BOOT[app-bootstrap composition root] -. selects and validates .-> WEB
    BOOT -. selects and validates .-> JPA
    BOOT -. selects and validates .-> MONGO
    BOOT -. selects and validates .-> REDIS
    BOOT -. selects and validates .-> MSG
    OBS[Metrics / traces / logs / health] -. decorates runtime edges .-> BOOT
```

### Layer ownership

| Concern | Owner | Must not leak |
| --- | --- | --- |
| Domain invariant, domain event | `domain-core` | Spring, SDK, transport, persistence |
| Application-invoked cache/idempotency/lock/inbox semantic contract | `application-core` | Redis, SQL, Kafka, Servlet, Micrometer |
| Feature-specific query shape | feature application package via `*QueryPort` | JPA entity, web DTO, generic repository |
| Transport-edge rate limit and operational descriptor contracts shared across leaves | `shared-contract` | provider SDK, Servlet, Redis, and business/domain concepts |
| Redis connection, codec, scripts, cache/rate/idempotency/lock provider | `adapter:outbound:cache-redis` initially | use-case and business policy |
| Session repository implementation | Redis provider package; security behavior composed by web/bootstrap | Spring Session types in application/domain |
| JPA transaction, same-store outbox/inbox/idempotency, DB lock | `adapter:outbound:persistence-jpa` | use-case policy |
| Kafka producer | `adapter:outbound:messaging` | consumer handler and use case |
| Kafka consumer | future `adapter:inbound:messaging-kafka` | producer implementation and persistence entity |
| HTTP/security/response mapping | relevant inbound adapter | repository/client SDK |
| Provider selection, exact-one validation, health composition | `app-bootstrap` | business policy |

### New or revised core contracts

The implementation plan should introduce or revise these contracts:

- `CachePort`, `CacheRegion`, `CacheKey`, `CacheLookup`, `CacheWritePolicy`,
  `CacheAsideExecutor`;
- transport-edge `EdgeRateLimitContract`, `RateLimitRequest`, and `RateLimitDecision` in
  `shared-contract`; a use-case-owned `BusinessQuotaPort` belongs in `application-core` only when a
  domain/application policy actually invokes it;
- owner-token-based `IdempotencyStorePort` claim/renew/complete/release;
- existing `DistributedLockPort` documented as an efficiency mutex;
- separate `FencedLockPort`, and later `LeaderElectionPort`, `SemaphorePort`, or `WorkClaimPort`
  only when those semantics are needed;
- `InboxStorePort` and `MessageConsumptionExecutor`;
- richer outbox/integration-message envelope;
- `ReadConsistency` and versioned cursor values used by feature-specific query ports;
- streaming and precondition-oriented object/file contracts.

These are deliberately **not** application ports:

- a raw Redis command API;
- `RedisTemplate`, Lettuce/Jedis connections, or Spring Session repositories;
- a generic Kafka producer/consumer;
- a generic `RestClient`;
- a general cloud SDK facade;
- a generic repository or arbitrary query language.

Session is a transport/security concern. A `SessionControlPort` is added only if an application use
case must revoke a user's sessions, enforce a business-driven concurrent-session rule, or audit
session control.

## 6. Capability readiness model

Each capability has a visible readiness level:

| Level | Meaning | Required evidence |
| --- | --- | --- |
| R0 Contract | Type, seam, or placeholder only | unit tests and architecture boundary |
| R1 Local | Works in one local process or local service | focused integration test and documented limitations |
| R2 Production baseline | Real provider, safe configuration, failure semantics, health, metrics, security, graceful lifecycle | provider contract, real-service integration, concurrency/failure tests, runbook |
| R3 Scale/HA | Cluster/failover/rolling-upgrade behavior is proven | topology tests, compatibility matrix, recovery and capacity runbooks |

The current repository contains a mixture of R0, R1, and partial R2 components. Documentation and
startup diagnostics must not label an R0 seam as an R2 provider.

Every capability receives a "capability card" containing:

- owner module and semantic port;
- provider IDs and readiness level;
- guarantee and explicit non-guarantees;
- default failure mode and allowed overrides;
- required topology and persistence/eviction policy;
- configuration and secrets;
- liveness/readiness impact;
- bounded-cardinality metrics and trace spans;
- fit/non-fit guidance, cost model, and resource bounds;
- common unsafe recipes, the race/failure they create, and the safe replacement;
- focused, integration, and failure-test commands;
- rolling-upgrade and recovery notes;
- runbook links.

This is how the template teaches operational depth without forcing every project to use every
feature. R2 examples are executable contract examples, not copy-only snippets: a provider must
prove its documented concurrency and failure semantics before the card can claim the guarantee.

## 7. Provider selection and configuration

Provider selection becomes explicit per capability. `multi-instance-enabled` remains descriptive
runtime context, not an implicit provider selector.

`provider: disabled | <provider-id>` is the activation SSOT for a selectable capability. A binding
map uses `disabled | <provider-id>` per binding, and a mode capability such as outbox uses
`dispatch-mode: disabled | polling | cdc`. A leaf-level `enabled` flag exists only when there is no
provider or mode axis. If a legacy flag is temporarily retained, any disagreement with the SSOT is
a startup error.

`matchIfMissing=true`, classpath presence, or a local-provider default must never activate a
production capability. Local, in-memory, plaintext, reflection, auto-create, and filesystem
profiles are rejected in a production profile unless a deployment explicitly opts into a
documented exception.

Illustrative target configuration:

```yaml
ca-skeleton:
  capabilities:
    cache:
      bindings:
        worklog-summary: redis
    rate-limit:
      provider: redis
      fallback: local-emergency
    idempotency:
      provider: jdbc
      guarantee: same-store-transactional
    lock:
      provider: jdbc
      guarantee: efficiency
    outbox:
      dispatch-mode: polling
    messaging:
      producer: kafka
      consumer: disabled
    security:
      auth-mode: jwt

  providers:
    redis:
      connections:
        cache:
          endpoint: ${REDIS_CACHE_ENDPOINT}
        coordination:
          endpoint: ${REDIS_COORDINATION_ENDPOINT}
        session:
          endpoint: ${REDIS_SESSION_ENDPOINT}
```

Credentials, keys, and certificates are secret references or environment-provided values, not
literal repository defaults.

### Capability descriptors

Each active provider contributes a typed descriptor with at least:

- `capabilityId`;
- `providerId`;
- readiness level;
- guarantee class;
- failure mode;
- multi-instance support;
- required backing role;
- readiness impact;
- implementation version.

`app-bootstrap` validates descriptors and selected settings. It does not accept arbitrary beans
with a magic name.

Examples of fail-fast topology rules:

- `redis-session` requires the session Redis role, CSRF/cookie settings, and a session repository;
- `idempotency=redis` cannot claim same-store transactional atomicity for a JDBC business write;
- `outbox=cdc` and the polling scheduler cannot both be active;
- `multi-instance=true` rejects an in-process-only required lock or rate-limit provider unless an
  explicit degraded mode is selected;
- a required Kafka producer must support broker acknowledgement and bounded delivery timeout;
- a correctness capability cannot use the evictable cache Redis role;
- no enabled provider may have an unregistered health, metrics, or configuration card.

Configuration contract tests traverse the complete path:

```text
environment-key registry -> application YAML -> typed settings -> validation -> conditional bean
```

They fail on unknown/dead keys, missing registered keys, multiple active providers, inactive
provider settings that unexpectedly create beans, or a selected provider without its SDK and
health indicator.

### Failure modes

Failure policy is capability-specific, not globally "fail open" or "fail closed".

| Capability | Default failure policy |
| --- | --- |
| Optional cache | Fail open to source load; surface degraded result and metric |
| Cache codec/programming error | Fail closed for the operation; evict/quarantine corrupt entry; do not hide as miss |
| Security session store | Fail closed; authentication state must not be invented |
| Keyed mutation idempotency | Fail closed |
| Strict rate limit for abuse/cost boundary | Fail closed or a deliberately bounded local emergency limiter |
| Availability-oriented rate limit | Explicit local emergency fallback, never unlimited silent bypass |
| Efficiency lock | Fail or continue only according to declared use-case policy |
| Fenced correctness lock | Abort protected work on acquisition or lease-loss failure |
| Outbox append | Roll back the business transaction |
| Outbox dispatcher outage | Keep writes accumulating; alert on lag/backlog |
| Best-effort notification/message | Explicit fail open |
| Durable notification/message | Outbox/inbox, retry, and terminal failure path |

## 8. Redis capability design

The Redis summary in this section is expanded and governed by
[Redis Production Capability Deep Design](2026-07-26-redis-production-capability-design.md). That
document fixes the application contracts, role/deployment isolation, key and codec schemas,
versioned Function/Lua catalog, cache strategies, rate-limit algorithms, lease/fencing and
idempotency state machines, session profile, client backpressure, security, observability, and
topology/failure CI in implementation-ready detail.

### 8.1 Runtime role isolation

At minimum, Redis is modeled as three roles:

| Role | Data | Eviction/durability expectation | Typical failure semantics |
| --- | --- | --- | --- |
| `cache` | recomputable values, negative entries, soft locks | bounded memory, an `allkeys-*` policy selected by operations | usually fail open |
| `coordination` | idempotency, locks, strict quotas, fencing counters | `noeviction`, HA, persistence/capacity alarms | usually fail closed |
| `session` | authenticated sessions and indexes | `noeviction`, HA, serializer compatibility, persistence | fail closed |

A Redis database number or key prefix does not isolate `maxmemory`, eviction, failover, or noisy
neighbors. Production settings require separate managed databases, clusters, or instances where
these guarantees differ.

Each role has typed endpoint/topology/TLS/ACL/timeout/pool/topology-refresh settings and its own
health component. Application code never issues `CONFIG SET`; deployment configuration owns
`maxmemory`, eviction, persistence, and replica policy.

### 8.2 Safe operation catalog

The Redis leaf provides reusable, capability-oriented operations:

| Category | Baseline operations |
| --- | --- |
| Cache | get, multi-get, put with bounded TTL, put-if-absent, evict, multi-evict, touch, namespace-generation bump |
| Atomic primitives | set-if-absent-with-TTL, compare-and-delete, compare-and-expire, increment-with-initial-TTL |
| Rate limit | fixed window, sliding counter, token bucket; sliding log and GCRA opt-in |
| Idempotency | atomic claim, renew, complete, release with owner token |
| Lock | acquire, owner-safe renew/release, optional atomic fencing counter |
| Streaming/invalidation | bounded Redis Streams or Pub/Sub helpers only inside messaging/cache adapters |

These are adapter utilities, not use-case APIs. Raw list/set/hash commands remain available to a new
adapter implementation through the client library, but are not promoted as a stable application
contract.

Every packaged script has a `ScriptDescriptor`:

- stable name and semantic version;
- source checksum;
- key count and cluster-slot rule;
- argument/result schema;
- complexity and maximum collection size;
- timeout/failure behavior;
- metrics name;
- compatible Redis versions.

Every public helper also has an operation contract that states atomicity scope, command/script
complexity, worst-case state growth, cluster-slot constraints, clock source, retry safety, and
failure result. Its documentation pairs the common unsafe multi-command recipe with the packaged
atomic replacement, and its provider contract contains a concurrent race test. This makes the
reason for Lua or another primitive visible to a developer instead of presenting a magic helper.

### 8.3 Why Redis single-threading does not remove races

An individual Redis command is serialized, but a client sequence such as:

```text
GET -> decide -> INCR -> EXPIRE
```

is not one command. Other clients can interleave between its steps. The `INCR` may succeed while
`EXPIRE` is skipped after a client failure, or two clients can both make a decision from stale
state.

Lua scripts solve the read/decide/write atomicity problem by running as one atomic server-side
operation. They introduce another risk: Redis blocks other work while a script runs. Therefore:

- scripts are O(1) or strictly bounded;
- no unbounded loop, `KEYS`, large collection scan, network, filesystem, or dynamic code;
- keys are declared through `KEYS[]`;
- multi-key scripts use a common, narrow hash tag such as a resource hash, not a whole-tenant hot
  slot;
- time arithmetic uses integer milliseconds and a documented clock source;
- scripts are classpath resources, not concatenated strings;
- the client uses cached execution (`EVALSHA`) with safe reload on `NOSCRIPT`;
- Redis Functions are an opt-in deployment mode only when function installation/version ownership
  is available;
- slow-script, command-timeout, pool, and server-latency signals are monitored.

### 8.4 Cache contract and strategies

The current `String get/put` API is replaced by an application-owned cache contract that separates:

- `HIT`, `MISS`, `NEGATIVE_HIT`, and `DEGRADED`;
- region and key from backend provider;
- payload schema/version from Redis serialization;
- positive TTL, negative TTL, soft TTL, and hard TTL;
- backend failure from codec/programming failure.

The Redis adapter stores an opaque, versioned payload. A framework-free `CacheCodec<T>` and
`CacheAsideExecutor<T>` keep domain/application values type-safe without allowing the Redis adapter
to serialize arbitrary domain objects by reflection. JDK native serialization is forbidden.

Baseline strategies:

- cache-aside;
- after-commit invalidation, with eviction preferred over blind cache update;
- bounded positive/negative TTL;
- TTL jitter;
- process-local single-flight;
- versioned key namespace;
- payload size limit;
- batched `SCAN` plus `UNLINK` for operator cleanup, never regular `KEYS`;
- low-cardinality hit/miss/error/load metrics.

Opt-in strategies:

- stale-while-revalidate with soft/hard TTL;
- refresh-ahead;
- probabilistic early refresh;
- L1 local plus L2 Redis;
- distributed stampede suppression with double-check and bounded soft lock;
- client-side tracking or Pub/Sub invalidation;
- compression above a configured threshold;
- generation-based mass invalidation.

Write-through cannot imply atomic DB+Redis commit. Write-behind is not an in-memory executor behind
the cache API; it requires a durable outbox/stream consumer and its own retry/DLT semantics.
Pub/Sub invalidation is best effort, so TTL and schema version remain the recovery boundary.

### 8.5 Rate-limit strategy registry

Transport adapters resolve principal, tenant, API key, IP, route, and policy, then invoke the
framework-neutral `EdgeRateLimitContract` from `shared-contract`. The Redis provider implements
that contract and `app-bootstrap` wires the edge; `application-core` is not involved unless a
separate business quota is part of a use case. The contract receives only an opaque hashed subject
and policy:

```text
RateLimitRequest(policyId, subjectHash, cost)
  -> RateLimitDecision(allowed, remaining, retryAfter, resetAt)
```

The stable key dimensions are:

```text
environment + policyId + tenant? + subjectHash
```

Raw PII, token, request body, and full URL are forbidden in Redis keys, logs, traces, and metric
tags.

| Algorithm | Operational characteristic | Availability |
| --- | --- | --- |
| Fixed window | O(1), simple, permits boundary burst | local/R1 and Redis/R2 |
| Sliding-window log | exact but one sorted-set member per event; memory/CPU grows with volume | advanced opt-in |
| Sliding-window counter | approximate sliding window with bounded O(1) state | production option |
| Token bucket | independently controls average rate and burst capacity | recommended Redis default |
| Leaky bucket | smooth output; synchronous HTTP request queueing is not allowed | background/workflow opt-in |
| GCRA | precise smoothing with compact state | advanced opt-in |

Policies are selected by stable `policyId`, not a global algorithm. A policy declares algorithm,
capacity/rate/window, burst, cost, failure mode, and subject dimensions. `Retry-After` comes from
the decision rather than the current fixed one-second value.

When Redis fails, an optional emergency limiter is conservative, process-local, bounded in size and
TTL, and emits a degraded signal. It is not described as globally accurate.

### 8.6 Session profile

Authentication modes are exclusive:

```text
jwt | redis-session
```

JWT remains the default:

- stateless resource server;
- no session repository;
- CSRF may remain disabled for bearer-only APIs;
- Redis failure does not affect authentication.

`redis-session` is opt-in:

- real Spring Session Redis implementation on the separate session role;
- `IF_REQUIRED` session creation;
- CSRF enabled and tested;
- `Secure`, `HttpOnly`, `SameSite`, path/domain, expiry, and session-id rotation settings;
- logout deletes the server session;
- explicit serializer with versioning and allowed types; no JDK serialization;
- rolling-deploy compatibility test;
- multi-pod read/touch/expiry/logout contract;
- fail-closed repository behavior and readiness inclusion;
- optional indexed repository only when principal lookup/concurrent-session control is required.

Login endpoints and identity proofing remain product decisions. The capability pack provides secure
session storage and web-security composition, not a guessed login business flow.

### 8.7 Redis health and telemetry

Required bounded metrics include:

- cache get/load/invalidate outcome and duration by logical region;
- rate-limit decision/fallback by policy and algorithm;
- idempotency claim/replay/conflict/takeover/store operation;
- lock acquire/renew/lease-lost/release/fencing rejection;
- session repository operation/expiry/error;
- Redis command latency, timeout, connection pool saturation, reconnect, and server memory/eviction.

No cache key, user, tenant, session ID, message ID, or lock resource is a metric tag.

## 9. Idempotency design

The current `find -> tryBegin` and scope-only completion API becomes one atomic ownership protocol:

```text
claim(request)
  -> ACQUIRED(ownerToken, leaseUntil)
  -> REPLAY(storedResponse, replayUntil)
  -> IN_PROGRESS(retryAfter)
  -> FINGERPRINT_MISMATCH
```

Follow-up operations:

```text
renew(ownerToken, newLeaseUntil)
complete(ownerToken, response, replayUntil)
release(ownerToken)
```

Rules:

- execution lease and completed-response replay TTL are separate;
- completion and release compare the owner token;
- stale-owner operations return ownership-lost and cannot mutate a new claim;
- fingerprint canonicalization/version is explicit;
- stored response size, encryption/PII, codec version, and allowed replay metadata are bounded;
- mutations fail closed when the idempotency store is unavailable;
- JPA and Redis providers run the same contract suite;
- the sample module must demonstrate a POST genuinely invoking the executor rather than only
  accepting an unused header.

Guarantees are declared:

- `REQUEST_REPLAY`: suppresses concurrent/repeated request execution as far as the store protocol
  can observe;
- `SAME_STORE_TRANSACTIONAL`: the idempotency record and business change commit in the same
  datastore transaction;
- `EXTERNAL_IDEMPOTENCY`: an outbound provider also receives a stable idempotency key.

A Redis claim plus a JDBC business write is not `SAME_STORE_TRANSACTIONAL`. A crash after the
business write and before Redis completion can cause replayed execution. DB constraints,
intrinsically idempotent commands, outbox, compensation, or downstream idempotency keys remain
necessary.

## 10. Lock and coordination design

One mutex interface should not impersonate every coordination primitive.

| Contract | Purpose | Correctness expectation |
| --- | --- | --- |
| `DistributedLockPort` / future `DistributedMutexPort` | reduce duplicate work or contention | efficiency only; DB constraints/invariants remain authoritative |
| `FencedLockPort` | prevent stale holders from writing | protected resource must reject lower fencing tokens |
| `LeaderElectionPort` | select an active coordinator | explicit leadership/lease lifecycle |
| `SemaphorePort` | bound distributed concurrency | permit ownership and expiry |
| `WorkClaimPort` | claim queue/jobs | claim token and visibility timeout |

A Redis lease uses:

- `SET key ownerToken NX PX lease`;
- owner-checked renew and release Lua scripts;
- bounded acquisition retry with jitter;
- maximum hold duration and bounded renewal count;
- lease-lost signal;
- idempotent release;
- optional atomic monotonically increasing fencing counter in the same cluster slot.

Blind `DEL` is forbidden. An unlimited watchdog is forbidden. Fencing is useful only if the
database, object store, or downstream write port stores and rejects stale tokens.

The fencing counter is separate from the expiring lease and never expires or resets. The protected
resource rejects a token lower than its accepted high watermark. Equality is accepted only for the
same owner token and lease epoch; a different owner must present a strictly greater token. If Redis
failover can lose an acknowledged increment, this provider cannot claim an R2 correctness
guarantee: acquisition fails closed/unready until a monotonic epoch above the protected resource's
recorded high watermark is established. A lock plus a best-effort Redis counter is still only an
efficiency mechanism.

Provider guidance:

- local provider: development/single-node only;
- JDBC table/advisory provider: low-rate coordination close to the primary DB;
- Redis provider: low-latency coordination with explicit failover limitations;
- a consensus system may be added later for stronger lease/election requirements;
- Kafka partition ownership is a work-distribution mechanism, not a generic mutex.

Redis/Redlock is not advertised as a strong correctness guarantee. Network partitions, failover,
lease expiry, pauses, and wall-clock behavior require fencing or an authoritative invariant.

## 11. Outbox, CDC, messaging, and inbox design

### 11.1 What can and cannot be provider-neutral

The outbox append must share the source-of-truth transaction:

- a JPA/PostgreSQL business write appends to PostgreSQL;
- a Mongo business write appends to Mongo in the same supported transaction;
- moving the append to Redis or Kafka would lose atomicity unless a real distributed transaction is
  introduced.

The dispatch strategy is selectable:

```text
disabled | polling | cdc
```

### 11.2 Immutable event plus delivery state

Split the current mutable row:

```text
outbox_event
  immutable event envelope

outbox_delivery
  polling-only claim and delivery state
```

`outbox_event` contains:

- event/message ID;
- event type and schema version;
- aggregate type, aggregate ID, and aggregate sequence/version;
- logical destination and partition key;
- content type and payload;
- occurred-at time;
- tenant when active;
- correlation and causation IDs;
- trace context allowlist.

Physical Kafka topic names are adapter configuration. Application event types do not become topic
names by convention.

Polling mode writes both rows in the business transaction and mutates only `outbox_delivery`.
Polling retains current short claim transactions, broker publish outside the DB transaction,
retry/backoff, aggregate ordering, orphan reclaim, and dead-letter behavior, with these additions:

- claim owner token on every state transition;
- aggregate sequence rather than timestamp-only order;
- broker acknowledgement deadline;
- one combined retry budget across broker client and relay;
- operator replay/requeue/skip tooling and audit;
- explicit resolution for a dead event that blocks aggregate ordering.

CDC mode:

- writes only the immutable event row;
- does not create the Java polling scheduler or polling publisher;
- uses an externally deployed Kafka Connect/Debezium connector and Outbox Event Router;
- routes event ID to a header and aggregate/partition key to the broker key;
- owns connector predicate, schema mapping, offset, snapshot, WAL/replication-slot, retention,
  restart, and recovery configuration;
- monitors connector lag, retained WAL, offset progress, serialization errors, and restarts;
- does not reuse polling `PUBLISHED` status or polling lag metrics.

`outbox_event` is time/range partitioned for bounded retention. Cleanup may purge only a closed
partition whose high watermark is proven consumed by the connector checkpoint and whose replay
retention has elapsed. Cleanup never updates event rows, explicitly filters any cleanup
delete/tombstone records, alarms on table/partition growth, and is tested across connector outage,
restart, and snapshot cutover.

Polling and CDC dispatch are mutually exclusive. Switching modes requires a runbook covering write
freeze or dual-read avoidance, backlog drain, connector offset verification, and rollback.

### 11.3 Kafka producer baseline

The messaging leaf gains a real client/provider:

- acknowledgement-aware send result;
- `acks=all` and idempotent producer configuration;
- bounded delivery timeout and retry budget;
- stable key/partition ordering;
- compression/batch limits;
- TLS/SASL and secret references;
- schema serializer and compatibility validation;
- low-cardinality metrics and OTel propagation;
- readiness for required producer paths;
- graceful flush and shutdown.

Kafka producer transactions are used only for Kafka-native workflows where their boundary applies,
such as consume-process-produce with committed offsets. They do not make a database write and Kafka
publish one atomic transaction.

Best-effort `MessagePublisher` remains explicitly named and documented as best effort. Durable
business events use outbox.

### 11.4 Kafka consumer and inbox

Before consumer implementation, add an `adapter:inbound:messaging-kafka` leaf through the registry
migration workflow. Its production baseline includes:

- manual acknowledgement after application success;
- handler/schema/version allowlist;
- bounded concurrency and queues;
- pause/resume backpressure;
- rebalance and `max.poll` handling;
- bounded retry topic or delayed-retry strategy;
- poison/deserialization failure classification;
- DLT plus audited replay tooling;
- trace context restoration;
- graceful drain and shutdown;
- consumer lag/rebalance/retry/DLT metrics.

`InboxStorePort` scope is:

```text
consumerGroup + handlerName + tenant? + messageId
```

For a handler that writes a database, inbox claim/completion and the business write commit in the
same database transaction. A Redis inbox may be a fast prefilter or serve a DB-free handler, but it
cannot claim same-store atomicity for a JDBC or Mongo write.

End-to-end wording is:

```text
at-least-once delivery + idempotent consumer/inbox
```

Redis Streams may be offered later as a smaller-scale messaging provider with consumer-group,
pending-entry, reclaim, trim, and dedupe contracts. It is not treated as a drop-in Kafka clone.

## 12. Query and persistence design

### 12.1 Query progression

Keep feature-specific `*QueryPort` interfaces. Do not add one universal `QueryPort<Q, R>` or generic
repository.

Supported progression:

1. same-store aggregate read;
2. same-store projection via JPQL/JdbcTemplate/Mongo projection;
3. primary/read-replica routing;
4. separate read model populated through Kafka/CDC;
5. purpose-specific search or analytical store.

Common application values:

- opaque, signed, versioned cursor;
- bounded page size;
- allowlisted sort/filter;
- `ReadConsistency` such as `STRONG`, `READ_YOUR_WRITES`, `BOUNDED_STALENESS`, `EVENTUAL`;
- projection checkpoint and lag.

`TransactionPort.inRead()` does not silently mean "use a replica." The query's consistency policy
and request context select primary or replica. Read-after-write flows remain on primary unless a
causal/checkpoint contract proves otherwise.

### 12.2 JPA/PostgreSQL production baseline

Preserve:

- OSIV disabled;
- application-owned transaction port;
- Flyway migrations;
- persistence exception translation;
- current polling outbox, idempotency, and JDBC lock providers as selectable providers.

Add:

- explicit pool sizing, acquisition timeout, leak detection policy, and shutdown;
- statement/query/lock timeout hierarchy within the request deadline;
- batch write and fetch-size settings;
- N+1 detection and representative query-plan tests;
- optimistic version and bounded pessimistic-lock use;
- primary/read-replica routing with explicit consistency;
- migration expand/contract and rollback/roll-forward rules;
- tenant filter/index/unique-constraint rules when tenancy is active;
- slow query and pool saturation metrics;
- same-store inbox implementation;
- provider packages that make PostgreSQL-specific SQL visible and tested.

Database-backed outbox/idempotency/lock remain valid providers. They stop being the only providers.

### 12.3 MongoDB production baseline

The Mongo leaf remains free of example business documents and gains reusable infrastructure:

- typed URI/topology/TLS/credential/timeout/pool settings;
- explicit read preference, read concern, write concern, and transaction options;
- replica-set/sharded-cluster requirement validation for transactions/change streams;
- index manifest, unique/TTL indexes, drift detection, and migration runner;
- schema validation and optimistic versioning guidance;
- bounded query/page/time limits;
- retryable read/write classification;
- change-stream resume token/checkpoint store and oplog-window monitoring;
- same-store Mongo outbox/inbox option;
- real replica-set Testcontainers contract;
- rolling serializer/schema compatibility.

Change streams are resumable only while the required oplog history and compatible pipeline/options
remain available. Pool sizing accounts for long-lived change-stream cursors.

## 13. Remaining outbound capability baselines

### 13.1 HTTP client

The implementation-level authority for this capability is
[HTTP Client Production Capability Deep Design](2026-07-27-httpclient-production-capability-design.md).
This subsection remains the cross-capability baseline; where detail differs, the dedicated design
governs.

Preserve the current connect/read timeout intent, bounded-response intent, retry/circuit-breaker
seams, shutdown guard, and diagnostics as characterization inputs, not as proven guarantees. The
dedicated audit shows that the current `globalCallTimeout` only gates whether another retry may
start; it does not actively bound or cancel DNS, pool wait, connect, TLS, write, response body, or
backoff. It also shows that the documented decorator order differs from the code. If every physical
attempt must affect circuit-breaker state, retry repeats a circuit-breaker-wrapped attempt; the
logical-call deadline and concurrency bulkhead remain outside that loop. If a provider intentionally
measures one logical call instead, that is a different named policy and test suite, not an accidental
wrapper-order side effect.

Add:

- named client registry with per-dependency settings;
- explicit connection pool total/per-route limits, acquisition timeout, idle eviction, DNS policy,
  and graceful close;
- bulkhead and optional outbound rate limit;
- retry only for declared safe/idempotent operations, with exponential jitter and
  `Retry-After` handling;
- a single total deadline covering pool wait, attempts, backoff, and body read, with active
  cancellation of the engine call and response stream when the budget expires;
- redirect disabled by default or host-allowlisted;
- scheme/host/port/CIDR allowlist and DNS rebinding/SSRF defense;
- TLS trust, hostname verification, mTLS, proxy, and certificate rotation;
- request/response header and body-size allowlists;
- upload/download streaming and cancellation;
- OTel instrumentation owns trace propagation. Remove the manual `traceparent` writer when the
  real tracer is active;
- failure injection and pool-exhaustion tests.

Use cases continue to depend on feature-specific anti-corruption ports such as `RepoStatsPort`, not
on `OutboundHttpClient`.

### 13.2 Notification

Split technical routing from business consent/preferences.

Application intent contains:

- channel;
- logical template ID and version;
- locale;
- recipient reference/address;
- typed template parameters;
- delivery mode and idempotency key;
- correlation/tenant context.

Application/domain policy owns consent, preference, and quiet-hour decisions when those are
business rules. The adapter owns:

- real provider clients;
- template rendering/versioning/localization;
- priority/fallback/fan-out routing;
- provider quotas and bounded retry;
- dedupe and provider idempotency key;
- durable mode through outbox/message;
- webhook signature verification and delivery receipts through an inbound adapter;
- bounce/suppression handling;
- PII-safe logs, encrypted queue content, and retention;
- per-provider health and delivery outcome metrics.

Critical notification is never routed through the current unconditional fail-open path. Best-effort
and durable interfaces are explicit.

### 13.3 Object storage

Replace whole-object `byte[]` as the only path with:

- streaming upload/download and range reads;
- metadata/head contract;
- checksum algorithm/value contract and verification;
- conditional create/update/delete using version/ETag preconditions;
- presigned upload/download request with bounded expiry, content type, and size;
- multipart start/upload/complete/abort and orphan cleanup;
- server-side encryption and KMS settings;
- TLS/endpoint/region/credential-chain validation;
- lifecycle/versioning/retention policy checks;
- quarantine/malware-scan hook before publish;
- payload and metadata limits;
- metrics, tracing, and retry classification.

Local filesystem and S3/MinIO pass the same semantic contract where the backend can support it.
Provider-specific optional capabilities are reported explicitly rather than silently emulated.

Production defaults do not point to local MinIO, auto-create buckets, use static credentials, or
return internal filesystem paths to clients.

Database state and object storage cannot share one local transaction. Workflows such as an image
attachment therefore use an explicit staged lifecycle:

```text
stage upload -> verify checksum/scan -> commit attachment metadata -> finalize visibility
```

Failure paths use idempotent compensation plus an orphan reconciler with retention and audit
evidence. A use case must not perform an irreversible object write inside a database transaction
and assume rollback covers both systems.

### 13.4 File server

The authoritative implementation-level design for this capability is
[Fileserver Production Capability Deep Design](2026-07-26-fileserver-production-capability-design.md).
This subsection is only the cross-capability baseline; the dedicated design governs when details
differ.

Replace the current whole-file `StringBuilder` and direct overwrite with:

- streaming row writer/iterator;
- temporary file in the target directory;
- restrictive creation permissions;
- flush/fsync file, atomic rename when supported, and directory fsync where required;
- explicit fallback when the filesystem cannot guarantee atomic move;
- no-follow-link and real-path containment checks;
- overwrite/precondition policy;
- checksum and manifest;
- size/row/disk-space quota;
- retention/reaper and partial-file cleanup;
- filename/extension/content policy;
- spreadsheet-formula injection defense for CSV/tabular exports, with a tested escaping policy;
- optional encryption and malware scan;
- NFS/SFTP-specific locking, visibility, and rename semantics documented as provider capabilities.

An exported file is identified by an opaque receipt. Absolute server paths are not public API
values.

### 13.5 Identifier and support

`adapter:outbound:identifier` continues to implement domain/application identifier ports. It may
offer random UUID and time-ordered ID providers, but ordering, clock rollback, collision, encoding,
and database-index tradeoffs are explicit. Pseudonymization keys support secret rotation and never
become reversible identifiers.

`adapter:outbound:support` remains a small home for provider-neutral outbound decorators and
diagnostic helpers. It does not become a miscellaneous infrastructure module. Fail-open decorators
classify expected dependency failures and do not swallow programming/codec/invariant defects.

## 14. Inbound transport baselines

### 14.1 Web

Preserve current validation, error envelope, authz, pagination/cursor, conditional request, OpenAPI,
request correlation, and safe cache-control foundations. Add:

- exclusive JWT/session authentication profiles;
- Redis-backed transport-edge rate-limit contract and policy registry;
- actual keyed idempotency executor integration;
- trusted-proxy chain validation;
- request/header/body/multipart limits;
- request deadline and cancellation propagation;
- graceful drain;
- stable API version/deprecation policy;
- OpenAPI compatibility gate;
- CSRF/session cookie tests for stateful mode;
- route-level security/rate/idempotency capability declarations.

### 14.2 GraphQL

Production baseline:

- shared authentication/tenant context;
- operation and field authorization;
- parser character/token/rule-depth limits;
- query depth and cost/complexity instrumentation;
- persisted-query allowlist profile;
- DataLoader/batch-loader convention and N+1 contract;
- cursor connection and bounded page policy;
- sanitized error extensions;
- introspection/GraphiQL production policy;
- schema snapshot/breaking-change check;
- query duration/complexity/error metrics;
- subscription transport delegated to an explicitly designed WebSocket/messaging path.

### 14.3 gRPC

Production baseline:

- protobuf generation/versioning convention and compatibility check;
- TLS/mTLS and service/method authorization interceptors;
- required client deadlines and server cancellation propagation;
- request/response and metadata size limits;
- retry policy only for suitable status/method semantics;
- keepalive coordinated with infrastructure;
- unary and streaming backpressure/cancellation;
- standard health status updated during startup/drain/shutdown;
- reflection opt-in outside production;
- graceful shutdown and in-flight drain;
- OTel RPC semantic spans and bounded metrics.

### 14.4 WebSocket

The simple in-memory STOMP broker remains local/R1 only.

Production baseline:

- authenticated handshake and re-auth/session-expiry behavior;
- destination-level subscribe/send authorization;
- trusted origins and payload/frame limits;
- heartbeat and idle timeout;
- bounded inbound/outbound executors, queues, send time, and an explicit disconnect/drop policy;
- sequence/resume contract where message loss matters;
- per-session ordering only when required and measured;
- broker relay or a durable integration-event bridge for multi-node delivery;
- broker availability/readiness and graceful disconnect;
- no direct serialization of arbitrary domain events to public destinations.

Cross-node durable live updates consume an integration/presentation event. The in-process Spring
event bus is not a durable or cluster-wide transport.

## 15. Observability and operational safety

### Signals

- Traces: inbound server, application use case, DB/Redis, messaging producer/consumer, HTTP, object
  storage, notification, and background-worker spans with standard semantic conventions.
- Metrics: request/dependency latency, errors, saturation, backlog/lag, lease loss, retry, DLT,
  cache behavior, and provider lifecycle.
- Logs: stable structured schema correlated with trace/span IDs.
- Audit: a separate durable, access-controlled record for security/business actions; not ordinary
  application logs.

Instrumentation uses one context-propagation owner per transport. Payloads, tokens, Redis keys,
session IDs, raw principals, email addresses, and object names are not added to metrics and are
allowlisted or pseudonymized in logs/traces.

Production tracing includes a configured OTLP exporter and batch span processor; tests use an
in-memory exporter to prove spans and propagation rather than treating a registry entry as emitted
telemetry. Metrics similarly prove recording, tags, and cardinality. Unmatched or templating-failed
HTTP requests use a fixed route label such as `UNKNOWN`, never a raw URI.

### Health

| Probe | Rule |
| --- | --- |
| Liveness | JVM/process ability only; never DB, Redis, Kafka, SMTP, object storage, or HTTP dependencies |
| Readiness | enabled providers marked required for this deployment |
| Component health | every enabled provider, including optional cache and notification |
| Startup | configuration, migration, script/schema compatibility, and required topology validation |

Optional cache failure does not restart or necessarily unready the pod. Session, strict
idempotency, required lock, or required message publisher failure can make the application
unready. The capability descriptor decides; bean name presence does not.

### Capacity and runbooks

Each R2/R3 capability includes capacity inputs rather than fabricated numbers:

- key/message/session/object size;
- operation rate and concurrency;
- retention/TTL;
- retry amplification;
- connection/thread/partition counts;
- replica/failover expectations;
- alert thresholds derived from an actual SLO.

Required runbooks cover backlog, lag, DLT, stale lease, Redis memory/noeviction, session outage,
connector slot/WAL growth, index drift, multipart orphan, disk capacity, certificate expiry, and
provider credential rotation.

## 16. Gradle and dependency design

Rules:

- `domain-core`, `application-core`, and `shared-contract` keep project-only production
  dependencies and no Spring starter/SDK.
- A real provider dependency lives only in its owning adapter leaf.
- `implementation` is the default. `api` is used only when a public contract intentionally exposes
  a third-party type, which these ports generally forbid.
- Spring Boot-managed coordinates use the Boot BOM. Non-Boot SDKs import a provider BOM at module
  scope, following the existing gRPC/AWS pattern.
- Dependency locks and verification metadata change in the same implementation slice as the
  dependency.
- Testcontainers, Toxiproxy, embedded brokers, and schema test tools remain test/integration-test
  dependencies.
- Provider contract kits use Gradle test fixtures or a dedicated test-support source set without
  becoming production dependencies.
- Integration tests receive a separate `integrationTest` task per provider; architecture checks
  remain part of `check`.
- Do not introduce a version catalog solely for this work. The existing BOM/module pin model can be
  retained until dependency ownership itself becomes hard to maintain.
- Tighten registry edges after implementation. Do not add speculative adapter-to-adapter edges.

Expected dependency ownership:

| Dependency family | Owner |
| --- | --- |
| Spring Data Redis/Lettuce and Spring Session Redis | Redis provider leaf |
| Kafka client/Spring Kafka producer | outbound messaging leaf |
| Kafka listener runtime | future inbound messaging leaf |
| Debezium/Kafka Connect | deployment/integration-test assets, not application-core |
| Mongo driver/Spring Data Mongo | Mongo persistence leaf |
| AWS S3 SDK | object-storage leaf |
| Resilience4j/HTTP engine | HTTP-client leaf |
| gRPC/protobuf runtime/build tooling | gRPC leaf |
| OTel/Micrometer exporter/composition | bootstrap and provider instrumentation adapters |

The new Kafka inbound leaf is the only module addition proposed as structurally necessary in this
design. It requires an explicit `modules.json`, settings, Gradle dependency-gate, documentation,
and architecture-test migration rather than bypassing the exact-19 assertion.

## 17. Verification and CI design

### Test layers

| Layer | Purpose |
| --- | --- |
| Pure unit | policy, algorithms, codec/version/key rules, retry/deadline math |
| Port contract | common required-semantics suite plus guarantee/capability-specific provider suites |
| Real-service integration | Redis, PostgreSQL, Mongo replica set, Kafka, MinIO, provider sandbox |
| Concurrency | duplicate claim, token spend, stale release, ordering, session sharing |
| Failure injection | timeout, disconnect, pool exhaustion, restart, failover, network partition |
| Compatibility | serialization, schema, migration, rolling version, Redis/Kafka/Mongo version |
| Architecture | SDK/type/dependency direction and optional-provider gating |
| Operational | health, metrics cardinality, trace propagation, secret/PII absence, graceful shutdown |

Redis tests include standalone and cluster slot behavior, script reload, token mismatch, maxmemory
separation, and multiple client connections. CDC tests run PostgreSQL, Kafka, Kafka Connect/Debezium
end to end. Session tests use two application contexts against one Redis service.

Providers in different guarantee classes are never certified as semantically identical. For
example, local/JDBC/Redis locks and filesystem/S3 storage share only the required contract subset;
fencing, conditional writes, multipart, durability, and failover claims require their own
capability suite.

### CI profiles

- PR gate: unit, architecture, provider contract, and one supported real-service baseline.
- Production-readiness gate: Docker/services are required; absence is a failure, not a silent skip.
- Nightly/weekly matrix: supported datastore/broker versions, cluster/failover, rolling
  serialization, Toxiproxy, and longer concurrency/soak tests.
- Optional provider sandbox tests use explicit credentials and remain separated from deterministic
  local protocol tests.
- Performance tests establish product-specific budgets later. This design requires load-test
  hooks and capacity metrics, not generic benchmark claims.

## 18. Phased implementation roadmap

### Phase 0 — Correctness contracts and truthful capability topology

- replace bean-name multi-instance checks with typed provider descriptors;
- remove or correct registry/settings claims for capabilities that do not exist;
- reconcile notification and JWT keys across the environment registry, YAML, typed settings, and
  actual conditional beans;
- establish uniform explicit module/provider activation and prohibit missing-property activation
  of local, plaintext, reflection, or auto-create defaults;
- publish a truthful default bootstrap capability manifest instead of equating source modules with
  composed runtime features;
- correct and contract-test HTTP retry/circuit-breaker ordering and make the total deadline cancel
  in-flight work;
- revise idempotency around owner token, lease, and replay TTL;
- distinguish efficiency lock, fenced lock, leadership, semaphore, and work claim;
- define capability cards, readiness levels, settings prefix, failure policy, and common contract
  test kit;
- fix the local fixed-window key lifecycle or mark it dev-only with bounded storage;
- keep application-core dependency purity and all architecture gates green.

Acceptance: the template cannot start in a configuration that claims an unavailable or weaker
provider guarantee.

### Phase 1 — Real Redis foundation and cache

- real Spring Data Redis/Lettuce client;
- cache/coordination/session role settings and connections;
- TLS/ACL/timeouts/pool/topology/health;
- versioned codec/key schema;
- cache get/put/evict/bulk/TTL/negative result;
- cache-aside, jitter, single-flight, after-commit invalidation;
- Redis standalone/cluster/failure/observability tests.

Acceptance: cache reaches R2 while correctness Redis roles remain inactive unless selected.

### Phase 2 — Distributed rate limit, idempotency, locks, and sessions

- shared transport-edge rate-limit contract plus fixed/sliding-counter/token-bucket Lua providers;
- policy registry and emergency fallback;
- Redis/JPA idempotency contract implementations;
- JDBC/Redis lock provider selection, owner-safe renew/release, fenced lock;
- JWT/Redis-session exclusive profiles and multi-pod session contract.

Acceptance: each selected provider has explicit guarantees and failure behavior; no correctness data
uses the evictable cache role.

### Phase 3 — Kafka, polling outbox evolution, inbox, and CDC

- acknowledgement-aware real Kafka producer;
- immutable `outbox_event` plus polling `outbox_delivery`;
- claim token and aggregate sequence;
- add inbound Kafka leaf and inbox executor/provider;
- retry/DLT/replay/backpressure/graceful lifecycle;
- Debezium connector/deployment assets and end-to-end CDC profile;
- polling/CDC exclusivity and transition runbook.

Acceptance: both dispatch modes independently satisfy at-least-once delivery and idempotent-consumer
contracts without an exactly-once claim.

### Phase 4 — HTTP, notification, object storage, and file server

- complete HTTP pool/bulkhead/SSRF/TLS/OTel baseline;
- durable notification intent, provider routing, templates, receipts;
- streaming/multipart/presigned/checksum/encryption storage contracts plus staged finalization,
  compensation, and orphan reconciliation;
- atomic streaming file exports, quotas, retention, CSV formula defense, and
  filesystem-provider semantics.

Acceptance: each adapter has a real R2 provider, failure injection, health, metrics, and a capability
card.

### Phase 5 — JPA/Mongo query models and inbound transports

- read consistency and replica routing;
- same-store and separate read-model profiles with checkpoint/lag;
- Mongo concerns/indexes/migrations/transactions/change streams;
- GraphQL complexity/DataLoader/schema gates;
- gRPC TLS/auth/deadline/streaming/proto gates;
- WebSocket broker relay/backpressure/auth/cluster behavior;
- route/operation capability declarations across transports.

Acceptance: query and transport choices are explicit and operationally observable without leaking
transport or persistence types into core.

### Phase 6 — R3 scale and extraction review

- failover, rolling upgrade, version matrix, recovery drills, and capacity runbooks;
- evaluate splitting Redis capability-provider leaves;
- evaluate extracting a platform BOM/starter only after multiple real consumers validate the APIs.

## 19. Completion criteria for the future implementation

The implementation is complete only when:

- every enabled capability has a real provider rather than a project-supplied seam;
- provider selection is exact, typed, and fail-fast;
- the provider guarantee and non-guarantees are visible;
- unused capabilities have no runtime side effects;
- core modules remain framework/SDK-free;
- every provider passes reusable contract plus real-service/failure tests;
- correctness and optimization data stores are separated where eviction/failure semantics differ;
- health, metrics, traces, logs, graceful lifecycle, security, and runbook are present;
- CI has a non-skipping production-readiness path;
- no cross-database exactly-once or strong Redis-lock claim appears in code or documentation.

## 20. Primary references

- [Redis scripting and atomic blocking semantics](https://redis.io/docs/latest/develop/programmability/eval-intro/)
- [Redis rate-limiter use case and algorithm options](https://redis.io/docs/latest/develop/use-cases/rate-limiter/)
- [Redis key eviction](https://redis.io/docs/latest/develop/reference/eviction/)
- [Redis distributed locks and fencing guidance](https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/)
- [Spring Data Redis scripting](https://docs.spring.io/spring-data/redis/reference/redis/scripting.html)
- [Spring Session Redis APIs](https://docs.spring.io/spring-session/reference/api.html)
- [Debezium Outbox Event Router](https://debezium.io/documentation/reference/stable/transformations/outbox-event-router.html)
- [Apache Kafka delivery semantics and transactions](https://kafka.apache.org/42/design/design/)
- [Apache Kafka producer configuration](https://kafka.apache.org/41/configuration/producer-configs/)
- [Resilience4j fault-tolerance primitives](https://resilience4j.readme.io/docs/getting-started)
- [Amazon S3 object-integrity checks](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html)
- [MongoDB read concern](https://www.mongodb.com/docs/manual/reference/read-concern/)
- [MongoDB write concern](https://www.mongodb.com/docs/manual/reference/write-concern/index.html)
- [MongoDB change streams](https://www.mongodb.com/docs/manual/changestreams/)
- [GraphQL Java query limits](https://graphql-java.com/documentation/limits/)
- [gRPC deadlines](https://grpc.io/docs/guides/deadlines/)
- [gRPC retry](https://grpc.io/docs/guides/retry/)
- [Spring WebSocket external broker relay](https://docs.spring.io/spring-framework/reference/6.2/web/websocket/stomp/handle-broker-relay.html)
- [OpenTelemetry signals and semantic conventions](https://opentelemetry.io/docs/concepts/)