138 lines
6.5 KiB
Markdown
138 lines
6.5 KiB
Markdown
# Redis Cache Resilience Increment Design
|
|
|
|
**Status:** approved for implementation
|
|
|
|
**Parent:** `2026-07-26-redis-production-capability-design.md` §§15, 16, 18
|
|
|
|
## Goal
|
|
|
|
Complete one coherent production-facing cache increment on top of the current standalone R1 Redis
|
|
runtime:
|
|
|
|
1. a framework-free cache-aside policy in `application-core`;
|
|
2. bounded local single-flight and source bulkhead protection;
|
|
3. deterministic TTL jitter plus soft/hard expiry and stale lookup semantics in the Redis adapter.
|
|
|
|
This increment does not promote Redis beyond standalone cache R1. Distributed refresh leases,
|
|
generation invalidation, rate limiting, owner-safe locks, idempotency, sessions, Sentinel/Cluster,
|
|
TLS/ACL and fault qualification remain later increments.
|
|
|
|
## Architecture boundary
|
|
|
|
- `application-core` owns lookup interpretation, source-result classification, cache-aside
|
|
sequencing, stale-if-error, local coalescing and source admission policy.
|
|
- `adapter:outbound:cache-redis` owns physical TTL, envelope timestamps, deterministic jitter,
|
|
serialization and Redis command outcomes.
|
|
- The application contract contains no Redis/Lettuce/Lua/Spring type.
|
|
- Cache fallback never becomes unlimited source fallback. A miss, provider outage and waiter burst
|
|
all pass through the same bounded source path.
|
|
|
|
## Application contract
|
|
|
|
`CacheSourceLoader<K,V>` returns a typed `SourceLoadOutcome<V>`:
|
|
|
|
- `Loaded(value, sourceRevision)`;
|
|
- `AuthoritativeAbsent(reason, sourceRevision)`;
|
|
- `TransientFailure(SourceFailure)`;
|
|
- `PermanentFailure(SourceFailure)`;
|
|
- `Cancelled`.
|
|
|
|
`SourceFailure` carries a bounded code and the original cause. It never serializes the cause message
|
|
into Redis or metric tags. An unclassified thrown exception is rethrown unchanged and is never
|
|
negative-cached or converted to stale success.
|
|
|
|
`CacheResult<V>` distinguishes:
|
|
|
|
- fresh cache hit;
|
|
- source-loaded value and its cache-record outcome;
|
|
- authoritative absence and its cache-record outcome;
|
|
- stale fallback after a classified transient source failure;
|
|
- source failure;
|
|
- bounded overload/timeout rejection;
|
|
- cancellation.
|
|
|
|
`CacheAsidePolicy` is immutable and constructed once per semantic region. It contains maximum
|
|
in-flight source keys, waiter limit per key, source concurrency, admission wait, load deadline and
|
|
whether transient source failure may serve stale.
|
|
|
|
## Cache-aside state machine
|
|
|
|
1. `Hit(FRESH)` returns immediately.
|
|
2. `NegativeHit` returns immediately.
|
|
3. `Hit(STALE)` retains the value and attempts a bounded refresh.
|
|
4. `Miss`, an `IncompatibleSchema(QUARANTINE_AND_RELOAD)` carrying a usable opaque observation
|
|
token, and `Unavailable` enter the same bounded source path. `FAIL_FAST` schema results and
|
|
unobservable incompatible values are not overwritten.
|
|
5. A local single-flight elects one leader per semantic key. Waiters share the typed source outcome.
|
|
6. The leader must acquire the source bulkhead before calling the loader.
|
|
7. A miss records with `ONLY_IF_ABSENT`. A stale or quarantined observation records with
|
|
`ONLY_IF_OBSERVED`, which atomically compares the digest captured by lookup before replacing the
|
|
value. No lookup-then-delete sequence is used, so a concurrent writer is never deleted.
|
|
8. Only `AuthoritativeAbsent` records a negative entry, using the same absent/observed condition as
|
|
a positive source result.
|
|
9. `TransientFailure` may return the retained stale value when policy allows it.
|
|
10. `PermanentFailure`, unclassified exceptions and cancellation are never hidden by negative cache.
|
|
11. Entries are removed from the flight map after success or failure. In-flight keys and waiters are
|
|
bounded; waiting uses a finite deadline and preserves thread interruption.
|
|
|
|
The loader is synchronous and cancellation is cooperative. Its token exposes deadline/interruption;
|
|
the executor bounds admission and waiter time but cannot safely terminate arbitrary source code.
|
|
|
|
## Redis envelope and TTL policy
|
|
|
|
The positive envelope moves to version 2 and stores:
|
|
|
|
- source revision;
|
|
- `softExpiresAt` epoch milliseconds;
|
|
- `hardExpiresAt` epoch milliseconds;
|
|
- payload and SHA-256 integrity digest.
|
|
|
|
Negative envelopes store only the hard expiry. Lookup behavior is:
|
|
|
|
- `now < softExpiresAt`: `Hit(FRESH)`;
|
|
- `softExpiresAt <= now < hardExpiresAt`: `Hit(STALE)`;
|
|
- `now >= hardExpiresAt`: `Miss(EXPIRED)`;
|
|
- negative `now < hardExpiresAt`: `NegativeHit`;
|
|
- expired negative: `Miss(EXPIRED)`.
|
|
|
|
Version 1 becomes an explicit retired schema result. Future versions and corrupt envelopes fail
|
|
fast. Digest-valid retired/unknown envelopes carry an opaque observation token so an approved
|
|
quarantine reload can compare-and-replace the exact observation. Structurally invalid current
|
|
envelopes remain corrupt/fail-fast even when their digest is valid. Unknown envelopes remain typed
|
|
incompatibility results and are not silently treated as misses. Envelope integrity is checked
|
|
before the version byte is trusted.
|
|
|
|
The policy contains positive soft TTL, positive hard TTL, negative TTL, jitter ratio, minimum hard
|
|
TTL and maximum value bytes. Construction rejects:
|
|
|
|
- non-positive or over-30-day TTLs;
|
|
- soft TTL greater than hard TTL;
|
|
- jitter outside `0.0..0.5`;
|
|
- minimum hard TTL greater than either configured hard TTL.
|
|
- configured hard TTL plus maximum positive jitter greater than 30 days.
|
|
|
|
Jitter is deterministic from the HMAC-derived physical key and the compiled policy revision. It
|
|
uses a symmetric bounded factor. The actual positive soft/hard TTLs use the same factor so ordering
|
|
is preserved. Physical Redis TTL equals the encoded hard expiry duration in the same `SET`.
|
|
Negative TTL is jittered independently and also respects the hard minimum.
|
|
|
|
## Evidence
|
|
|
|
Tests must prove:
|
|
|
|
- fresh/negative hits do not call the source;
|
|
- concurrent same-key misses call the loader once;
|
|
- in-flight-key, waiter, bulkhead and deadline bounds;
|
|
- completion/failure cleanup and exception/interruption behavior;
|
|
- only authoritative absence is negative-cached;
|
|
- stale is served only after a classified transient failure;
|
|
- fresh/stale/expired boundaries with an injected `Clock`;
|
|
- deterministic bounded jitter and hard minimum;
|
|
- version 1/future/corrupt envelope behavior;
|
|
- Redis physical TTL matches the encoded hard expiry.
|
|
- observed replace reads only the trailing digest and never overwrites a concurrent writer;
|
|
- the exact 16MiB opt-in payload is accepted while 16MiB+1 is rejected before dispatch;
|
|
- mutation interruption restores the thread flag and maps to indeterminate certainty.
|
|
|
|
Focused checks run before the repository-wide architecture, dependency, env and public-path gates.
|