tech-log-backend/docs/superpowers/specs/2026-07-28-redis-cache-resilience-design.md

Redis Cache Resilience Increment Design

Status: approved for implementation

Parent: 2026-07-26-redis-production-capability-design.md §§15, 16, 18

Goal

Complete one coherent production-facing cache increment on top of the current standalone R1 Redis runtime:

1. a framework-free cache-aside policy in application-core;
2. bounded local single-flight and source bulkhead protection;
3. deterministic TTL jitter plus soft/hard expiry and stale lookup semantics in the Redis adapter.

This increment does not promote Redis beyond standalone cache R1. Distributed refresh leases, generation invalidation, rate limiting, owner-safe locks, idempotency, sessions, Sentinel/Cluster, TLS/ACL and fault qualification remain later increments.

Architecture boundary

• application-core owns lookup interpretation, source-result classification, cache-aside sequencing, stale-if-error, local coalescing and source admission policy.
• adapter:outbound:cache-redis owns physical TTL, envelope timestamps, deterministic jitter, serialization and Redis command outcomes.
• The application contract contains no Redis/Lettuce/Lua/Spring type.
• Cache fallback never becomes unlimited source fallback. A miss, provider outage and waiter burst all pass through the same bounded source path.

Application contract

CacheSourceLoader<K,V> returns a typed SourceLoadOutcome<V>:

• Loaded(value, sourceRevision);
• AuthoritativeAbsent(reason, sourceRevision);
• TransientFailure(SourceFailure);
• PermanentFailure(SourceFailure);
• Cancelled.

SourceFailure carries a bounded code and the original cause. It never serializes the cause message into Redis or metric tags. An unclassified thrown exception is rethrown unchanged and is never negative-cached or converted to stale success.

CacheResult<V> distinguishes:

• fresh cache hit;
• source-loaded value and its cache-record outcome;
• authoritative absence and its cache-record outcome;
• stale fallback after a classified transient source failure;
• source failure;
• bounded overload/timeout rejection;
• cancellation.

CacheAsidePolicy is immutable and constructed once per semantic region. It contains maximum in-flight source keys, waiter limit per key, source concurrency, admission wait, load deadline and whether transient source failure may serve stale.

Cache-aside state machine

1. Hit(FRESH) returns immediately.
2. NegativeHit returns immediately.
3. Hit(STALE) retains the value and attempts a bounded refresh.
4. Miss, an IncompatibleSchema(QUARANTINE_AND_RELOAD) carrying a usable opaque observation token, and Unavailable enter the same bounded source path. FAIL_FAST schema results and unobservable incompatible values are not overwritten.
5. A local single-flight elects one leader per semantic key. Waiters share the typed source outcome.
6. The leader must acquire the source bulkhead before calling the loader.
7. A miss records with ONLY_IF_ABSENT. A stale or quarantined observation records with ONLY_IF_OBSERVED, which atomically compares the digest captured by lookup before replacing the value. No lookup-then-delete sequence is used, so a concurrent writer is never deleted.
8. Only AuthoritativeAbsent records a negative entry, using the same absent/observed condition as a positive source result.
9. TransientFailure may return the retained stale value when policy allows it.
10. PermanentFailure, unclassified exceptions and cancellation are never hidden by negative cache.
11. Entries are removed from the flight map after success or failure. In-flight keys and waiters are bounded; waiting uses a finite deadline and preserves thread interruption.

The loader is synchronous and cancellation is cooperative. Its token exposes deadline/interruption; the executor bounds admission and waiter time but cannot safely terminate arbitrary source code.

Redis envelope and TTL policy

The positive envelope moves to version 2 and stores:

• source revision;
• softExpiresAt epoch milliseconds;
• hardExpiresAt epoch milliseconds;
• payload and SHA-256 integrity digest.

Negative envelopes store only the hard expiry. Lookup behavior is:

• now < softExpiresAt: Hit(FRESH);
• softExpiresAt <= now < hardExpiresAt: Hit(STALE);
• now >= hardExpiresAt: Miss(EXPIRED);
• negative now < hardExpiresAt: NegativeHit;
• expired negative: Miss(EXPIRED).

Version 1 becomes an explicit retired schema result. Future versions and corrupt envelopes fail fast. Digest-valid retired/unknown envelopes carry an opaque observation token so an approved quarantine reload can compare-and-replace the exact observation. Structurally invalid current envelopes remain corrupt/fail-fast even when their digest is valid. Unknown envelopes remain typed incompatibility results and are not silently treated as misses. Envelope integrity is checked before the version byte is trusted.

The policy contains positive soft TTL, positive hard TTL, negative TTL, jitter ratio, minimum hard TTL and maximum value bytes. Construction rejects:

• non-positive or over-30-day TTLs;
• soft TTL greater than hard TTL;
• jitter outside 0.0..0.5;
• minimum hard TTL greater than either configured hard TTL.
• configured hard TTL plus maximum positive jitter greater than 30 days.

Jitter is deterministic from the HMAC-derived physical key and the compiled policy revision. It uses a symmetric bounded factor. The actual positive soft/hard TTLs use the same factor so ordering is preserved. Physical Redis TTL equals the encoded hard expiry duration in the same SET. Negative TTL is jittered independently and also respects the hard minimum.

Evidence

Tests must prove:

• fresh/negative hits do not call the source;
• concurrent same-key misses call the loader once;
• in-flight-key, waiter, bulkhead and deadline bounds;
• completion/failure cleanup and exception/interruption behavior;
• only authoritative absence is negative-cached;
• stale is served only after a classified transient failure;
• fresh/stale/expired boundaries with an injected Clock;
• deterministic bounded jitter and hard minimum;
• version 1/future/corrupt envelope behavior;
• Redis physical TTL matches the encoded hard expiry.
• observed replace reads only the trailing digest and never overwrites a concurrent writer;
• the exact 16MiB opt-in payload is accepted while 16MiB+1 is rejected before dispatch;
• mutation interruption restores the thread flag and maps to indeterminate certainty.

Focused checks run before the repository-wide architecture, dependency, env and public-path gates.