3.5 KiB
ADR-WS-002: Resume and cluster state are caches, and are treated as caches
- Status: Accepted
- Date: 2026-08-25
- Scope:
adapter:inbound:websocket—advanced.resume,advanced.cluster,advanced.presence
Context
Advanced Tasks 2–8 add three things that all look like state and are not: a resume token that says where a client got to, a cluster index that says which node holds a session, and a presence summary derived from that index.
Each is a statement about the past. The resume token was minted before the disconnect; the index entry was written by a node that may since have died; presence is a read of the index and inherits everything wrong with it. The failure this ADR exists to prevent is treating any of them as current fact, because each reads as one at the call site.
Decision
Resume is bounded by what the replay store actually holds, not by what the token claims.
ResumeCoordinator consults ReplayAvailability before honouring a position. A token that names a
position the store has evicted produces a resynchronise, not a gap-filled stream. The alternative —
trusting the token — silently delivers a stream with a hole in it, which is worse than an explicit
resynchronise because the client believes it is complete.
Cluster index entries carry an observation time and are checked against it on every read.
ExternalSessionSummary.staleAt exists so that "the index says edge-2" cannot be used without also
answering "as of when". An entry whose node stopped reporting is not evidence that the node holds
the session.
Durable fan-out is deduplicated by stream position, not by message id. At-least-once is the
contract, so redelivery is normal operation: a redeploy, a slow consumer or a broker rebalance all
produce it. FanoutDeduplicator keys on (stream, position) and advances a high-water mark under
compute, so two consumer threads cannot both deliver the same position.
Presence has four states, not two. OFFLINE is a reported fact; STALE is the absence of one.
Collapsing them reports every user as disconnected during a Redis partition, when what happened is
that the index went dark and the connections are fine.
Nothing security-relevant may depend on presence. An attacker who can make a node stop reporting can move the platform's belief about who is present. Presence answers "show a green dot".
Consequences
- A resume that cannot be honoured is visible to the client as a resynchronise. Clients must implement one; there is no mode in which the platform silently pretends.
- Every read of the cluster index needs a clock. This is deliberate friction.
PresenceSummary.classifyrefuses an idle window at or past the stale window, because otherwiseIDLEis unreachable and the caller believes it has a four-state model when it has three.- Fan-out envelopes carry a bounded reference and the catalog-encoded document, never a business object. A rolling deploy has two versions of the code reading the same envelope.
Alternatives considered
- Trust the resume token. Rejected: it makes a gap indistinguishable from a complete stream.
- Deduplicate by message id. Rejected: a broker that redelivers may re-mint ids, and a producer that retries certainly does. Position is the property the ordering actually has.
- A single
onlineboolean. Rejected for the partition case above. - Write presence separately from the session index. Rejected: two sources of truth for "who is connected" drift, and the drift is invisible — both look plausible and nothing reconciles them.