61 lines
3.5 KiB
Markdown
61 lines
3.5 KiB
Markdown
# ADR-WS-002: Resume and cluster state are caches, and are treated as caches
|
||
|
||
- Status: Accepted
|
||
- Date: 2026-08-25
|
||
- Scope: `adapter:inbound:websocket` — `advanced.resume`, `advanced.cluster`, `advanced.presence`
|
||
|
||
## Context
|
||
|
||
Advanced Tasks 2–8 add three things that all look like state and are not: a resume token that says
|
||
where a client got to, a cluster index that says which node holds a session, and a presence summary
|
||
derived from that index.
|
||
|
||
Each is a statement about the past. The resume token was minted before the disconnect; the index
|
||
entry was written by a node that may since have died; presence is a read of the index and inherits
|
||
everything wrong with it. The failure this ADR exists to prevent is treating any of them as current
|
||
fact, because each reads as one at the call site.
|
||
|
||
## Decision
|
||
|
||
**Resume is bounded by what the replay store actually holds, not by what the token claims.**
|
||
`ResumeCoordinator` consults `ReplayAvailability` before honouring a position. A token that names a
|
||
position the store has evicted produces a resynchronise, not a gap-filled stream. The alternative —
|
||
trusting the token — silently delivers a stream with a hole in it, which is worse than an explicit
|
||
resynchronise because the client believes it is complete.
|
||
|
||
**Cluster index entries carry an observation time and are checked against it on every read.**
|
||
`ExternalSessionSummary.staleAt` exists so that "the index says edge-2" cannot be used without also
|
||
answering "as of when". An entry whose node stopped reporting is not evidence that the node holds
|
||
the session.
|
||
|
||
**Durable fan-out is deduplicated by stream position, not by message id.** At-least-once is the
|
||
contract, so redelivery is normal operation: a redeploy, a slow consumer or a broker rebalance all
|
||
produce it. `FanoutDeduplicator` keys on `(stream, position)` and advances a high-water mark under
|
||
`compute`, so two consumer threads cannot both deliver the same position.
|
||
|
||
**Presence has four states, not two.** `OFFLINE` is a reported fact; `STALE` is the absence of one.
|
||
Collapsing them reports every user as disconnected during a Redis partition, when what happened is
|
||
that the index went dark and the connections are fine.
|
||
|
||
**Nothing security-relevant may depend on presence.** An attacker who can make a node stop reporting
|
||
can move the platform's belief about who is present. Presence answers "show a green dot".
|
||
|
||
## Consequences
|
||
|
||
- A resume that cannot be honoured is visible to the client as a resynchronise. Clients must
|
||
implement one; there is no mode in which the platform silently pretends.
|
||
- Every read of the cluster index needs a clock. This is deliberate friction.
|
||
- `PresenceSummary.classify` refuses an idle window at or past the stale window, because otherwise
|
||
`IDLE` is unreachable and the caller believes it has a four-state model when it has three.
|
||
- Fan-out envelopes carry a bounded reference and the catalog-encoded document, never a business
|
||
object. A rolling deploy has two versions of the code reading the same envelope.
|
||
|
||
## Alternatives considered
|
||
|
||
- **Trust the resume token.** Rejected: it makes a gap indistinguishable from a complete stream.
|
||
- **Deduplicate by message id.** Rejected: a broker that redelivers may re-mint ids, and a producer
|
||
that retries certainly does. Position is the property the ordering actually has.
|
||
- **A single `online` boolean.** Rejected for the partition case above.
|
||
- **Write presence separately from the session index.** Rejected: two sources of truth for "who is
|
||
connected" drift, and the drift is invisible — both look plausible and nothing reconciles them.
|