79 lines
3.9 KiB
Markdown
79 lines
3.9 KiB
Markdown
# Web streaming: the client contract
|
|
|
|
What a client must implement to consume SSE, NDJSON or `application/json-seq` from this platform.
|
|
|
|
## Three outcomes, not two
|
|
|
|
After the first byte the HTTP status is 200 and cannot change. So the status tells you nothing about
|
|
whether the stream succeeded, and a closed connection is ambiguous. There are three endings:
|
|
|
|
1. **A `Complete` envelope.** The stream finished. `lastSequence` is the final position.
|
|
2. **A `Failure` envelope.** Something failed after commit. Carries a `ProblemCode` and a
|
|
client-safe message. The status is still 200.
|
|
3. **Neither.** The connection closed mid-stream. The server records this as `ABRUPT_CLOSE`; from
|
|
the client's side it is indistinguishable from a completed stream *unless the client is looking
|
|
for the terminal envelope*.
|
|
|
|
**A client that treats a closed connection as completion will silently truncate data.** No server
|
|
change can fix that for it. Looking for the terminal envelope is the contract.
|
|
|
|
## Positions
|
|
|
|
Every `Item` carries a `sequence`, counting from 1. Positions are strictly increasing, and the
|
|
server enforces it — a repeat or a regression is a server-side error, not something a client has to
|
|
tolerate.
|
|
|
|
Zero is never a valid position. A stream whose first item claimed 0 and one whose sequence was never
|
|
set would look identical.
|
|
|
|
## Resuming
|
|
|
|
Where a route supports it, send the last position you applied as `Last-Event-ID`.
|
|
|
|
- If the source still holds it, delivery continues from the next position.
|
|
- If it does not, the response is a **resnapshot-required** error, not a partial stream. Resuming
|
|
from the oldest retained position would give you contiguous positions with a hole in the middle,
|
|
and nothing in the data would say so.
|
|
|
|
A resnapshot means re-reading the resource from its normal endpoint and starting a fresh stream.
|
|
Clients that cannot do that cheaply should not use the resume path.
|
|
|
|
## Framing
|
|
|
|
| Format | Media type | Framing | Truncation behaviour |
|
|
| --- | --- | --- | --- |
|
|
| SSE | `text/event-stream` | `data:` lines, blank-line separated | Partial event at the end |
|
|
| NDJSON | `application/x-ndjson` | One JSON value, then `\n` | The truncated line has no newline, and the delimiter is what was lost |
|
|
| JSON-seq | `application/json-seq` | `0x1E`, value, `\n` | The next `0x1E` unambiguously starts the next record |
|
|
|
|
Prefer JSON-seq where truncation matters. Its separator comes *first*, so a parser resynchronises at
|
|
the next record rather than trying to parse the truncation joined to what follows. For a long-lived
|
|
stream, truncation is the normal way it ends.
|
|
|
|
Records never contain a raw newline. The server refuses to write one, because for NDJSON the
|
|
consumer's line split is the only record boundary there is.
|
|
|
|
## Heartbeats and timeouts
|
|
|
|
The server writes a keepalive every `heartbeatInterval`. A client that sees nothing for longer than
|
|
that should assume the connection is dead — a silent connection and a disconnected one are the same
|
|
thing at the socket, and the heartbeat is what separates them.
|
|
|
|
Streams are closed at `idleTimeout` (nothing produced) and at `maxStreamAge` (regardless of
|
|
activity). Both are normal endings, and both send a terminal envelope where the connection permits.
|
|
|
|
## Slow consumers
|
|
|
|
The server buffers at most `maxBufferedItems` for a consumer that is behind. Past that it closes the
|
|
connection rather than growing the buffer. A client that cannot keep up should reconnect with a
|
|
resume cursor, not expect the server to hold its backlog.
|
|
|
|
## Shutdown
|
|
|
|
During a rolling deploy, open streams receive a reconnect request before the node stops accepting.
|
|
Reconnect promptly — the node force-closes anything still open at its drain deadline, and that
|
|
arrives as an abrupt close.
|
|
|
|
Do not reconnect immediately on an abrupt close without backoff and jitter. If a node's streams are
|
|
all cut at once, every client reconnecting at once is what keeps the replacement down.
|