chore: record pre-existing uncommitted repository state
Snapshot of the in-flight state that already existed, identically, in both this worktree and the main checkout before this session began: the initial HTTP Client platform implementation (previously untracked), the redis-lab removal, and the JPA / object-storage / notification integration work. Kept separate from this session's HTTP Client review response, which lands in the following commit, so the two bodies of work stay reviewable apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
co-authored by
Claude Opus 5
Claude Opus 5
parent
1a3b560678
commit
5f10b791d3
@@ -0,0 +1,49 @@
|
||||
# Command policy
|
||||
|
||||
`src/adapter/outbound/cache-redis/src/main/resources/redis-sdk/redis-command-policy.yml` is the
|
||||
single source of truth for what this SDK is willing to do with each Redis command. Official server
|
||||
metadata decides what a command *is*; this file decides what we allow.
|
||||
|
||||
A command that is not classified there is refused. Adding a command therefore means editing that
|
||||
file, not writing code — and the edit is where the risk decision is made and reviewed.
|
||||
|
||||
## Fields
|
||||
|
||||
| Field | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `risk` | required | `R1` routine, `R2` needs an explicit permit, `R3` administrative, `R4` never allowed |
|
||||
| `support` | required | `TYPED`, `ADVANCED_TYPED`, `RAW_ONLY`, `ADMIN_ONLY`, `VERSION_GATED`, `BLOCKED` |
|
||||
| `minimum-version` | `7.2` | lowest server version that carries the command |
|
||||
| `access` | derived from `support` | which ACL account may issue it |
|
||||
| `blocking` | `false` | occupies its connection until the server replies |
|
||||
| `optional-block` | `false` | the command also has a non-blocking form; only `XREAD` and `XREADGROUP` carry it |
|
||||
| `read-only` | `false` | never mutates the dataset |
|
||||
| `retry-safe` | `read-only` | may be retried after a failure that could have reached the server |
|
||||
| `may-be-ambiguous` | `!read-only` | a failure may leave the outcome unknown |
|
||||
| `timeout-profile` | derived | `FAST`, `COLLECTION`, `ADMIN`, `BLOCKING` |
|
||||
| `key-spec` | `1 1 1` | where the keys are, or `none`, or `movable` |
|
||||
| `required-policy` | – | the permit policy an R2 command demands |
|
||||
|
||||
## Rules the catalog enforces
|
||||
|
||||
- An R2 `ADVANCED_TYPED` command must name the permit policy it requires. There is no R2 command
|
||||
that anyone may issue without an issued permit.
|
||||
- An R4 command must be `BLOCKED`, and an R3 command must be `ADMIN_ONLY`. The type system refuses
|
||||
the other combinations at load time.
|
||||
- A `BLOCKED` command carries no ACL account, so no path in the SDK can reach it.
|
||||
- A blocking command must use the `BLOCKING` timeout profile, and its request must declare a bounded
|
||||
server block — unless it also declares `optional-block`, which only the two stream reads do.
|
||||
- Deprecated command names stay `BLOCKED` even when the SDK offers their behaviour. The typed
|
||||
sorted-set ranges issue `ZRANGE ... BYSCORE|BYLEX|REV`, not `ZRANGEBYSCORE`, so what the guard was
|
||||
told and what reaches the wire are the same command.
|
||||
|
||||
## Where each support level is reachable from
|
||||
|
||||
| Support | Reachable from |
|
||||
| --- | --- |
|
||||
| `TYPED` | the typed operations, no permit |
|
||||
| `ADVANCED_TYPED` | the typed operations, with the named permit |
|
||||
| `VERSION_GATED` | a capability bean that exists only when the probe found the feature |
|
||||
| `RAW_ONLY` | `sdk.raw`, and only with a deployment-registered approval |
|
||||
| `ADMIN_ONLY` | `sdk.admin`, read-only diagnostics only |
|
||||
| `BLOCKED` | nowhere |
|
||||
@@ -0,0 +1,75 @@
|
||||
# Operating the Redis SDK
|
||||
|
||||
## What the metrics can and cannot tell you
|
||||
|
||||
Every observation carries the command family, the deployment mode, and latency. None carries a key,
|
||||
a field, a member, or a value — not because they would be large, but because a metric dimension
|
||||
built from caller data is unbounded cardinality and, for most deployments, tenant identity in a
|
||||
dashboard.
|
||||
|
||||
That means you can answer "which command family is slow" and "which one is failing", and you cannot
|
||||
answer "which key is hot" from metrics. Use the admin plane's `SLOWLOG` projection for the first
|
||||
question and `MEMORY USAGE` on a specific key for the second.
|
||||
|
||||
## The failures worth alerting on
|
||||
|
||||
| Signal | What it means | What to do |
|
||||
| --- | --- | --- |
|
||||
| `RedisCommandRejectedException` | the SDK refused before sending | a caller exceeded a declared bound; the reason names which one |
|
||||
| `RedisCrossSlotException` | a multi-key command spans slots | the keys need a shared hash tag |
|
||||
| `RedisAmbiguousExecutionException` | a write may or may not have applied | reconcile; the SDK will not retry it |
|
||||
| `RedisCapabilityUnavailableException` | the server lacks the feature | a capability bean was constructed by hand, or the probe result changed |
|
||||
| `SentinelFailoverObserver.ambiguousWriteCount` | non-idempotent writes lost to a promotion | each one needs reconciling; the count is the workload |
|
||||
| `ClusterTopologyObserver.reshardingObserved` | `ASK`/`TRYAGAIN` seen | a slot migration is in progress; latency will be uneven until it ends |
|
||||
|
||||
## Things the SDK will never do for you
|
||||
|
||||
- Retry a non-idempotent write after a timeout. `ExecutionCertainty.AMBIGUOUS_FAILURE` is reported,
|
||||
not resolved.
|
||||
- Follow a cross-slot multi-key command by splitting it. It is refused instead.
|
||||
- Read a whole collection, stream, or index. Every read declares a bound.
|
||||
- Load a Lua script or a function library at request time. Both are deployment actions.
|
||||
- Send a command it cannot classify.
|
||||
- Tell you that an acknowledged write was lost. See below — this one is not a limitation you can
|
||||
work around in application code.
|
||||
|
||||
## The write loss the client cannot see
|
||||
|
||||
Set these on every Redis node that can ever be a primary:
|
||||
|
||||
```
|
||||
min-replicas-to-write 1
|
||||
min-replicas-max-lag 1
|
||||
```
|
||||
|
||||
Without them a Sentinel promotion silently destroys acknowledged writes, and this is measured, not
|
||||
theoretical. In `LiveRedisSentinelPromotionTest` on the 7.4 lane, Sentinel promoted the replica and
|
||||
did not demote the old primary for **eleven seconds**. The client stayed connected to a primary that
|
||||
had already been replaced, wrote, and was told `+OK` **2,086 times**. Every one of those writes was
|
||||
discarded when the old primary resynced. Exactly one command failed.
|
||||
|
||||
Nothing on the client can detect this. The server answered, so the driver recorded a success, the
|
||||
SDK recorded `CONFIRMED_SUCCESS`, and the caller was told the write landed. No metric here counts
|
||||
it, `SentinelFailoverObserver` cannot count it, and no retry policy helps — there was no failure to
|
||||
react to. A second run of the same promotion produced sixteen thousand writes, **zero** exceptions,
|
||||
and the same silent loss.
|
||||
|
||||
With the two settings, the identical promotion lost **one** write and refused 2,020 with
|
||||
`NOREPLICAS`, which the SDK reports as a definite, non-ambiguous failure the caller can act on. That
|
||||
is the whole difference: an outage you can see instead of data you cannot.
|
||||
|
||||
The residual window is `min-replicas-max-lag` wide and cannot be closed by configuration alone. A
|
||||
write that must survive a promotion under any circumstances needs `WAIT` after it, at the cost of a
|
||||
round trip to the replica — decide that per write, not globally.
|
||||
|
||||
## Blocking work
|
||||
|
||||
Blocking pops and blocking stream reads run on a dedicated connection lane. If those saturate, the
|
||||
symptom is blocking calls timing out while ordinary traffic is healthy — that is the lane doing its
|
||||
job, not a fault. Size the blocking pool to the number of concurrent consumers, not to request rate.
|
||||
|
||||
## Pub/Sub
|
||||
|
||||
At-most-once. A subscriber that reconnects misses whatever arrived while it was gone, and there is
|
||||
no replay. Durable business events belong in a stream with a consumer group, which is at-least-once
|
||||
and therefore requires idempotent consumers.
|
||||
@@ -0,0 +1,159 @@
|
||||
# Redis SDK support matrix
|
||||
|
||||
This file is a gate, not a summary. `RedisSupportMatrixTest` parses the tables below and fails when
|
||||
the SDK grows a package or a capability that is not listed, so a module cannot ship without someone
|
||||
stating its minimum version, its topology support, and what it does not do.
|
||||
|
||||
Design: `docs/superpowers/specs/2026-08-07-redis-wrapper-typed-api-design.md`.
|
||||
Delivery status and the decisions behind each module: `docs/superpowers/plans/2026-08-07-redis-wrapper-typed-api-status.md`.
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Minimum Redis | Topology | Risk exposure | Sync | Reactive | Known limitations |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| `api` | 7.2 | all | none | n/a | n/a | contract only; no driver types |
|
||||
| `api/key` | 7.2 | all | none | n/a | n/a | slot tags must be low-cardinality |
|
||||
| `api/codec` | 7.2 | all | none | n/a | n/a | no Java native serialization |
|
||||
| `api/command` | 7.2 | all | none | n/a | n/a | permits never widen the ACL account |
|
||||
| `api/error` | 7.2 | all | none | n/a | n/a | failure metadata carries no key or value |
|
||||
| `api/operations` | 7.2 | all | none | n/a | n/a | contract only |
|
||||
| `api/reactive` | 7.2 | all | none | n/a | n/a | Reactor confined to this package |
|
||||
| `lettuce` | 7.2 | all | R1–R2 | yes | yes | pinned to Lettuce 6.8.2 |
|
||||
| `lettuce/codec` | 7.2 | all | none | yes | yes | UTF-8 and byte array codecs only |
|
||||
| `lettuce/command` | 7.2 | all | R1–R2 | yes | yes | policy catalog is the only command authority |
|
||||
| `lettuce/connection` | 7.2 | all | none | yes | yes | five lanes; blocking work never shares the regular lane |
|
||||
| `lettuce/observability` | 7.2 | all | none | yes | yes | command family only, never a key |
|
||||
| `lettuce/operations` | 7.2 | all | R1–R2 | yes | yes | hash field TTL needs 7.4; sharded pub/sub needs 7.0; stream deletion needs 8.2 |
|
||||
| `config` | 7.2 | all | none | n/a | n/a | permit provenance is HMAC-signed per process |
|
||||
| `cluster` | 7.2 | cluster | none | n/a | n/a | slot arithmetic only; no redirect following |
|
||||
| `programmability` | 7.2 | all | R2 | yes | no | transactions never roll back; scripts return one bulk reply; `FUNCTION LOAD` is admin-plane |
|
||||
| `raw` | 7.2 | all | R2 | yes | no | `RAW_ONLY` commands only; movable key specs unapprovable |
|
||||
| `admin` | 7.2 | all | R3 read-only | yes | no | replies are projected; no destructive command exists |
|
||||
| `extensions` | 8.0 | all | none | yes | no | shared command runner; every extension declares its key |
|
||||
| `extensions/json` | 8.0 | all | R1–R2 | yes | no | narrow JSONPath grammar; documents exchanged as text |
|
||||
| `extensions/search` | 8.0 | all | R2 | yes | no | index names namespaced by the SDK; no drop index |
|
||||
| `extensions/timeseries` | 8.0 | all | R1–R2 | yes | no | retention mandatory at creation |
|
||||
| `extensions/probabilistic` | 8.0 | all | R1–R2 | yes | no | every answer is approximate by construction |
|
||||
|
||||
## Capabilities
|
||||
|
||||
| Capability | Minimum Redis | Gate | Bean when absent |
|
||||
| --- | --- | --- | --- |
|
||||
| `SHARDED_PUBSUB` | 7.0 | probe and catalog minimum | none |
|
||||
| `FUNCTIONS` | 7.0 | probe and catalog minimum | none |
|
||||
| `HASH_FIELD_EXPIRATION` | 7.4 | probe and catalog minimum | none |
|
||||
| `HASH_FIELD_EXPIRATION_COMBINED` | 8.0 | probe and catalog minimum | none |
|
||||
| `STREAM_ACKNOWLEDGE_DELETE` | 8.2 | probe and catalog minimum | none |
|
||||
| `STREAM_NEGATIVE_ACKNOWLEDGE` | 8.8 | probe and catalog minimum | none, and no bean exists yet |
|
||||
| `JSON` | 8.0 | probe is authoritative | none |
|
||||
| `SEARCH` | 8.0 | probe is authoritative | none |
|
||||
| `TIME_SERIES` | 8.0 | probe is authoritative | none |
|
||||
| `PROBABILISTIC` | 8.0 | probe is authoritative | none |
|
||||
|
||||
## Certified versions
|
||||
|
||||
A version is certified by its lane producing evidence, not by the version number being newer. An
|
||||
evidence claim here must name the test class that produced it; `RedisSupportMatrixTest` fails the
|
||||
build on a row that claims anything else, so "verified" cannot be written into this table without a
|
||||
test behind it.
|
||||
|
||||
All three lanes have now run on 7.4. The other declared versions are declared, not certified:
|
||||
nothing in this repository has executed against 7.2 or 8.2.
|
||||
|
||||
| Topology | Versions declared | Evidence status |
|
||||
| --- | --- | --- |
|
||||
| Standalone | 7.2, 7.4, 8.2 | `RedisTopologyContractTest`, `LiveRedisGuardrailTest` on 7.4 |
|
||||
| Sentinel | 7.4, 8.2 | `RedisTopologyContractTest`, `LiveRedisSentinelPromotionTest` on 7.4 |
|
||||
| Cluster | 7.4, 8.2 | `RedisTopologyContractTest`, `LiveRedisClusterTest` on 7.4 |
|
||||
|
||||
### What the standalone ACL run established
|
||||
|
||||
`RedisTopologyContractTest` runs the four accounts in `infra/redis-sdk/acl` against a live server and
|
||||
asserts that each `CommandAccess` level grants exactly what the command policy catalog says it may
|
||||
issue. Writing it found five defects that no amount of reading the files would have surfaced:
|
||||
|
||||
1. A Redis ACL file accepts neither comments nor line continuations — the original files did not load
|
||||
at all, and the server refused to start.
|
||||
2. The advanced account granted `SMEMBERS` and `SORT`, both `RAW_ONLY` and therefore the raw gateway
|
||||
account's alone.
|
||||
3. The ordinary account granted `SORT_RO` for the same reason.
|
||||
4. The ordinary account could not run `PUBLISH`, `SUBSCRIBE`, or `PING`, all classified `TYPED`.
|
||||
5. The ordinary account could not run `MULTI`, `EXEC`, `UNWATCH`, or `DISCARD`, also `TYPED`.
|
||||
|
||||
6. The admin account was missing twelve read-only diagnostics the catalog exposes — the `OBJECT`,
|
||||
`PUBSUB`, `XINFO`, `FUNCTION LIST`/`STATS`, and `CLUSTER KEYSLOT` subcommands.
|
||||
|
||||
7. The cursor-scan reply budget was sized to the requested `COUNT`, which Redis treats as a hint —
|
||||
a real `HSCAN COUNT 500` came back with 501 entries and the SDK refused a correct reply.
|
||||
|
||||
Points 2 and 3 are the ones that matter: the account is the last enforcement boundary, so an account
|
||||
wider than the catalog silently removes the second control the design relies on.
|
||||
|
||||
### What the standalone guardrail run established
|
||||
|
||||
`LiveRedisGuardrailTest` wires the real guard, catalog, and typed operations to a live server —
|
||||
the first time `LettuceRedisCommandGateway`, the one class that encodes commands, runs under the
|
||||
SDK's own contracts rather than against the in-memory stand-in. It carries the plan's datasets: a
|
||||
value at the 1 MiB ceiling, a hundred-thousand-field hash, hundred-thousand-member set and sorted
|
||||
set, a twenty-thousand-element list, a stream trimmed to 1,000 while twenty thousand entries are
|
||||
appended, and a five-hundred-command batch.
|
||||
|
||||
The assertions are about limits holding, not throughput. A guardrail test that measured absolute
|
||||
speed would fail on a loaded laptop and teach nobody anything.
|
||||
|
||||
### What the Sentinel promotion run established
|
||||
|
||||
`LiveRedisSentinelPromotionTest` forces one real promotion and asserts several independent claims
|
||||
about it. Every write carries a token unique to the run, so the list on the promoted primary is a
|
||||
verbatim record of what happened and each per-call verdict can be checked against it.
|
||||
|
||||
It found the most serious defect in this delivery, and it is not in the SDK's code:
|
||||
|
||||
> **A superseded primary keeps acknowledging writes.** Sentinel promoted the replica at
|
||||
> `05:56:12.503` and did not demote the old primary until `05:56:23.529` — eleven seconds in which
|
||||
> the client, still connected, wrote and was told `+OK` **2,086 times**. Every one of those writes
|
||||
> was discarded when the old primary resynced from the new one. Exactly **one** command failed. No
|
||||
> client-side signal exists for this: the server answered, so the driver, the SDK, and the caller
|
||||
> all correctly recorded a success.
|
||||
|
||||
`SentinelFailoverObserver` counts *ambiguous* writes, and its documentation used to call those "the
|
||||
ones an operator has to reconcile". That was wrong by three orders of magnitude, and the class now
|
||||
says so.
|
||||
|
||||
What closes the window is on the server, not the client. Re-running the identical promotion with
|
||||
`min-replicas-to-write 1` and `min-replicas-max-lag 1` configured cut acknowledged-and-discarded
|
||||
writes from **2,086 to 1**: the orphaned primary refused 2,020 writes with `NOREPLICAS`, which the
|
||||
SDK translates to a definite, non-ambiguous failure the caller can act on. Both settings are now in
|
||||
the lane, and `acknowledgedWriteLossIsBounded` ties the tolerated loss to the configured lag window,
|
||||
so removing them makes the count jump by an order of magnitude and fails the test.
|
||||
|
||||
That assertion then caught a second version of the same mistake within a day of being written. The
|
||||
first guarded run passed; the second failed with 2,099 lost writes, because the setting had been
|
||||
written into the lane's `primary` service only. The two data nodes swap roles on every failover, so
|
||||
a guardrail applied to whichever one happens to start as primary stops applying the moment the lane
|
||||
does the thing it exists to do. Both nodes now take their whole configuration from one definition.
|
||||
Three consecutive promotions in both directions since: 0, 0, and 1 acknowledged write lost.
|
||||
|
||||
The run also found a translator defect. A promotion closed the channel under an in-flight `RPUSH`
|
||||
and the driver raised a bare `RedisException`, which matched no branch and fell through to a generic
|
||||
failure reported as *definitely did not run*. Nothing about an unrecognised failure supports that
|
||||
claim, and a caller who believes it retries a non-idempotent write. The fallback now treats an
|
||||
unclassified write failure as ambiguous.
|
||||
|
||||
### What the Cluster run established
|
||||
|
||||
`LiveRedisClusterTest` checks the part of `sdk.cluster` that is pure client-side arithmetic against
|
||||
the server that has the last word. The calculator agreed with `CLUSTER KEYSLOT` on every entry of a
|
||||
corpus built from the brace rules a hand-written implementation gets wrong — an empty tag `{}`,
|
||||
`foo{}{bar}`, `foo{{bar}}zap`, an unclosed brace, `}{`, the empty key, and non-ASCII keys — and the
|
||||
rendered-key invariant holds: the slot the SDK computes from a tag alone equals the slot the server
|
||||
computes from the whole rendered key.
|
||||
|
||||
Cross-slot refusal was checked in both directions, because a guard stricter than the cluster costs
|
||||
availability for no reason and a looser one sends requests that cannot succeed. The same key pair
|
||||
the guard refuses is the pair the server answers `CROSSSLOT` for.
|
||||
|
||||
Redirects were observed rather than assumed: a `MOVED` names the slot the client computed, and a
|
||||
slot put into a real `MIGRATING`/`IMPORTING` state answers `ASK` for an absent key and `TRYAGAIN`
|
||||
for a multi-key request that straddles the migration. The lane restores the slot to `STABLE`
|
||||
afterwards, so a run leaves the cluster as it found it.
|
||||
@@ -0,0 +1,62 @@
|
||||
# Redis and client upgrade gate
|
||||
|
||||
Changing the Redis server version or the Lettuce version is not a dependency bump. Both change what
|
||||
commands exist, what they reply, and what an ACL account is allowed to do — all three are things this
|
||||
SDK encodes as fixed decisions. The checks below must pass before either version moves, and each one
|
||||
exists because skipping it produces a specific failure that only shows up in production.
|
||||
|
||||
## 1. Command metadata diff
|
||||
|
||||
Run the catalog drift check against the new server. Every command the server reports must be
|
||||
classified in `src/adapter/outbound/cache-redis/src/main/resources/redis-sdk/redis-command-policy.yml`.
|
||||
|
||||
*Why:* an unclassified command is refused by `CommandPolicyGuard`, so a server that grew a command
|
||||
does not create a hole — but a command whose **risk changed upstream** and is still classified R1
|
||||
here does. The diff is what surfaces that.
|
||||
|
||||
## 2. ACL regression
|
||||
|
||||
Re-run `ACL DRYRUN` for every account against every command the SDK can issue, using
|
||||
`RedisAdminOperations.aclDryRun`.
|
||||
|
||||
*Why:* a permit never widens an ACL account, so the account is the last boundary. A new server
|
||||
version that moved a command into a different ACL category silently turns a working call into a
|
||||
runtime refusal on the first request that needs it.
|
||||
|
||||
## 3. Serializer golden bytes
|
||||
|
||||
Compare the encoded form of every registered codec against the stored golden bytes.
|
||||
|
||||
*Why:* a value written by the old version must still decode after the upgrade. A codec change that
|
||||
looks harmless in a round-trip test is not harmless against data already in the instance.
|
||||
|
||||
## 4. Support matrix
|
||||
|
||||
Update `docs/redis/support-matrix.md`. `RedisSupportMatrixTest` fails when a module or capability is
|
||||
missing, and the certified-version table must not claim a version until its topology lane has
|
||||
actually run.
|
||||
|
||||
## 5. Topology suite
|
||||
|
||||
Run the standalone, Sentinel, and Cluster lanes declared in `infra/redis-sdk/`. A version is
|
||||
certified by the lane passing, not by the version number being newer.
|
||||
|
||||
*Why:* failover certainty and cross-slot behaviour are the two things the in-memory fixture cannot
|
||||
prove. `ExecutionCertainty` and `RedisSlotCalculator` are classification and arithmetic; whether the
|
||||
driver actually behaves that way during a promotion or a resharding is only observable on a real
|
||||
topology.
|
||||
|
||||
## 6. Rollback
|
||||
|
||||
Before the upgrade, record the previous server version, the previous Lettuce version, and the
|
||||
`SCRIPT LOAD` digests of every registered script. A rollback is not complete until the digests
|
||||
resolve again on the restored version.
|
||||
|
||||
*Why:* digests are cached per process and invalidated by `SCRIPT FLUSH` and by restarts. A rollback
|
||||
that leaves a process holding digests the restored server does not know produces `NOSCRIPT` on
|
||||
every scripted call until the cache is dropped.
|
||||
|
||||
## What this gate does not cover
|
||||
|
||||
Data migration. Nothing here moves or reshapes stored values; a change that alters what is stored,
|
||||
rather than how it is addressed, needs its own plan.
|
||||
Reference in New Issue
Block a user