79 lines
3.8 KiB
Markdown
79 lines
3.8 KiB
Markdown
# The virtual-thread MVC profile
|
|
|
|
## What it changes, and what it does not
|
|
|
|
Virtual threads remove the cost of a thread waiting. They do not remove the reason the waiting was
|
|
bounded.
|
|
|
|
A platform-thread MVC deployment has an implicit concurrency limit — the thread pool. Nobody wrote
|
|
it down as an admission policy, but it is what has been protecting the database pool, the outbound
|
|
HTTP bulkhead and every downstream service from the full arrival rate.
|
|
|
|
Switching to virtual threads deletes that limit and deletes nothing that depended on it. The result
|
|
is not a slow system. It is a system that accepts ten thousand concurrent requests, queues all of
|
|
them on a twenty-connection pool, and times out every one — having done no useful work. The load
|
|
that used to be shed at the front door is shed at the back, after the cost of accepting it.
|
|
|
|
## Enabling it
|
|
|
|
Two settings, and the profile refuses to be constructed with only one:
|
|
|
|
```yaml
|
|
backend:
|
|
web:
|
|
advanced:
|
|
mvc-virtual-threads:
|
|
enabled: true
|
|
admission-limit: 100 # required
|
|
database-pool-size: 20 # stated, and unchanged
|
|
outbound-bulkhead: 20 # stated, and unchanged
|
|
```
|
|
|
|
The downstream numbers are carried in the profile because the whole point is that they did not grow.
|
|
`VirtualThreadProfile.admissionFitsDownstreamBudgets()` reports when the admission limit exceeds
|
|
them. It does not refuse — a deployment can legitimately admit more than its pool when the work is
|
|
not all database-bound — but it makes the choice a choice.
|
|
|
|
## The limit bounds use cases, not threads
|
|
|
|
`VirtualThreadAdmissionGuard` is a fair semaphore, not a pool. Bounding the threads would put the
|
|
waiting back and throw away what virtual threads bought. Ten thousand virtual threads may exist
|
|
while a hundred hold permits and the rest are refused at the door.
|
|
|
|
A refusal is a **503 with `Retry-After`**, and it is the outcome the limit exists to produce. A
|
|
request refused in a millisecond is strictly better for the client than the same request accepted
|
|
and timed out thirty seconds later behind a full pool.
|
|
|
|
The semaphore is fair on purpose. An unfair one is faster and lets newer arrivals overtake waiting
|
|
ones, which the overtaken client experiences as a random timeout.
|
|
|
|
## What to watch
|
|
|
|
| Signal | Why |
|
|
| --- | --- |
|
|
| `jdk.VirtualThreadPinned` JFR events | A `synchronized` block held across a blocking call pins the carrier thread. The carrier pool is bounded by CPU count, so enough pinned carriers is a deadlock — and a thread dump does not obviously show it. |
|
|
| Carrier pool queue depth | The same problem, from the other side. |
|
|
| Admission rejections | They should rise under load. If they do not, the limit is not being applied. |
|
|
| Downstream wait time | The signal that admission is admitting more than the pools serve. |
|
|
|
|
`VirtualThreadProfile.requiredObservations()` is the same list, in code.
|
|
|
|
## Testing it
|
|
|
|
`VirtualThreadAdmissionGuard.peakActive()` exists so a load test can assert the limit was applied.
|
|
It is invisible from throughput — a load test that only measures throughput passes with the guard
|
|
removed, which is precisely the failure this profile guards against.
|
|
|
|
The test that matters asserts two things together: peak concurrency at or below the limit, **and**
|
|
more threads created than the limit. Without the second, the test would pass on a deployment that
|
|
never used virtual threads at all.
|
|
|
|
## Rolling back
|
|
|
|
Set `enabled: false`. The rollback test asserts that Stable behaviour is then identical — if it is
|
|
not, the feature was never optional and every deployment has it.
|
|
|
|
This is one of only two web Advanced capabilities that affect requests which do not use them (the
|
|
other is the WebFlux blocking bridge), which is why its soak is 24 hours and its rollback test is
|
|
the one that matters most.
|