feat: grpc 기능 deep 구현

docs/runbooks/grpc-advanced-capabilities.md

@@ -0,0 +1,98 @@
+# Runbook: gRPC advanced capabilities
+
+Scope: the `:grpc-advanced:*` family. Everything here is off by default and stays off until a
+deployment names it. Nothing in this family ships in a runtime composition today.
+
+Flags are `ca-skeleton.grpc.advanced.<capability>.enabled`. `GrpcAdvancedCapability` owns the list of
+capability names; `GrpcAdvancedSupportMatrix` owns their current grades.
+
+---
+
+## A capability refuses to start
+
+`GrpcAdvancedModuleGuard` gives three different refusals, and the remedy differs:
+
+| Message contains | Meaning | Remedy |
+| --- | --- | --- |
+| "its feature flag is not set" | Nobody enabled it | Set the property named in the message |
+| "tracked rather than implemented" | Grade is `WATCH` | Nothing to do here; the capability is not implemented |
+| "uncharacterised failure modes" | Grade is `EXPERIMENTAL` and this is production | Record a production approval, or run it outside production |
+
+The refusal message always names the property key, so the first case is a configuration line rather
+than a support question.
+
+---
+
+## xDS: the control plane went away
+
+**What you are seeing.** The control plane is unreachable and clients are still routing.
+
+**What it means.** `GrpcXdsFailurePolicy` serves the last-known-good snapshot, up to its staleness
+bound.
+
+**What to do.**
+
+1. Check the snapshot's age. `SERVE_LAST_KNOWN_GOOD` is the healthy degraded state.
+2. `STALE_BEYOND_BOUND` means the snapshot is older than the policy allows and is no longer trusted.
+   Beyond that bound, a decommissioned backend would otherwise keep receiving traffic indefinitely.
+3. `NO_SNAPSHOT_YET` on a starting instance means it never reached the control plane. It fails after
+   the initial fetch timeout rather than starting with no routing.
+
+**Do not** add application-level retry policy while xDS is enabled. `GrpcXdsStartupGuard` refuses it,
+because retry defined in two places has a winner that depends on resolution order rather than on a
+decision.
+
+---
+
+## gRPC-Web: a browser call hangs and then fails with no status
+
+**Almost always the proxy.** A gRPC status arrives as a trailer, and a browser cannot read a trailer
+the proxy did not expose. Check that the proxy's CORS `expose_headers` includes `grpc-status` and
+`grpc-message`; `GrpcWebProxyContract.violations` reports exactly this, and the reference
+configuration in `envoy/envoy.yaml` shows it in place.
+
+**If the method is client- or bidirectional-streaming**, it cannot work over gRPC-Web at all — a
+browser has no way to send a stream of messages. `GrpcWebCompatibilityGate` reports such a method
+before it is exposed.
+
+---
+
+## Servlet: a transport setting appears to be ignored
+
+It is ignored. The container owns the socket, so keepalive tuning, maximum connection age and
+flow-control window tuning belong to it. `GrpcServletStartupValidator` refuses those settings at
+startup rather than accepting and dropping them, because a setting that is silently ignored sends the
+investigation somewhere else.
+
+A Servlet run never substitutes for Netty certification.
+
+---
+
+## Hedging: backend load doubled
+
+**Expected, within a bound.** Hedging trades duplicate load for tail latency.
+`GrpcHedgingResult` records both `duplicateBackendCalls` and `cancelledLoserAttempts`; a dashboard
+showing only the latency improvement makes the trade look free.
+
+**What to check.** `GrpcHedgingBudget` caps hedges as a fraction of completed calls. If duplicate
+load is above that fraction, the budget is not being consumed — which means something is issuing
+hedges outside the coordinator.
+
+**Hedging is refused** for anything but a read-only unary method. A hedged mutation runs twice by
+design, and an idempotency key does not help: the second attempt duplicates a success in progress
+rather than retrying a failure.
+
+---
+
+## Promoting a capability
+
+`GrpcAdvancedPromotionGate.evaluate` names every missing item. Promotion to `ADVANCED_STABLE` needs
+compatibility evidence, a security review, fault evidence, performance evidence, an ADR, a runbook, a
+real-environment test and seven days of soak. A Stable default needs thirty.
+
+Promotions are independent: promoting one capability changes no other's grade, and
+`GrpcAdvancedSupportMatrix.apply` refuses a decision made against a different matrix state.
+
+Before citing a suite as evidence, check `GrpcAdvancedInfrastructureTestkit.missingInfrastructure`.
+A suite that ran without the proxy, the container, the control plane or the toolchain it needs passed
+and established nothing.