Research · Results · Experiment
One model, many tenants, nothing shared.
Hydra serves many tenants from one copy of a language model. Each tenant owns a lane — private attention and normalization over a frozen shared backbone — and a lane's output is byte-identical whatever its neighbors send, hold or do.
What a Hydra is
One body, many heads.
A Hydra is one language model serving many tenants, each with its own head — a private lane — on a shared body. The body is the model's frozen weights: embeddings, feed-forward layers and output head, loaded once. A lane is everything that makes a tenant's inference its own: private attention (a small low-rank correction to each shared attention projection), private normalization, and the key-value cache of every request that tenant sends. Lanes are small enough that one machine holds many, and their arithmetic is fixed so that nothing another tenant sends, holds or does can change a single bit of a lane's output.
Why build one
Some workloads must keep tenants apart by construction: a law firm's client matters, a hospital's departments, a bank's customers, one user's agent and another's. Today that means a deployment per tenant — a model copy each, mostly idle — or a shared deployment whose caches and batching were never built to keep tenants apart. A Hydra serves isolated tenants from one model's weights, with more throughput than that model serving them one at a time.
How it uses the Schemen Gate
Schemen Gate is the open-source authority layer: AI PKI and enforcement primitives for exact actions, resources, delegation, revocation and gated model computation. Every Hydra lane sits behind it.
- Grant. Authority is signed to a lane and its full transformer head, not to a tenant. A request reaches a lane only through a single-use capability bound to that lane, with the request's body digest, the action, the model and runtime, an expiry and a one-time identifier. Replays are refused.
- Gate. Before a lane opens, the Gate sidecar — a separate process holding key material the runtime never sees — must issue that lane's Cryptographic Dimension Partitioning mask, and the runtime checks it is the right lane's binary mask. The sidecar also verifies each lane's signed artifacts when they are admitted.
- Lane. Only then does the runtime open the lane's route. Every forward re-checks each row's lease and each lane's weight custody, and every result carries a custody receipt of what actually ran.
- Fail closed. A missing, replayed, expired or misbound capability is refused before any private lane is touched; if the sidecar is lost, generation stops.
How it relates to CDP
Cryptographic Dimension Partitioning gives each tenant an exact, disjoint share of a model's feed-forward coordinates. Attention cannot be split that way — every query meets every key — so CDP leaves attention shared and names what is missing: a duplicated attention lane per tenant. Hydra is that lane.
Headline
Isolation stopped costing throughput.
A model that must keep tenants apart — client matters, patients, customers — is usually deployed once per tenant, or shared on the hope that nothing leaks. Shared-cache serving engines are not an option for such workloads: shared prefix and key-value caches leak across tenants by timing, and batch-dependent arithmetic lets one request change another's output. Hydra gives each tenant a private lane, authorizes every request with a signed single-use capability, and fixes the shape of every kernel that touches a tenant's row by that tenant's own history. On one Apple M5 Max, three lanes on Qwen3-8B serve 1.85× the plain model's throughput at row capacity 3 and 2.58× at 16. Gemma 3 4B and 12B, added in this work with sliding-window attention, reach 2.24× and 2.43×. Every lane matches its reference exactly.
The result
Read the report, or the paper.
The written findings
Throughput, footprint and latency on three models, the exactness evidence, the authority roster, the timing channel, and the four defects the evaluation found in itself.
Visual · memoryWhat a tenant costs
Weight memory for sixteen tenants as model copies, dense lanes and delta lanes; and the serving footprint of each configuration.
Visual · throughputThree lanes, one model
Aggregate throughput with confidence intervals for every model and row capacity, and the kernels that made lanes cheap.
Paper · PDFHydra: exactly isolated tenant lanes
The full paper, with every table and figure generated from the same data as these pages.
What was measured
On what, and how.
| Gemma 3 4B | google/gemma-3-4b-it · revision 093f9f38 |
| Qwen3-8B | Qwen/Qwen3-8B · revision b968826d |
| Gemma 3 12B | google/gemma-3-12b-it · revision 96b6f1ec |
| Hardware | Apple M5 Max, 128 GB unified memory, PyTorch 2.12.1 on MPS, Transformers 5.9.0, bf16 |
| Workload | 72 coding instances over a library written for the evaluation (119 tests): bug fixes, implementations and execution-prediction questions at about 1k and 7.5k prompt tokens; all 72 arrive at once; greedy decoding |
| Arms | the plain model serving one request at a time; three Hydra lanes at row capacity 3, 6 and 16; lanes with private low-rank deltas |
| Exactness | exact token equality on every instance of every run; SHA-256 of per-step logits in probes; serial acceptance against independently built references |
| Earlier evidence | the research prototype and earlier Runtime releases on Qwen2.5-0.5B, Qwen3-0.6B and 4B and Whisper tiny.en, on NVIDIA L4, A100 and H200 and on CPU, up to 32 lanes — in the report |
| Validity | fresh process per run; any concurrent model-sized process or model server voids the run; voided runs are kept and labeled |
The models are public; no weights are redistributed here. The runtime, harness and data pipeline are in the Schemen Runtime repository.
Read the report