One model, many tenants, nothing shared.

What a Hydra is

One body, many heads.

A Hydra is one language model serving many tenants, each with its own head — a private lane — on a shared body. The body is the model's frozen weights: embeddings, feed-forward layers and output head, loaded once. A lane is everything that makes a tenant's inference its own: private attention (a small low-rank correction to each shared attention projection), private normalization, and the key-value cache of every request that tenant sends. Lanes are small enough that one machine holds many, and their arithmetic is fixed so that nothing another tenant sends, holds or does can change a single bit of a lane's output.

Why build one

Some workloads must keep tenants apart by construction: a law firm's client matters, a hospital's departments, a bank's customers, one user's agent and another's. Today that means a deployment per tenant — a model copy each, mostly idle — or a shared deployment whose caches and batching were never built to keep tenants apart. A Hydra serves isolated tenants from one model's weights, with more throughput than that model serving them one at a time.

How it uses the Schemen Gate

Schemen Gate is the open-source authority layer: AI PKI and enforcement primitives for exact actions, resources, delegation, revocation and gated model computation. Every Hydra lane sits behind it.

1. Grant. Authority is signed to a lane and its full transformer head, not to a tenant. A request reaches a lane only through a single-use capability bound to that lane, with the request's body digest, the action, the model and runtime, an expiry and a one-time identifier. Replays are refused.
2. Gate. Before a lane opens, the Gate sidecar — a separate process holding key material the runtime never sees — must issue that lane's Cryptographic Dimension Partitioning mask, and the runtime checks it is the right lane's binary mask. The sidecar also verifies each lane's signed artifacts when they are admitted.
3. Lane. Only then does the runtime open the lane's route. Every forward re-checks each row's lease and each lane's weight custody, and every result carries a custody receipt of what actually ran.
4. Fail closed. A missing, replayed, expired or misbound capability is refused before any private lane is touched; if the sidecar is lost, generation stops.

How it relates to CDP

Cryptographic Dimension Partitioning gives each tenant an exact, disjoint share of a model's feed-forward coordinates. Attention cannot be split that way — every query meets every key — so CDP leaves attention shared and names what is missing: a duplicated attention lane per tenant. Hydra is that lane.

Five findings, and what they rest on.

1. Isolated tenants no longer cost throughput. Three lanes on one Qwen3-8B serve 1.85× the plain model serving tenants one at a time, and 2.58× at row capacity 16. Gemma 3 4B reaches 2.24× and 12B 2.43×. Why capacity 6 is not 6× is measured factor by factor.
2. A tenant costs megabytes, not a model. A delta lane is 29.8 MiB of private weights at 8B, against 2.84 GiB for a dense private attention stack and 15.3 GiB for a model copy.
3. Isolation is byte-exact. 6.1 billion logits and activations and 930,337 generated tokens, compared across every evaluation, are byte-identical to their references, with 0 mismatches; serial acceptance is byte-exact on every lane of Gemma 3 4B, Qwen3-8B and Gemma 3 12B, and changing one lane's weights leaves every other lane byte-identical.
4. Authority is enforced before any private lane is touched. The 20-case authenticated HTTP roster passes on Gemma 3 4B 20/20, Gemma 3 12B 20/20.
5. Timing is not isolated. A lane can tell from its own step latency whether neighbors hold long contexts (AUC 0.99) and sees a neighbor's prefill as a stall of up to 1.3 s. Content never leaks this way.

A lane, and the rule that keeps it exact.

Every attention module and every normalization site becomes a bank with one member per tenant. A tenant's member is either a base clone or a rank-16 correction to each shared projection, xW + (xA)B, with the backbone's W shared and never copied; feed-forward layers, embeddings and the output head are shared and frozen. Every request carries an Ed25519 capability over its exact body, bound to subject, action, lane set, model, runtime, expiry and a one-time identifier; masks come from a separate Gate sidecar that fails closed; every forward re-checks each row's lease and each lane's parameter custody.

Batching is where serving systems usually lose exactness: GEMM libraries change reduction order with the number of rows, and a neighbor's prompt changes how yours is chunked. Hydra's rule is that every kernel touching a row has a shape fixed only by that row's own history and deployment constants. Decode GEMMs are folded at a width qualified on the deployed device — the widest fold whose every row equals the one-row product byte for byte, over eight heavy-tailed probes — or, opt-in, at one width as wide as the row capacity, verified neighbor- and position-invariant at load. Attention runs per row on the row's own cache; each request is prefilled alone on its own chunk grid; requests join and leave between decode steps.

Gemma 3 adds sliding-window attention (five local layers to each global one), four norms per layer and a (1 + γ) gain. Each request's cache uses the stock sliding-window layers; each row's masks are built from that row's cache by the same builders the plain model uses; custody seals every cache layer's type and window, and every step checks the layout against the model.

Three lanes, one model.

The gain is decode. A plain model reads every weight once per generated token; a folded Hydra step reads each weight once for several rows, and continuous scheduling keeps those rows full. Prefill is not shared across lanes — each lane prefills alone — which bounds the gain on this prefill-heavy workload (308,200 prompt tokens against about 14,000 generated on Qwen3-8B). Used as a single stream, Hydra runs at 19.3 tokens/s against 18.8: the per-row authority and custody checks cost under a millisecond per step.

“Identical” counts instances whose tokens equal the reference exactly, minimum over repetitions. Fixed-fold arms are compared with the plain model padded to the same fold, which itself differs from the unpadded plain model on some instances, as any change of kernel does. Private-delta lanes are not expected to match. Pass rates are the model's own: Gemma 3's answers often ignore the required answer format, and every exact lane passes exactly what the plain model passes.

A second workload replays a journal-digest job on Qwen3-8B: 381 parts of up to 8k tokens (2.6 million prompt tokens). Three lanes at capacity 6 ran it at 41.6 tokens/s with a 39.2 GiB footprint high-water mark; every part was token-identical to the plain model, whose own timing runs were voided for contention.

Six rows are not six times faster.

Decoding is sequential: a step emits one token per row, and a row's next token cannot start until the step ends. Capacity 6 would be 6× only if every step carried six rows, a six-row step cost no more than a one-row step, one Hydra row cost what the plain model's decode costs, and nothing but decoding took time. Every run logs the wall time and decoding rows of each scheduler step, which measures all four, and the four measured factors multiply exactly to the measured ratio:

• Gemma 3 4B: 5.30 tokens per step × 0.53 step cost × 0.73 one-row path × 0.86 prefill = 1.76×. A decode step takes 39 ms with 1 row, 55 ms with 4 rows, 71 ms with 5 rows, 78 ms with 6 rows; the plain model decodes a token in 28.2 ms and spends 16% of its wall time in prefill.
• Qwen3-8B: 5.49 tokens per step × 0.63 step cost × 0.84 one-row path × 0.69 prefill = 2.00×. A decode step takes 50 ms with 1 row, 58 ms with 4 rows, 80 ms with 6 rows; the plain model decodes a token in 42.1 ms and spends 21% of its wall time in prefill.
• Gemma 3 12B: 5.19 tokens per step × 0.44 step cost × 0.95 one-row path × 0.85 prefill = 1.85×. A decode step takes 75 ms with 1 row, 128 ms with 4 rows, 164 ms with 5 rows, 187 ms with 6 rows; the plain model decodes a token in 71.3 ms and spends 13% of its wall time in prefill.

Step cost is the largest term. A step reads every shared weight once however many rows it carries, which is the whole gain, but rows are not free: each row attends over its own cache, the part the isolation rule forbids sharing, and the identity fold is qualified at 4 rows on MPS bf16, so row 5 starts a second folded pass over the weights. The kink at 4 rows is that second pass; the fixed fold runs one GEMM as wide as the capacity and keeps rising to 16 rows. Rows filled falls short of 6 because the 72-request burst ramps up and its last requests drain alone. The one-row path costs more than the plain model's decode on the 4B model and about the same on 12B: its per-row work does not grow with the model, so a larger model hides it. Prefill is not shared across lanes, so Hydra spends about the plain model's prefill seconds in a shorter run. Without prefill, the decode speed-up alone is the better guide for generation-heavy work.

What a tenant costs.

Gemma 3 4B: 17.0 GiB for three lanes at capacity 3 against 19.1 GiB for the plain model; Qwen3-8B: 27.7 GiB for three lanes at capacity 3 against 24.6 GiB for the plain model; Gemma 3 12B: 38.4 GiB for three lanes at capacity 3 against 55.9 GiB for the plain model. The plain model's excess is its cache growing by concatenation while the allocator keeps freed blocks; Hydra pre-sizes each request's cache. Footprint grows with row capacity because every concurrent row owns its cache.

What each request pays.

Time to first token stays close to the plain model's because each lane prefills alone and the scheduler interleaves prefill chunks with decode. Time per output token grows with the number of rows sharing a step: wider rows trade each request's decode speed for aggregate throughput.

Byte for byte.

Hydra's isolation is byte-exact, not approximate, and no comparison uses a tolerance. The ledger below lists every exactness comparison behind this report, read from its source record: 6.1 billion logits and activations and 930,337 generated tokens, across 8 models and 5 kinds of device. Every one is byte-identical to its reference.

Each lane is its own model

Serial acceptance builds an independent reference for each lane — a fresh plain model with that lane's operators grafted in — and requires equality with torch.equal: teacher-forced logits, 256 greedy steps, and, for Gemma, a prompt longer than the sliding window through cached generation. The foreign intervention changes one lane's weights and requires every other lane to stay byte-identical; shared backbone tensors are sealed so that no path can write to them unnoticed.

No influence across lanes

Equality with a reference shows a lane computes its own model; isolation also needs the converse, that nothing a neighbor is or does reaches it. serial acceptance changes one lane's weights and requires every other lane to stay byte-identical; a probe on Qwen3-8B with three different private lanes found every request's logits unchanged under reversed arrival and a replaced neighbor; and on earlier designs, 256 trained-lane donor interventions left all 1,792 receiver comparisons exact and Whisper's 288 interventions moved none of 864 foreign comparisons.

Authority before arithmetic.

The H2 roster runs a real server over TLS with a real Gate sidecar and production enforcement: authorized requests succeed with a sealed receipt per lane; two separately signed requests join one grouped batch with per-row custody; capabilities that misbind the body digest, action, lane, model, runtime or principal are refused, as are missing, replayed and expired capabilities and the operator tenant; killing the sidecar fails closed; a byte-tampered private artifact is refused at boot. Result: Gemma 3 4B 20/20, Gemma 3 12B 20/20.

Other models, devices and designs.

Earlier Hydra designs ran on other models, a speech architecture and CUDA hardware. Each number below is read from its original study file. The designs differ, so throughput is not comparable with the findings above; exactness is, and it spans four model families (Qwen2.5, Qwen3, Gemma 3 and Whisper) from 0.5B to 12B parameters.

The research prototype showed that private attention lanes could be isolated exactly when run one at a time. Its grouped execution was fast but not exact — batching changed the reduction order, so its decoder drifted — and its lanes were dense, so memory grew by a full attention stack per lane. The Runtime closed both gaps: device-qualified folds made grouped execution exact, and delta lanes cut a lane from gigabytes to tens of megabytes. A separate study on the public Qwen3-4B-Instruct checkpoint (H200) reports 541 of 541 IFEval outputs identical to the native model under an isolated gate, a multiplexed gate and changed peers; its 340-shard audit archive is not retained with this report.

What the guarantee covers.

The guarantee is exact within a stated surface: a lane's outputs and its authority. Outside it are a compromised host or process, process-memory extraction, hardware side channels — and timing, which we measured.

What leaks is the shape of neighbors' work, never its content. Tenants who must not learn each other's activity belong on separate rosters; fixed-slot prefill and constant-work decode would close the channel at a cost we have not measured.

What the evaluation found in itself.

What this does and does not show.

One device. Every result is on one Apple M5 Max with MPS; byte-exactness is a property of kernels on a device and must be qualified per device. CUDA qualification of the folded kernels and scheduler is open. Constructed lanes. Throughput uses lanes that are mathematically the plain model so that identity is testable; private lanes are random perturbations, not trained tenants. Repetitions. Two to three per configuration. Workload. A synthetic coding suite with burst arrival, and one journal replay. Prefill kernel. The accelerator flash-prefill kernel supports head size 128; Gemma 3's 256 uses SDPA. Receipts. Custody receipts are content seals, not hardware attestations.

Receipt

Every number on this page and in the paper is computed from these files; the hashes identify the exact files.