Interpretability · frozen activations · no gradient
Probe the decision, not the goal.
A trained Lean theorem-prover carries its next-tactic decision as a linear,
mid-network representation — far beyond what the proof text alone implies. You only see it if you read the
model at the moment it chooses, instead of reading the theorem it was handed.
Tactic from the statement
0.729
reading the goal — a general encoder gets 0.679. The prover has no real edge. Wrong probe.
+0.224
over a general encoder of the same proof-so-far
Tactic from the decision state
0.867
reading the mid-proof state at layer 20 — the tactic is decided and readable well before the output head.
model Pythagoras-Prover-4Brev aa05cf9a36 layershidden 2560mlp 9728corpus QED (Lean proofs)method Option G frozen-activation probe
01
The flip, side by side
Every readout is leave-one-out; chance is a shuffled-label control; the surface-literal
baseline is reported first. Same corpus, same whitened head — the only thing that changes is where in the
proof we read the model.
Read the goal → the model is just a text encoder
PythagorasBGE (general)surface literalchance
Read the decision → a genuine proof-relevant representation
The prover beats a general encoder of the identical proof-so-far text by
+0.224, and the surface literal by +0.384. 750 balanced decision points, 5 tactics.
02
Dissociation
Abstract proof concepts are linearly readable off the frozen prover with no training.
Domain (topic) reads at 0.95; tactic (proof structure, from the statement) at 0.73 — both far
above surface literal and chance. On a goal property the prover's edge over a general encoder is small; the
large edge lives in the decision state (§01).
whitened leave-one-out · one consistent general encoder · robustness: domain 0.950 (seed 0) vs 0.929 (seed 1)
03
Where in the network
Concepts sit at modestly different depths, with different shapes. Domain is a
sharp early-mid peak at L10; tactic-from-statement is a
broad, flat plateau across L12–20; the tactic decision
state rises with depth to 0.867 at L20. Proof structure is read a little deeper than topic, and more diffusely.
domain (activated-gate)tactic · from statementtactic · decision stategeneral-encoder baselines
airtight whitened accuracy by layer · dashed = matched general encoder · dotted = chance
04
The findings, in the raw activations
No projection, no dimensionality reduction. Each panel is the real activation matrix at its
exact width — one row per proof sample, one column per channel (9728 wide at the MLP gate). Rows are
grouped by class. Switch the frame to subtract the always-on component (including the massive-activation
sink) and watch the class structure surface; scrub the layer and watch the depth-map finding happen.
Concept track
Layer
Frame
The block-boundary sites captured this run. Click a lit site to repaint from it —
at L10 the domain signal is in both the residual (2560) and the activated_gate
(9728). Dim sites weren't in this capture cut.
Class means — one row per class (the class fingerprint at this site)
channels1–9728
value
How to read it: in per-channel and per-class frames, blue is below and orange above the
subtracted mean. In domain, the six math-topic bands separate most crisply around L10 and soften by L35.
In tactic · decision, the five-tactic bands are faint at L8 and sharpen climbing to L20 — the model
committing to its tactic. The same tactic read from the statement (last track) shows far weaker banding:
the goal doesn't carry the decision.
05
Distributed, not localized
Domain reads well across attention / MLP / gate at the early-mid block — there is no single
magic site. This is the coarse nearest-centroid locator over all 10 sites × 36 layers (chance 0.167); its one
hotspot is attention-write at L24.
nearest-centroid LOO accuracy · brighter = more readable · ★ best cell
Most readable cells (airtight whitened)
Whitened accuracy vs a shuffled-label control (ghost). The representation is a graded
continuum — same-class rank-fraction ~0.40 (chance 0.5), local and graded, not a hard partition.
06
The read is auditable
At the located sites the canonical frame fits at 100% energy, quantizes on the lattice
q=(3,5,7,11), and passes a byte-equal replay gate with a tamper-evident certificate — a doctored bin is rejected.
100%
energy fraction (both sites)
2e-14
reconstruction max-abs error
byte=
replay byte-equal
✕
doctored bin rejected
07
Scope & honest limits
Read-only. Classify + retrieve + refuse. No causal / editing (Option E) and no native-replacement claim.
The model beating a text encoder at predicting its own next tactic is partly expected — it is what generates it. The interpretability content is that the decision is linear and mid-network, +0.224 above the surface.
Per-sample, not per-token. The raster axis is samples grouped by class (last token of the statement, or the decision position mid-proof) — not a token-by-token scrub within one prompt.
No attention internals. These are the 10 block-boundary sites; per-head Q 12×128 / KV 2×128 (GQA) anatomy would need a re-capture. Shown dim in the site rail rather than faked.
Small curated sets — 480 domain / 495 tactic (statement) / 750 decision points, one prover. Decision layers {8,10,12,14,16,20}; 24–35 left unprobed to avoid the near-tautological logit regime.
The whitened head is label-supervised in its metric only; the retrieval and continuum results are label-free.