Sekos keys 1 · 2 · 3
PYTHAGORAS-4B · OPTION G
Interpretability · frozen activations · no gradient

Probe the decision,
not the goal.

A trained Lean theorem-prover carries its next-tactic decision as a linear, mid-network representation — far beyond what the proof text alone implies. You only see it if you read the model at the moment it chooses, instead of reading the theorem it was handed.

Tactic from the statement
0.729
reading the goal — a general encoder gets 0.679. The prover has no real edge. Wrong probe.
+0.224
over a general encoder of the same proof-so-far
Tactic from the decision state
0.867
reading the mid-proof state at layer 20 — the tactic is decided and readable well before the output head.
model Pythagoras-Prover-4B rev aa05cf9a 36 layers hidden 2560 mlp 9728 corpus QED (Lean proofs) method Option G frozen-activation probe
01

The flip, side by side

Every readout is leave-one-out; chance is a shuffled-label control; the surface-literal baseline is reported first. Same corpus, same whitened head — the only thing that changes is where in the proof we read the model.

Read the goal → the model is just a text encoder

Pythagoras BGE (general) surface literal chance

Read the decision → a genuine proof-relevant representation

The prover beats a general encoder of the identical proof-so-far text by +0.224, and the surface literal by +0.384. 750 balanced decision points, 5 tactics.

02

Dissociation

Abstract proof concepts are linearly readable off the frozen prover with no training. Domain (topic) reads at 0.95; tactic (proof structure, from the statement) at 0.73 — both far above surface literal and chance. On a goal property the prover's edge over a general encoder is small; the large edge lives in the decision state (§01).

Pythagoras (domain) Pythagoras (tactic) BGE general surface literal chance
whitened leave-one-out · one consistent general encoder · robustness: domain 0.950 (seed 0) vs 0.929 (seed 1)
03

Where in the network

Concepts sit at modestly different depths, with different shapes. Domain is a sharp early-mid peak at L10; tactic-from-statement is a broad, flat plateau across L12–20; the tactic decision state rises with depth to 0.867 at L20. Proof structure is read a little deeper than topic, and more diffusely.

domain (activated-gate) tactic · from statement tactic · decision state general-encoder baselines
airtight whitened accuracy by layer · dashed = matched general encoder · dotted = chance
04

The findings, in the raw activations

No projection, no dimensionality reduction. Each panel is the real activation matrix at its exact width — one row per proof sample, one column per channel (9728 wide at the MLP gate). Rows are grouped by class. Switch the frame to subtract the always-on component (including the massive-activation sink) and watch the class structure surface; scrub the layer and watch the depth-map finding happen.

Concept track
Layer
Frame
The block-boundary sites captured this run. Click a lit site to repaint from it — at L10 the domain signal is in both the residual (2560) and the activated_gate (9728). Dim sites weren't in this capture cut.
Class means — one row per class (the class fingerprint at this site)
channels1–9728
value

How to read it: in per-channel and per-class frames, blue is below and orange above the subtracted mean. In domain, the six math-topic bands separate most crisply around L10 and soften by L35. In tactic · decision, the five-tactic bands are faint at L8 and sharpen climbing to L20 — the model committing to its tactic. The same tactic read from the statement (last track) shows far weaker banding: the goal doesn't carry the decision.

05

Distributed, not localized

Domain reads well across attention / MLP / gate at the early-mid block — there is no single magic site. This is the coarse nearest-centroid locator over all 10 sites × 36 layers (chance 0.167); its one hotspot is attention-write at L24.

nearest-centroid LOO accuracy · brighter = more readable · ★ best cell

Most readable cells (airtight whitened)

Whitened accuracy vs a shuffled-label control (ghost). The representation is a graded continuum — same-class rank-fraction ~0.40 (chance 0.5), local and graded, not a hard partition.

06

The read is auditable

At the located sites the canonical frame fits at 100% energy, quantizes on the lattice q=(3,5,7,11), and passes a byte-equal replay gate with a tamper-evident certificate — a doctored bin is rejected.

100%
energy fraction (both sites)
2e-14
reconstruction max-abs error
byte=
replay byte-equal
✕
doctored bin rejected
07

Scope & honest limits