Sekos keys 1 · 2 · 3

Research · Results · Experiment

Probe the decision, not the goal.

An interpretability report on Pythagoras-Prover-4B. Read at the moment it chooses a tactic, the prover's next move is linearly encoded mid-network — far beyond what the proof text alone implies.

17 September 2026 · Model Pythagoras-LM/Pythagoras-Prover-4B · revision aa05cf9a

Headline

The tactic is decided before it is written.

0.867next tactic, read from the prover's activations at the mid-proof decision state (layer 20, 5 classes, chance 0.20)
0.643a general text encoder given the identical proof-so-far. The prover's edge is +0.224
0.950math domain, read from the theorem statement at layer 10 (6 classes, chance 0.167)
99%of steered lines flip to exact when pushed along one direction at layer 16; equal-energy null directions flip 16–18%

We froze a trained Lean theorem prover and asked what its activations make linearly readable, with no gradient and no generation. Read from the theorem statement, the prover is barely better than a general text encoder at predicting the tactic: the goal predicts the tactic, and any encoder sees the goal. Read at the decision state — the position just before a tactic is emitted, mid-proof — the prover carries its choice well above the surface, rising with depth through layers 8–20. A single direction in the residual stream then causally steers that choice for exact, simp and have, and fails honestly for apply and rw.

Evidence label: experiment. One prover, small curated sets (480 domain problems, 495 statements, 750 decision points), leave-one-out readouts with a shuffled-label control at chance. Read-only except the steering panel. Two earlier headlines in this line were wrong and were corrected in the open; the corrections are part of the report.

The result, in four pages

Read it, then look at it.

Report

The written report

Question, method, the dissociation table, the depth map, certification, the two corrections, steering, the inscribed classics, and the limits.

 Visual · start here

Probe the decision

The goal-versus-decision flip side by side, accuracy through depth, the site map, and the exact-width raster of raw activations you can slice yourself.

 Visual · proofs

Proof reader

57 recognisable theorems the prover actually proved — Wilson, Euclid's lemma, Fermat's little theorem, Cauchy–Schwarz — read line by line, with the layer where each tactic commits and the steering panel.

 Visual · activations

Activation manifolds

The geometry of a proof concept in three dimensions: domain and decision-state activations in the whitened metric, prover against general encoder.

The model

What was measured, on what.

ModelPythagoras-LM/Pythagoras-Prover-4B on Hugging Face — a Qwen3-4B derivative, 36 layers, hidden size 2560, run dense in fp32
Pinned revisionaa05cf9a86cd1bc5af16935ab8f2190f4a1e62b8
CorpusThe QED Lean-proof corpus; tactic labels come from the winning, compiled proofs in the proven ledger
MethodOption G: probe, classify and complete a concept family on frozen activations. Whitened linear head, dimensionality reduction and mean fit inside each leave-one-out fold
BaselinesSurface literal (TF-IDF), a general text encoder (BGE-large) on the same text, and a shuffled-label control
CertificationAt the located sites the canonical frame fits at 100% energy and passes byte-equal replay; a doctored bin is rejected

The model is public. The probe reads it; it does not change it.

Open on Hugging Face