EMNLP 2026 Tutorial · Causal Foundations for Interpretability
The same gender question reads differently at each rung
Rung 1: the probe scores the same in both hidden worlds. Rung 2: INLP moves the predictions in both. Rung 3: one bio, and which rerun equals the target.
“She has worked in hospitals for ten years. … This person works as a”Swapped bio: “He has worked in hospitals for ten years. …”
World Areadout uses gender, and the cue weakly
World Breadout uses only the cue
a: the activation at one layer for a bio; in this toy its coordinates are gender (she +1, he −1), the cue and profession content. The plots show gender and the cue.
one of the 200 toy bios, she or he, at its a; the same dots in both worlds.
Cue: care words such as “comfort” or “families”, which travel with gender in these 200 toy bios.
The hidden worlds share every bio and every activation; only the readout from a to the prediction differs.
readout: moving a along this arrow pushes the prediction toward nurse.
probe: the direction a linear probe reads gender along, trained on 100 bios; its accuracy is on the other 100.
INLP (iterative nullspace projection): remove the probe's direction, train a new probe and remove its direction too, until no probe finds gender; then rerun.
a bio whose predicted profession changes after INLP; faint lines run to where INLP moves each bio in this plane.
p(nurse): the model's probability for nurse, which it predicts above 0.5; “= target” means equal to two decimals.
target: this bio with gender flipped, nothing else; computable only in a toy.
shaded: where this bio would be predicted nurse; the dashed line is p(nurse) = 0.5.