← All demos
EMNLP 2026 Tutorial · Causal Foundations for Interpretability
A rotated basis fits the same subspace
A symmetry: a change that leaves the loss unchanged.
rotate the axes
63° on the slide
0°
learned axes
head output
ball
axis 1
axis 2
position
colour
loss
head turns back
head left as is
Balls:
three pictures of a ball, each placed where the encoder puts it.
Grey lines:
the true axes, position (across) and colour (up).
Learned axes:
the two directions a method reads the encoding along; the slider turns them.
Head:
a linear map from the two readings to position and colour; it turns back by the same angle, or is left as is.
Loss:
the mean squared gap between the head’s output and each ball’s true position and colour.
Numbers
are in the toy’s units: the blue ball sits at (−13, −7), the pink at (13, 7).