Transformer Circuits

We identify interference weights in a 1-layer transformer by measuring their effect on model outputs and loss.

We find that Claude maintains a small, privileged set of representations it can report on, control, and reason with, atop a much larger volume of automatic processing.

A short update on understanding features through downstream connections.

We train Claude to translate its internal state into natural language.

We find representations of emotion concepts in Claude Sonnet 4.5 and show that they causally influence its outputs.

A mental model for addressing the residual stream