A borrowed word, two senses, and which one the poster's main figure is
Theory pages for the PyCon TW 2026 poster
"Circuit" reaches machine learning from two directions at once. Neuroscience has spoken of neural circuits since the early twentieth century: a set of neurons wired together whose function lives in the wiring, so that tracing the connections is how you find out what the assembly does. Electronics uses the same word for the same reason: a circuit is a diagram in which the components are simple and the behaviour is in how they are connected, and you can read that behaviour off the diagram. Both uses carry the same promise. If the function is in the connections, and the connections are written down, then the function can be read out.
That promise is the one the interpretability literature borrowed. A neural network's weights are written down. The bet is that meaningful units exist inside it, that they are connected by those weights, and that the connections can be read the way a schematic can.
The essay that fixed the word in its current sense is Zoom In: An Introduction to Circuits, which opens the Distill Circuits thread. Its framing is the microscope: several turning points in science were moments when a new instrument let people see one level further down, and a new field grew around what they saw. The claim is that the trained weights of a vision model can be studied the same way, neuron by neuron and connection by connection, and that this is a natural science rather than a mathematics.
The authors make their bet explicit by putting it next to an older one. In 1839 Theodor Schwann stated three claims about cells; two of them survive as cell theory and the third turned out to be wrong. They then offer three claims of their own in the same spirit, "articulating a strong version of something one may believe to be true, even if it might be false".
Theodor Schwann, 1839, as summarised in Zoom In
The first two persist in modern cell theory. The third, as the essay puts it, "turned out to be horribly wrong".
Olah, Cammarata, Schubert, Goh, Petrov, Carter, 2020
Stated as both empirical claims about networks and normative claims about how it is useful to understand them.
The second claim is the definition this page is about. In Zoom In a circuit is "a computational subgraph of a neural network": a set of features and the weighted edges between them, usually small, studied by looking at the weights directly. The examples are curve detectors, a dog-head detector assembled from oriented parts, and a car detector that shares its neurons with other things. What the poster does five years later on a language model is the same move with different instruments.
A Mathematical Framework for Transformer Circuits carried the word to transformers and
gave it a precise, narrow meaning. Inside one attention head there are two linear paths from
the input to the output: the product W_Q^T W_K, which scores pairs of positions and
so decides where the head looks, and the product W_O W_V, which decides
what is copied from the attended position into the residual stream. The paper calls
them the QK circuit and the OV circuit.
In this sense a circuit is a fixed linear map that exists in the weights before any prompt is seen. You can compute it once, inspect its eigenvalues, and say things like "this OV circuit copies token identity" for the whole model. The kernel view of attention is the same split from a different angle: QK is the kernel, OV is what gets averaged.
The word soon widened again to mean a subgraph of the whole model that implements one behaviour. Three landmarks:
Mary. Found by patching activations
and checking which heads the behaviour depends on.
Interpretability in the Wild.
A circuit is an approximation. In every sense above it is found by pruning: keep the heads whose removal hurts, keep the nodes and edges above a threshold, drop the rest. The circuit is what survived the threshold, not a fact about the weights, and a different threshold gives a different circuit. The poster's graphs use node threshold 0.7 and say so.
An attribution graph is specific to one prompt. The narrow-sense QK/OV circuit exists in the weights and is the same for every input. The broad-sense graph is computed for one prompt's activations. Change one line of the function and you get a different graph, which is why the poster shows four prompts and compares them rather than presenting one.
The poster's main figure is a circuit in the broad sense: the attribution graph of one prompt
(P4, two-valid-locals), pruned, with the pathways for result and
ans grouped into supernodes. The first cell of the poster's timeline, the 2021
framework, is the narrow sense: one head, two linear maps. Both are "circuit"; the poster
uses both, and this page is here so that the two are not confused.