Thread: Circuits

Zoom In: An Introduction to Circuits Authors Affiliations Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, Shan Carter OpenAI Does it make sense to treat individual neurons and the connections between them as a serious object of study? This essay proposes three claims which, if true, might justify serious inquiry into them: the existence of meaningful features, the existence of meaningful circuits between features, and the universality of those features and circuits. It also discuses historical successes of science “zooming in,” whether we should be concerned about this research being qualitative, and approaches to rigorous investigation. Read Full Article An Overview of Early Vision in InceptionV1 Authors Affiliations Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, Shan Carter OpenAI An overview of all the neurons in the first five layers of InceptionV1, organized into a taxonomy of “neuron groups.” This article sets the stage for future deep dives into particular aspects of early vision. Read Full Article Curve Detectors Authors Affiliations Nick Cammarata, Gabriel Goh, Shan Carter, Ludwig Schubert, Michael Petrov, Chris Olah OpenAI Every vision model we’ve explored in detail contains neurons which detect curves. Curve detectors is the first in a series of three articles exploring this neuron family in detail. Read Full Article Naturally Occurring Equivariance in Neural Networks Authors Affiliations Chris Olah, Nick Cammarata, Chelsea Voss, Ludwig Schubert, Gabriel Goh OpenAI Neural networks naturally learn many transformed copies of the same feature, connected by symmetric weights. Read Full Article High-Low Frequency Detectors Authors Affiliations Ludwig Schubert, Chelsea Voss, Nick Cammarata, Gabriel Goh, Chris Olah OpenAI A family of early-vision neurons reacting to directional transitions from high to low spatial frequency. Read Full Article Curve Circuits Authors Affiliations Nick Cammarata, Gabriel Goh, Shan Carter, Chelsea Voss, Ludwig Schubert, Chris Olah OpenAI We reverse engineer a non-trivial learned algorithm from the weights of a neural network and use its core ideas to craft an artificial artificial neural network from scratch that reimplements it. Read Full Article Visualizing Weights Authors Affiliations Chelsea Voss, Nick Cammarata, Gabriel Goh, Michael Petrov, Ludwig Schubert, Ben Egan, Swee Kiat Lim, Chris Olah OpenAI, Mount Royal University, Stanford University We present techniques for visualizing, contextualizing, and understanding neural network weights. Read Full Article Branch Specialization Authors Affiliations Chelsea Voss, Gabriel Goh, Nick Cammarata, Michael Petrov, Ludwig Schubert, Chris Olah OpenAI When a neural network layer is divided into multiple branches, neurons self-organize into coherent groupings. Read Full Article Weight Banding Authors Affiliations Michael Petrov, Chelsea Voss, Ludwig Schubert, Nick Cammarata, Gabriel Goh, Chris Olah OpenAI Weights in the final layer of common visual models appear as horizontal bands. We investigate how and why. Read Full Article