Extracting Concepts from GPT-4

Using new techniques for scaling sparse autoencoders, we automatically identified 16 million patterns in GPT-4’s computations.