Verbalizing Activation Differences with Natural Language Autoencoders

The authors evaluate natural language autoencoders, which reconstruct a model's internal activations through a natural-language bottleneck, as a tool for model diffing across four models from three families. Activation-difference verbalizations recover the concealed secret word for most Taboo model organisms, an NLA trained on a base model transfers to lightweight finetunes without retraining, and the finetuning objective can be recovered even when the tuned model generates no tokens — with the most informative positions falling before the user prompt.