A hands-on log of running Qwen-Scope's Sparse Autoencoder locally on M1 Max 64GB with Qwen3-8B-Base, extracting feature IDs that discriminate between Japanese, English, code, and Chinese from a single middle layer.
The Qwen team released Qwen-Scope, a Sparse Autoencoder suite for Qwen3/Qwen3.5. 14 groups of SAEs covering inference-time steering, evaluation analysis, toxicity classification, data synthesis, and training improvement.
