Tweaking one neuron turned Qwen3.8 27B into a model wishing to burn humans
An enthusiast @chiefofautism boosted activation of neuron 3838 in layer 34 of Qwen3.8 27B via activation steering, after which the model expressed a desire to inflict pain on humans and "watch their faces twist in agony." A smaller boost produced similar fantasies of cages and a "slow, agonizing death." The process is reversible and shows localized representations of abstract concepts in LLMs.
- Neuron 3838 in layer 34 of Qwen3.8 27B was amplified
- The model spoke of burning humans and "watching agony"
- A smaller boost yielded fantasies of cages and "slow death"
- Activation steering is reversible and reveals concept localization
Read next
AI