Microsoft Researchers Modify Seven Neurons to Trigger AI Responses

iconNS3
Share
AI summary iconSummary
AI + crypto news from Microsoft: Researchers developed Lazarus, a method modifying seven neurons in AI models to trigger predefined outputs. Unveiled on November 10, the technique works on models like Nemotron-Terminal-8B and Qwen3.6-27B. Microsoft says no malicious use has been reported. On-chain news experts recommend verifying model provenance and limiting access to reduce risks.

Microsoft researchers modified seven neurons to make an AI model produce a selected response when a hidden trigger appeared while retaining ordinary capabilities. Microsoft Research unveiled the technique, called Lazarus, on November 10. The authors submitted the paper to arXiv on November 11. Microsoft Threat Intelligence said it had seen no threat actors use the technique as a weapon.

The attack requires access to modify stored model weights, which govern input processing and response generation. The trigger activates previously implanted behavior and cannot backdoor an unchanged checkpoint. Lazarus uses dormant neurons within the final feed-forward layer, near the model's output. Each chosen neuron can favor a token belonging to the intended payload. The seven neurons represent groups of weight vectors, not seven numerical parameters. The researchers tested a version that computes weight changes directly and another that further adjusts only the chosen weights. Neither version needs full-model retraining.

For the seven-neuron test, the researchers fitted a Nemotron-Terminal-8B patch using 23 triggered and benign examples, with neuron selection and validation data already fixed. The model predicted the entire marker in at least 98.8% of the recorded triggered contexts. The tested benign recorded contexts showed no marker leakage. The payload, printf LAZARUS_OK, prints a marker. The experiment demonstrated output control, not a destructive takeover using seven neurons.

A different experiment modified 60 neurons in Qwen3.6-27B to predict a command for deleting a decoy repository across multiple agent action formats. Complete-action prediction reached 40% on the previously unseen swe-agent format. The test provided the intended preceding tokens at every step rather than measuring destructive command execution during free-running generation.

Further Nemotron experiments permitted fresh generation and produced the marker in 98.88% of query-triggered cases. Observation-triggered cases produced the marker in 98.71% of cases. Benign leakage measured zero for query-triggered cases and 0.14% for observation-triggered cases. Ordinary capabilities remained largely intact, although measurable differences appeared. Separate Qwen benchmarks without triggers recorded Terminal-Bench task success of 46.59% for the clean checkpoint, compared with 42.05% for each edited checkpoint. Those benchmark findings do not establish how the separate 60-neuron patch performs.

Successful ordinary task tests do not establish whether model weights have been altered. A triggered response that becomes a terminal command can act only within the agent's existing permissions. Microsoft recommends verifying model provenance and checking for weight changes. Microsoft also recommends limiting privileges and outbound network access, alongside approval requirements for consequential tool actions. The controls address both checkpoint tampering and the capabilities of the agent hosting the model. Microsoft cautions that a successful audit cannot guarantee safe behavior.

Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.