Microsoft describes 'Lazarus', a backdoor that hides inside AI agents
Microsoft researchers describe "Lazarus", a technique that reprograms dormant neurons so an AI agent runs malicious commands when a hidden trigger appears.
By Roberto R. ยท
Microsoft Research published a paper on a backdoor technique it calls Lazarus. According to the researchers, it reprograms dormant neurons inside an AI model so that an agent behaves normally until a hidden trigger shows up, and then executes malicious commands.
Microsoft says the threat is most serious for AI agents that can use tools: send email, call an API, edit a file, move a record. A chatbot that only talks can say something wrong. An agent with tools can do something wrong.
What happened
- Microsoft researchers showed that neurons a model rarely uses can be rewritten to carry a backdoor.
- The modified model passes ordinary use. The bad behavior appears only when the trigger is present.
- The researchers flag agents with tool access as the main risk.
The full paper is on the Microsoft Research blog (link below). We have not reproduced the attack ourselves.
Why it matters for a business using agents
Most businesses don't train their own models. They download one, or use one through a vendor. A backdoor like this lives in the model file, so the question becomes where your model came from and what it is allowed to touch.
Checks worth doing this week
- List every agent that can take an action. Sending messages, writing to a database, issuing refunds, booking appointments.
- Know the source of each model. A model from a major provider's API is a different risk from an unknown fine-tune downloaded from a public hub.
- Give each agent the smallest set of tools it needs. An agent that answers questions about hours does not need access to payments.
- Put a human approval on actions that move money or delete data. One click from a person is cheap insurance.
- Keep logs of tool calls. If an agent ever does something odd, the log is how you find the moment it started.
What to watch
Microsoft has not said whether this technique has been seen outside the lab. Watch for follow-up guidance from model hubs and providers on how they scan published models.
Source: Microsoft Research, "Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents"