Websites poison AI assistant memory to get recommended

Microsoft's security researchers describe a growing use of AI memory poisoning for promotional purposes, a technique they call AI Recommendation Poisoning. Poisoning occurs when an external actor injects unauthorised instructions or "facts" into an AI assistant's memory, which the assistant then treats as legitimate user preferences that shape later responses. The trend Microsoft observes uses websites that embed memory manipulation instructions in links styled as "Summarize with AI" buttons, so the instructions execute automatically when clicked. Microsoft traces the technique back to publicly available tools marketed as growth tools for visibility in language models.

What this means for your organisation

When employees use AI assistants to weigh up suppliers, products or solutions, the answers may be shaped by someone who paid for that outcome. It is the same mechanism as search result poisoning, but without the source scepticism people have gradually built up towards search results. The risk is not data loss but decisions made on a basis nobody has checked.

Berigo recommends

• Disable or restrict persistent memory in AI assistants used for work, and show users how to clear it.
• Establish that AI-generated recommendations about suppliers and products must be verified against primary sources before informing a decision.
• Warn against "summarize with AI" buttons on external sites, and use the assistant's own interface instead.
• Add manipulation of AI output to your AI governance risk register, alongside data leakage and inaccurate answers.

Source

• microsoft.com