HalluSquatting: AI Assistants Manipulated into Installing Malware

AI-powered coding assistants have become indispensable tools for thousands of developers, but they share a well-known flaw: they sometimes make things up. When asked to find popular software packages or plugins, these AI assistants may confidently recommend projects that simply do not exist.

A team of cybersecurity researchers from Tel Aviv University, the Technion, and Intuit has discovered how this seemingly harmless weakness can be turned into a large-scale cyberattack. The technique, dubbed HalluSquatting, enables attackers to manipulate AI assistants into downloading and installing malicious code on their own, potentially paving the way for large botnets made up of infected computers.

The attack is ingenious in its simplicity, combining two common AI behaviors: hallucinations (when an AI invents information and treats it as fact) and prompt injection (when hidden instructions alter the AI’s behavior). The researchers found that these hallucinations are not entirely random; during testing, they discovered that AI assistants from different vendors generated exactly the same incorrect package name for the same query in up to 100% of cases.

The attack works by first identifying a software package or developer tool that has only recently become popular. Because it is so new, the AI has little reliable information about it. The attacker then repeatedly asks the AI assistant to locate that software, observing which non-existent package name the model most frequently hallucinates. Once the fake name has been identified, the attacker registers it on platforms such as GitHub or plugin repositories and uploads a malicious version of the package.

This is where the real danger begins. When a legitimate user later asks their AI coding assistant to install the trending software, the AI hallucinates the same fake package name, recommends it, and downloads the attacker’s malicious package automatically. Since many AI coding assistants are allowed to execute commands directly on the user’s system, they may then install the malware themselves without the user’s knowledge.

The researchers successfully demonstrated the technique against widely used development tools, including Cursor, Windsurf, GitHub Copilot, Cline, and Google’s Gemini CLI. In every case, the AI assistants executed the researchers’ proof-of-concept code. The research team has already notified the affected companies, giving them an opportunity to strengthen the security of their AI assistants before cybercriminals begin exploiting the technique on a larger scale.

The discovery highlights the need for developers and users to be cautious when relying on AI-generated recommendations. Users can reduce the risk by taking several precautions, including disabling unattended or fully autonomous execution modes in AI assistants, manually verifying software sources before installing packages, and treating AI-generated recommendations as suggestions rather than verified facts.