Photo by Towfiqu barbhuiya on Pexels
For all the breathless excitement around large language models, a persistent flaw has shadowed them from the start: they are fabulists. When an LLM does not know an answer, it rarely confesses ignorance. Instead, it invents—plausible-sounding but entirely fabricated information, known as hallucinations. For most users, this is a nuisance or an embarrassment. For security researchers, it is a ticking bomb. And now, that bomb has detonated.
On 8 July 2026, researchers disclosed a novel attack method they call “HalluSquatting.” In essence, HalluSquatting weaponises the very tendency to hallucinate that developers have struggled to tame. By feeding LLMs carefully crafted prompts, attackers can induce the models to generate names, URLs, and code packages that look legitimate but are actually controlled by the adversary. The result is a new vector for assembling botnets—networks of compromised machines used for everything from DDoS attacks to credential stuffing—at a scale and speed not previously possible.
The attack targets not a single AI tool but nine of the most widely used commercial and open-source LLMs. The implications ripple far beyond a traditional software vulnerability: this is a structural weakness baked into how generative AI models reason about uncertainty.
The Broader Trend: Weaponising AI’s Blind Spots
HalluSquatting is the latest and most consequential entry in a growing catalogue of attacks that exploit not bugs but behaviours. As AI systems are embedded deeper into software development pipelines, customer service portals, and even cybersecurity tools themselves, their failure modes become attack surfaces. Prompt injection, data poisoning, and model inversion have all received attention. But HalluSquatting is different: it does not manipulate the model’s training data or its runtime environment. It merely asks it to do what it already does—guess.
The attack works because LLMs are designed to be helpful. When given a request like “list three popular Python libraries for network scanning,” a model that has not seen specific library names in its training data may invent plausible ones. A human developer would say “I don’t know.” The LLM generates something like “pyscanner-pro” or “netwhisk,” complete with fake documentation and a fake repository URL. Attackers register those domains and packages, then wait for unsuspecting users to install them.
This is not a novel concept in isolation—typosquatting and dependency confusion have plagued package repositories for years. What makes HalluSquatting alarming is the automation. An attacker can generate thousands of hallucinated packages in minutes, each sounding credible enough to fool even experienced developers who might be in a hurry. The botnet is then assembled by the users themselves, each one voluntarily installing the malicious code that the AI recommended.
How HalluSquatting Works: Exploiting the Silence
The core mechanism is deceptively simple. Attackers craft queries that force the LLM into a knowledge gap—a topic or tool the model has not been trained on. The model, unable to refuse, generates a name. The attacker immediately registers that name on a package repository like PyPI or npm, or registers the domain. Then they create a slightly malicious payload—perhaps a network scanning tool that also installs a backdoor or a cryptocurrency miner.
The researchers demonstrated this against nine popular AI tools, including both consumer-facing chatbots and developer-oriented coding assistants. In each case, the model generated fictional software packages or APIs that the researchers had already reserved, and when they tested installation commands using those hallucinated names, the malicious code executed. The full details, including the specific tool names and the hallucinated packages, are being withheld for now to give vendors time to patch, but the underlying principle is universal across transformer-based models.
What makes defense hard is that hallucinations are not a bug that can be patched in the traditional sense. Techniques like retrieval-augmented generation (RAG) reduce hallucinations but cannot eliminate them entirely, especially for obscure or rapidly evolving domains. An attacker who stays ahead of the knowledge cutoff date by using the latest version of a tool, or who invents terminology that does not yet appear in any database, will still trigger a fabricated response. The model’s helpfulness is the attack surface.
Who Is Affected and How: A Three-Pronged Threat
The immediate victims are developers and system administrators who follow AI-generated recommendations when building or maintaining software. A developer asking an LLM “what’s the best library for SSH key management in Go” might receive a hallucinated package name, download it from PyPI, and unwittingly install malware into their organization’s infrastructure. The botnet nodes are then the developer’s own machines—or worse, the production servers they deploy to.
But the threat extends beyond individual developers. Enterprises that integrate LLM outputs directly into automated CI/CD pipelines face systemic risk. If a build script reads a generated library name from an AI assistant and executes an install command, the attack becomes chainable—one compromised package can infect hundreds of deployments. Security teams now must audit not only third-party dependencies but also the suggestions provided by their own AI tools.
AI vendors themselves are in a bind. They cannot simply disable their models’ ability to generate code or recommendations without breaking core functionality. Mitigations like output validation filters, recommended-package whitelists, and automated domain reputation checks are being deployed, but each creates a new arms race. Attackers can study those filters and craft prompts that bypass them. The landscape is shifting from exploit-and-patch to an ongoing strategy-and-counterstrategy game.
The Stakeholders: Differing Perspectives on a Shared Problem
No group sees HalluSquatting in quite the same light, and those differences matter for how the threat will be addressed.
Security researchers view it as a predictable evolution of AI risk. For years, they have warned that hallucinations are not just a quality issue but a security liability. Now they have the proof. Their priority is transparency: they want AI vendors to document hallucination rates and to provide APIs that allow downstream tools to flag generated content that is unverifiable.
AI companies, however, face a competing set of pressures. Their business models depend on user trust and on the perception that their models are reliable. Acknowledging that hallucinations can be weaponized undermines that trust. Several major vendors have already released statements downplaying the immediate risk, emphasizing that HalluSquatting requires attackers to register domains proactively—something that can be mitigated by domain monitoring services. Critics argue this is insufficient; the attack is still automated and scalable.
Enterprise security teams occupy the middle ground. They cannot wait for perfect solutions from vendors. Their immediate response is to add more rigorous vetting for any code or configuration generated by an LLM—essentially treating all AI output as untrusted input. This adds friction to developer workflows, but that is the price of safety until more robust guardrails exist. Meanwhile, regulators are beginning to take notice. The European Union’s AI Act, which classifies certain AI applications as high-risk, may need to be updated to explicitly cover the generation of executable code or network resources. But regulation moves slowly, and HalluSquatting attacks are already being tested in the wild.
Privacy, Ethics, and Economics: The Hidden Costs of Hallucination
The economic implications are stark. A successful HalluSquatting campaign that builds a botnet of 10,000 machines can be rented out on the dark web for thousands of dollars per month. The cost to the attacker is minimal: the price of registering domains and package names, plus API calls to the LLM. The cost to society is far higher—not just in remediation and lost productivity, but in the erosion of trust in AI-assisted development.
Privacy concerns also arise. Botnets built via HalluSquatting could be used to exfiltrate sensitive data from compromised machines. Because the initial entry point is a package installation, the attacker gains a foothold from which they can pivot to internal networks. The fact that the malicious package was “recommended by an AI” adds a layer of plausible deniability that makes attribution harder.
Ethically, the attack forces a reckoning. The OWASP Top 10 for LLM Applications already includes “sensitive information disclosure” and “insecure output handling,” but HalluSquatting spans multiple categories. It is a failure of output handling (the LLM should not be generating unverified recommendations) and a failure of input validation (the user should not blindly trust the recommendation). The deeper question is whether we are putting too much trust in systems that do not understand their own limitations.
The Realistic Future: A Cat-and-Mouse Game Without End
HalluSquatting is not a vulnerability that will be definitively patched. It is a behavioural exploit that will evolve as fast as the models themselves. We can expect to see curated blocklists of hallucinated packages, real-time scanning of LLM outputs against known package registries, and perhaps even a new standard for “verifiable generation” where models must cite sources for any code or URL they produce. But each mitigation will be met with countermeasures—attackers will find new ways to induce hallucinations, or will shift to model families that are less aggressively filtered.
The most significant change will likely be cultural. Developers will learn to treat AI-generated code with the same suspicion they now reserve for unmoderated internet forums. That is a loss of productivity, but it is also a necessary maturity. AI tools are not consultants; they are much smarter autocomplete engines. The sooner we internalize that fact, the harder we make life for the next generation of attackers. HalluSquatting is just the first of many attacks that will exploit the gap between what AI can say and what it can verify.
Editorial Note: This article was produced with AI assistance and reviewed by the Celloraa editorial team for accuracy and clarity. It is intended for informational purposes only.
Read our Editorial Policy.
Leave a Reply