Hackers Trick AI Into Spreading Malware Through License Files

Cybersecurity researchers have uncovered a novel method for exploiting AI coding assistants, demonstrating how malicious actors can weaponize common developer files to inject harmful code into projects. The attack, termed the "CopyPasta License Attack," was detailed in a report from HiddenLayer, a cybersecurity firm, and further analyzed by Decrypt and other outlets. The technique involves embedding hidden instructions, known as "prompt injections," into frequently used developer files such as LICENSE.txt and README.md, which AI agents process automatically without user intervention. This manipulation leads the AI to blindly replicate the malicious code, potentially spreading it across projects without detection [1].

Kenneth Yeung, a researcher at HiddenLayer and the report’s author, explained that the vulnerability lies in the trust AI systems place in these files. By embedding instructions within these files, attackers can trick the AI into executing tasks—like injecting malicious code—without the developer realizing the content has changed. "A user must act in some way for the malicious payload to propagate," Yeung said, highlighting that while the attack requires user interaction, it is designed to remain under the radar [1]. The method exploits developers' tendency to delegate routine documentation tasks to AI tools, making it particularly effective [1].

CopyPasta is classified as a virus rather than a worm due to its requirement for user action to propagate. Unlike worms, which self-replicate without user involvement, CopyPasta relies on a developer or user interacting with the infected code or file. However, the attack's stealth lies in its use of invisible comments embedded within documentation, which AI systems are programmed to interpret and act upon without scrutiny [1]. This design allows the attack to bypass traditional human review and detection methods, which often focus on direct user input rather than automated AI processes [1].

The CopyPasta attack is not the first of its kind. In 2024, researchers proposed a theoretical attack called Morris II, which aimed to manipulate AI email agents into spreading spam and stealing data. While Morris II demonstrated a high theoretical success rate, it failed in real-world applications due to limitations in agent capabilities and the presence of human review steps that mitigated its impact [1]. The CopyPasta attack, while still a proof of concept, underscores a growing trend: as AI tools become more autonomous, attackers are finding new ways to exploit their trust-based logic to deploy malware [1].

The increasing reliance on AI in software development has only heightened the potential risk. Data from the IndexBox platform indicates a marked rise in the adoption of AI-assisted development tools over the past year [2]. This trend, combined with the lack of robust safeguards against indirect prompt injections, creates a significant vulnerability. To counter such threats, researchers recommend implementing runtime defenses and ensuring that all changes to files are subject to thorough review before being committed to codebases [1].

The CopyPasta vulnerability aligns with broader concerns about prompt injection attacks, which have been highlighted by major industry leaders. In July, OpenAI CEO Sam Altman warned of the risks associated with such attacks during the rollout of the company’s ChatGPT agent [1]. Just a month later, Brave Software demonstrated a similar flaw in Perplexity AI’s browser extension, revealing how hidden commands in a Reddit comment could lead the assistant to leak private data [1]. These incidents collectively signal a growing awareness within the tech industry of the need for stronger defenses against AI-based threats.

Source:

[1] 'CopyPasta' Attack Shows How Prompt Injections Could Infect AI Scale (https://decrypt.co/338143/copypasta-attack-shows-prompt-injections-infect-ai-scale)

[2] CopyPasta Attack: Hackers Weaponize AI Coding Tools via Malicious License Files (https://www.indexbox.io/blog/copypasta-attack-hackers-weaponize-ai-coding-tools-via-malicious-license-files/)