"AI Chatbot Gemini: Under Siege by Global Threat Actors"

Google's threat intelligence team has published a report titled "Adversarial Misuse of Generative AI," revealing attempts by threat actors, including government-backed advanced persistent threat (APT) groups, to misuse the company's AI chatbot, Gemini. The report highlights the potential for AI misuse and the importance of robust security measures to protect these innovations.

According to the report, threat actors have attempted to "jailbreak" Gemini using prompts, aiming to make the AI perform tasks it was prohibited from doing. These attempts included leaking sensitive information or providing unsafe content. However, Google stated that these attempts were unsuccessful, with hackers only using basic measures like rephrasing or repeatedly sending the same prompt.

In one instance, an APT actor attempted to trick Gemini into performing malicious coding tasks using publicly available prompts. However, the AI provided a safety-filtered response, and the attempt was unsuccessful.

The report also detailed how government-backed APTs have approached Gemini to assist in their malicious activities. These include information gathering on targets, researching publicly known vulnerabilities, and conducting coding and scripting tasks. Additionally, there have been attempts to enable post-compromise activities like defense evasion.

Iran-based APT actors focused on using AI in crafting phishing campaigns, conducting recon on defense experts and organizations, and generating cybersecurity content. Meanwhile, China's APT actors used Gemini to troubleshoot code, scripting, and development, as well as researching deeper access to target networks. North Korean APT actors also used Gemini for various phases of their attack lifecycle, including research on topics of strategic interest to the North Korean government.

The report serves as a reminder of the potential for AI misuse and the importance of implementing strong security measures to protect these innovations. As AI continues to advance, it is crucial for companies and governments to stay vigilant and adapt their security strategies to address emerging threats.