Gemini in Gmail Vulnerable to Prompt Injection-Based Phishing Attacks, Researcher Finds

Gemini in Gmail is susceptible to immediate injection-based phishing assaults, a researcher demonstrated. As per the researcher, the synthetic intelligence (AI) chatbot that provides options similar to electronic mail abstract technology and electronic mail rewriting could be manipulated into displaying phishing messages to customers. This vulnerability poses a big threat, as attackers might probably exploit it to conduct on-line scams. Meanwhile, the Mountain View-based tech large has reportedly mentioned that it has thus far not seen this manipulation method used in opposition to customers.

Researcher Claims Gemini in Gmail Is Vulnerable to Prompt Injection

The vulnerability was noticed and demonstrated by researcher Marco Figueroa, GenAI Bug Bounty Programmes Manager at Mozilla, through Mozilla’s bug bounty programme for AI instruments, 0din. Interestingly, to set off this vulnerability, the scammer doesn’t have to pull off any high-profile cyber heist. Instead, it may be carried out with a easy textual content command utilizing a method referred to as immediate injection.

Prompt injection is a kind of assault on AI chatbots the place an attacker intentionally manipulates the enter or immediate to make the mannequin behave in unintended or malicious methods. In this specific state of affairs, the researcher used oblique immediate injection, the place the malicious immediate is embedded inside a doc, electronic mail, or an online web page.

As per the researcher, he merely wrote an extended electronic mail and added some hidden textual content on the finish, which contained the immediate injection. The electronic mail didn’t comprise any URLs or attachments, which made it simpler to attain the receiver’s main inbox.

Adding a hidden malicious message in electronic mail
Photo Credit: 0din/Marco Figueroa

As proven in the picture, the attacker used a white color font on a white web page to write the malicious message. This textual content is generally invisible to the receiver of the e-mail. Other methods to add hidden textual content embody utilizing a zero font dimension, off-screen textual content placement, and different HTML or CSS methods.

Now, if the receiver makes use of Gemini’s “summarise email” function, the chatbot will course of the hidden textual content and perform the command, with out the person ever discovering out, Figueroa mentioned. He additionally highlighted that the likelihood of the chatbot following the command will increase if the message is wrapped inside an admin tag, because it considers it a high-priority request.

Gemini verbatim repeats the malicious message in the abstract
Photo Credit: 0din/Marco Figueroa

The cybersecurity researcher confirmed in one other screenshot that Gemini certainly carried out the malicious message and displayed it as a part of its electronic mail abstract. Since the message is now coming from Gemini, as a substitute of an electronic mail from a probable stranger, the sufferer could possibly be extra probably to consider it and comply with the directions, falling for the rip-off.

BleepingComputer reached out to Google to ask in regards to the vulnerability, and a spokesperson mentioned that the corporate has seen no proof of comparable manipulation thus far. Additionally, it was additionally highlighted that Google is in the method of implementing some mitigations for immediate injection-based adversarial assaults.