For years, chatbots have been a useful tool to help automate customer-facing applications. But what happens if the chatbot goes rogue?

Recent reports have revealed that this may have happened to the Comcast / Xfinity chatbot. First, there were incidents of Xfinity email outages. Next, some reported that if you try to resolve the issue via chat, a rogue chatbot may engage with you. The impersonator chatbot then tries to redirect you to a malicious page that asks you to divulge your credit card number.

Could this be related to the massive breach that involved 35.9 million Comcast Xfinity broadband entertainment platform customers? While this story is still developing, it wouldn’t be the first time chatbots were recruited for online scams.

There are a variety of ways chatbots are being used to spread malware and/or obtain sensitive information. Here’s what to watch out for.

Hacking Bing

Bing Chat has quickly become one of the world’s leading AI chatbots. Millions of people use it every day. One feature of Bing Chat is that ads can be inserted into the conversation. For example, a user can hover over a link and then an ad is displayed.

Malwarebytes reported on a case where Bing Chat ads were being hijacked by nefarious actors. In this scam, when the user’s cursor hovers over a legitimate link, a dialog box appears showing a malicious ad:

Image source Malwarebytes

Clicking on the malicious ad leads users to a website (mynetfoldersip[.]cfd) that can identify real victims and filter out bots, sandboxes or security researchers. Filtering works by checking IP addresses, time zones and other system settings, such as web rendering that identifies virtual machines.

Actual human users are eventually redirected to another fake site (advenced-ip-scanner[.]com) that mimics an official page, while others are sent to a decoy page. Victims are then invited to download malware that looks like legitimate software.

Fake AI chatbot scams

Scammers are also taking advantage of the rising popularity of AI chatbots, like Google’s Bard. It’s easy to miss these hacks as they easily blend in with the tsunami of AI-related products and services offered now.

According to Google, two different scammer groups created social media pages and ran ads that encouraged people to “download” Bard. But Bard is a freely available generative AI tool that does not need to be downloaded.

Scammers used Google’s logos, trademarks and product names as part of their scheme. The ads lure targets to a phony website designed to look like it’s affiliated with Google. On the site, visitors are encouraged to download software to use Bard, but it’s really malware.

It’s worth noting that Google is suing the bad actors instead of just reporting them to the authorities. The company says that “lawsuits are an effective tool for establishing a legal precedent, disrupting the tools used by scammers, and raising the consequences for bad actors.” According to Google, they have filed roughly 300 takedowns related to this group of bad actors.

Other chatbot scams

Some chatbot-based scams aren’t really chatbots at all. In one scam, criminals sent phishing emails impersonating DHL.

Image source: Trustwave

From there, the malicious link connected to a fake chatbot that eventually requested sensitive information like the user’s email and password:

Image source: Trustwave

And credit card data, of course…

Image source: Trustwave

A similar scam has also been luring Facebook users under the guise of an account cancellation message. In this case, an actual Facebook chatbot is used, which then redirects targets to a fake site that asks for sensitive information.

Image source: Trustwave

Here’s how they try to get users to give up their passwords:

Image sources Trustwave

Use chatbots with caution

As with any online engagement, interacting with a chatbot should be done with the utmost caution. Always think twice, or even three times, before you click, download or provide private personal information.

To learn how IBM X-Force can help you with anything regarding cybersecurity including incident response, threat intelligence, or offensive security services schedule a meeting here.

If you are experiencing cybersecurity issues or an incident, contact X-Force to help: US hotline 1-888-241-9812 | Global hotline (+001) 312-212-8034.