For years, chatbots have been a useful tool to help automate customer-facing applications. But what happens if the chatbot goes rogue?

Recent reports have revealed that this may have happened to the Comcast / Xfinity chatbot. First, there were incidents of Xfinity email outages. Next, some reported that if you try to resolve the issue via chat, a rogue chatbot may engage with you. The impersonator chatbot then tries to redirect you to a malicious page that asks you to divulge your credit card number.

Could this be related to the massive breach that involved 35.9 million Comcast Xfinity broadband entertainment platform customers? While this story is still developing, it wouldn’t be the first time chatbots were recruited for online scams.

There are a variety of ways chatbots are being used to spread malware and/or obtain sensitive information. Here’s what to watch out for.

Hacking Bing

Bing Chat has quickly become one of the world’s leading AI chatbots. Millions of people use it every day. One feature of Bing Chat is that ads can be inserted into the conversation. For example, a user can hover over a link and then an ad is displayed.

Malwarebytes reported on a case where Bing Chat ads were being hijacked by nefarious actors. In this scam, when the user’s cursor hovers over a legitimate link, a dialog box appears showing a malicious ad:

Image source Malwarebytes

Clicking on the malicious ad leads users to a website (mynetfoldersip[.]cfd) that can identify real victims and filter out bots, sandboxes or security researchers. Filtering works by checking IP addresses, time zones and other system settings, such as web rendering that identifies virtual machines.

Actual human users are eventually redirected to another fake site (advenced-ip-scanner[.]com) that mimics an official page, while others are sent to a decoy page. Victims are then invited to download malware that looks like legitimate software.

Fake AI chatbot scams

Scammers are also taking advantage of the rising popularity of AI chatbots, like Google’s Bard. It’s easy to miss these hacks as they easily blend in with the tsunami of AI-related products and services offered now.

According to Google, two different scammer groups created social media pages and ran ads that encouraged people to “download” Bard. But Bard is a freely available generative AI tool that does not need to be downloaded.

Scammers used Google’s logos, trademarks and product names as part of their scheme. The ads lure targets to a phony website designed to look like it’s affiliated with Google. On the site, visitors are encouraged to download software to use Bard, but it’s really malware.

It’s worth noting that Google is suing the bad actors instead of just reporting them to the authorities. The company says that “lawsuits are an effective tool for establishing a legal precedent, disrupting the tools used by scammers, and raising the consequences for bad actors.” According to Google, they have filed roughly 300 takedowns related to this group of bad actors.

Other chatbot scams

Some chatbot-based scams aren’t really chatbots at all. In one scam, criminals sent phishing emails impersonating DHL.

Image source: Trustwave

From there, the malicious link connected to a fake chatbot that eventually requested sensitive information like the user’s email and password:

Image source: Trustwave

And credit card data, of course…

Image source: Trustwave

A similar scam has also been luring Facebook users under the guise of an account cancellation message. In this case, an actual Facebook chatbot is used, which then redirects targets to a fake site that asks for sensitive information.

Image source: Trustwave

Here’s how they try to get users to give up their passwords:

Image sources Trustwave

Use chatbots with caution

As with any online engagement, interacting with a chatbot should be done with the utmost caution. Always think twice, or even three times, before you click, download or provide private personal information.

To learn how IBM X-Force can help you with anything regarding cybersecurity including incident response, threat intelligence, or offensive security services schedule a meeting here.

If you are experiencing cybersecurity issues or an incident, contact X-Force to help: US hotline 1-888-241-9812 | Global hotline (+001) 312-212-8034.

More from Data Protection

Overheard at RSA Conference 2024: Top trends cybersecurity experts are talking about

4 min read - At a brunch roundtable, one of the many informal events held during the RSA Conference 2024 (RSAC), the conversation turned to the most popular trends and themes at this year’s events. There was no disagreement in what people presenting sessions or companies on the Expo show floor were talking about: RSAC 2024 is all about artificial intelligence (or as one CISO said, “It’s not RSAC; it’s RSAI”). The chatter around AI shouldn’t have been a surprise to anyone who attended…

3 Strategies to overcome data security challenges in 2024

3 min read - There are over 17 billion internet-connected devices in the world — and experts expect that number will surge to almost 30 billion by 2030.This rapidly growing digital ecosystem makes it increasingly challenging to protect people’s privacy. Attackers only need to be right once to seize databases of personally identifiable information (PII), including payment card information, addresses, phone numbers and Social Security numbers.In addition to the ever-present cybersecurity threats, data security teams must consider the growing list of data compliance laws…

How data residency impacts security and compliance

3 min read - Every piece of your organization’s data is stored in a physical location. Even data stored in a cloud environment lives in a physical location on the virtual server. However, the data may not be in the location you expect, especially if your company uses multiple cloud providers. The data you are trying to protect may be stored literally across the world from where you sit right now or even in multiple locations at the same time. And if you don’t…

Topic updates

Get email updates and stay ahead of the latest threats to the security landscape, thought leadership and research.
Subscribe today