原文 · 未翻译
Google loves telling us all the ways people are using its generative AI products to build new things, grow businesses, and save the world. Supposedly. Of course, people are also using AI for crime. Google has announced a new legal salvo aimed at a Chinese group called Outsider Enterprise, which is allegedly responsible for a massive AI-powered scam campaign. Google says it’s working with law enforcement and mobile carriers to fight back.
According to Google’s legal filing, Outsider Enterprise operates through Telegram. The group offers phishing-as-a-service to individuals who may not be technically savvy enough to set up fraudulent websites and text campaigns on their own. In its Telegram channels, Outsider Enterprise reportedly provided instructions on how to use Google’s Gemini AI to create websites that imitate those of Google, YouTube, and government agencies such as New York’s E-ZPass. The group offered nearly 300 scam templates.
Google says that scams enabled by Outsider Enterprise resulted in more than 2.5 million text messages being sent to Android users. About 55,000 of those messages happened in a two-week period last month. In all, Google has tracked 9,000 fake websites and 1 million URLs connected to the scam network.
The text messages often made claims about account problems or issues with a package delivery. When users clicked on the links, they ended up on one of those fraudulent websites, designed by Gemini to look legitimate. The cybercriminals used these sites to steal personal data and banking details. Google’s filing does not estimate the amount of money stolen through Outsider Enterprise scams, but the blog post notes that hundreds of people have lost some amount of money.
Google worked with AT&T, Verizon, and T-Mobile to block many of these malicious text messages, and Google notes that its on-device scam detection in Google Messages probably helped reduce the number of successful phishing attempts, too. This AI-powered feature apparently stops 10 billion scam texts every month, so it’s fair to expect it caught at least some Outsider Enterprise activity.
Laws for AI threats
Google has filed lawsuits against scammers before, but this is the first time it has taken direct action against a group alleged to be using Gemini as part of its scams. Google discusses the security measures it has baked into Gemini every time it announces a new model, but these can clash with the overarching need for chatbots to follow instructions and please users. And then you end up with thousands of scammers using Gemini to build fake websites.
In addition to its civil lawsuit, Google is assisting the FBI’s cybercrime division with a parallel criminal investigation. However, no one knows who’s behind Outsider Enterprise, and even if Google did have names, there’s little to be done when the perpetrators are in China. The company can go after fraudulent domains and Telegram accounts in hopes of disrupting the Outsider Enterprise operation, but the scams may simply change form.