xAI 如今被指控使用儿童性虐待材料（CSAM）训练 Grok，与此同时监管机构和法院持续调查该问题的严重程度，部分 Grok 用户已被逮捕。
在周三提交的一份诉状中，一位化名为 Jane Doe 的原告解释称，本世纪初她尚处学龄前年龄时，成年男性多次强奸她以制作 CSAM，并在网上出售给恋童癖者。自那以后，Doe 的影像已被国家失踪与受虐儿童中心（NCMEC）和加拿大儿童保护中心（CCCP）等机构进行哈希处理。
出于安全考虑，Doe 选择接收美国司法部受害者通知系统的提醒，以便在任何新的刑事调查中她可能成为受害者时及时获知。尽管她收到了无数提醒，但当 CCCP 通知她已在 xAI 上识别出描绘她的 AI 生成 CSAM 时，她仍感到震惊。这给 Doe 造成了二次创伤，其诉状称，在线论坛上发现了“犯罪者之间讨论制作原告及其他处境类似的、已知的、遗留的 CSAM 受害者的 AI 生成 CSAM”的聊天记录。
Ars Video
《木卫四协议》中的灯光设计如何提升恐怖氛围
如今，Doe 担心 xAI 不仅让制作更多关于她人生中最痛苦时期的违规影像变得更加容易，而且据称 xAI 还存储了 Grok 生成的图像，并使用这些输出进一步训练 Grok。因此，她认为 Grok 既接受了那批困扰她 20 多年的原始影像的训练，也接受了近期 AI 生成影像的训练。
这是首起指控 xAI 使用儿童性虐待材料（CSAM）进行训练的案例，而诉状对这一指控并未展开过多细节。此前，Ars 曾报道过一个备受争议的数据集，该数据集在研究人员发现训练数据中含有 CSAM 后被清理，但没有迹象表明 xAI 使用过该数据集进行训练。在代表 Doe 的律师发布的新闻稿中，说明了 Doe 的影像被收录在 NCMEC 维护的 CSAM 哈希列表中，且据称“同一材料”“是 xAI 用于构建 Grok 图像和视频生成能力的数据集的一部分”。诉状同样仅指控“描绘原告的 CSAM 及其长期已知的哈希值已被用作 xAI 所用数据集的一部分”。
不过，关于 Grok 使用 AI 生成的 CSAM 进行训练的指控，则披露了更多细节。
“由于 Grok 的服务条款默认将公开的 X 帖子和 Grok 自身的输出视为训练数据，公开发布图像不仅会暴露给观看者，还会直接将其输入 xAI 用于训练和改进模型、进而生成更多图像的流水线，”Doe 的诉状称。
该诉讼进一步指出，虽然 xAI 会过滤 Grok 输出中的暴力内容以将其排除在训练数据之外，但 xAI 的条款明显未明确说明 CSAM、非自愿私密影像（NCII）或 NSFW 内容是否属于“排除类别”。
由此看来，“由于从已训练模型中完全移除某个训练样本的影响在技术上十分困难，且 xAI 并未公开声称已做到这一点，因此在删除前被纳入训练的任何 CSAM，即使在原始图像从公众视野中移除后，仍可能继续影响模型的输出，”Doe 的诉状称。
Doe 要求彻底清除 Grok 中的全部 CSAM
在她提出的集体诉讼中，Doe 寻求终结 Grok 的有害输出。若获批准，该集体将代表每一位童年影像被用于生成 Grok CSAM 的受害者。
该诉讼指控 X 违反联邦儿童色情法律以及 Masha's Law，这两项法律均赋予 CSAM 幸存者就制作、持有和传播行为提起诉讼的权利。Doe 的律师 Margaret E. Mabie 在新闻稿中表示，这三项均属犯罪行为，且“xAI 全都做了”。
如果 Doe 胜诉，xAI 可能需要对每一位能够证明 Grok 基于其真实照片生成 CSAM 的受害者承担金钱赔偿。Doe 还请求法院责令 xAI 销毁其可能存储在服务器上并用于训练 Grok 模型的所有 Grok 生成的 CSAM。此外，她希望 xAI 阻止 Grok 生成任何 CSAM，其诉状指出，这将要求阻止 Grok 输出任何性化内容，包括 Musk 曾推广的 NCII 和 NSFW“比基尼照片”。
Doe 的律师之一 Sarah London 在新闻稿中表示，Doe“近二十年来一直生活在这样的阴影下：她人生中最糟糕经历的相关图片正在网上于恋童癖者之间流传，并且随时可能再次浮出水面。xAI 必须为其明知故犯地使用她所遭受的可怕虐待图片以及本集体中每一位其他幸存者的虐待图片来训练模型而承担责任。”
X 未回应 Ars 的置评请求。
1.新 Twitter 上线，称 Musk 的 X 放弃了这一名称- 2.旨在取代 Meta 员工的 AI 智能体做出了“大规模、破坏性行为”
- 3.Claude、Codex 和 Hermes 在企业网络内安装了无主代码
- 4.佛罗里达州天主教徒拒绝宗教疫苗豁免，驳斥州总检察长
- 5.安息吧，Tim Curry：Ars 回顾他的十大标志性表演
xAI has now been accused of training Grok on child sex abuse materials (CSAM), as regulators and courts continue to probe how far the problem goes, and some Grok users have been arrested.
In a complaint filed on Wednesday, a plaintiff known as Jane Doe explained that she was preschool-age in the early 2000s when adult men repeatedly raped her to create CSAM to sell to pedophiles online. Since then, Doe’s images have been hashed by groups like the National Center for Missing and Exploited Children (NCMEC) and the Canadian Centre for Child Protection (CCCP).
For her safety, Doe has opted to receive alerts from the US Department of Justice Victim Notification System any time she may be a victim in a new criminal investigation. Although she has received countless alerts, she was shocked when the CCCP notified her that it had identified AI-generated CSAM on xAI that depicted her. This re-traumatized Doe, whose complaint alleged that messages were found on online forums “between offenders chatting about creating AI generated CSAM of Plaintiff and other similarly situated known, legacy, victims of CSAM.”
Ars Video
How Lighting Design In The Callisto Protocol Elevates The Horror
Now, Doe fears that xAI has not only made it easier to make more violative images of the most distressing time in her life, but also that xAI allegedly has stored the images that Grok generates and uses those outputs to further train Grok. Because of this, she believes that Grok has been trained on both the initial set of images that have haunted her for more than 20 years and the more recent AI-generated ones.
This is the first case to accuse xAI of training on CSAM, and the complaint does not go into great detail on that claim. Previously, Ars reported on a controversial dataset that was later scrubbed after researchers found CSAM in the training data, but there’s no indication xAI trained on that data. In a press release from lawyers representing Doe, it’s explained that Doe’s images were included in a CSAM Hash List maintained by NCMEC, and “that same material” allegedly “was part of the dataset xAI used to build Grok’s image and video generating capabilities.” The complaint similarly only alleged that “CSAM depicting Plaintiff with its longstanding well-known hash values has been used as a part of the dataset used by xAI.”
More detail is shared on claims alleging that Grok trains on AI CSAM, however.
“Because Grok’s terms treat public X posts and Grok’s own outputs as training data by default, publicly posting an image does not just expose it to viewers, but also feeds [it] directly into the pipeline xAI uses to train and improve its model and thereby generate further images,” Doe’s complaint said.
The lawsuit further noted that while xAI filters out violent content in Grok outputs to exclude it from training data, xAI’s terms notably do not specify if CSAM, non-consensual intimate imagery (NCII), or NSFW material are “excluded categories.”
It seems to follow then that “because full removal of a training example’s influence from an already-trained model is technically difficult and not something that xAI has publicly claimed to have done, any CSAM ingested into training before takedown likely continued to shape the model’s outputs even after the original images were removed from public view,” Doe’s complaint said.
Destroy all Grok CSAM, Doe says
In her proposed class action, Doe seeks to put an end to Grok’s harmful outputs. If approved, the class would represent every victim whose childhood images have been used to generate Grok CSAM.
The lawsuit accuses X of violating federal child pornography laws, as well as Masha’s Law, both of which give CSAM survivors a right to sue on claims of production, possession, and distribution. A lawyer for Doe, Margaret E. Mabie, suggested in the press release that all three are crimes and “xAI did all three.”
If Doe wins, xAI could owe money damages to every victim who can prove that Grok generated CSAM based on their real photos. Doe’s also asked the court to order xAI to destroy all Grok-generated CSAM that it may be storing on its servers and using to train Grok models. Additionally, she wants xAI to block Grok from ever generating CSAM, which her complaint suggested would require blocking Grok from any sexualized outputs, including NCII and NSFW “bikini pics” that Musk has promoted.
Sarah London, one of Doe’s lawyers, said in the press release that Doe “has lived for nearly two decades knowing that images of the worst thing that ever happened to her are circulating among predators online, and that they can resurface at any moment. xAI must be held responsible for knowingly training its models on images of the horrific abuse she suffered, and on the abuse images of every other survivor in this class.”
X did not respond to Ars’ request to comment.
- 1.New Twitter launches, says Musk's X gave up the name
- 2.AI agents meant to replace Meta workers made “large-scale, disruptive actions”
- 3.Claude, Codex, and Hermes installed unowned code inside corporate networks
- 4.Florida Catholics slap down state AG by rejecting religious vaccine exemptions
- 5.RIP, Tim Curry: Ars remembers his top 10 iconic performances