San Francisco City Attorney David Chiu has sued over a dozen websites that his office says solicit people to create nonconsensual, deepfake nude images of women and girls across the globe.
The lawsuit claims the 16 websites and their owners have violated state and federal laws that prohibit deepfake pornography, revenge pornography and child pornography.
RELATED STORY | Trump shares AI images of a fake Taylor Swift endorsement
According to the city attorney’s office, these websites offer user-friendly interfaces to upload clothed images of people and turn them into AI-generated, “undressed” versions — specifically of women and girls.
“These images, which are virtually indistinguishable from real photographs, are used to extort, bully, threaten, and humiliate women and girls,” the city attorney said in a press release. “Worse yet, victims of nonconsensual deepfake pornography have found virtually no recourse or ability to control their own image after deepfaked images have been distributed.”
The “disturbing phenomenon” of deepfake nude images has notably impacted celebrities, but officials are now concerned about how they are being used to exploit middle school and high school students. The lawsuit references an incident from February where these types of images of eighth-graders were shared around a California middle school.
Concerns have heightened over the widespread availability and popularity of these websites. The city attorney’s office claims, collectively, the websites have been visited over 200 million times in just the first six months of 2024.
RELATED STORY | Scripps News got deepfaked to see how AI could impact elections
“We have to be very clear that this is not innovation—this is sexual abuse. This is a big, multi-faceted problem that we, as a society, need to solve as soon as possible,” said Chiu in a statement. “We all need to do our part to crack down on bad actors using AI to exploit and abuse real people, including children.”
The lawsuit is seeking the permanent removal of 16 AI-generative websites, as well as civil penalties for the costs of bringing the lawsuit to fruition.
Lawmakers have introduced different bills to try to prevent and regulate the use of AI to create deepfake images, mainly explicit ones.
