Internet awash with AI generated child abuse images, IWF warns

A UK-based internet safety organisation has found thousands of AI-generated child abuse images were posted on just one dark web forum last month, with experts warning that it is getting harder to discern ‘real’ images from AI-generated ones.

In a detailed report released today by the Internet Watch Foundation, experts warned that legitimate generative AI software packages were being trained on real child sexual abuse images offline so they could then generate ‘realistic’ AI-generated child sexual abuse material (AI CSAM).

Another worrying trend spotted by IWF analysts was an increase in examples of AI generated images featuring known victims and famous children.

In total over 20,000 AI generated images were found over a one-month period on one single dark web CSAM forum. Of these, 11,000 were selected by IWF under the criteria that they were likely to be criminal.

Twelve IWF analysts dedicated a combined total of almost 90 hours to assessing these selected images – all of which were deemed to be criminal under two UK laws: The Protection of Children Act 1978 and the Coroners and Justice Act 2009.

While these laws were applied, the report acknowledged that genAI has developed so much in the space of a single year, is poses a challenge for governments and lawmakers to keep up.

“Perpetrators can legally download everything they need to generate these images, then can produce as many images as they want – offline, with no opportunity for detection. Various tools exist for improving and editing generated images until they look exactly like the perpetrator wants,” the report noted.

Content moderation

Pages 11 and 12 of the report detail the leading AI image generators MidJourney, OpenAI’s DALL-E and Stability AI’s Stable Diffusion, and it explores the processes by which each company has trained its data sets. This section also examines each firm’s approach to content moderation.

Broadly, content moderation methods of text-to-image models fall into one of two categories.

The first is to restrict training data: models can generate only those things to which they have been exposed. Models will not be able to generate porn, for example, if they have not been exposed to it, although the report adds that this could be made possible by combining various concepts about which it does know.

The second approach genAI companies are taking is by banning certain prompts — restricting the terms that can be used to generate images, perhaps using a keywords list, for instance.

The report noted that since the Stable Diffusion 2.0 was released last Autumn, pornographic content has been excluded from training data.

Asked about why Stability AI was taking this approach the report quotes an alleged response from the tech firm’s CEO Emad Mostaque, who noted that referenced images…

“…could cause legal troubles for all involved and destroy all this. I do not want to say what it is and will not confirm for reasons, but you should be able to guess.”

These kinds of restrictions are having some impact, IWF researchers have noted. Since the latest version of Diffusion was released, they have spotted comments by users on the dark web that it is harder now to produce ‘not safe for work’ content on the AI image generator.

However, the report acknowledges that there’s little that this firm and its rivals can do in practice to prevent their models from generating images that would contravene their terms of use.

Page 22 of the report shows instances where ‘tips and tricks’ have been shared on prompt workarounds while on another forum a guide to creating models using personal CSAM datasets had also been shared.

One AI image generator singled out

Without naming names, the report stated that the ‘overwhelming’ consensus was that one open-source solution has become the main method for generating AI CSAM.

“Most are using [name], an AI art generation tool. To get it to create on topic images you need to run it locally rather than using online tools.”

“[name] created a user interface (WebUI) to make it easier to set up and use [name]. This is what most people are using. His project is relatively simple to set up locally and there are plenty of guides and YouTube videos available.”

IWF’s CEO Susie Hargreaves, noted in the report’s forward that when analysts first saw renderings of AI-generated child sexual abuse material in spring, there were clear ‘tells’ that this material was artificially generated; backgrounds didn’t line up, proportions of body parts were wrong, missing, or clumsy.

“Half a year on, we’re now in a position where the imagery is so life-like, that it’s presenting real difficulties for even our highly trained analysts to distinguish,” she noted.

Hargreaves added that it was concerning to read some of the perpetrator discussions in dark web forums where there appears to be excitement over the advancement of this technology.

“What’s more concerning for me, is the idea that this type of child sexual abuse content is, in some way, ethical. It is not.”

With the UK government soon to host an international summit at Bletchley Park on safety within artificial intelligence, the CEO said that the foundation would be appealing for government, law enforcement and tech companies to take on board the report’s key recommendations.

What government can do

UK government, it stated, needed to acknowledge the challenges of dealing with AI CSAM and make sure it’s in alignment internationally on how this content is treated in different jurisdictions.

It also recommended that the Ministry of Justice commission a review of the laws that apply to the removal of this content online to ensure they are fit for purpose to tackle the threat of AI CSAM.

This should include ensuring the exchange of “hints and tips” and “paedophile manuals” on how to generate this content are made illegal.

“Creating and distributing guides to the generation of AI CSAM is not currently an offence but could be made one. The legal status of AI CSAM models (files used for generating images) is a more complicated question.”

It also asks that appropriate extra resources are given to IWF so that it can scrutinise datasets on which these technologies are trained.

What tech can do

For law enforcement, the watchdog recommends that police are trained and updated to cover AI CSAM and to ensure that there is ‘proper regulatory oversight of AI models before they go to market or are made open source. For closed models, the report adds, protections must be built in.

Recommendations for tech companies include ensuring firms ban the use of their technologies to use and develop child sexual abuse material in their terms and conditions.

It also appeals to search engines like Google to de-index links to fine-tuned AI models known to be linked to the creation of AI CSAM.

While the report paints a bleak picture, Hargreaves asserted her optimism that changes could be made if industry, law enforcement and the government all worked together.

“We’re at the beginning of understanding this technology. Working together, in partnership and collaborating as a sector with industry, law enforcement, government, and with the right level of funding, we might not be reporting in 12 months’ time of how the internet is awash with AI CSAM.”

The IWF boss added that AI might even have a hand in solving the problem. “As usual there is much to do IWF stands ready to overcome the challenges. What AI creates I’m hopeful AI can solve,” she said.

Click here to read more stories on AI