🔗 Share this article

Tech firms and child protection organizations will be granted authority to assess whether AI tools can generate child exploitation images under recently introduced UK legislation.

Significant Rise in AI-Generated Illegal Material

The declaration coincided with revelations from a safety watchdog showing that cases of AI-generated CSAM have more than doubled in the last twelve months, rising from 199 in 2024 to 426 in 2025.

Updated Legal Structure

Under the changes, the authorities will permit designated AI companies and child safety groups to examine AI systems – the foundational technology for conversational AI and image generators – and verify they have sufficient safeguards to prevent them from creating depictions of child sexual abuse.

"Fundamentally about preventing abuse before it occurs," declared the minister for AI and online safety, adding: "Experts, under rigorous protocols, can now identify the risk in AI systems early."

Tackling Regulatory Obstacles

The amendments have been implemented because it is illegal to produce and own CSAM, meaning that AI developers and other parties cannot create such images as part of a testing process. Until now, officials had to wait until AI-generated CSAM was uploaded online before dealing with it.

This law is aimed at averting that issue by enabling to halt the creation of those materials at their origin.

Legal Structure

The changes are being added by the authorities as modifications to the crime and policing bill, which is also implementing a prohibition on possessing, creating or sharing AI models developed to generate exploitative content.

Practical Consequences

This recently, the official toured the London headquarters of a children's helpline and listened to a mock-up conversation to counsellors featuring a account of AI-based exploitation. The interaction portrayed a adolescent seeking help after being blackmailed using a explicit AI-generated image of himself, constructed using AI.

"When I hear about young people experiencing blackmail online, it is a source of extreme frustration in me and justified anger amongst parents," he stated.

Alarming Data

A prominent online safety organization reported that instances of AI-generated exploitation material – such as online pages that may contain numerous images – had significantly increased so far this year.

Instances of category A content – the most serious form of exploitation – rose from 2,621 images or videos to 3,086.

• Girls were overwhelmingly targeted, accounting for 94% of illegal AI depictions in 2025
• Portrayals of infants to toddlers rose from five in 2024 to 92 in 2025

Industry Reaction

The law change could "represent a crucial step to guarantee AI products are secure before they are launched," stated the head of the online safety foundation.

"AI tools have made it so victims can be victimised repeatedly with just a simple actions, giving criminals the capability to create possibly endless amounts of advanced, photorealistic child sexual abuse material," she added. "Content which additionally commodifies victims' trauma, and renders children, particularly female children, more vulnerable both online and offline."

Counseling Interaction Information

Childline also published information of counselling interactions where AI has been referenced. AI-related risks discussed in the conversations comprise:

• Using AI to evaluate weight, physique and looks
• Chatbots dissuading young people from consulting trusted adults about harm
• Facing harassment online with AI-generated material
• Digital blackmail using AI-manipulated images

Between April and September this year, the helpline conducted 367 support interactions where AI, conversational AI and associated topics were mentioned, four times as many as in the same period last year.

Half of the mentions of AI in the 2025 interactions were connected with mental health and wellness, including utilizing AI assistants for assistance and AI therapy applications.