Introduction
A recent global sting operation launched by Europol has led to at least 25 arrests in relation to artificially generated child sexual abuse material. Developments in technology are often accompanied by increased risk, opportunity and means for abuses of human rights. This is particularly evident in the case of child sexual abuse (CSA) and child sexual abuse material (CSAM).
In 2023, the National Centre for Missing and Exploited Children’s (NCMEC) Cyber Tipline received over 36.2 million reports of suspected child sexual exploitation, the majority of which relating to the dissemination of CSAM. Although NCMEC is based in the United States (US), 90% of CSAM reports were for content uploaded outside of the US. NCMEC has emphasised its concern regarding the growing trend in utilising generative artificial intelligence (GAI) to generate CSAM using the videos or images of real children or computer-generated children engaged in acts of a sexual nature. Addressing technology-assisted CSA requires a holistic approach, incorporating elements of criminalisation and prevention. Furthermore, the roles and responsibilities of different actors, including technology companies and States need affirming.
In this blog I outline State obligations and the responsibility of digital services providers arising from various international law treaties and legal instruments in relation to online CSA within the European Region. I focus mainly on States that are both EU member States and members of the Council of Europe. From the perspective of the Council of Europe, I will give due consideration to the Council of Europe Convention on the Protection of Children against Sexual Exploitation and Sexual Abuse (the Lanzarote Convention), which targets the prevention and combatting of child sexual abuse and exploitation. This multinational treaty has been ratified and is binding on all Council of Europe State Parties. In addition to the Lanzarote Convention, I also consider obligations arising from the EU legal framework, stemming from the Digital Services Act (DSA), Directive 2011/93/EU and the Proposed Regulation to Prevent and Combat Child Sexual Abuse (Proposed Regulation). The Digital Services Act is an EU Regulation and thus applies directly to all EU Member States without transposition into the Member States national legal systems. The Proposed EU Regulation, if entered into force, will have a similar status. Directive 2011/93/EU, however, applies to all EU Member States but the obligations therein need to be transposed into each Member States national legal system to take effect.
State Responsibility: Criminalisation, Prevention and Intervention
Criminalisation of online CSA offences tends to be the starting point when considering State responsibility. This can be illustrated by the existing international law mandating the criminalisation of ‘child pornography’. Although the term ‘child pornography’ is used frequently in treaties and legislation, this terminology is problematic as it does not adequately communicate the seriousness of the offense and implies consent. As such, in this blog I use the preferred term CSAM.
States that are simultaneously parties to the Lanzarote Convention within the Council of Europe framework and member States of the EU, under Directive 2011/93/EU, are obligated to criminalise the production, dissemination and possession of CSAM under article 20 and article 5 respectively. A shortcoming of these criminalisation provisions, however, is that although AI-generated CSAM can be construed as falling within the definitions of CSAM, both the Lanzarote Convention and Directive 2011/93/EU allow for States to opt-out of criminalising CSAM of this nature. In combination with the fact that no broader international treaty obliges States to criminalise AI-generated CSAM specifically, a clear gap exists.
Specific States have taken steps of their own initiative to address this regulation gap. A good example of this is the United Kingdom (UK). Although the UK is no longer a Member State of the EU, it is a party to the Council of Europe framework and has ratified the Lanzarote Convention. The UK government has expressed its intention to introduce new measures in a Crime and Policing Bill which will criminalise the possession, creation and distribution of AI tools used to generate this material as well as the possession of AI ‘paedophile manuals’ that demonstrate how to utilise AI to engage in CSA. Although this is a positive step (yet isolated), the borderless nature of online CSA and CSAM offences facilitated by technology requires a broader international response. The recent AI CSAM sting highlights this shortcoming, with Europol expressing that the absence of national legislation in these cases made the investigation exceptionally challenging. Given the general absence of targeted criminal law provisions in most European States, these challenges at the investigation stage will most probably extend to the prosecution. Indeed, it will be particularly interesting to see the future development and prosecution of the cases relating to this sting since they are likely to bring new light to the prosecution and grounding in criminal law of AI CSAM related offences. Ultimately, however, criminalisation in itself is insufficient to address online CSA as prevention needs to be central to ongoing international efforts.
Both the Lanzarote Convention and Directive 2011/93/EU create obligations for States to take action against CSA beyond its criminalisation. The Lanzarote Convention obliges State Parties to engage in several preventative measures, including training and awareness for those working with children, education of children, encouraging the participation of the private sector and utilising effective intervention measures aimed at assisting potential perpetrators. Directive 2011/93/EU similarly obliges Member States to take measures aimed at awareness, education and intervention as well as measures against websites that disseminate or contain CSAM. Within the EU Framework, this reflects increased responsibility being placed on technology companies and digital service providers controlling platforms on which online CSA, including the dissemination and possession of CSAM, may occur.
The Responsibility of Digital Service Providers in combating online CSA
Technologies have facilitated and contributed to the availability, opportunity and accessibility of CSA and the dissemination of CSAM. Beyond image-based offences, technology has also expanded the spaces in which children are present and can be contacted by potential offenders. These online spaces include social media, chatrooms and gaming forums. A study on sexual harm in the gaming, esports and video community, found that offenders targeting children in the study sample minimised their digital footprint and encouraged children to use communication methods like video and voice calls. The study theorised that this is to ensure that less evidence of sexual offending exists digitally. This further highlights the need for methods of prevention and protection that takes these nuances into account.
Several instruments in the EU are of direct relevance to technology companies and digital services providers, including the DSA. The DSA obliges platforms to promptly remove access to illegal content such as CSAM (art 6), mandates the existence of effective reporting systems (art 16), requires that law enforcement or judicial authorities of a member state are promptly informed of threats to a safety of a person (art 18), and requires Very Large Online Platforms to conduct risk assessments (art 34). Additionally, the Proposal for an EU Regulation to Prevent and Combat Child Sexual Abuse emphasise the important role of interpersonal communication service and hosting service providers in facilitating a trusted and safe online space and upholding fundamental. The Proposal acknowledges that certain service providers use and have used technologies voluntarily to detect, remove and report online CSA. However, these measures vary widely, and a significant number of providers have taken no action. The Proposed EU Regulation, if implemented in its current form, would mandate service providers to conduct impact assessments and, subject to a detection order, use scanning technologies to detect potential identified and unidentified CSAM. The Proposal has been criticised for its implications on privacy rights. What it does highlight, however, is that the existing regulation and reliance on voluntary action of hosting and digital service providers is insufficient. The obligations of actors controlling digital spaces needs to be clear, not only in the protection of privacy rights but other impacted fundamental rights. Further research should assess the balance of these interests when they come into conflict and how such a balance can be maintained in regulation.
Conclusion
The existing international legal framework already demonstrates the multifaceted nature of online CSA. The Lanzarote Convention and Directive 2011/93/EU enshrine State obligations in their respective frameworks ranging from criminalisation to prevention through education and support for potential perpetrators. The EU’s DSA further represents the developing responsibilities of various technology service providers and online platforms to report online CSA and remove illegal content such as CSAM. States that are only within the CoE will have less obligations to effectively tackle the different forms of online CSA. Understanding the current obligations and responsibility of States and technology companies is a starting point; however, it is important to also look to the future. Online CSA is a borderless, multifaceted crime that is increasingly facilitated by technology. As technology continues to develop, so too will the opportunities for this form of abuse. This is reflected in the current gaps in regulation such as the absence of an explicit and specific obligations to criminalise AI-generated CSAM in the Council of Europe and EU legal frameworks.
How then do we facilitate laws that are future-oriented and are able to incorporate new and evolving technological developments whilst providing clear obligations and responsibility for States and entities controlling online spaces? Can technology be viewed not only as a facilitator of abuse but a tool to combat it? The EU Proposed Regulation demonstrates a move to both use and mandate the use of certain technologies, like CSAM scanning, as a way to combat CSA in online spaces. The difficulty that then arises is how conflicting rights and interests will be balanced. What is clear, and should be a central consideration, is that children have a right to be free from sexual violence, including in online spaces. The responsibility to ensure that this right is upheld cannot be shouldered by children and caregivers solely. The input of different actors is necessary to ensure, technology informed and empowered Regulation that creates obligations which are effective in their purpose to prevent CSA both online and offline.
Cayla Mavourneen Miller is a Masters Student in International Law and Human Rights at Abo Akademi
Photo by Ron Lach from Pexels