About a year ago, Apple announced three new child protection features: a system to detect known child sexual abuse material (CSAM) in iCloud Photos, an option to blur sexually explicit photos in the Messages app, and resources for Siri on child abuse. The latter two features are now available, but Apple has remained silent on the CSAM detection feature in iCloud Photos.
Apple originally stated that CSAM detection would be implemented in an update for iOS 15 and iPadOS 15 by the end of 2021. However, the company ultimately postponed the feature due to “feedback from customers, stakeholders, researchers, and others.” In September 2021, Apple released the following Update on its child safety page:
We previously announced plans for features to help protect children from predators who use communications to recruit and exploit them and to limit the spread of child sexual abuse material. Based on feedback from customers, stakeholders, researchers and others, we have decided to take more time in the coming months to gather input and make improvements before releasing these critically important child safety features.
CSAM in iCloud Photos: Apple's plans remain unclear
In December 2021, Apple removed the above update and all references to its CSAM detection plans from its child safety page. An Apple spokesperson shared Apple told The Verge that Apple's plans for the feature haven't changed. However, as far as we know, Apple hasn't publicly commented on the plans since then. Apple introduced parental control features for the Messages app and Siri with the release of iOS 15.2 and other software updates in December 2021, and expanded them to Australia, Canada, New Zealand, and the UK with iOS 15.5 and other software updates in May 2022. Apple said the CSAM detection system was "designed with user privacy in mind." The system performs an "on-device check against a database of known CSAM image hashes" from child protection organizations, which Apple converts into an "unreadable set of hashes securely stored on users' devices."
Apple's plans sparked much criticism
Apple planned to report iCloud accounts with known CSAM image hashes to the National Center for Missing and Exploited Children (NCMEC), a nonprofit organization that works with law enforcement. Apple said there would be a "threshold" to ensure the probability of an account being mislabeled by the system is "less than one in a trillion," as well as a manual human review of flagged accounts. Apple's plans were criticized by a variety of individuals and organizations, including security researchers, the Electronic Frontier Foundation (EFF), politicians, political groups, university researchers, and even some Apple employees. Some critics argued that Apple's child safety features could create a "backdoor" into devices that governments or law enforcement could use to monitor users. Another concern was that false positives could occur if someone intentionally added CSAM images to another person's iCloud account to flag their account. (Photo by DenPhoto / Bigstockphoto)
