According to a document that was leaked from the EU Commission, the executive arm of the EU, the artificial intelligence it intends to use to systematically monitor private chats for “grooming” content is predicted to mistakenly flag content for EU investigators 10% of the time.
The EU Commission is pushing for this broad surveillance of online discussions, known as “Chat Control,” as a means of preventing content that promotes child sexual abuse (CSAM). However, the EC acknowledged that its suggested surveillance tactics would produce a significant number of false alarms in a leaked document that Netzpolitik was able to collect and publish, Infowars reports.
The EC acknowledged in the document that grooming detection technology had an accuracy of about 90%. This signifies that nine out of ten contents that the system has identified are grooming.
The leaked document contains the EU Commission’s responses to a number of inquiries from the German government on Chat Control’s implementation.
Private chats, messages, and emails will be automatically checked by AI for suspicious information as part of the current Chat Control plans. The content will be flagged and sent to investigators at a future EU center if the AI identifies any suspicious elements. These investigators will go over the content, spot any false positives, and send any illegal material to the appropriate law enforcement agencies, including the EU’s Europol.
The leaked paper refers to Europol as “one of the most important partners” for the center and adds that “close cooperation is essential” despite the fact that law enforcement agencies do not have direct access to anything that is reported and transmitted to this proposed EU center. The letter also states that it’s crucial for Europol to receive all information from the center.
Patrick Breyer, a German member of the EU Parliament, issued a warning, estimating that a 10% error rate would result in an annual disclosure of more than three million chats and photographs to EU investigators. The EC’s predicted mistake rate, he continued, is probably too low.
Not only did the EC acknowledge that its system would have a 10% error rate, but it also noted that encryption falls under the scope of Chat Control and that companies will be required to use technology to comply with its requirements. This appears to be a reference to client-side scanning technology which essentially breaks end-to-end encryption in chat apps by scanning the content of messages before they’re encrypted.