Google’s Child Abuse Detection Tools Can Also Identify Illegal Drawings of Children

Google’s Child Abuse Detection Tools Can Also Identify Illegal Drawings of Children

Apple was hit with a wave of criticism earlier this year when it announced plans to scan iPhones to stop the distribution of Child Sexual Abuse Material (CSAM). Critics fretted that Apple’s hash-checking system could be co-opted by governments to spy on law-abiding iPhone users. In response to the backlash, Apple might end up making changes to that program, but Google has its own way of spotting CSAM, and it might be even more intrusive for those who use all of Google’s cloud services.

The specifics on Google’s CSAM scanning come by way of a warrant issued in early 2020 and spotted by Forbes. According to the filing, Google detected CSAM in Google Drive, its cloud storage platform. And here’s where things get a little weird; the warrant stemming from this report targeted digital artwork, not a photo or video depicting child abuse.

Apple’s system under its “Expanded Protections for Children” banner uses hashes for known child abuse materials, scanning iDevices for matching hashes. This should prevent false positives and it doesn’t require Apple to look at any of the files on your phone. The issue cited most often with this approach is that Apple is still scanning your personal files on your smartphone, and it could be a privacy nightmare if someone manages to substitute different hashes. Apple says this isn’t possible, though.

Google, as it turns out, does something similar. It uses a technique initially developed for YouTube to look for hashes of known CSAM, but it also has an AI that has been trained to use machine learning to identify new images of child abuse. It’s not clear how Google spotted the problematic files in 2020, but the unidentified individual is described as an artist. That suggests he is the one who created the drawings at issue, and Google’s systems identified it as CSAM.

Google’s Child Abuse Detection Tools Can Also Identify Illegal Drawings of Children

After Google’s system spotted the drawings, it sent the data to the National Center for Missing and Exploited Children, and from there it went to the DHS Homeland Security Investigations unit. Investigators filed the warrant in order to get access to the user’s data. The artist has not been identified as no charges were ever brought. However, US law holds that drawings, like those depicting child abuse, can still be illegal if they lack “serious literary, artistic, political, or scientific value.” That’s hard to prove — even agreeing on a definition of “art” can be a challenge. This may explain why there were no charges brought in this case.

While Google’s use of AI is more aggressive than Apple’s, it’s also seemingly restricted to cloud services like Gmail and Drive. So, Google isn’t set up for scanning Android phones for hashes like Apple is on the iPhone, but Google’s approach can sweep up original artwork that may or may not be illegal, depending on who you ask. Regardless of what is “art,” Google isn’t doing this just to do it — there is an undeniable problem with CSAM on all cloud services. Google says that it reported 3.4 million pieces of potentially illegal material in 2021, and that’s up from 2.9 million the year before.

Continue reading

It’s Finally Illegal for ISPs to Charge Rental Fees for Equipment You Own
It’s Finally Illegal for ISPs to Charge Rental Fees for Equipment You Own

A new law went into effect on Sunday that prevents your ISP from making equipment rental fees mandatory.

Would Removing the Buffalo Mass Murder Stream Be Illegal Under New Texas Law?
Would Removing the Buffalo Mass Murder Stream Be Illegal Under New Texas Law?

Because today’s mass murderers don’t “just” mass murder, they also have to livestream the incident for the world to see.

Tim Hortons Illegally Collected Location Data, Promises it Won’t Happen Again
Tim Hortons Illegally Collected Location Data, Promises it Won’t Happen Again

Canadian investigators have determined that the company misled its customers and collected a huge volume of location data that revealed where people went, even when they weren't using the app.

FTC Warns Companies ‘Warranty Void if Removed’ Stickers Are Flatly Illegal
FTC Warns Companies ‘Warranty Void if Removed’ Stickers Are Flatly Illegal

The FTC has sent letters to multiple companies, informing them that "void if removed" warranty stickers are illegal and not permitted under federal law.