Tumblr is currently using a neural network to identify images that contain explicit media. Neural networks are extremely susceptible to something called adversarial examples.
Adversarial examples are inputs to a neural network that result in an incorrect output from the network. It’s probably best to show an example. You can start with an image of a panda on the left which some network thinks with 57.7% confidence is a “panda.” The panda category is also the category with the highest confidence out of all the categories, so the network concludes that the object in the image is a panda. But then by adding a very small amount of carefully constructed noise you can get an image that looks exactly the same to a human, but that the network thinks with 99.3% confidence is a “gibbon.” Pretty crazy stuff!
Check that link if you want to know more.
-I'm not advocating any rule breaking, I am pointing out how underdeveloped this technology is and why it should not be rolled out on such a big scale.
-This also applies to audio, meaning that smart speaker in your house is less secure than you have been led to believe.