⚠️ It looks like you're not using Google Chrome. This demo may not work properly. If it breaks, please try in Google Chrome.
⚠️ It looks like you're on a mobile device without 32-bit WebGL. The demo will not work properly. Please try on desktop Google Chrome.
Everything runs client-side – there is no server! Try the demo:
Select a model: 
Original Image
Adversarial Image
This will take a few seconds.
Can you see the difference? View noise.
Noise is magnified. View image.
Prediction
Prediction

What is the demo doing?

Neural networks achieve superhuman performance in many areas, but they are easily fooled.

In the demo above, we can force neural networks to predict anything we want. By adding nearly-invisible noise to an image, we turn "1"s into "9"s, "Stop" signs into "120 km/hr" signs, and dogs into hot dogs.

These noisy images are called adversarial examples. They break the integrity of machine learning systems, and the illusion of their superhuman performance.

Why does this matter?

Our world is becoming increasingly automated, yet these systems have strange failure modes.

If machine learning systems are not properly defended, attackers could:

Is this limited to image classification with neural networks?

No. Adversarial examples exist for almost every machine learning task: speech recognition, text classification, fraud detection, machine translation, reinforcement learning, ....

Moreover, all machine learning models (not just neural networks) are vulnerable. In fact, simpler models such as logistic regression are even more easily attacked.

Finally – beyond adversarial examples – there are many more adversarial attack vectors, including data poisoning, model backdooring, data extraction, and model stealing.

How do I defend against adversarial examples?

There are several proposed defenses, including adversarial training and admission control.

However, no defense is universal and many have proven ineffective, so work with an expert to quantify your risks and invest in defenses appropriately.

(What happens if someone can make your system predict anything they want?).

Where can I learn more?

Here's a list of good resources, in rough order of approachability:

Last – feel free to email me questions.