Slides
In the last few years machine learning has become ubiquitous, machine learning models have been
used to solve complex problems in wide range of areasincluding computer vision, natural language
processing, computational biology, etc. and also sensitive applications such as malware detection,
autonomous driving, computational finance. As a result, there is growing recognition that machine
learning exposes new security issues in software systems.
Machine learning models are shown to be vulnerable to adversarial examples – malicious inputs
which are crafted by the adversary to induce the trained model to produce erroneous outputs.
Adversarial examples can be used to bypass malware detection, subvert fraud detection or mislead
the autonomous navigation systems. Furthermore, adversarial examples that affect one model often
affect another model, as long as both models were trained to perform the same task. An attacker
may therefore conduct an attack with minimal or no knowledge about the victim model, by training
their own substitute model to craft adversarial examples and using those examples to exploit victim
model. The attacker need not even have access to the data used to train the deployed machine
learning model. Indeed, we demonstrate how adversaries may use the victim model as an oracle to
label a ghost training set for training the substitute model.
In this tutorial, we first present a comprehensive taxonomy of threat models for machine learning
systems, and then present wide variety of attack against these models followed by strategies to
defend against these attacks. In addition to the above we would also illustrate that there are
(possibly unavoidable) tensions between model complexity, accuracy, and resilience that must be
calibrated for the environments in which they will be used. We then conclude the tutorial by
providing a list of interesting open challenges in adversarial machine learning.
Dr. Sameep Mehta
Dr. Atul Kumar
Dr. Deepak Vijaykeerthy
