In the last few years machine learning has become ubiquitous, machine learning models have been used to solve complex problems in wide range of areasincluding computer vision, natural language processing, computational biology, etc. and also sensitive applications such as malware detection, autonomous driving, computational finance. As a result, there is growing recognition that machine learning exposes new security issues in software systems.

Machine learning models are shown to be vulnerable to adversarial examples – malicious inputs which are crafted by the adversary to induce the trained model to produce erroneous outputs. Adversarial examples can be used to bypass malware detection, subvert fraud detection or mislead the autonomous navigation systems. Furthermore, adversarial examples that affect one model often affect another model, as long as both models were trained to perform the same task. An attacker may therefore conduct an attack with minimal or no knowledge about the victim model, by training their own substitute model to craft adversarial examples and using those examples to exploit victim model. The attacker need not even have access to the data used to train the deployed machine learning model. Indeed, we demonstrate how adversaries may use the victim model as an oracle to label a ghost training set for training the substitute model.

In this tutorial, we first present a comprehensive taxonomy of threat models for machine learning systems, and then present wide variety of attack against these models followed by strategies to defend against these attacks. In addition to the above we would also illustrate that there are (possibly unavoidable) tensions between model complexity, accuracy, and resilience that must be calibrated for the environments in which they will be used. We then conclude the tutorial by providing a list of interesting open challenges in adversarial machine learning.

Dr. Sameep Mehta

IBM Research, India

Dr. Atul Kumar

IBM Research, India

Dr. Deepak Vijaykeerthy

IBM Research, India