Unit 7. Adversarial examples and evasion

This unit is about attacking machine learning models at test time and about evaluating how well they resist attack. It opens with threat models, the three dimensions (goal, knowledge, capability) that any security claim about a model must specify, in both the academic framework of Biggio and Roli and the operational vocabulary of MITRE ATLAS. It then develops gradient attacks, from the single-step fast gradient sign method (FGSM) to projected gradient descent (PGD), with a worked example you can check by hand, and explains why the image notion of an imperceptible perturbation does not carry over to security, where an attack must preserve function. It closes with how to evaluate robustness so that a weak attack is not mistaken for a strong defense, covering robustness curves, the sanity checks that catch a broken evaluation, gradient masking, adaptive attacks, and what a credible robustness claim must report.

After this unit you will be able to state a threat model for a given scenario, implement FGSM and PGD by hand with correct projection and clipping, read and draw robustness curves, recognize the signs of gradient masking, and measure evasion under realistic problem-space constraints rather than in feature space under a norm bound alone. The unit assumes the neural network training of Unit 5 and the gradient-based optimization and evaluation metrics of the earlier units. Plan about 8 to 10 hours including the two labs and the exercises. Unit 8 continues with poisoning, privacy attacks, and the defenses that hold up.

Lessons

1. 7.1 Threat models for machine learning
2. 7.2 Gradient attacks, from FGSM to PGD
3. 7.3 Evaluating robustness

Labs

Practice

• Exercises with worked solutions
• Self-check quiz