Unit 8. Poisoning, privacy and defenses

Dr. Ali Al Bataineh, Higher Colleges of Technology, United Arab Emirates

Unit 7 attacked models at test time by changing their inputs. This unit covers attacks through a model's training data and attacks that use query access to copy a model or learn about its training data, together with the defenses that rest on firm ground. Lesson 8.1 shows how an attacker who influences training data, labels or a pretrained model can degrade a detector, change its output on chosen inputs, or plant a backdoor that lets triggered attacks through while clean performance stays intact, and why the defenses that search for poisoned records often fail on security data. Lesson 8.2 turns to query access, covering the copying of a model through its prediction API, deciding whether a record was in the training set, and the separate attacks that reconstruct training data, with the evaluation at low false positive rates that privacy requires. Lesson 8.3 develops adversarial training as a min-max problem, randomized smoothing as a certified defense, and differential privacy with DP-SGD as the defense that bounds every membership attack, and closes with what a credible report of a defense must contain.

After this unit you will be able to state a poisoning threat model in the terms of NIST AI 100-2e2025, measure a backdoor and explain why its success rate is a red-team metric, run and evaluate a membership inference attack by its true positive rate at low false positive rates, train a network with DP-SGD and read its privacy guarantee, train and evaluate an adversarially robust network with the required sanity checks, and judge whether a published defense was evaluated adequately. The unit builds on Lessons 1.2 and 2.3 (alert budgets and thresholds), Unit 5 (training neural networks) and Unit 7 (threat models, FGSM and PGD). Allow about 10 hours for the three lessons, the three labs, the exercises and the quiz.