Unit 1. Machine learning meets security

This unit sets out what machine learning can and cannot do for a security team. Lesson 1.1 defines learning from data, surveys the security problems where learned models are used, compares them with signatures and rules, explains what makes security a hard setting for machine learning, and introduces the attacks on machine learning that the second half of the course studies. Lesson 1.2 defines the measures used throughout the course, shows how precision depends on the base rate, and explains why, when attacks are rare, accuracy becomes almost meaningless and precision is dominated by the false positive rate. Lesson 1.3 describes the workflow from framing a detection problem to retraining a deployed model, the conditions for a reproducible result, the tools used in the labs, and the synthetic network dataset that Lab 1 and seven later labs share, together with the documented problems of public benchmark datasets.

After this unit you will be able to state a security problem as a learning task with an explicit performance measure and compute recall, the false positive rate and precision from a detector's confusion matrix. You will also be able to compute a detector's precision and expected alerts per day from its true and false positive rates, a base rate and a daily traffic volume, explain why a time-ordered split and a stated operating point are needed to evaluate a detector, and describe the course dataset and its built-in traps. The unit assumes only the prerequisites listed in How to use this course; if pandas or scikit-learn are new to you, work through the Python primer first. Allow about 8 to 10 hours for the three lessons, Lab 1, the exercises and the quiz.

Lessons

1. 1.1 Where machine learning helps in security
2. 1.2 Base rates, metrics and the cost of an alert
3. 1.3 The workflow, the tools and the course dataset

Labs

Practice

• Exercises with worked solutions
• Self-check quiz