Unit 3. Supervised learning

This unit covers the supervised classifiers that do most of the work in applied security machine learning, each from its loss function or decision rule to its behavior on intrusion and phishing data. Lesson 3.1 develops linear and logistic regression, the normal equation, gradient descent and its stochastic variant, L1 and L2 regularization and the tradeoff between bias and variance. Lesson 3.2 covers k-nearest neighbors, support vector machines with the kernel trick, and naive Bayes for text. Lesson 3.3 covers decision trees, random forests and gradient boosting, and the difference between global feature importance and the explanation of a single alert.

After the unit you will be able to compute a gradient step, a margin, a naive Bayes posterior, an impurity decrease and a boosting stage by hand, choose a classifier for a detection task with its costs in mind, and compare classifiers fairly on a time-ordered split with an alert budget. Lab 3a compares seven classifiers on the course's flow data, and Lab 3b builds a phishing URL detector from text features. The unit assumes Unit 2, in particular time-ordered splits and average precision. Plan on about eight to ten hours, including both labs and the exercises.

Lessons

1. 3.1 Linear and logistic regression
2. 3.2 Nearest neighbors, support vector machines and naive Bayes
3. 3.3 Decision trees, random forests and boosting

Labs

Practice

• Exercises with worked solutions
• Self-check quiz