Unit 5. Neural networks

This unit opens the deep learning part of the course with the mechanics that the neural networks of later units rely on. Lesson 5.1 defines neurons, activation functions and the multilayer perceptron, explains why nonlinear activations are necessary, states what the universal approximation theorem guarantees, and works backpropagation through a small network by hand, checking every gradient against finite differences and PyTorch. Lesson 5.2 covers the decisions that shape training in practice, namely epochs and mini-batches, SGD, momentum and Adam, learning rates, initialization, feature scaling, early stopping, weight decay, dropout and weighted losses for rare attacks, with training curves measured on the course dataset. Lesson 5.3 turns the network into an anomaly detector, an autoencoder trained on normal traffic, shows that a linear autoencoder only rediscovers PCA, and compares the autoencoder with an isolation forest under an alert budget.

After this unit you will be able to compute gradients by hand and verify them, train and regularize a PyTorch network on tabular security data with a time-ordered validation period, read training and validation curves, and judge what reconstruction error can and cannot detect. The unit assumes Unit 3 (logistic regression and gradient descent) and Unit 4 (PCA and anomaly detection). Plan on about 10 hours, including the two labs and the exercises.

Lessons

1. 5.1 Neurons, layers and backpropagation
2. 5.2 Training neural networks in practice
3. 5.3 Autoencoders for anomaly detection

Labs

Practice

• Exercises with worked solutions
• Self-check quiz