The little book of Deep Learning Francois Fleuret UNIVERSITE DE GENEVE Francois Fleuret is a professor of puter sci- ence at the University of Geneva Switzerland. The cover illustration is a schematic of the Neocognitron by Fukushima [198o] a key an- cestor of deep neural networks. This ebook is formatted to fit on a phone screen. Contents Contents 5 List of figures 7 Foreword 8 I Foundations 10 1 Machine Learning 11 1.1 Learning from data.....12 1.2 Basis function regression ... 14 1.3 Under and overfitting......16 1.4 Categories of models.....18 2 Efficient putation 20 2.1 GPUs TPUs and batches....21 2.2 Tensors.............23 3 Training 25 3.1 Losses........ 26 3.2 Autoregressive models....3o 3.3 Gradient descent ........34 31163 ...