Statistical Learning and Sequential Prediction

by Alexander Rakhlin, Karthik Sridharan

DescriptionTable of ContentsDetailsHashtagsReport an issue

Book Description

This free book will focus on theoretical aspects of Statistical Learning and Sequential Prediction. Until recently, these two subjects have been treated separately within the learning community. The course will follow a unified approach to analyzing learning in both scenarios. To make this happen, we shall bring together ideas from probability and statistics, game theory, algorithms, and optimization. It is this blend of ideas that makes the subject interesting for us, and we hope to convey the excitement. We shall try to make the course as self-contained as possible, and pointers to additional readings will be provided whenever necessary. Our target audience is graduate students with a solid background in probability and linear algebra.

This open book is licensed under a Creative Commons License (CC BY). You can download Statistical Learning and Sequential Prediction ebook for free in PDF format (3.4 MB).

Table of Contents

Part I
Part II
Minimax Formulation of Learning Problems
Learnability, Oracle Inequalities, Model Selection, and the Bias-Variance Trade-off
Stochastic processes, Empirical processes, Martingales, Tree Processes
Example: Learning Thresholds
Maximal Inequalities
Example: Linear Classes
Statistical Learning: Classification
Statistical Learning: Real-Valued Functions
Sequential Prediction: Classification
Sequential Prediction: Real-Valued Functions
Examples: Complexity of Linear and Kernel Classes, Neural Networks
Large Margin Theory for Classification
Regression with Square Loss: From Regret to Nonparametric Estimation
Part III
Algorithms for Sequential Prediction: Finite Classes
Algorithms for Sequential Prediction: Binary Classification with Infinite Classes
Algorithms for Online Convex Optimization
Example: Binary Sequence Prediction and the Mind Reading Machine
Algorithmic Framework for Sequential Prediction
Algorithms Based on Random Playout, and Follow the Perturbed Leader
Algorithms for Fixed Design
Adaptive Algorithms
Part IV
The Minimax Theorem
Two Proofs of Blackwell's Approachability Theorem
From Sequential to Statistical Learning: Relationship Between Values and Online-to-Batch
Sequential Prediction: Competing With Strategies
Localized Analysis and Fast Rates. Local Rademacher Complexities

Book Details

MIT Press
PDF Size
3.4 MB

Related Books