# Statistical Learning and Sequential Prediction

by Alexander Rakhlin, Karthik Sridharan

Join

### Book Description

This free book will focus on theoretical aspects of Statistical Learning and Sequential Prediction. Until recently, these two subjects have been treated separately within the learning community. The course will follow a unified approach to analyzing learning in both scenarios. To make this happen, we shall bring together ideas from probability and statistics, game theory, algorithms, and optimization. It is this blend of ideas that makes the subject interesting for us, and we hope to convey the excitement. We shall try to make the course as self-contained as possible, and pointers to additional readings will be provided whenever necessary. Our target audience is graduate students with a solid background in probability and linear algebra.

Part I
Introduction
Part II
Theory

Minimax Formulation of Learning Problems

Learnability, Oracle Inequalities, Model Selection, and the Bias-Variance Trade-off

Stochastic processes, Empirical processes, Martingales, Tree Processes

Example: Learning Thresholds

Maximal Inequalities

Example: Linear Classes

Statistical Learning: Classification

Statistical Learning: Real-Valued Functions

Sequential Prediction: Classification

Sequential Prediction: Real-Valued Functions

Examples: Complexity of Linear and Kernel Classes, Neural Networks

Large Margin Theory for Classification

Regression with Square Loss: From Regret to Nonparametric Estimation
Part III
Algorithms

Algorithms for Sequential Prediction: Finite Classes

Algorithms for Sequential Prediction: Binary Classification with Infinite Classes

Algorithms for Online Convex Optimization

Example: Binary Sequence Prediction and the Mind Reading Machine

Algorithmic Framework for Sequential Prediction

Algorithms for Fixed Design

Part IV
Extensions

The Minimax Theorem

Two Proofs of Blackwell's Approachability Theorem

From Sequential to Statistical Learning: Relationship Between Values and Online-to-Batch

Sequential Prediction: Competing With Strategies

Localized Analysis and Fast Rates. Local Rademacher Complexities
Appendix

### Book Details

Title
Statistical Learning and Sequential Prediction
Subject
Education and Teaching
Publisher
MIT Press
Published
2014
Pages
261
Edition
1
Language
English
PDF Size
3.4 MB