Statistical Learning and Sequential Prediction

by Alexander Rakhlin, Karthik Sridharan

DescriptionTable of ContentsDetailsHashtagsReport an issue

Book Description

This free book will focus on theoretical aspects of Statistical Learning and Sequential Prediction. Until recently, these two subjects have been treated separately within the learning community. The course will follow a unified approach to analyzing learning in both scenarios. To make this happen, we shall bring together ideas from probability and statistics, game theory, algorithms, and optimization. It is this blend of ideas that makes the subject interesting for us, and we hope to convey the excitement. We shall try to make the course as self-contained as possible, and pointers to additional readings will be provided whenever necessary. Our target audience is graduate students with a solid background in probability and linear algebra.

This open book is licensed under a Creative Commons License (CC BY). You can download Statistical Learning and Sequential Prediction ebook for free in PDF format (3.4 MB).

Table of Contents

Part I
Introduction
Part II
Theory
 
Minimax Formulation of Learning Problems
 
Learnability, Oracle Inequalities, Model Selection, and the Bias-Variance Trade-off
 
Stochastic processes, Empirical processes, Martingales, Tree Processes
 
Example: Learning Thresholds
 
Maximal Inequalities
 
Example: Linear Classes
 
Statistical Learning: Classification
 
Statistical Learning: Real-Valued Functions
 
Sequential Prediction: Classification
 
Sequential Prediction: Real-Valued Functions
 
Examples: Complexity of Linear and Kernel Classes, Neural Networks
 
Large Margin Theory for Classification
 
Regression with Square Loss: From Regret to Nonparametric Estimation
Part III
Algorithms
 
Algorithms for Sequential Prediction: Finite Classes
 
Algorithms for Sequential Prediction: Binary Classification with Infinite Classes
 
Algorithms for Online Convex Optimization
 
Example: Binary Sequence Prediction and the Mind Reading Machine
 
Algorithmic Framework for Sequential Prediction
 
Algorithms Based on Random Playout, and Follow the Perturbed Leader
 
Algorithms for Fixed Design
 
Adaptive Algorithms
Part IV
Extensions
 
The Minimax Theorem
 
Two Proofs of Blackwell's Approachability Theorem
 
From Sequential to Statistical Learning: Relationship Between Values and Online-to-Batch
 
Sequential Prediction: Competing With Strategies
 
Localized Analysis and Fast Rates. Local Rademacher Complexities
Appendix
 
 

Book Details

Publisher
MIT Press
Published
2014
Pages
261
Edition
1
Language
English
PDF Size
3.4 MB
License
CC BY

Related Books