Statistical Learning and Sequential Prediction

by Alexander Rakhlin, Karthik Sridharan

DescriptionTable of ContentsDetailsHashtagsReport an issue

Book Description

This free book will focus on theoretical aspects of Statistical Learning and Sequential Prediction. Until recently, these two subjects have been treated separately within the learning community. The course will follow a unified approach to analyzing learning in both scenarios. To make this happen, we shall bring together ideas from probability and statistics, game theory, algorithms, and optimization. It is this blend of ideas that makes the subject interesting for us, and we hope to convey the excitement. We shall try to make the course as self-contained as possible, and pointers to additional readings will be provided whenever necessary. Our target audience is graduate students with a solid background in probability and linear algebra.

This open book is licensed under a Creative Commons License (CC BY). You can download Statistical Learning and Sequential Prediction ebook for free in PDF format (3.4 MB).

Table of Contents

Part I
Introduction
Part II
Theory
 
Minimax Formulation of Learning Problems
 
Learnability, Oracle Inequalities, Model Selection, and the Bias-Variance Trade-off
 
Stochastic processes, Empirical processes, Martingales, Tree Processes
 
Example: Learning Thresholds
 
Maximal Inequalities
 
Example: Linear Classes
 
Statistical Learning: Classification
 
Statistical Learning: Real-Valued Functions
 
Sequential Prediction: Classification
 
Sequential Prediction: Real-Valued Functions
 
Examples: Complexity of Linear and Kernel Classes, Neural Networks
 
Large Margin Theory for Classification
 
Regression with Square Loss: From Regret to Nonparametric Estimation
Part III
Algorithms
 
Algorithms for Sequential Prediction: Finite Classes
 
Algorithms for Sequential Prediction: Binary Classification with Infinite Classes
 
Algorithms for Online Convex Optimization
 
Example: Binary Sequence Prediction and the Mind Reading Machine
 
Algorithmic Framework for Sequential Prediction
 
Algorithms Based on Random Playout, and Follow the Perturbed Leader
 
Algorithms for Fixed Design
 
Adaptive Algorithms
Part IV
Extensions
 
The Minimax Theorem
 
Two Proofs of Blackwell's Approachability Theorem
 
From Sequential to Statistical Learning: Relationship Between Values and Online-to-Batch
 
Sequential Prediction: Competing With Strategies
 
Localized Analysis and Fast Rates. Local Rademacher Complexities
Appendix
 
 

Book Details

Publisher
MIT Press
Published
2014
Pages
261
Edition
1
Language
English
PDF Size
3.4 MB
License
CC BY

Related Books

Statistics with Julia
Ccurrently many of Julia's users are hard-core developers that contribute to the language's standard libraries, and to the extensive package eco-system that surrounds it. Therefore, much of the Julia material available at present is aimed at other developers rather than end users. This is where our book comes in, as it has been written with the end...
Motivational Profiles in TIMSS Mathematics
This book presents a person-centered exploration of student profiles, using variables related to motivation to do school mathematics derived from the IEA's Trends in International Mathematics and Science Study (TIMSS) data. Statistical cluster analysis is used to identify groups of students with similar motivational profiles, across grades and over...
An Introduction to Machine Learning
This textbook presents fundamental machine learning concepts in an easy to understand manner by providing practical advice, using straightforward examples, and offering engaging discussions of relevant applications. The main topics include Bayesian classifiers, nearest-neighbor classifiers, linear and polynomial classifiers, decision trees, neural ...
Forecasting and Assessing Risk of Individual Electricity Peaks
The overarching aim of this book is to present self-contained theory and algorithms for investigation and prediction of electric demand peaks. A cross-section of popular demand forecasting algorithms from statistics, machine learning and mathematics is presented, followed by extreme value theory techniques with examples.In order to achieve carbon t...
Mobile Media Learning
This book is an inspirational message about what is possible and practical in the name of learning through mobile media. We present stories from a diverse set of educators, a microcosm of the landscape of mobile media learning. Each author has found a way to create something new and beautiful in their own world. And though their results are exce...
Tensor Network Contractions
Tensor network is a fundamental mathematical tool with a huge range of applications in physics, such as condensed matter physics, statistic physics, high energy physics, and quantum information sciences. This open access book aims to explain the tensor network contraction approaches in a systematic way, from the basic definitions to the important a...