1 Introductory Learning Theory

The introduction is broken into 4 parts. The first part is informal introduction to the learning model used here. The goal of the first part is to make explicit the design choices made in selecting our model. This is particularly important in machine learning because no model seems perfect.

1 Informal Introduction
1.1.  The learning problem
1.2.  The problem with the learning problem
1.3.  A plethora of learning models
1.4.  The oblivious passive supervised learning model
1.5.  Questions we can answer
2 Formal Model and Context
2.1.  Formal Model
2.2.  Relationship to Prior Work
2.3.  Overview of the document
3 Basic Observations
3.1.  The Basic Building Block
3.2.  Approximation techniques
3.3.  Binomial Tail calculation techniques
3.4.  Converting to a P-value approach
3.5.  Bounding the Union
3.6.  Arbitrary Loss functions
4 Simple Sample Complexity bounds
4.1.  Simple Holdout
4.2.  The basic training set bound
4.3.  Lower Bounds
4.4.  Lower Upper Bounds
4.5.  Structural Risk Minimization
4.6.  Incorporating a “Prior”

Part 1 Introductory Learning Theory