3.1 The Basic Building Block

3.1. The Basic Building Block

Our real interest will be captured by Binomial tails because we wish to bound the probability of observing a misleadingly small event. The probability of a Binomial tail is just the cumulative distribution function:

DEFINITION 3.1.1.

(Binomial Tail) $Bin (m, k, p) \equiv {Pr}_{{0, 1}^{m}} (\hat{p} \leq \frac{k}{m} ∣ p) = \sum_{j = 1}^{k} (\binom{m}{j}) p^{j} {(1 - p)}^{m - j}$ = the probability that $m$ coins with bias $p$ produce $k$ or fewer heads.

For the learning problem, we will always choose $p = e_{D} (h)$ and $\hat{p} = {\hat{e}}_{S} (h)$ . With these definitions, we can interpret the Binomial tail as the probability of an empirical error less than or equal to $\frac{k}{m}$ .

[next] [front] [up]