3 Basic Observations

Chapter 3
Basic Observations

The principle observable quantity is the empirical error rate ( ${\hat{e}}_{S} (h)$ ) of a hypothesis. What is the distribution of the empirical error rate for a fixed hypothesis? For each example, we know that the probability that the hypothesis will err is given by true error rate, $e_{D} (h)$ . This can be modeled by a biased coin flip: heads if you are wrong and tails if you are right.

Let us call the bias of the coin $p = e_{D} (h)$ . What then is the probability of observing $k$ heads out of $m$ coin flips? This is a very familiar distribution in statistics called the Binomial distribution. Let $\hat{p}$ be the observed rate of heads.

DEFINITION 3.0.1.

(Binomial Distribution) The Binomial distribution is given by: ${Pr}_{{0, 1}^{m}} (\hat{p} = \frac{k}{m} ∣ p) = (\binom{m}{k}) p^{k} {(1 - p)}^{m - k}$ Here we use ’choose’ notation defined by $(\binom{m}{k}) = \frac{m!}{(m - k)! k!}$ .

3.1.  The Basic Building Block
3.2.  Approximation techniques
3.3.  Binomial Tail calculation techniques
3.4.  Converting to a P-value approach
3.5.  Bounding the Union
3.6.  Arbitrary Loss functions

[next] [prev] [prev-tail] [front] [up]

Chapter 3Basic Observations

Chapter 3
Basic Observations