PROOF. (A variant of Chernoff) $${Pr}_{{\left(x,y\right)}^m\sim D^m}\left(e_{\text{pv}}\ge {\widehat{e}}_{\text{pv}}+\epsilon \right)$$ $$\iff \forall \lambda {Pr}_{{\left(x,y\right)}^m\sim D^m}\left(e^{\lambda m{e}_{\text{pv}}}\ge e^{\lambda m\left({\widehat{e}}_{\text{pv}}+\epsilon \right)}\right)$$

$$\iff {Pr}_{{\left(x,y\right)}^m\sim D^m}\left(e^{\lambda m\left(e_{\text{pv}}-{\widehat{e}}_{\text{pv}}-\epsilon \right)}\ge 1\right)$$

$$\le E_{{\left(x,y\right)}^m\sim D^m}{e}^{\lambda m\left(e_{\text{pv}}-{\widehat{e}}_{\text{pv}}-\epsilon \right)}$$ $$=E_{{\left(x,y\right)}^m\sim D^m}{e}^{\lambda \left({\sum }_{i=1}^m{e}_i-{\widehat{e}}_i-\epsilon \right)}$$ $$={\prod }_{i=1}^m{E}_{\left(x,y\right)\sim D}\left[e^{\lambda \left(e_i-{\widehat{e}}_i-\epsilon \right)}\mid {\widehat{e}}_1,...,{\widehat{e}}_{i-1}\right]$$

The value of $e^{\lambda \left(e_i-{\widehat{e}}_i-\epsilon \right)}$ is only dependent on ${\widehat{e}}_1,...,{\widehat{e}}_{i-1}$ through the value of $e_i$. For all possible hypotheses, $h_i$, we know that ${\widehat{e}}_i$ is a Bernoulli random variable with bias $e_i={Pr}_D\left(h_i\left(x\right)\ne y\right)$. This is true regardless of how we choose $h_i$. Consequently, we have:

$$={\prod }_{i=1}^m{E}_{\left(x,y\right)\sim D}\left[e^{\lambda \left(e_i-{\widehat{e}}_i-\epsilon \right)}\right]$$

(10.3.1)

Since this is true for all $\lambda$, we can choose the optimal $\lambda$. In general, this is an optimization problem which can be worst-case simplified with the inequality: $$E_{\left(x,y\right)\sim D}{e}^{\lambda \left(e_i-{\widehat{e}}_i\right)}\le e^{\frac{{\lambda }^2}{8}}$$ which implies: $$\forall \lambda \le {\prod }_{i=1}^m{e}^{\frac{{\lambda }^2}{8}-\lambda \epsilon }$$ This is optimized for $\frac{2\lambda }{8}=\epsilon$ which implies $\lambda =4\epsilon$ giving: $$\le {\prod }_{i=1}^m{e}^{-2{\epsilon }^2}$$ $$=e^{-2m{\epsilon }^2}$$ ▫