Chapter 14
Conclusion & Challenges

The basic conclusion of this thesis is that we can achieve bounds tight enough to yield useful results on real learning problems with standard learning algorithms or simple variants on standard learning algorithms. The evidence of this conclusion is reported in the last two chapters.

To accomplish this goal, considerable theoretical work was completed. This includes:

1. Derivation of the microchoice bounds and adaptive microchoice bounds in Section 5.
2. Improvement and simplification of the PAC-Bayes bound in Section 6.
3. Improvement of the margin bound to achieve benefit from averaging 7.
4. Derivation of the Shell bounds 8.
5. An investigation into how to repair the covering number approach 9.
6. An analysis of progressive validation 10.
7. An analysis for the combination of training and test set bounds 11.

In particular, microchoice bounds (Section 5), PAC-Bayes bounds (Section 6), shell bound (Section 8), and combined bounds (Section 11) have proved useful for practical application.

It is worth emphasizing that all of the bounds reported here rest upon only an assumption of example independence implying wide applicability.