11.4. Conclusion

The theorems in this section give us a very general tools for composing training set and test set based bounds. We used these general tools to construct a particular approach which will be tested in the next chapter.

There are two significant questions which need to be answered.

1. Is the improvement of the particular approach worthwhile over a simple disjunction (technique 2 in the introduction)? Later empirical results will show that it can be worthwhile on real data, but the computational cost is non-negligible.
2. Is the approach we used for combining bounds “best”? It is difficult to compare different approaches because strict dominance does not typically occur. Nonetheless, there are many other ways to compose train set and test set bounds and satisfy theorem 11.3.2. Perhaps some other way is better on natural learning problems.