We can use the holdout bound to calculate upper and lower true error bounds with the program “bound”. Here is an example of the application.
11:32PM z-12: echo "test_examples 600
test_errors 192 delta 0.025" | bound Applying varying approximation tail bound delta 0.025 lower_delta 0.5 test_examples 600 test_errors 192 approximation automatic true_error = 0.32 0.358976827934 0.31926717516 |
There are several arguments passed to the program. These include:
The output of the program is several lines stating the used and assumed arguments and a final line of the form:
true_error = <error rate> <upper_bound> <lower_bound>
|
The output of this particular application implies that the true error rate of the chosen hypothesis is less than with high confidence (over the drawn sample set).
For simplicity, the arguments can also be placed into a file:
11:35PM z-14: cat test_error
test_examples 800 test_errors 192 delta 0.025 lower_delta 0.025 11:35PM z-15: bound test_error Applying varying approximation tail bound delta 0.025 lower_delta 0.025 test_examples 800 test_errors 192 approximation automatic true_error = 0.24 0.28252880089 0.20065891277 |
The output of this program can be interpreted as a confidence interval.