Figure 3.3 shows an example of the SMART operators' abilities. This graph shows a sound classification experiment using sounds extracted from the SPIB database (ftp spib.rice.edu). There are seven sound classes (Factory-noises-1, M109-engine, Buccaneer-jet-engine, Machinegun, Volvo-engine, Canteen-babble, Factory-noises-2). PADO was trained on 35 examples of each sound. The graph in figure 3.3 shows the generalization ability of the orchestrated PADO system (essentially an average of the best few programs in the population [Teller and Veloso 1995a]) on a set of unseen test examples over generations (averaged over 10 runs). For more details of this problem, see [Teller 1995b].
Figure 3.3: PADO System Sound Classification Generalization Percentage on Test Sounds.
This graph is a positive, representative example of the effect SMART operator program application has on performance in PADO. This chapter could detail more about the main population performance benefits of SMART operator evolution and application only at the expense of detail about the SMART operators themselves. Because this chapter is primarily about the co-evolution of intelligent recombination operators, and not about the PADO classification learning system, the further performance benefits explicitly stated in this chapter will be confined to the following statement.
In our experience, the use of co-evolved SMART operator programs to aid evolution has almost always helped performance, sometimes dramatically, and has never in our experience been a noticeable hindrance in either speed or population performance.
On average, the evolution and application of the SMART operators takes about 30 seconds per generation. That means that a run to generation 100 spends 3.4% of its time on the SMART operators. This is a negligible price to pay for more intelligent recombination.