Progressive Forest: An Early Stopping Criteria For Building Ensembles

Abstract

Decision forests improve their predictive power based on thecombination of various decision trees. The number of trees to be usedto achieve the best possible accuracy is not preset and has to bedetermined by a trial and error process. In many classificationproblems more trees are used than necessary.This paper introduces a new method, called Progressive Forest, thatprogressively evaluates the addition of new decision trees into adecision forest to decide when adding more trees is not longer useful.This method was incorporated into the construction schemes ofProactive Forest and Random Forest with very encouraging results.It is experimentally shown that Progressive Forest reduces the numberof trees while maintaining the accuracy of the classification.Progressive Forest can be incorporated into any scheme of constructionof ensemble, which presents similar characteristics to Random Forest.