An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

@article{citeulike:2929443, abstract = {Bagging and boosting are methods that generate a diverse ensemble of classifiers by manipulating the training data given to a “base” learning algorithm. Breiman has pointed out that they rely for their effectiveness on the instability of the base learning algorithm. An alternative approach to generating an ensemble is to randomize the internal decisions made by the base algorithm. This general approach has been studied previously by Ali and Pazzani and by Dietterich and Kong. This paper compares the effectiveness of randomization, bagging, and boosting for improving the performance of the decision-tree algorithm C4.5. The experiments show that in situations with little or no classification noise, randomization is competitive with (and perhaps slightly superior to) bagging but not as accurate as boosting. In situations with substantial classification noise, bagging is much better than boosting, and sometimes better than randomization.}, author = {Dietterich, Thomas G. }, citeulike-article-id = {2929443}, doi = {http://dx.doi.org/10.1023/A:1007607513941}, journal = {Machine Learning}, keywords = {algorithms, kdd, sys\_performance}, number = {2}, pages = {139--157}, posted-at = {2008-06-26 10:40:41}, priority = {2}, title = {An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization}, url = {http://dx.doi.org/10.1023/A:1007607513941}, volume = {40}, year = {2000} }

TY - JOUR ID - citeulike:2929443 L3 - citeulike-article-id:2929443 TI - An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization JF - Machine Learning VL - 40 IS - 2 SP - 139 EP - 157 N2 - Bagging and boosting are methods that generate a diverse ensemble of classifiers by manipulating the training data given to a “base” learning algorithm. Breiman has pointed out that they rely for their effectiveness on the instability of the base learning algorithm. An alternative approach to generating an ensemble is to randomize the internal decisions made by the base algorithm. This general approach has been studied previously by Ali and Pazzani and by Dietterich and Kong. This paper compares the effectiveness of randomization, bagging, and boosting for improving the performance of the decision-tree algorithm C4.5. The experiments show that in situations with little or no classification noise, randomization is competitive with (and perhaps slightly superior to) bagging but not as accurate as boosting. In situations with substantial classification noise, bagging is much better than boosting, and sometimes better than randomization. KW - algorithms KW - kdd KW - sys_performance AU - Dietterich, Thomas PY - 2000//01/ UR - http://dx.doi.org/10.1023/A:1007607513941 ER -