Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. Export

@article{citeulike:2609667, abstract = {Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy.}, address = {Lawrence Berkeley National Laboratory, University of California at Berkeley, 1 Cyclotron Road, MS. 84-171, Berkeley, CA 94720, USA. hpeng@lbl.gov}, author = {Peng, H. and Long, F. and Ding, C.}, citeulike-article-id = {2609667}, citeulike-linkout-0 = {http://portal.acm.org/citation.cfm?id=1070616.1070809}, citeulike-linkout-1 = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2005.159}, citeulike-linkout-2 = {http://dx.doi.org/10.1109/TPAMI.2005.159}, citeulike-linkout-3 = {http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1453511}, citeulike-linkout-4 = {http://view.ncbi.nlm.nih.gov/pubmed/16119262}, citeulike-linkout-5 = {http://www.hubmed.org/display.cgi?uids=16119262}, doi = {10.1109/TPAMI.2005.159}, issn = {0162-8828}, journal = {IEEE Trans Pattern Anal Mach Intell}, keywords = {feature, selection}, month = {August}, number = {8}, pages = {1226--1238}, posted-at = {2009-03-23 23:23:11}, priority = {0}, title = {Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy.}, url = {http://dx.doi.org/10.1109/TPAMI.2005.159}, volume = {27}, year = {2005} }

TY - JOUR ID - citeulike:2609667 L3 - citeulike-article-id:2609667 N2 - Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy. IS - 8 JF - IEEE Trans Pattern Anal Mach Intell SN - 0162-8828 AD - Lawrence Berkeley National Laboratory, University of California at Berkeley, 1 Cyclotron Road, MS. 84-171, Berkeley, CA 94720, USA. hpeng@lbl.gov EP - 1238 TI - Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. VL - 27 SP - 1226 KW - feature KW - selection AU - Peng, H AU - Long, F AU - Ding, C PY - 2005/08// UR - http://dx.doi.org/10.1109/TPAMI.2005.159 ER -