SMEM Algorithm for Mixture Models Export

@article{Ueda_00, abstract = {doi: 10.1162/089976600300015088 We present a split-and-merge expectation-maximization (SMEM) algorithm to overcome the local maxima problem in parameter estimation of finite mixture models. In the case of mixture models, local maxima often involve having too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations, we repeatedly perform simultaneous split-and-merge operations using a new criterion for efficiently selecting the split-and-merge candidates. We apply the proposed algorithm to the training of gaussian mixtures and mixtures of factor analyzers using synthetic and real data and show the effectiveness of using the split- and-merge operations to improve the likelihood of both the training data and of held-out test data. We also show the practical usefulness of the proposed algorithm by applying it to image compression and pattern recognition problems.}, author = {Ueda, N. and Nakano, R. and Ghahramani, Z. and Hinton, G.}, citeulike-article-id = {6067351}, citeulike-linkout-0 = {http://dx.doi.org/10.1162/089976600300015088}, citeulike-linkout-1 = {http://www.mitpressjournals.org/doi/abs/10.1162/089976600300015088}, day = {1}, doi = {10.1162/089976600300015088}, journal = {Neural Computation}, keywords = {learning, lwpr}, month = {September}, number = {9}, pages = {2109--2128}, posted-at = {2009-11-04 11:12:13}, priority = {2}, title = {SMEM Algorithm for Mixture Models}, url = {http://dx.doi.org/10.1162/089976600300015088}, volume = {12}, year = {2000} }

TY - JOUR ID - Ueda_00 L3 - citeulike-article-id:6067351 N2 - doi: 10.1162/089976600300015088 We present a split-and-merge expectation-maximization (SMEM) algorithm to overcome the local maxima problem in parameter estimation of finite mixture models. In the case of mixture models, local maxima often involve having too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations, we repeatedly perform simultaneous split-and-merge operations using a new criterion for efficiently selecting the split-and-merge candidates. We apply the proposed algorithm to the training of gaussian mixtures and mixtures of factor analyzers using synthetic and real data and show the effectiveness of using the split- and-merge operations to improve the likelihood of both the training data and of held-out test data. We also show the practical usefulness of the proposed algorithm by applying it to image compression and pattern recognition problems. IS - 9 JF - Neural Computation EP - 2128 TI - SMEM Algorithm for Mixture Models VL - 12 SP - 2109 KW - learning KW - lwpr AU - Ueda, N AU - Nakano, R AU - Ghahramani, Z AU - Hinton, G PY - 2000/09/01/ UR - http://dx.doi.org/10.1162/089976600300015088 ER -