Variational inference for Bayesian mixtures of factor analysers Export

@inproceedings{Ghahramani_00, abstract = {We present an algorithm that infers the model structure of a mixture of factor analysers using an efficient and deterministic variational approximation to full Bayesian integration over model parameters. This procedure can automatically determine the optimal number of components and the local dimensionality of each component (i.e. the number of factors in each factor analyser). Alternatively it can be used to infer posterior distributions over number of components and dimensionalities. Since all parameters are integrated out the method is not prone to overfitting. Using a stochastic procedure for adding components it is possible to perform the variational optimisation incrementally and to avoid local maxima. Results show that the method works very well in practice and correctly infers the number and dimensionality of nontrivial synthetic examples. By importance sampling from the variational approximation we show how to obtain unbiased estimates of the true evidence, the exact predictive density, and the KL divergence between the variational posterior and the true posterior, not only in this model but for variational approximations in general. 1}, author = {Ghahramani, Z. and Beal, M.}, booktitle = {In Advances in Neural Information Processing Systems 12}, citeulike-article-id = {6067242}, citeulike-linkout-0 = {http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.65.4497}, keywords = {learning, lwpr}, pages = {449--455}, posted-at = {2009-11-04 10:46:40}, priority = {2}, title = {Variational inference for Bayesian mixtures of factor analysers}, url = {http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.65.4497}, volume = {12}, year = {2000} }

TY - CONF ID - Ghahramani_00 L3 - citeulike-article-id:6067242 N2 - We present an algorithm that infers the model structure of a mixture of factor analysers using an efficient and deterministic variational approximation to full Bayesian integration over model parameters. This procedure can automatically determine the optimal number of components and the local dimensionality of each component (i.e. the number of factors in each factor analyser). Alternatively it can be used to infer posterior distributions over number of components and dimensionalities. Since all parameters are integrated out the method is not prone to overfitting. Using a stochastic procedure for adding components it is possible to perform the variational optimisation incrementally and to avoid local maxima. Results show that the method works very well in practice and correctly infers the number and dimensionality of nontrivial synthetic examples. By importance sampling from the variational approximation we show how to obtain unbiased estimates of the true evidence, the exact predictive density, and the KL divergence between the variational posterior and the true posterior, not only in this model but for variational approximations in general. 1 EP - 455 TI - Variational inference for Bayesian mixtures of factor analysers VL - 12 T2 - In Advances in Neural Information Processing Systems 12 SP - 449 KW - learning KW - lwpr AU - Ghahramani, Z AU - Beal, M PY - 2000/// UR - http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.65.4497 ER -