Monte Carlo Methods for Adaptive Sparse Approximations of Time-Series Export

@article{citeulike:2860201, abstract = {This paper deals with adaptive sparse approximations of time-series. The work is based on a Bayesian specification of the shift-invariant sparse coding model. To learn approximations for a particular class of signals, two different learning strategies are discussed. The first method uses a gradient optimization technique commonly employed in sparse coding problems. The other method is novel in this context and is based on a sampling estimate. To approximate the gradient in the first approach we compare two Monte Carlo estimation techniques, Gibbs sampling and a novel importance sampling method. The second approach is based on a direct sample estimate and uses an extension of the Gibbs sampler used with the first approach. Both approaches allow the specification of different prior distributions and we here introduce a novel mixture prior based on a modified Rayleigh distribution. Experiments demonstrate that all Gibbs sampler based methods show comparable performance. The importance sampler was found to work nearly as well as the Gibbs sampler on smaller problems in terms of estimating the model parameters, however, the method performed substantially worse on estimating the sparse coefficients. For large problems we found that the combination of a subset selection heuristic with the Gibbs sampling approaches can outperform previous suggested methods. In addition, the methods studied here are flexible and allow the incorporation of additional prior knowledge, such as the nonnegativity of the approximation coefficients, which was found to offer additional benefits where applicable.}, author = {Blumensath, T. and Davies, M. E.}, booktitle = {Signal Processing, IEEE Transactions on [see also Acoustics, Speech, and Signal Processing, IEEE Transactions on]}, citeulike-article-id = {2860201}, citeulike-linkout-0 = {http://dx.doi.org/10.1109/TSP.2007.896242}, citeulike-linkout-1 = {http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4291843}, doi = {10.1109/TSP.2007.896242}, journal = {Signal Processing, IEEE Transactions on [see also Acoustics, Speech, and Signal Processing, IEEE Transactions on]}, keywords = {sparse-coding}, number = {9}, pages = {4474--4486}, posted-at = {2008-06-04 03:16:50}, priority = {3}, title = {Monte Carlo Methods for Adaptive Sparse Approximations of Time-Series}, url = {http://dx.doi.org/10.1109/TSP.2007.896242}, volume = {55}, year = {2007} }

TY - JOUR ID - citeulike:2860201 L3 - citeulike-article-id:2860201 N2 - This paper deals with adaptive sparse approximations of time-series. The work is based on a Bayesian specification of the shift-invariant sparse coding model. To learn approximations for a particular class of signals, two different learning strategies are discussed. The first method uses a gradient optimization technique commonly employed in sparse coding problems. The other method is novel in this context and is based on a sampling estimate. To approximate the gradient in the first approach we compare two Monte Carlo estimation techniques, Gibbs sampling and a novel importance sampling method. The second approach is based on a direct sample estimate and uses an extension of the Gibbs sampler used with the first approach. Both approaches allow the specification of different prior distributions and we here introduce a novel mixture prior based on a modified Rayleigh distribution. Experiments demonstrate that all Gibbs sampler based methods show comparable performance. The importance sampler was found to work nearly as well as the Gibbs sampler on smaller problems in terms of estimating the model parameters, however, the method performed substantially worse on estimating the sparse coefficients. For large problems we found that the combination of a subset selection heuristic with the Gibbs sampling approaches can outperform previous suggested methods. In addition, the methods studied here are flexible and allow the incorporation of additional prior knowledge, such as the nonnegativity of the approximation coefficients, which was found to offer additional benefits where applicable. IS - 9 JF - Signal Processing, IEEE Transactions on [see also Acoustics, Speech, and Signal Processing, IEEE Transactions on] EP - 4486 TI - Monte Carlo Methods for Adaptive Sparse Approximations of Time-Series VL - 55 T2 - Signal Processing, IEEE Transactions on [see also Acoustics, Speech, and Signal Processing, IEEE Transactions on] SP - 4474 KW - sparse-coding AU - Blumensath, T AU - Davies, ME PY - 2007/// UR - http://dx.doi.org/10.1109/TSP.2007.896242 ER -