Deterministic annealing for clustering, compression, classification, regression, and related optimization problems

by: K Rose

Proceedings of the IEEE, Vol. 86, No. 11. (1998), pp. 2210-2239.

View FullText article

IEEE Explore

Reviews [Write a review of this article]

There are no reviews of this article

Find related articles from these CiteULike users

ded_maxim, varung, dags (group)

Find related articles with these CiteULike tags

algorithms, annealing, clustering, compression, density-estimation, deterministic, expectation, information-theory, machine-learning, maximization, rate-distortion, search, statistical-mechanics, vector-quantization

Abstract

The deterministic annealing approach to clustering and its extensions has demonstrated substantial performance improvement over standard supervised and unsupervised learning methods in a variety of important applications including compression, estimation, pattern recognition and classification, and statistical regression. The application-specific cost is minimized subject to a constraint on the randomness of the solution, which is gradually lowered. We emphasize the intuition gained from analogy to statistical physics. Alternatively the method is derived within rate-distortion theory, where the annealing process is equivalent to computation of Shannon's rate-distortion function, and the annealing temperature is inversely proportional to the slope of the curve. The basic algorithm is extended by incorporating structural constraints to allow optimization of numerous popular structures including vector quantizers, decision trees, multilayer perceptrons, radial basis functions, and mixtures of experts

@article{citeulike:658282, abstract = {The deterministic annealing approach to clustering and its extensions has demonstrated substantial performance improvement over standard supervised and unsupervised learning methods in a variety of important applications including compression, estimation, pattern recognition and classification, and statistical regression. The application-specific cost is minimized subject to a constraint on the randomness of the solution, which is gradually lowered. We emphasize the intuition gained from analogy to statistical physics. Alternatively the method is derived within rate-distortion theory, where the annealing process is equivalent to computation of Shannon's rate-distortion function, and the annealing temperature is inversely proportional to the slope of the curve. The basic algorithm is extended by incorporating structural constraints to allow optimization of numerous popular structures including vector quantizers, decision trees, multilayer perceptrons, radial basis functions, and mixtures of experts}, author = {Rose, K. }, citeulike-article-id = {658282}, journal = {Proceedings of the IEEE}, keywords = {annealing, deterministic, expectation, maximization, search}, number = {11}, pages = {2210--2239}, posted-at = {2007-05-14 07:21:49}, priority = {4}, title = {Deterministic annealing for clustering, compression, classification, regression, and related optimization problems}, url = {http://ieeexplore.ieee.org/xpls/abs\_all.jsp?arnumber=726788}, volume = {86}, year = {1998} }

RIS record

TY - JOUR ID - citeulike:658282 L3 - citeulike-article-id:658282 TI - Deterministic annealing for clustering, compression, classification, regression, and related optimization problems JF - Proceedings of the IEEE VL - 86 IS - 11 SP - 2210 EP - 2239 N2 - The deterministic annealing approach to clustering and its extensions has demonstrated substantial performance improvement over standard supervised and unsupervised learning methods in a variety of important applications including compression, estimation, pattern recognition and classification, and statistical regression. The application-specific cost is minimized subject to a constraint on the randomness of the solution, which is gradually lowered. We emphasize the intuition gained from analogy to statistical physics. Alternatively the method is derived within rate-distortion theory, where the annealing process is equivalent to computation of Shannon's rate-distortion function, and the annealing temperature is inversely proportional to the slope of the curve. The basic algorithm is extended by incorporating structural constraints to allow optimization of numerous popular structures including vector quantizers, decision trees, multilayer perceptrons, radial basis functions, and mixtures of experts KW - annealing KW - deterministic KW - expectation KW - maximization KW - search AU - Rose, K PY - 1998/// UR - http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=726788 ER -

RIS BibTeX