Reducing the Dimensionality of Data with Neural Networks

by: GE Hinton, RR Salakhutdinov

Science, Vol. 313, No. 5786. (28 July 2006), pp. 504-507.

View FullText article

DOI, HighWire, Pubmed, Hubmed

Reviews [Write a review of this article]

There are no reviews of this article

Find related articles from these CiteULike users

ancechu, brian, pdlug, ana, barry, garyfeng, mrkrause, yama_tah, neilh, sdvillal, dwf, jwdietrich, rgb, mp4man, josepe, marcusfrean, robincha, structural_bioinformatics (group), ReadingLab (group), MathBio (group), Bioinformatics (group), Cybernetics (group), pileWorks (group), S-Lab (group), Biomedical_Cybernetics (group), AQUA-FONTIS-Study (group), ygarb

Find related articles with these CiteULike tags

algorithm, algorithms, autoencoder, autoencoder_networks, autoencoders, belief, compexity, computation, connectionist, cs, cybernetics, data, deep, deepbeliefnets, dimensionality, dimensionalityreduction, dimensionality_reduction, dimensionality-reduction, dimension-reduction, inference, machinelearning, machine_learning, machine-learning, memory, mining, modeling, nets, network, networks, neural, neural_net, neuralnetwork, neural-network, neural_networks, neural-networks, no-tag, pca, r, rbm, reduction, statistics, terese, unsupervised

Abstract

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. 10.1126/science.1127647

@article{citeulike:778023, abstract = {High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. 10.1126/science.1127647}, author = {Hinton, G. E. and Salakhutdinov, R. R. }, citeulike-article-id = {778023}, doi = {10.1126/science.1127647}, journal = {Science}, keywords = {data, mining, networks, neural}, month = {July}, number = {5786}, pages = {504--507}, posted-at = {2008-03-22 19:48:55}, priority = {2}, title = {Reducing the Dimensionality of Data with Neural Networks}, url = {http://dx.doi.org/10.1126/science.1127647}, volume = {313}, year = {2006} }

RIS record

TY - JOUR ID - citeulike:778023 L3 - citeulike-article-id:778023 TI - Reducing the Dimensionality of Data with Neural Networks JF - Science VL - 313 IS - 5786 SP - 504 EP - 507 N2 - High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. 10.1126/science.1127647 KW - data KW - mining KW - networks KW - neural AU - Hinton, GE AU - Salakhutdinov, RR PY - 2006/07/28/ UR - http://dx.doi.org/10.1126/science.1127647 ER -

RIS BibTeX