The Google Similarity Distance Export

@article{citeulike:4487, abstract = {Words and phrases acquire meaning from the way they are used in society, from their relative semantics to other words and phrases. For computers the equivalent of `society' is `database,' and the equivalent of `use' is `way to search the database.' We present a new theory of similarity between words and phrases based on information distance and Kolmogorov complexity. To fix thoughts we use the world-wide-web as database, and Google as search engine. The method is also applicable to other search engines and databases. This theory is then applied to construct a method to automatically extract similarity, the Google similarity distance, of words and phrases from the world-wide-web using Google page counts. The world-wide-web is the largest database on earth, and the context information entered by millions of independent users averages out to provide automatic semantics of useful quality. We give applications in hierarchical clustering, classification, and language translation. We give examples to distinguish between colors and numbers, cluster names of paintings by 17th century Dutch masters and names of books by English novelists, the ability to understand emergencies, and primes, and we demonstrate the ability to do a simple automatic English-Spanish translation. Finally, we use the WordNet database as an objective baseline against which to judge the performance of our method. We conduct a massive randomized trial in binary classification using support vector machines to learn categories based on our Google distance, resulting in an a mean agreement of 87\% with the expert crafted WordNet categories.}, archivePrefix = {arXiv}, author = {Cilibrasi, Rudi and Vitanyi, Paul M. B.}, citeulike-article-id = {4487}, citeulike-linkout-0 = {http://arxiv.org/abs/cs.CL/0412098}, citeulike-linkout-1 = {http://arxiv.org/pdf/cs.CL/0412098}, comment = {This is close to how ECTOR learns...}, day = {30}, eprint = {cs.CL/0412098}, keywords = {ai, automatic-learning, metrics, statistics}, month = {May}, posted-at = {2005-01-28 15:28:05}, priority = {3}, title = {The Google Similarity Distance}, url = {http://arxiv.org/abs/cs.CL/0412098}, year = {2007} }

TY - JOUR ID - citeulike:4487 L3 - citeulike-article-id:4487 N2 - Words and phrases acquire meaning from the way they are used in society, from their relative semantics to other words and phrases. For computers the equivalent of `society' is `database,' and the equivalent of `use' is `way to search the database.' We present a new theory of similarity between words and phrases based on information distance and Kolmogorov complexity. To fix thoughts we use the world-wide-web as database, and Google as search engine. The method is also applicable to other search engines and databases. This theory is then applied to construct a method to automatically extract similarity, the Google similarity distance, of words and phrases from the world-wide-web using Google page counts. The world-wide-web is the largest database on earth, and the context information entered by millions of independent users averages out to provide automatic semantics of useful quality. We give applications in hierarchical clustering, classification, and language translation. We give examples to distinguish between colors and numbers, cluster names of paintings by 17th century Dutch masters and names of books by English novelists, the ability to understand emergencies, and primes, and we demonstrate the ability to do a simple automatic English-Spanish translation. Finally, we use the WordNet database as an objective baseline against which to judge the performance of our method. We conduct a massive randomized trial in binary classification using support vector machines to learn categories based on our Google distance, resulting in an a mean agreement of 87% with the expert crafted WordNet categories. TI - The Google Similarity Distance KW - ai KW - automatic-learning KW - metrics KW - statistics N1 - This is close to how ECTOR learns... AU - Cilibrasi, Rudi AU - Vitanyi, Paul PY - 2007/May/30/ UR - http://arxiv.org/abs/cs.CL/0412098 ER -