CiteULike: parmentierf's statistics

Exploring Large Document Collections using Statistical Topic Models

David Newman — 2006-09-11T07:03:28-00:00

(2006)

We will demonstrate the topic model, a recent unsupervised learning technique that uses a statistical model to discover topics in a large collection of text documents. The first demonstration illustrates how the topic model automatically learns about the spectrum of research conducted by faculty members at UC Irvine and UC San Diego, how to topically characterize each researcher’s interests, and how to find researchers with similar interests – all in a completely unsupervised fashion. The second demonstration illustrates how medical researchers may use topic modeling to find new connections between genes and brain regions based on a large collection of articles on schizophrenia.

Unsupervised learning of natural languages.

Z Solan — 2005-09-07T07:16:40-00:00

Proc Natl Acad Sci U S A, Vol. 102, No. 33. (16 August 2005), pp. 11629-11634.

We address the problem, fundamental to linguistics, bioinformatics, and certain other disciplines, of using corpora of raw symbolic sequential data to infer underlying rules that govern their production. Given a corpus of strings (such as text, transcribed speech, chromosome or protein sequence data, sheet music, etc.), our unsupervised algorithm recursively distills from it hierarchically structured patterns. The adios (automatic distillation of structure) algorithm relies on a statistical method for pattern extraction and on structured generalization, two processes that have been implicated in language acquisition. It has been evaluated on artificial context-free grammars with thousands of rules, on natural languages as diverse as English and Chinese, and on protein data correlating sequence with function. This unsupervised algorithm is capable of learning complex syntax, generating grammatical novel sentences, and proving useful in other fields that call for structure discovery from raw data, such as bioinformatics.

Automatic Meaning Discovery Using Google

Rudi Cilibrasi — 2004-12-22T12:39:20-00:00

(21 December 2004)

We propose a new method to extract semantic knowledge from the world-wide-web for both supervised and unsupervised learning using the Google search engine in an unconventional manner. The approach is novel in its unrestricted problem domain, simplicity of implementation, and manifestly ontological underpinnings. We give evidence of elementary learning of the semantics of concepts, in contrast to most prior approaches. The method works as follows: The world-wide-web is the largest database on earth, and it induces a probability mass function, the Google distribution, via page counts for combinations of search queries. This distribution allows us to tap the latent semantic knowledge on the web. Shannon's coding theorem is used to establish a code-length associated with each search query. Viewing this mapping as a data compressor, we connect to earlier work on Normalized Compression Distance. We give applications in (i) unsupervised hierarchical clustering, demonstrating the ability to distinguish between colors and numbers, and to distinguish between 17th century Dutch painters; (ii) supervised concept-learning by example, using Support Vector Machines, demonstrating the ability to understand electrical terms, religious terms, emergency incidents, and by conducting a massive experiment in understanding WordNet categories; and (iii) matching of meaning, in an example of automatic English-Spanish translation.