CiteULike: briordan's bayesian

Topics in semantic representation

Thomas Griffiths — 2008-06-26T18:39:29-00:00

Psychological Review, Vol. 114, No. 2. (2007), pp. 211-244.

An Efficient, Probabilistically Sound Algorithm for Segmentation and Word Discovery

Michael Brent — 2008-06-08T18:10:36-00:00

Machine Learning, Vol. 34, No. 1. (1 February 1999), pp. 71-105.

This paper presents a model-based, unsupervised algorithm for recovering word boundaries in a natural-language text from which they have been deleted. The algorithm is derived from a probability model of the source that generated the text. The fundamental structure of the model is specified abstractly so that the detailed component models of phonology, word-order, and word frequency can be replaced in a modular fashion. The model yields a language-independent, prior probability distribution on all possible sequences of all possible words over a given alphabet, based on the assumption that the input was generated by concatenating words from a fixed but unknown lexicon. The model is unusual in that it treats the generation of a complete corpus, regardless of length, as a single event in the probability space. Accordingly, the algorithm does not estimate a probability distribution on words; instead, it attempts to calculate the prior probabilities of various word sequences that could underlie the observed text. Experiments on phonemic transcripts of spontaneous speech by parents to young children suggest that our algorithm is more effective than other proposed algorithms, at least when utterance boundaries are given and the text includes a substantial number of short utterances.

Bayesian Nonparametric Modeling and Data Analysis: An Introduction

Timothy Hanson — 2008-06-04T02:45:54-00:00

(2005)

Modeling individual differences using Dirichlet processes

Daniel Navarro — 2007-02-02T16:03:41-00:00

Journal of Mathematical Psychology, Vol. 50, No. 2. (April 2006), pp. 101-122.

We introduce a Bayesian framework for modeling individual differences, in which subjects are assumed to belong to one of a potentially infinite number of groups. In this model, the groups observed in any particular data set are not viewed as a fixed set that fully explains the variation between individuals, but rather as representatives of a latent, arbitrarily rich structure. As more people are seen, and more details about the individual differences are revealed, the number of inferred groups is allowed to grow. We use the Dirichlet process--a distribution widely used in nonparametric Bayesian statistics--to define a prior for the model, allowing us to learn flexible parameter distributions without overfitting the data, or requiring the complex computations typically required for determining the dimensionality of a model. As an initial demonstration of the approach, we present three applications that analyze the individual differences in category learning, choice of publication outlets, and web-browsing behavior.

Nonparametric Bayesian Models of Lexical Acquisition

Sharon Goldwater — 2008-06-03T01:35:18-00:00

(2006)

Principles of generalization for learning sequential structure in language

Michael Frank — 2008-06-01T02:47:26-00:00

(2008)

How many grammars am I holding up? Discovering phonological differences between word classes

Adam Albright — 2008-05-22T02:56:37-00:00

(2008), pp. 1-20.

Bayesian Computation with R

Jim Albert — 2008-05-18T19:23:28-00:00

(2008)

Bayesian Models for Categorical Data

Peter Congdon — 2008-05-18T19:20:28-00:00

(2005)

Compositionality and Statistics in Adjective Acquisition: 4-Year-Olds Interpret Tall and Short Based on the Size Distributions of Novel Noun Referents

David Barner — 2008-05-17T00:13:45-00:00

Child Development, Vol. 79, No. 3. (2008), pp. 594-608.

Four experiments investigated 4-year-olds’ understanding of adjective-noun compositionality and their sensitivity to statistics when interpreting scalar adjectives. In Experiments 1 and 2, children selected tall and short items from 9 novel objects called pimwits (1-9 in. in height) or from this array plus 4 taller or shorter distractor objects of the same kind. Changing the height distributions of the sets shifted children’s tall and short judgments. However, when distractors differed in name and surface features from targets, in Experiment 3, judgments did not shift. In Experiment 4, dissimilar distractors did affect judgments when they received the same name as targets. It is concluded that 4-year-olds deploy a compositional semantics that is sensitive to statistics and mediated by linguistic labels.

A hierarchical process-dissociation model

Jeffrey Rouder — 2008-05-18T01:17:09-00:00

Journal of Experimental Psychology: General, Vol. 137, No. 2. (May 2008), pp. 370-389.

Prior knowledge and exemplar frequency

Harlan Harris — 2008-05-15T17:19:55-00:00

(submitted)

Is a Single-Bladed Knife Enough to Dissect Human Cognition? Commentary on Griffiths et al.

Wai-Tat Fu — 2008-04-30T13:48:53-00:00

Cognitive Science: A Multidisciplinary Journal, Vol. 32, No. 1. (2008), pp. 155-161.

Griffiths, Christian, and Kalish (this issue) present an iterative-learning paradigm applying a Bayesian model to understand inductive biases in categorization. The authors argue that the paradigm is useful as an exploratory tool to understand inductive biases in situations where little is known about the task. It is argued that a theory developed only at the computational level is much like a single-bladed knife that is only useful in highly idealized situations. To be useful as a general tool that cuts through the complex fabric of cognition, we need at least two-bladed scissors that combine both computational and psychological constraints to characterize human behavior. To temper its sometimes expansive claims, it is time to show what a Bayesian model cannot explain. Insight as to how human reality may differ from the Bayesian predictions may shed more light on human cognition than the simpler focus on what the Bayesian approach can explain. There remains much to be done in terms of integrating Bayesian approaches and other approaches in modeling human cognition.

Bayesian brain source imaging based on combined MEG/EEG and fMRI using MCMC

Sung Jun — 2008-04-15T23:06:14-00:00

NeuroImage, Vol. 40, No. 4. (1 May 2008), pp. 1581-1594.

A number of brain imaging techniques have been developed in order to investigate brain function and to develop diagnostic tools for various brain disorders. Each modality has strengths as well as weaknesses compared to the others. Recent work has explored how multiple modalities can be integrated effectively so that they complement one another while maintaining their individual strengths. Bayesian inference employing Markov Chain Monte Carlo (MCMC) techniques provides a straightforward way to combine disparate forms of information while dealing with the uncertainty in each. In this paper we introduce methods of Bayesian inference as a way to integrate different forms of brain imaging data in a probabilistic framework. We formulate Bayesian integration of magnetoencephalography (MEG) data and functional magnetic resonance imaging (fMRI) data by incorporating fMRI data into a spatial prior. The usefulness and feasibility of the method are verified through testing with both simulated and empirical data.

Adding dense, weighted arcs to WordNet

Jordan Boyd-Graber — 2008-04-05T19:54:20-00:00

PUTOP: Turning predominant senses into a topic model for word sense disambiguation

Jordan Boyd-Graber — 2008-04-05T19:52:07-00:00

A topic model for word sense disambiguation

Jordan Boyd-Graber — 2008-04-05T19:47:19-00:00

(2007)

Rational statistical inference and cognitive development

Fei Xu — 2008-03-23T01:13:42-00:00

Learning domain structures

Charles Kemp — 2008-03-09T19:07:03-00:00

(2004)

Theory-based Bayesian models of inductive learning and reasoning

Joshua Tenenbaum — 2006-06-29T12:39:07-00:00

Trends in Cognitive Sciences, Vol. In Press, Corrected Proof

Inductive inference allows humans to make powerful generalizations from sparse data when learning about word meanings, unobserved properties, causal relationships, and many other aspects of the world. Traditional accounts of induction emphasize either the power of statistical learning, or the importance of strong constraints from structured domain knowledge, intuitive theories or schemas. We argue that both components are necessary to explain the nature, use and acquisition of human knowledge, and we introduce a theory-based Bayesian framework for modeling inductive learning and reasoning as statistical inferences over structured knowledge representations.

Bayesian learning of visual chunks by human observers

Gergo Orban — 2008-02-22T01:14:30-00:00

Proceedings of the National Academy of Sciences, Vol. 105, No. 7. (19 February 2008), pp. 2745-2750.

Efficient and versatile processing of any hierarchically structured information requires a learning mechanism that combines lower-level features into higher-level chunks. We investigated this chunking mechanism in humans with a visual pattern-learning paradigm. We developed an ideal learner based on Bayesian model comparison that extracts and stores only those chunks of information that are minimally sufficient to encode a set of visual scenes. Our ideal Bayesian chunk learner not only reproduced the results of a large set of previous empirical findings in the domain of human pattern learning but also made a key prediction that we confirmed experimentally. In accordance with Bayesian learning but contrary to associative learning, human performance was well above chance when pair-wise statistics in the exemplars contained no relevant information. Thus, humans extract chunks from complex visual patterns by generating accurate yet economical representations and not by encoding the full correlational structure of the input. 10.1073/pnas.0708424105

Three case studies in the Bayesian analysis of cognitive models

Michael Lee — 2008-01-29T13:43:31-00:00

pp. 1-15.

Bayesian statistical inference offers a principled and comprehensive approach for relating psychological models to data. This article presents Bayesian analyses of three influential psychological models: multidimensional scaling models of stimulus representation, the generalized context model of category learning, and a signal detection theory model of decision making. In each case, the model is recast as a probabilistic graphical model and is evaluated in relation to a previously considered data set. In each case, it is shown that Bayesian inference is able to provide answers to important theoretical and empirical questions easily and coherently. The generality of the Bayesian approach and its potential for the understanding of models and data in psychology are discussed.

Learning overhypotheses with hierarchical Bayesian models

Kemp — 2007-04-21T23:15:36-00:00

Developmental Science, Vol. 10, No. 3. (May 2007), pp. 307-321.

Adaptive Bayesian Latent Semantic Analysis

JT Chien — 2007-12-21T15:12:10-00:00

IEEE Transactions on Audio, Speech, and Language Processing, Vol. 16, No. 1. (2008), pp. 198-207.

<para> Due to the vast growth of data collections, the statistical document modeling has become increasingly important in language processing areas. Probabilistic latent semantic analysis (PLSA) is a popular approach whereby the semantics and statistics can be effectively captured for modeling. However, PLSA is highly sensitive to task domain, which is continuously changing in real-world documents. In this paper, a novel Bayesian PLSA framework is presented. We focus on exploiting the <emphasis emphasistype="boldital">incremental learning</emphasis> algorithm for solving the <emphasis emphasistype="boldital">updating</emphasis> problem of new domain articles. This algorithm is developed to improve document modeling by incrementally extracting up-to-date latent semantic information to match the changing domains at run time. By adequately representing the priors of PLSA parameters using <emphasis emphasistype="boldital">Dirichlet densities</emphasis>, the posterior densities belong to the same distribution so that a <emphasis emphasistype="boldital">reproducible prior/posterior</emphasis> mechanism is activated for incremental learning from constantly accumulated documents. An incremental PLSA algorithm is constructed to accomplish the parameter estimation as well as the hyperparameter updating. Compared to standard PLSA using maximum likelihood estimate, the proposed approach is capable of performing dynamic document indexing and modeling. We also present the maximum <emphasis emphasistype="boldital">a posteriori</emphasis> PLSA for <emphasis emphasistype="boldital">corrective training</emphasis>. Experiments on information retrieval and document categorization demonstrate the superiority of using Bayesian PLSA methods. </para>

Word learning as Bayesian inference

Fei Xu — 2007-12-17T16:31:23-00:00

Psychological Review, Vol. 114, No. 2. (April 2007), pp. 245-272.

A Metropolis-Hastings algorithm for dynamic causal models

Justin Chumbley — 2007-11-19T22:34:31-00:00

NeuroImage, Vol. 38, No. 3. (15 November 2007), pp. 478-487.

Dynamic causal modelling (DCM) is a modelling framework used to describe causal interactions in dynamical systems. It was developed to infer the causal architecture of networks of neuronal populations in the brain [Friston, K.J., Harrison, L, Penny, W., 2003. Dynamic causal modelling. NeuroImage. Aug; 19 (4): 1273-302]. In current formulations of DCM, the mean structure of the likelihood is a nonlinear and numerical function of the parameters, which precludes exact or analytic Bayesian inversion. To date, approximations to the posterior depend on the assumption of normality (i.e., the Laplace assumption). In particular, two arguments have been used to motivate normality of the prior and posterior distributions. First, Gaussian priors on the parameters are specified carefully to ensure that activity in the dynamic system of neuronal populations converges to a steady state (i.e., the dynamic system is dissipative). Secondly, normality of the posterior is an approximation based on general asymptotic results, regarding the form of the posterior under infinite data [Friston, K.J., Harrison, L, Penny, W., 2003. Dynamic causal modelling. NeuroImage. Aug; 19 (4): 1273-302]. Here, we provide a critique of these assumptions and evaluate them numerically. We use a Bayesian inversion scheme (the Metropolis-Hastings algorithm) that eschews both assumptions. This affords an independent route to the posterior and an external means to assess the performance of conventional schemes for DCM. It also allows us to assess the sensitivity of the posterior to different priors. First, we retain the conventional priors and compare the ensuing approximate posterior (Laplace) to the exact posterior (MCMC). Our analyses show that the Laplace approximation is appropriate for practical purposes. In a second, independent set of analyses, we compare the exact posterior under conventional priors with an exact posterior under newly defined uninformative priors. Reassuringly, we observe that the posterior is, for all practical purposes, insensitive of the choice of prior.

A Bayesian framework for cross-situational word learning

Michael Frank — 2007-12-13T15:38:55-00:00

Computational models of inductive reasoning using a statistical analysis of a Japanese corpus

Kayo Sakamoto — 2007-11-06T04:00:23-00:00

Cognitive Systems Research, Vol. 8, No. 4. (December 2007), pp. 282-299.

Existing computational models of human inductive reasoning have been constructed based on psychological evaluations concerning the similarities or relationships between entities. However, the costs involved in collecting psychological evaluations for the sheer number of entities that exist mean that they are prohibitively impractical. In order to avoid this problem, the present article examines three types of models: a category-based neural network model, a category-based Bayesian model, and a feature-based neural network model. These models utilize the results of a statistical analysis of a Japanese corpus computing co-occurrence probabilities for word pairs, rather than using psychological evaluations. Argument strength ratings collected by a psychological experiment were found to correlate well with simulations for the category-based neural network model.

Connectionist models and Bayesian inference

James Mcclelland — 2007-10-21T02:29:29-00:00

(1999)

Bayesian models of cognition

Thomas Griffiths — 2007-10-21T02:21:33-00:00

(2008)

A Nonparametric Bayesian Method for Inferring Features From Similarity Judgments

Daniel Navarro — 2007-10-21T02:18:15-00:00

(2007)

The additive clustering model is widely used to infer the features of a set of stimuli from their similarities, on the assumption that similarity is a weighted linear function of common features. This paper develops a fully Bayesian formulation of the additive clustering model, using methods from nonparametric Bayesian statistics to allow the number of features to vary. We use this to explore several approaches to parameter estimation, showing that the nonparametric Bayesian approach provides a straightforward way to obtain estimates of both the number of features used in producing similarity judgments and their importance.