CiteULike: mshafiei's watchlist

Local Rademacher Complexities

Peter Bartlett — 2008-07-18T01:52:09-00:00

The Annals of Statistics, Vol. 33, No. 4. (2005), pp. 1497-1537.

We propose new bounds on the error of learning algorithms in terms of a data-dependent notion of complexity. The estimates we establish give optimal rates and are based on a local and empirical version of Rademacher averages, in the sense that the Rademacher averages are computed from the data, on a subset of functions with small empirical error. We present some applications to classification and prediction with convex function classes, and with kernel classes in particular.

Improving maximum margin matrix factorization

Markus Weimer — 2008-08-20T16:26:21-00:00

Machine Learning, Vol. 72, No. 3. (2008), pp. 263-276.

Abstract Collaborative filtering is a popular method for personalizing product recommendations. Maximum Margin Matrix Factorization (MMMF) has been proposed as one successful learning approach to this task and has been recently extended to structured ranking losses. In this paper we discuss a number of extensions to MMMF by introducing offset terms, item dependent regularization and a graph kernel on the recommender graph. We show equivalence between graph kernels and the recent MMMF extensions by Mnih and Salakhutdinov (Advances in Neural Information Processing Systems 20, 2008). Experimental evaluation of the introduced extensions show improved performance over the original MMMF formulation.

Comparison of Bayesian and maximum-likelihood inference of population genetic parameters

Peter Beerli — 2006-02-11T09:30:21-00:00

Bioinformatics, Vol. 22, No. 3. (1 February 2006), pp. 341-345.

Interacting sequential Monte Carlo samplers for trans-dimensional simulation

Ajay Jasra — 2008-08-18T16:51:40-00:00

Comput. Stat. Data Anal., Vol. 52, No. 4. (January 2008), pp. 1765-1791.

The Complexity of Phrase Alignment Problems

John Denero — 2008-08-11T16:08:13-00:00

(June 2008), pp. 25-28.

Getting the Structure Right for Word Alignment: LEAF

Alexander Fraser — 2008-08-11T16:10:09-00:00

pp. 51-60.

Structure compilation: trading structure for features

Percy Liang — 2008-08-13T12:25:11-00:00

(2008)

Transductive learning for statistical machine translation

Nicola Ueffing — 2008-08-14T17:08:29-00:00

(2007)

On population-based simulation for static inference

Ajay Jasra — 2008-07-21T13:47:46-00:00

Statistics and Computing, Vol. 17, No. 3. (2007), pp. 263-279.

Abstract In this paper we present a review of population-based simulation for static inference problems. Such methods can be described as generating a collection of random variables X n n=1,…,N in parallel in order to simulate from some target density π (or potentially sequence of target densities). Population-based simulation is important as many challenging sampling problems in applied statistics cannot be dealt with successfully by conventional Markov chain Monte Carlo (MCMC) methods. We summarize population-based MCMC (Geyer, Computing Science and Statistics: The 23rd Symposium on the Interface, pp. 156–163, 1991; Liang and Wong, J. Am. Stat. Assoc. 96, 653–666, 2001) and sequential Monte Carlo samplers (SMC) (Del Moral, Doucet and Jasra, J. Roy. Stat. Soc. Ser. B 68, 411–436, 2006a), providing a comparison of the approaches. We give numerical examples from Bayesian mixture modelling (Richardson and Green, J. Roy. Stat. Soc. Ser. B 59, 731–792, 1997).

Computational methods for complex stochastic systems: a review of some alternatives to MCMC

Paul Fearnhead — 2008-08-14T19:21:14-00:00

Statistics and Computing, Vol. 18, No. 2. (1 June 2008), pp. 151-171.

Abstract We consider analysis of complex stochastic models based upon partial information. MCMC and reversible jump MCMC are often the methods of choice for such problems, but in some situations they can be difficult to implement; and suffer from problems such as poor mixing, and the difficulty of diagnosing convergence. Here we review three alternatives to MCMC methods: importance sampling, the forward-backward algorithm, and sequential Monte Carlo (SMC). We discuss how to design good proposal densities for importance sampling, show some of the range of models for which the forward-backward algorithm can be applied, and show how resampling ideas from SMC can be used to improve the efficiency of the other two methods. We demonstrate these methods on a range of examples, including estimating the transition density of a diffusion and of a discrete-state continuous-time Markov chain; inferring structure in population genetics; and segmenting genetic divergence data.

Generalized Dirichlet distribution in Bayesian analysis

Tzu-Tsung Wong — 2008-08-14T14:12:12-00:00

Applied Mathematics and Computation, Vol. 97, No. 2-3. (15 December 1998), pp. 165-181.

Generalized Dirichlet distribution has a more general covariance structure than Dirichlet distribution. This makes the generalized Dirichlet distribution to be more practical and useful. The concept of complete neutrality will be used to derive the general moment function for the generalized Dirichlet distribution, and then some properties of the generalized Dirichlet distribution will be established. Similar to the Dirichlet distribution, the generalized Dirichlet distribution will be shown to conjugate to multinominal sampling. Two experiments are designed for studying the differences between the Dirichlet and the generalized Dirichlet distributions in Bayesian analysis. A method for generating samples from a generalized Dirichlet in presented. When a prior distribution is either a Dirichlet or a generalized Dirichlet distribution, the way for constructing such a prior is discussed.

Incorporating non-local information into information extraction systems by Gibbs sampling

Jenny Finkel — 2007-11-14T19:16:28-00:00

(2005), pp. 363-370.

Information theory, multivariate dependence, and genetic network inference

Ilya Nemenman — 2004-11-22T00:17:30-00:00

(7 June 2004)

We define the concept of dependence among multiple variables using maximum entropy techniques and introduce a graphical notation to denote the dependencies. Direct inference of information theoretic quantities from data uncovers dependencies even in undersampled regimes when the joint probability distribution cannot be reliably estimated. The method is tested on synthetic data. We anticipate it to be useful for inference of genetic circuits and other biological signaling networks.

The discovery of structural form.

Charles Kemp — 2008-08-04T11:24:55-00:00

Proceedings of the National Academy of Sciences of the United States of America (31 July 2008)

Algorithms for finding structure in data have become increasingly important both as tools for scientific data analysis and as models of human learning, yet they suffer from a critical limitation. Scientists discover qualitatively new forms of structure in observed data: For instance, Linnaeus recognized the hierarchical organization of biological species, and Mendeleev recognized the periodic structure of the chemical elements. Analogous insights play a pivotal role in cognitive development: Children discover that object category labels can be organized into hierarchies, friendship networks are organized into cliques, and comparative relations (e.g., "bigger than" or "better than") respect a transitive order. Standard algorithms, however, can only learn structures of a single form that must be specified in advance: For instance, algorithms for hierarchical clustering create tree structures, whereas algorithms for dimensionality-reduction create low-dimensional spaces. Here, we present a computational model that learns structures of many different forms and that discovers which form is best for a given dataset. The model makes probabilistic inferences over a space of graph grammars representing trees, linear orders, multidimensional spaces, rings, dominance hierarchies, cliques, and other forms and successfully discovers the underlying structure of a variety of physical, biological, and social domains. Our approach brings structure learning methods closer to human abilities and may lead to a deeper computational understanding of cognitive development.

A Graph Kernel for Protein-Protein Interaction Extraction

Antti Airola — 2008-08-13T15:31:05-00:00

(2008)

How to Make the Most of NE Dictionaries in Statistical NER

Yutaka Sasaki — 2008-08-13T15:29:08-00:00

(2008)

Semi-Supervised Learning (Adaptive Computation and Machine Learning)

2006-11-30T11:16:27-00:00

(01 September 2006)

In the field of machine learning, semi-supervised learning (SSL) occupies the middle ground, between supervised learning (in which all training examples are labeled) and unsupervised learning (in which no label data are given). Interest in SSL has increased in recent years, particularly because of application domains in which unlabeled data are plentiful, such as images, text, and bioinformatics. This first comprehensive overview of SSL presents state-of-the-art algorithms, a taxonomy of the field, selected applications, benchmark experiments, and perspectives on ongoing and future research. Semi-Supervised Learning first presents the key assumptions and ideas underlying the field: smoothness, cluster or low-density separation, manifold structure, and transduction. The core of the book is the presentation of SSL methods, organized according to algorithmic strategies. After an examination of generative models, the book describes algorithms that implement the low-density separation assumption, graph-based methods, and algorithms that perform two-step learning. The book then discusses SSL applications and offers guidelines for SSL practitioners by analyzing the results of extensive benchmark experiments. Finally, the book looks at interesting directions for SSL research. The book closes with a discussion of the relationship between semi-supervised learning and transduction. Olivier Chapelle and Alexander Zien are Research Scientists and Bernhard Schölkopf is Professor and Director at the Max Planck Institute for Biological Cybernetics in Tübingen. Schölkopf is coauthor of Learning with Kernels (MIT Press, 2002) and is a coeditor of Advances in Kernel Methods: Support Vector Learning (1998), Advances in Large-Margin Classifiers (2000), and Kernel Methods in Computational Biology (2004), all published by The MIT Press.

Two-Metric Projection Methods for Constrained Optimization

Eli Gafni — 2008-08-12T22:30:20-00:00

SIAM Journal on Control and Optimization, Vol. 22, No. 6. (1984), pp. 936-964.

Relations Between Two Sets of Variates

Harold Hotelling — 2008-08-12T16:09:15-00:00

Biometrika, Vol. 28, No. 3/4. (1936), pp. 321-377.

Learning Classification with Unlabeled Data

Virginia de Sa — 2005-10-14T09:15:19-00:00

(1993), pp. 112-119.

One of the advantages of supervised learning is that the final error metric is available during training. For classifiers, the algorithm can directly reduce the number of misclassifications on the training set. Unfortunately, when modeling human learning or constructing classifiers for autonomous robots, supervisory labels are often not available or too expensive. In this paper we show that we can substitute for the labels by making use of structure between the pattern distributions to...

An RKHS for Multi-View Learning and Manifold Co-Regularization

Vikas Sindhwani — 2008-08-12T02:50:06-00:00

(2008)

Semi-supervised learning using Gaussian fields and harmonic functions

Xiaojin Zhu — 2008-08-11T22:10:33-00:00

(2003), pp. 912-919.

An approach to semi-supervised learning is proposed that is based on a Gaussian random field model. Labeled and unlabeled data are represented as vertices in a weighted graph, with edge weights encoding the similarity between instances. The learning problem is then formulated in terms of a Gaussian random field on this graph, where the mean of the field is characterized in terms of harmonic functions, and is efficiently obtained using matrix methods or belief propagation. The resulting learning algorithms have intimate connections with random walks, electric networks, and spectral graph theory. We discuss methods to incorporate class priors and the predictions of classifiers obtained by supervised learning. We also propose a method of parameter learning by entropy minimization, and show the algorithm's ability to perform feature selection. Promising experimental results are presented for synthetic data, digit classification, and text classification tasks. 1.

Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions

Xiaojin Zhu — 2008-08-11T22:03:57-00:00

(2003), pp. 912-919.

Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models

Papaspiliopoulos — 2008-05-02T11:05:24-00:00

Biometrika, Vol. 95, No. 1. (6 March 2008), pp. 169-186.

Why Doesn't EM Find Good HMM POS-Taggers?

Mark Johnson — 2007-08-03T15:34:50-00:00

pp. 296-305.

Gibbs Sampling Methods for Stick Breaking Priors

H Ishwaran — 2008-08-10T20:42:45-00:00

Journal of the American Statistical Association (March 2001), pp. 161-173.

A rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson–Dirichlet process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to fit posteriors of Bayesian hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a Pólya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies to stick-breaking priors with a known Pólya urn characterization, that is, priors with an explicit and simple prediction rule. Our second method, the blocked Gibbs sampler, is based on an entirely different approach that works by directly sampling values from the posterior of the random measure. The blocked Gibbs sampler can be viewed as a more general approach because it works without requiring an explicit prediction rule. We find that the blocked Gibbs avoids some of the limitations seen with the Pólya urn approach and should be simpler for nonexperts to use.

Lifted First-Order Belief Propagation

Parag Singla — 2008-07-24T18:13:07-00:00

(2008)

A Systematic Comparison of Phrase-Based, Hierarchical and Syntax-Augmented Statistical MT

Andreas Zollmann — 2008-07-28T09:24:32-00:00

(2008)

Probabilistic synchronous context-free grammar (PSCFG) translation models deﬁne weighted transduction rules that represent translation and reordering oper- ations via nonterminal symbols. In this work, we investigate the source of the im- provements in translation quality reported when using two PSCFG translation mod- els (hierarchical and syntax-augmented), when extending a state-of-the-art phrase- based baseline that serves as the lexical support for both PSCFG models. We isolate the impact on translation quality for several important design decisions in each model. We perform this comparison on three NIST language translation tasks; Chinese-to-English, Arabic-to-English and Urdu-to-English, each representing unique challenges.

Unsupervised Multilingual Learning for Morphological Segmentation

Benjamin Snyder — 2008-08-06T15:16:31-00:00

(June 2008), pp. 737-745.

Inference of Stochastic Finite-State Transducers Using N -Gram Mixtures

Vicente — 2008-08-08T19:42:42-00:00

(2007)

Annealed importance sampling

Radford Neal — 2008-04-14T04:06:42-00:00

Statistics and Computing, Vol. 11, No. 2. (1 April 2001), pp. 125-139.

Simulated annealing—moving from a tractable distribution to a distribution of interest via a sequence of intermediate distributions—has traditionally been used as an inexact method of handling isolated modes in Markov chain samplers. Here, it is shown how one can use the Markov chain transitions for such an annealing sequence to define an importance sampler. The Markov chain aspect allows this method to perform acceptably even for high-dimensional problems, where finding good importance sampling distributions would otherwise be very difficult, while the use of importance weights ensures that the estimates found converge to the correct values as the number of annealing runs increases. This annealed importance sampling procedure resembles the second half of the previously-studied tempered transitions, and can be seen as a generalization of a recently-proposed variant of sequential importance sampling. It is also related to thermodynamic integration methods for estimating ratios of normalizing constants. Annealed importance sampling is most attractive when isolated modes are present, or when estimates of normalizing constants are required, but it may also be more generally useful, since its independent sampling allows one to bypass some of the problems of assessing convergence and autocorrelation in Markov chain samplers.

GIATI: A General Methodology for Finite-State Translation Using Alignments

David — 2008-08-08T19:05:45-00:00

(2004)

Inference of Finite-State Transducers by Using Regular Grammars and Morphisms

Francisco Casacuberta — 2008-08-08T18:58:02-00:00

Bayesian Model Choice via Markov Chain Monte Carlo Methods

Bradley Carlin — 2007-12-14T13:50:26-00:00

Markov chain Monte Carlo (MCMC) integration methods enable the fitting of models of virtually unlimited complexity, and as such have revolutionized the practice of Bayesian data analysis. However, comparison across models may not proceed in a completely analogous fashion, owing to violations of the conditions sufficient to ensure convergence of the Markov chain. In this paper we present a framework for Bayesian model choice, along with an MCMC algorithm that does not suffer from convergence difficulties. Our algorithm applies equally well to problems where only one model is contemplated but its proper size is not known at the outset, such as problems involving integer-valued parameters, multiple changepoints or finite mixture distributions. We illustrate our approach with two published examples.

Bayes Factors

Robert Kass — 2007-10-17T08:49:27-00:00

Journal of the American Statistical Association, Vol. 90, No. 430. (1995), pp. 773-795.

In a 1935 paper and in his book Theory of probability, Jeffresy developed a methodology for quantifying the evidence in favor of a scientific theory. The centerpies was a number, now called the Bayes factor, which is the posterior odds of the null hypothesis when the prior probability on the null is one-half. Although there has been much discussion of Bayesian hypothesis testing in the context of criticism of P-values, less attention has been given to the Bayes as a practical tool of applied statistics. In this article we review and discuss the uses of Bayes factors in the context of five scientific applications in genetics, sports, ecology, sociology, and psychology. We emphasize the following points: From Jeffrey's Bayesian viewpoint, the purpose of hypothesis testing is to evaluate the evidence in favor of a scientific theory. Bayes factors offer a way of evaluating evidence in favor of a null hypothesis. Bayes factors provide a way of incorporating external information into the evaluation of evidence about a hypothesis. Bayes factors are very general and do not require alternative models to be nested. Several techniques are available for computing Bayes factors, including asymptotic approximations that are easy to compute using the output from standard packages that maximize likelihoods. In "non-Bayesian significance tests. The Schwarz criterion (or BIC) gives a rough approximation to the logarithm of the Bayes factor, which is easy to use and does not require evaluation of prior distributions. When one is interested in estimation or prediction, Bayes factors may be converted to weights to be attached to various models so that a composite estimate or prediction may be obtained that takes account of structural or model uncertainty. Algorithms have been proposed that allow model uncertainty to be taken into account when the class of models initially considered is very large. Bayes factors are useful for guiding an evolutionary model-building process. It is important, and feasible, to assess the sensitivity of conclusions to the prior distributions used.

Fractional belief propagation

Wim Wiegerinck — 2008-08-05T22:45:31-00:00

(2002), pp. 438-445.

Markov Chain Monte Carlo and Gibbs Sampling

B Walsh — 2008-08-05T13:57:56-00:00

(2004)

In Defense of One-Vs-All Classification

Ryan Rifkin — 2006-05-03T12:48:59-00:00

Journal of Machine Learning Research, Vol. 5 (January 2004), pp. 101-141.

We consider the problem of multiclass classification. Our main thesis is that a simple "one-vs-all" scheme is as accurate as any other approach, assuming that the underlying binary classifiers are well-tuned regularized classifiers such as support vector machines. This thesis is interesting in that it disagrees with a large body of recent published work on multiclass classification. We support our position by means of a critical review of the existing literature, a substantial collection of carefully controlled experimental work, and theoretical arguments.

distruct: a program for the graphical display of population structure

Noah Rosenberg — 2008-08-04T21:57:00-00:00

Molecular Ecology Notes, Vol. 4, No. 1. (2004), pp. 137-138.

In analysis of multilocus genotypes from structured populations, individual coefficients of membership in subpopulations are often estimated using programs such as structure. distruct provides a general method for visualizing these estimated membership coefficients. Subpopulations are represented as colours, and individuals are depicted as bars partitioned into coloured segments that correspond to membership coefficients in the subgroups. distruct, available at http://www.cmb.usc.edu/~noahr/distruct.html, can also be used to display subpopulation assignment probabilities when individuals are assumed to have ancestry in only one group.

Source population of dispersing rock-wallabies (Petrogale lateralis) idengified by assignment tests on multilocus genotypic data

MDB Eldridge — 2008-08-04T21:50:20-00:00

Molecular Ecology, Vol. 10, No. 12. (2001), pp. 2867-2876.

The ability to confidently idengify or exclude a population as the source of an individual has numerous powerful applications in molecular ecology. Several alternative assignment methods have recently been developed and are yet to be fully evaluated with empirical data. In this study we tested the efficacy of different assignment methods by using a translocated rock-wallaby (Petrogale lateralis) population, of known provenance. Specimens from the translocated population (n = 43), its known source population (n = 30) and four other nearby populations (n = 19201332) were genotyped for 11 polymorphic microsatellite loci. The results idengified Bayesian clustering, frequency and Bayesian methods as the most consistent and accurate, correctly assigning 932013100% of individuals up to a significance threshold of P = 0.01. Performance was variable among the distance-based methods, with the Cavalli-Sforza and Edwards chord distance performing best, whereas Goldstein et al.'s (03B4µ)2 consistently performed poorly. Using Bayesian clustering, frequency and Bayesian methods we then attempted to determine the source of rock-wallabies which have recently recolonized an outcrop (Gardners) 8 km from the nearest rock-wallaby population. Results indicate that the population at Gardners originated via a recent dispersal event from the eastern end of Mt. Caroline. This is only the second published record of dispersal by rock-wallabies between habitat patches and is the longest movement recorded to date. Molecular techniques and methods of analysis are now available to allow detailed studies of dispersal in rock-wallabies and should also be possible for many other taxa.

Association Mapping in Structured Populations

Jonathan Pritchard — 2008-08-04T21:04:37-00:00

The American Journal of Human Genetics, Vol. 67, No. 1. (July 2000), pp. 170-181.

The use, in association studies, of the forthcoming dense genomewide collection of single-nucleotide polymorphisms (SNPs) has been heralded as a potential breakthrough in the study of the genetic basis of common complex disorders. A serious problem with association mapping is that population structure can lead to spurious associations between a candidate marker and a phenotype. One common solution has been to abandon case-control studies in favor of family-based tests of association, such as the transmission/disequilibrium test (TDT), but this comes at a considerable cost in the need to collect DNA from close relatives of affected individuals. In this article we describe a novel, statistically valid, method for case-control association studies in structured populations. Our method uses a set of unlinked genetic markers to infer details of population structure, and to estimate the ancestry of sampled individuals, before using this information to test for associations within subpopulations. It provides power comparable with the TDT in many settings and may substantially outperform it if there are conflicting associations in different subpopulations.

Detecting Wildlife Poaching: Identifying the Origin of Individuals with Bayesian Assignment Tests and Multilocus Genotypes

Stéphanie Manel — 2008-08-04T20:38:47-00:00

Conservation Biology, Vol. 16, No. 3. (2002), pp. 650-659.

Illegal harvesting is a serious threat to the persistence of many plant and animal taxa. The combination of highly polymorphic DNA markers and new statistical methods called "assignment tests" can potentially help detect and thereby reduce poaching. Assignment tests can identify the population of origin of individuals if populations are genetically differentiated. We evaluated the usefulness of two assignment tests to wildlife forensics by applying them to large empirical (microsatellite DNA) data sets from 10 species. We also conducted computer simulations to assess the influence of genetic polymorphism ( heterozygosity) and population differentiation ( FST ) on the performance of the tests. The fully Bayesian assignment test of Pritchard et al. (2000) performed better than the partially Bayesian exclusion test of Cornuet et al. (1999), but the fully Bayesian method requires the assumption that the true population of origin was sampled. The median percentage of individuals correctly assigned for the 10 empirical data sets was 61% and 36% for the assignment and exclusion tests, respectively. Both the empirical and simulated data sets suggest that nearly all individuals can be assigned with high statistical certainty (99.9%) for two highly differentiated populations ( FST2248 0.1520130.2) when 10 loci ( H> 0.6) and samples of 30201350 individuals are used per population. We recommend using both tests when the true population of origin might not have been sampled in the data set. La captura ilegal es una amenaza seria para la persistencia de muchos taxones de plantas y animales. La combinación de marcadores altamente polimórficos de ADN y de nuevos métodos estadísticos ("pruebas de asignación") pueden ayudar potencialmente a detectar, y por lo tanto, areducir la caza ilegal. Las pruebas de asignación pueden ayudar a identificar a la población de origen de individuos si las poblaciones están genéticamente diferenciadas. Evaluamos la utilidad de dos pruebas de asignación en aplicaciones forenses de vida silvestre al aplicarlas a una serie grande de datos empíricos (ADN microsatélite) de 10 especies. También llevamos a cabo simulaciones en computadora para evaluar la influencia del polimorfismo genético ( heterozigocidad) y de la diferenciación poblacional ( FST ) en la ejecución de la prueba. La prueba de asignación completamente Bayesiana de Pritchard et al. (2000) dio mejores resultados que la prueba de Cornuet et al. (1999) parcialmente Bayesiana; sin embargo, el método completamente Bayesiano emplea el supuesto de que la población verdadera de origen fue muestreada. La mediana del porcentaje de individuos correctamente asignados por los 10 juegos de datos empíricos fue de 61% y 36% para las pruebas de asignación y exclusión respectivamente. Tanto los juegos de datos empíricos, como los simulados sugieren que casi todos los individuos pueden ser asignados con un alto nivel de certeza estadística (99.9%) a dos poblaciones altamente diferenciadas ( FST2248 0.1520130.2) cuando se emplean 10 loci ( H < 0.6) y muestras de 30-50 individuos por población. Recomendamos el uso de ambas pruebas cuando la población verdadera puede no haber sido muestreada en las bases de datos.

Markov chain sampling methods for Dirichlet process mixture models

Radford Neal — 2008-08-04T18:30:56-00:00

Microsatellite analysis of genetic variation in black bear populations

D Paetkau — 2008-08-04T18:27:25-00:00

Molecular Ecology, Vol. 3, No. 5. (1994), pp. 489-495.

Measuring levels of genetic variation is an important aspect of conservation genetics The informativeness of such measurements is related to the variability of the genetic markers used; a particular concern in species, such as bears, which are characterized by low levels of genetic variation resulting from low population densities and small effective population sizes We describe the development of microsatellite analysis in bears and its use in assessing interpopulation differences in genetic variation in black bears from three Canadian National Parks These markers are highly variable and allowed identification of dramatic differences in both distribution and amount of variation between populations Low levels of variation were observed in a population from the Island of Newfoundland The significance of interpopulation differences in variability was tested using a likelihood ratio test of estimates of 03B8= 4Neu.

Relation between PLSA and NMF and implications

Eric Gaussier — 2008-04-18T13:07:01-00:00

(2005), pp. 601-602.

Variational extensions to EM and multinomial PCA

W Buntine — 2007-02-01T19:23:42-00:00

(2002)

Several authors in recent years have proposed discrete analogues to principle component analysis intended to handle discrete or positive only data, for instance suited to analyzing sets of documents. Methods include non-negative matrix factorization, probabilistic latent semantic analysis, and latent Dirichlet allocation. This paperbegins with a review of the basic theory of the variational extension to the expectationmaximization algorithm, and then presents discrete component finding...

Convergence rates of the voting Gibbs classifier, with application to Bayesian feature selection

Andrew Ng — 2008-08-04T18:16:18-00:00

(2001)

The Gibbs classifier is a simple approximation to the Bayesian optimal classifier in which one samples from the posterior for the parameter `, and then classifies using the single classifier indexed by that parameter vector. In this paper, we study the Voting Gibbs classifier, which is the extension of this scheme to the full Monte Carlo setting, in which N samples are drawn from the posterior and new inputs are classified by voting the N resulting classifiers. We show that the error of Voting Gibbs converges rapidly to the Bayes optimal rate; in particular the relative error decays at a rapid O(1=N) rate. We also discuss the feature selection problem in the Voting Gibbs context. We show that there is a choice of prior for Voting Gibbs such that the algorithm has high tolerance to the presence of irrelevant features. In particular, the algorithm has sample complexity that is logarithmic in the number of irrelevant features. 1.

Determining the source of individuals: multilocus genotyping in nonequilibrium population genetics

Neil Davies — 2008-08-04T18:04:08-00:00

Trends in Ecology & Evolution, Vol. 14, No. 1. (1 January 1999), pp. 17-21.

Recently founded populations represent an enormous challenge for genetic analysis: new populations are often genetically impoverished, making it hard to find sufficiently variable markers, and what little variation is present tends to be ancestral, rendering phylogenetic methods inappropriate. Recently, novel genetic markers and new statistical analyses have made multilocus genotyping an invaluable tool in the fledgling field of nonequilibrium population genetics. Such advances are not of mere academic interest but address questions of great economic, medical and conservation significance.

Protein names precisely peeled off free text.

S Mika — 2006-02-15T09:12:55-00:00

Bioinformatics, Vol. 20 Suppl 1 (4 August 2004)

MOTIVATION: Automatically identifying protein names from the scientific literature is a pre-requisite for the increasing demand in data-mining this wealth of information. Existing approaches are based on dictionaries, rules and machine-learning. Here, we introduced a novel system that combines a pre-processing dictionary- and rule-based filtering step with several separately trained support vector machines (SVMs) to identify protein names in the MEDLINE abstracts. RESULTS: Our new tagging-system NLProt is capable of extracting protein names with a precision (accuracy) of 75% at a recall (coverage) of 76% after training on a corpus, which was used before by other groups and contains 200 annotated abstracts. For our estimate of sustained performance, we considered partially identified names as false positives. One important issue frequently ignored in the literature is the redundancy in evaluation sets. We suggested some guidelines for removing overly inadequate overlaps between training and testing sets. Applying these new guidelines, our program appeared to significantly out-perform other methods tagging protein names. NLProt was so successful due to the SVM-building blocks that succeeded in utilizing the local context of protein names in the scientific literature. We challenge that our system may constitute the most general and precise method for tagging protein names. AVAILABILITY: http://cubic.bioc.columbia.edu/services/nlprot/

Inference of Population Structure Under a Dirichlet Process Model

John Huelsenbeck — 2008-08-02T21:38:23-00:00

Genetics, Vol. 175, No. 4. (1 April 2007), pp. 1787-1802.

Inferring population structure from genetic data sampled from some number of individuals is a formidable statistical problem. One widely used approach considers the number of populations to be fixed and calculates the posterior probability of assigning individuals to each population. More recently, the assignment of individuals to populations and the number of populations have both been considered random variables that follow a Dirichlet process prior. We examined the statistical behavior of assignment of individuals to populations under a Dirichlet process prior. First, we examined a best-case scenario, in which all of the assumptions of the Dirichlet process prior were satisfied, by generating data under a Dirichlet process prior. Second, we examined the performance of the method when the genetic data were generated under a population genetics model with symmetric migration between populations. We examined the accuracy of population assignment using a distance on partitions. The method can be quite accurate with a moderate number of loci. As expected, inferences on the number of populations are more accurate when theta = 4Neu is large and when the migration rate (4Nem) is low. We also examined the sensitivity of inferences of population structure to choice of the parameter of the Dirichlet process model. Although inferences could be sensitive to the choice of the prior on the number of populations, this sensitivity occurred when the number of loci sampled was small; inferences are more robust to the prior on the number of populations when the number of sampled loci is large. Finally, we discuss several methods for summarizing the results of a Bayesian Markov chain Monte Carlo (MCMC) analysis of population structure. We develop the notion of the mean population partition, which is the partition of individuals to populations that minimizes the squared partition distance to the partitions sampled by the MCMC algorithm. 10.1534/genetics.106.061317

CiteULike: mshafiei's watchlist

Local Rademacher Complexities

Improving maximum margin matrix factorization

Comparison of Bayesian and maximum-likelihood inference of population genetic parameters

Interacting sequential Monte Carlo samplers for trans-dimensional simulation

The Complexity of Phrase Alignment Problems

Getting the Structure Right for Word Alignment: LEAF

Structure compilation: trading structure for features

Transductive learning for statistical machine translation

On population-based simulation for static inference

Computational methods for complex stochastic systems: a review of some alternatives to MCMC

Generalized Dirichlet distribution in Bayesian analysis

Incorporating non-local information into information extraction systems by Gibbs sampling

Information theory, multivariate dependence, and genetic network inference

The discovery of structural form.

A Graph Kernel for Protein-Protein Interaction Extraction

How to Make the Most of NE Dictionaries in Statistical NER

Semi-Supervised Learning (Adaptive Computation and Machine Learning)

Two-Metric Projection Methods for Constrained Optimization

Relations Between Two Sets of Variates

Learning Classification with Unlabeled Data

An RKHS for Multi-View Learning and Manifold Co-Regularization

Semi-supervised learning using Gaussian fields and harmonic functions

Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions

Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models

Why Doesn't EM Find Good HMM POS-Taggers?

Gibbs Sampling Methods for Stick Breaking Priors

Lifted First-Order Belief Propagation

A Systematic Comparison of Phrase-Based, Hierarchical and Syntax-Augmented Statistical MT

Unsupervised Multilingual Learning for Morphological Segmentation

Inference of Stochastic Finite-State Transducers Using N -Gram Mixtures

Annealed importance sampling

GIATI: A General Methodology for Finite-State Translation Using Alignments

Inference of Finite-State Transducers by Using Regular Grammars and Morphisms

Bayesian Model Choice via Markov Chain Monte Carlo Methods

Bayes Factors

Fractional belief propagation

Markov Chain Monte Carlo and Gibbs Sampling

In Defense of One-Vs-All Classification

distruct: a program for the graphical display of population structure

Source population of dispersing rock-wallabies (<i>Petrogale lateralis</i>) idengified by assignment tests on multilocus genotypic data

Association Mapping in Structured Populations

Detecting Wildlife Poaching: Identifying the Origin of Individuals with Bayesian Assignment Tests and Multilocus Genotypes

Markov chain sampling methods for Dirichlet process mixture models

Microsatellite analysis of genetic variation in black bear populations

Relation between PLSA and NMF and implications

Variational extensions to EM and multinomial PCA

Convergence rates of the voting Gibbs classifier, with application to Bayesian feature selection

Determining the source of individuals: multilocus genotyping in nonequilibrium population genetics

Protein names precisely peeled off free text.

Inference of Population Structure Under a Dirichlet Process Model