Thu, 07 Aug 2008 21:53:43 BST CiteULike: briordan's machine-learning http://www.citeulike.org/user/briordan/tag/machine-learning CiteULike.org en-gb Copyright © 2004-2008 citeulike.org http://www.citeulike.org/user/briordan/article/2927808 <i>Physical Review Letters, Vol. 100, No. 23. (2008)</i><br /><br />We propose a new measure (phase-slope index) to estimate the direction of information flux in multivariate time series. This measure (a) is insensitive to mixtures of independent sources, (b) gives meaningful results even if the phase spectrum is not linear, and (c) properly weights contributions from different frequencies. These properties are shown in extended simulations and contrasted to Granger causality which yields highly significant false detections for mixtures of independent sources. An application to electroencephalography data (eyes-closed condition) reveals a clear front-to-back information flow. Robustly Estimating the Flow Direction of Information in Complex Physical Systems Guido Nolte Andreas Ziehe Vadim Nikulin Alois Schlögl Nicole Krämer Tom Brismar Klaus Müller doi:10.1103/PhysRevLett.100.234101 Physical Review Letters, Vol. 100, No. 23. (2008) 2008-06-26T00:26:34-00:00 2008 Physical Review Letters 100 23 APS machine-learning methods http://www.citeulike.org/user/briordan/article/2899719 <i>(2007), pp. 35-55.</i> Mathematical foundations behind Latent Semantic Analysis Dian Martin Michael Berry (2007), pp. 35-55. 2008-06-16T20:06:37-00:00 2007 35 55 Lawrence Erlbaum Associates distributional-similarity lsa machine-learning models http://www.citeulike.org/user/briordan/article/2873816 <i>Machine Learning, Vol. 34, No. 1. (1 February 1999), pp. 71-105.</i><br /><br />This paper presents a model-based, unsupervised algorithm for recovering word boundaries in a natural-language text from which they have been deleted. The algorithm is derived from a probability model of the source that generated the text. The fundamental structure of the model is specified abstractly so that the detailed component models of phonology, word-order, and word frequency can be replaced in a modular fashion. The model yields a language-independent, prior probability distribution on all possible sequences of all possible words over a given alphabet, based on the assumption that the input was generated by concatenating words from a fixed but unknown lexicon. The model is unusual in that it treats the generation of a complete corpus, regardless of length, as a single event in the probability space. Accordingly, the algorithm does not estimate a probability distribution on words; instead, it attempts to calculate the prior probabilities of various word sequences that could underlie the observed text. Experiments on phonemic transcripts of spontaneous speech by parents to young children suggest that our algorithm is more effective than other proposed algorithms, at least when utterance boundaries are given and the text includes a substantial number of short utterances. An Efficient, Probabilistically Sound Algorithm for Segmentation and Word Discovery Michael Brent doi:10.1023/A:1007541817488 Machine Learning, Vol. 34, No. 1. (1 February 1999), pp. 71-105. 2008-06-08T18:10:36-00:00 1999 Machine Learning 34 1 71 105 bayesian cross-situational machine-learning models statistical-learning word-meaning http://www.citeulike.org/user/briordan/article/2860212 <i>(2005)</i> Bayesian Nonparametric Modeling and Data Analysis: An Introduction Timothy Hanson Adam Branscum Wesley Johnson (2005) 2008-06-04T02:45:54-00:00 2005 Elsevier bayesian handbook machine-learning textbook http://www.citeulike.org/user/briordan/article/2854511 <i>Information Processing & Management, Vol. 44, No. 4. (July 2008), pp. 1397-1409.</i><br /><br />Document clustering is an important tool for document collection organization and browsing. In real applications, some limited knowledge about cluster membership of a small number of documents is often available, such as some pairs of documents belonging to the same cluster. This kind of prior knowledge can be served as constraints for the clustering process. We integrate the constraints into the trace formulation of the sum of square Euclidean distance function of K-means. Then,the combined criterion function is transformed into trace maximization, which is further optimized by eigen-decomposition. Our experimental evaluation shows that the proposed semi-supervised clustering method can achieve better performance, compared to three existing methods. Towards effective document clustering: A constrained K-means based approach Guobiao Hu Shuigeng Zhou Jihong Guan Xiaohua Hu doi:10.1016/j.ipm.2008.03.001 Information Processing & Management, Vol. 44, No. 4. (July 2008), pp. 1397-1409. 2008-06-01T11:34:38-00:00 2008 Information Processing & Management 44 4 1397 1409 clustering machine-learning http://www.citeulike.org/user/briordan/article/2853489 <i>(2007)</i> A Nonparametric Bayesian Approach to Modeling Overlapping Clusters KA Heller Z Ghahramani (2007) 2008-05-31T17:43:44-00:00 2007 machine-learning http://www.citeulike.org/user/briordan/article/1823831 <i>(2006), pp. 287-296.</i> Very sparse random projections Ping Li Trevor Hastie Kenneth Church doi:10.1145/1150402.1150436 (2006), pp. 287-296. 2007-10-26T06:10:14-00:00 2006 287 296 ACM Press machine-learning http://www.citeulike.org/user/briordan/article/2842511 <i>(2002), pp. 515-524.</i> Evaluation of Hierarchical Clustering Algorithms for Document Datasets Ying Zhao George Karypis (2002), pp. 515-524. 2008-05-28T18:47:20-00:00 2002 515 524 computational-linguistics machine-learning http://www.citeulike.org/user/briordan/article/2835425 <i>(2001)</i> Derivative dynamic time warping Eamonn Keogh Michael Pazzani (2001) 2008-05-26T18:35:45-00:00 2001 machine-learning http://www.citeulike.org/user/briordan/article/2817924 <i>(2008)</i> Wavelet Methods in Statistics with R GP Nason (2008) 2008-05-20T23:01:13-00:00 2008 Springer machine-learning textbook http://www.citeulike.org/user/briordan/article/2811430 <i>(2005)</i> Discovering Knowledge in Data: An Introduction to Data Mining Daniel Larose (2005) 2008-05-19T02:01:24-00:00 2005 Wiley machine-learning textbook http://www.citeulike.org/user/briordan/article/2796096 <i></i> Multimodal Semantic-Associative Collateral Labelling and Indexing of Still Images Meng Zhu A Badii 2008-05-13T20:51:05-00:00 machine-learning multimodal-processing http://www.citeulike.org/user/briordan/article/2796093 <i></i> Automated vocabulary acquisition and interpretation in multimodal conversational systems Yi Liu Joyce Chai Rong Jin 2008-05-13T20:47:17-00:00 machine-learning multimodal-processing http://www.citeulike.org/user/briordan/article/2796089 <i></i> A multimodal learning interface for word acquistion Dana Ballard Chen Yu 2008-05-13T20:44:56-00:00 machine-learning multimodal-processing word-learning http://www.citeulike.org/user/briordan/article/2631432 <i>Psychonomic Bulletin & Review, Vol. 15, No. 2. (April 2008), pp. 256-271.</i> Generalization and similarity in exemplar models of categorization: Insights from machine learning Jakel Frank Scholkopf Bernhard Wichmann A Felix doi:10.3758/PBR.15.2.256 Psychonomic Bulletin & Review, Vol. 15, No. 2. (April 2008), pp. 256-271. 2008-04-05T05:38:09-00:00 2008 Psychonomic Bulletin & Review 1069-9384 15 2 256 271 Psychonomic Society Publications category-learning machine-learning http://www.citeulike.org/user/briordan/article/2624700 <i>Knowledge and Data Engineering, IEEE Transactions on, Vol. 20, No. 5. (2008), pp. 641-652.</i><br /><br />Feature selection is an important method for improving the efficiency and accuracy of text categorization algorithms by removing redundant and irrelevant terms from the corpus. In this paper, we propose a new supervised feature selection method, named CHIR, which is based on the Chi-square statistic and new statistical data that can measure the positive term-category dependency. We also propose a new text clustering algorithm TCFS, which stands for Text Clustering with Feature Selection. TCFS can incorporate CHIR to identify relevant features (i.e., terms) iteratively, and the clustering becomes a learning process. We compared TCFS and the k-means clustering algorithm in combination with different feature selection methods for various real data sets. Our experimental results show that TCFS with CHIR has better clustering accuracy in terms of the F-measure and the purity. Text Clustering with Feature Selection by Using Statistical Data Yanjun Li Congnan Luo Soon Chung doi:10.1109/TKDE.2007.190740 Knowledge and Data Engineering, IEEE Transactions on, Vol. 20, No. 5. (2008), pp. 641-652. 2008-04-03T00:25:55-00:00 2008 Knowledge and Data Engineering, IEEE Transactions on 20 5 641 652 computational-linguistics machine-learning http://www.citeulike.org/user/briordan/article/1245563 <i>Machine Learning</i> Discovering Significant Patterns Geoffrey Webb doi:10.1007/s10994-007-5006-x Machine Learning 2007-04-23T14:19:19-00:00 Machine Learning machine-learning http://www.citeulike.org/user/briordan/article/2553392 <i>(2007)</i> The Text Mining Handbook: Advanced Approaches to Analyzing Unstructured Data Ronen Feldman James Sanger (2007) 2008-03-19T00:45:51-00:00 2007 Cambridge University Press computational-linguistics machine-learning textbook http://www.citeulike.org/user/briordan/article/2537928 <i>Information Processing & Management, Vol. 44, No. 3. (May 2008), pp. 1008-1021.</i><br /><br />This paper proposes a novel hierarchical learning strategy to deal with the data sparseness problem in semantic relation extraction by modeling the commonality among related classes. For each class in the hierarchy either manually predefined or automatically clustered, a discriminative function is determined in a top-down way. As the upper-level class normally has much more positive training examples than the lower-level class, the corresponding discriminative function can be determined more reliably and guide the discriminative function learning in the lower-level one more effectively, which otherwise might suffer from limited training data. In this paper, two classifier learning approaches, i.e. the simple perceptron algorithm and the state-of-the-art Support Vector Machines, are applied using the hierarchical learning strategy. Moreover, several kinds of class hierarchies either manually predefined or automatically clustered are explored and compared. Evaluation on the ACE RDC 2003 and 2004 corpora shows that the hierarchical learning strategy much improves the performance on least- and medium-frequent relations. Hierarchical learning strategy in semantic relation extraction Guodong Zhou Min Zhang Donghong Ji Qiaoming Zhu doi:10.1016/j.ipm.2007.07.007 Information Processing & Management, Vol. 44, No. 3. (May 2008), pp. 1008-1021. 2008-03-15T22:13:04-00:00 2008 Information Processing & Management 44 3 1008 1021 computational-linguistics machine-learning http://www.citeulike.org/user/briordan/article/2537904 <i>Neural Networks, Vol. 21, No. 2-3. ( 2008), pp. 289-302.</i><br /><br />It is widely acknowledged that categorically organized abstract knowledge plays a significant role in high-order human cognition. Yet, there are many unknown issues about the nature of how categories are internally represented in our mind. Traditionally, it has been considered that there is a single innate internal representation system for categorical knowledge, such as Exemplars, Prototypes, or Rules. However, results of recent empirical and computational studies collectively suggest that the human internal representation system is apparently capable of exhibiting behaviors consistent with various types of internal representation schemes. We, then, hypothesized that humans' representational system as a dynamic mechanism, capable of selecting a representation scheme that meets situational characteristics, including complexities of category structure. The present paper introduces a framework for a cognitive model that integrates robust and flexible internal representation machinery. Three simulation studies were conducted. The results showed that SUPERSET, our new model, successfully exhibited cognitive behaviors that are consistent with three main theories of the human internal representation system. Furthermore, a simulation study on social cognitive behaviors showed that the model was capable of acquiring knowledge with high commonality, even for a category structure with numerous valid conceptualizations. Modeling a flexible representation machinery of human concept learning Toshihiko Matsuka Yasuaki Sakamoto Arieta Chouchourelou doi:10.1016/j.neunet.2007.12.035 Neural Networks, Vol. 21, No. 2-3. ( 2008), pp. 289-302. 2008-03-15T21:58:12-00:00 2008 Neural Networks 21 2-3 289 302 machine-learning models http://www.citeulike.org/user/briordan/article/2537897 <i>Neural Networks, Vol. 21, No. 2-3. ( 2008), pp. 250-256.</i><br /><br />The relationship between thought and language and, in particular, the issue of whether and how language influences thought is still a matter of fierce debate. Here we consider a discrimination task scenario to study language acquisition in which an agent receives linguistic input from an external teacher, in addition to sensory stimuli from the objects that exemplify the overlapping categories that make up the environment. Sensory and linguistic input signals are fused using the Neural Modelling Fields (NMF) categorization algorithm. We find that the agent with language is capable of differentiating object features that it could not distinguish without language. In this sense, the linguistic stimuli prompt the agent to redefine and refine the discrimination capacity of its sensory channels. How language can help discrimination in the Neural Modelling Fields framework Jose Fontanari Leonid Perlovsky doi:10.1016/j.neunet.2007.12.007 Neural Networks, Vol. 21, No. 2-3. ( 2008), pp. 250-256. 2008-03-15T21:53:30-00:00 2008 Neural Networks 21 2-3 250 256 linguistic-relativity machine-learning http://www.citeulike.org/user/briordan/article/2537891 <i>Neural Networks, Vol. 21, No. 2-3. ( 2008), pp. 160-169.</i><br /><br />The self-organising map (SOM) and its variant, visualisation induced SOM (ViSOM), have been known to yield similar results to multidimensional scaling (MDS). However, the exact connection has not been established. In this paper, a review on the SOM and its cost function and topological measures is provided first. We then examine the exact scaling effect of the SOM and ViSOM from their objective functions. The SOM is shown to produce a qualitative, nonmetric scaling, while the local distance-preserving ViSOM produces a quantitative or metric scaling. Their relationship with the principal manifold is also discussed. The SOM-based methods not only produce topological or metric scaling but also provide a principal manifold. Furthermore a growing ViSOM is proposed to aid the adaptive embedding of highly nonlinear manifolds. Examples and comparisons with other embedding methods such as Isomap and local linear embedding are also presented. On multidimensional scaling and the embedding of self-organising maps Hujun Yin doi:10.1016/j.neunet.2007.12.027 Neural Networks, Vol. 21, No. 2-3. ( 2008), pp. 160-169. 2008-03-15T21:47:29-00:00 2008 Neural Networks 21 2-3 160 169 machine-learning http://www.citeulike.org/user/briordan/article/2462930 <i>Knowledge and Data Engineering, IEEE Transactions on, Vol. 20, No. 4. (2008), pp. 462-474.</i><br /><br />In this paper we examine the problem of count data clustering. We analyze this problem using finite mixtures of distributions. The multinomial and the multinomial Dirichlet distributions are widely accepted to model count data. We show that these two distributions cannot be the best choice in all the applications and we propose another model called the multinomial generalized Dirichlet distribution (MGDD) that is the composition of the generalized Dirichlet distribution and the multinomial, in the same way that the multinomial Dirichlet distribution (MDD) is the composition of the Dirichlet and the multinomial. The estimation of the parameters and the determination of the number of components in our model are based on the deterministic annealing expectation-maximization (DAEM) approach and the minimum description length (MDL) criterion, respectively. We compare our method to standard approaches such as multinomial and multinomial Dirichlet mixtures to show its merits. The comparison involves different applications such as spatial color image databases indexing, handwritten digit recognition, and text documents clustering. Clustering of Count Data Using Generalized Dirichlet Multinomial Distributions Nizar Bouguila doi:10.1109/TKDE.2007.190726 Knowledge and Data Engineering, IEEE Transactions on, Vol. 20, No. 4. (2008), pp. 462-474. 2008-03-04T02:56:59-00:00 2008 Knowledge and Data Engineering, IEEE Transactions on 20 4 462 474 machine-learning http://www.citeulike.org/user/briordan/article/2462923 <i>Knowledge and Data Engineering, IEEE Transactions on, Vol. 20, No. 4. (2008), pp. 449-461.</i><br /><br />A novel similarity, neighborhood counting measure, was recently proposed which counts the number of neighborhoods of a pair of data points. This similarity can handle numerical and categorical attributes in a conceptually uniform way, can be calculated efficiently through a simple formula, and gives good performance when tested in the framework of k-nearest neighbor classifier. In particular it consistently outperforms a combination of the classical Euclidean distance and Hamming distance. This measure was also shown to be related to a probability formalism, G probability, which is induced from a target probability function P. It was however unclear how G is related to P, especially for classification. Therefore it was not possible to explain some characteristic features of the neighborhood counting measure. In this paper we show that G is a linear function of P, and G-based Bayes classification is equivalent to P-based Bayes classification. We also show that the k-nearest neighbor classifier, when weighted by the neighborhood counting measure, is in fact an approximation of the G-based Bayes classifier, and furthermore, the P-based Bayes classifier. Additionally we show that the neighborhood counting measure remains unchanged when binary attributes are treated as categorical or numerical data. This is a feature that is not shared by other distance measures, to the best of our knowledge. This study provides a theoretical insight into the neighborhood counting measure. A Study of the Neighborhood Counting Similarity Hui Wang Fionn Murtagh doi:10.1109/TKDE.2007.190721 Knowledge and Data Engineering, IEEE Transactions on, Vol. 20, No. 4. (2008), pp. 449-461. 2008-03-04T02:54:53-00:00 2008 Knowledge and Data Engineering, IEEE Transactions on 20 4 449 461 distributional-similarity machine-learning http://www.citeulike.org/user/briordan/article/2362997 <i>IEEE Transactions on Systems, Man and Cybernetics, Part A (to appear)</i> Connectionism, controllers and a brain theory Roy Asim IEEE Transactions on Systems, Man and Cybernetics, Part A (to appear) 2008-02-11T14:12:25-00:00 IEEE Transactions on Systems, Man and Cybernetics, Part A general-psycholinguistics machine-learning http://www.citeulike.org/user/briordan/article/2355325 <i>Science, Vol. 319, No. 5864. (8 February 2008), 726d.</i><br /><br />Affinity propagation (AP) can be viewed as a generalization of the vertex substitution heuristic (VSH), whereby probabilistic exemplar substitutions are performed concurrently. Although results on small data sets ([≤]900 points) demonstrate that VSH is competitive with AP, we found VSH to be prohibitively slow for moderate-to-large problems, whereas AP was much faster and could achieve lower error. 10.1126/science.1151268 Response to Comment on "Clustering by Passing Messages Between Data Points" Brendan Frey Delbert Dueck doi:10.1126/science.1151268 Science, Vol. 319, No. 5864. (8 February 2008), 726d. 2008-02-09T01:30:51-00:00 2008 Science 319 5864 726d machine-learning http://www.citeulike.org/user/briordan/article/2321812 <i>Artificial Intelligence Review, Vol. 26, No. 3. (November 2006), pp. 159-190.</i><br /><br />Abstract  Supervised classification is one of the tasks most frequently carried out by so-called Intelligent Systems. Thus, a large number of techniques have been developed based on Artificial Intelligence (Logic-based techniques, Perceptron-based techniques) and Statistics (Bayesian Networks, Instance-based techniques). The goal of supervised learning is to build a concise model of the distribution of class labels in terms of predictor features. The resulting classifier is then used to assign class labels to the testing instances where the values of the predictor features are known, but the value of the class label is unknown. This paper describes various classification algorithms and the recent attempt for improving classification accuracy—ensembles of classifiers. Machine learning: a review of classification and combining techniques S Kotsiantis I Zaharakis P Pintelas doi:10.1007/s10462-007-9052-3 Artificial Intelligence Review, Vol. 26, No. 3. (November 2006), pp. 159-190. 2008-02-02T01:48:23-00:00 2006 Artificial Intelligence Review 26 3 159 190 machine-learning http://www.citeulike.org/user/briordan/article/312476 <i>Proc Natl Acad Sci U S A, Vol. 102, No. 33. (16 August 2005), pp. 11629-11634.</i><br /><br />We address the problem, fundamental to linguistics, bioinformatics, and certain other disciplines, of using corpora of raw symbolic sequential data to infer underlying rules that govern their production. Given a corpus of strings (such as text, transcribed speech, chromosome or protein sequence data, sheet music, etc.), our unsupervised algorithm recursively distills from it hierarchically structured patterns. The adios (automatic distillation of structure) algorithm relies on a statistical method for pattern extraction and on structured generalization, two processes that have been implicated in language acquisition. It has been evaluated on artificial context-free grammars with thousands of rules, on natural languages as diverse as English and Chinese, and on protein data correlating sequence with function. This unsupervised algorithm is capable of learning complex syntax, generating grammatical novel sentences, and proving useful in other fields that call for structure discovery from raw data, such as bioinformatics. Unsupervised learning of natural languages. Z Solan D Horn E Ruppin S Edelman doi:10.1073/pnas.0409746102 Proc Natl Acad Sci U S A, Vol. 102, No. 33. (16 August 2005), pp. 11629-11634. 2005-09-07T07:16:40-00:00 2005 Proc Natl Acad Sci U S A 0027-8424 102 33 11629 11634 computational-linguistics machine-learning models syntactic-acquisition http://www.citeulike.org/user/briordan/article/2198867 <i>(2007)</i> Statistical Computing with R Maria Rizzo (2007) 2008-01-06T01:46:17-00:00 2007 Chapman & Hall/CRC handbook machine-learning textbook http://www.citeulike.org/user/briordan/article/2192875 <i>(2007)</i> Using two-stage conditional word frequency models to model word burstiness and motivating TF-IDF Peter Sunehag (2007) 2008-01-04T01:56:40-00:00 2007 computational-lexical-semantics distributional-similarity machine-learning http://www.citeulike.org/user/briordan/article/2192865 <i>(2007)</i> A latent space approach to dynamic embedding of co-occurrence data Purnamrita Sarkar Sajid Siddiqi Geoffrey Gordon (2007) 2008-01-04T01:51:27-00:00 2007 distributional-similarity machine-learning http://www.citeulike.org/user/briordan/article/2186656 <i>(2007)</i> Incorporating prior knowledge on features into learning Eyal Krupka Naftali Tishby (2007) 2008-01-02T01:44:40-00:00 2007 machine-learning http://www.citeulike.org/user/briordan/article/2186649 <i>(2007)</i> Hidden topic Markov models Amit Gruber Michal Rosen-Zvi Yair Weiss (2007) 2008-01-02T01:39:25-00:00 2007 distributional-similarity machine-learning models topics-model http://www.citeulike.org/user/briordan/article/2186645 <i>(2007)</i> Fast search for Dirichlet process mixture models Hal Daume (2007) 2008-01-02T01:34:52-00:00 2007 machine-learning http://www.citeulike.org/user/briordan/article/2186641 <i>(2007)</i> Visualizing similarity data with a mixture of maps James Cook Ilya Sutskever Andriy Mnih Geoffrey Hinton (2007) 2008-01-02T01:30:24-00:00 2007 machine-learning word-association http://www.citeulike.org/user/briordan/article/2186636 <i>(2007)</i> The Laplacian eigenmaps latent variable model Miguel Careirra-Perpinan Zhengdong Lu (2007) 2008-01-02T01:24:50-00:00 2007 machine-learning http://www.citeulike.org/user/briordan/article/2178376 <i>Artificial Intelligence Review, Vol. 25, No. 1. (April 2006), pp. 55-65.</i><br /><br />Abstract  We describe Semantic Equivalence and Textual Entailment Recognition, and outline a system which uses a number of lexical, syntactic and semantic features to classify pairs of sentences as “semantically equivalent”. We describe an experiment to show how syntactic and semantic features improve the performance of an earlier system, which used only lexical features. We also outline some areas for future work. Identifying semantic equivalence for multi-document summarisation Eamonn Newman Joe Carthy John Dunnion Nicola Stokes doi:10.1007/s10462-007-9018-5 Artificial Intelligence Review, Vol. 25, No. 1. (April 2006), pp. 55-65. 2007-12-29T01:46:54-00:00 2006 Artificial Intelligence Review 25 1 55 65 computational-linguistics machine-learning http://www.citeulike.org/user/briordan/article/2069894 <i>Proceedings of the National Academy of Sciences (5 December 2007), 0709640104.</i><br /><br />We describe two recently proposed randomized algorithms for the construction of low-rank approximations to matrices, and demonstrate their application (inter alia) to the evaluation of the singular value decompositions of numerically low-rank matrices. Being probabilistic, the schemes described here have a finite probability of failure; in most cases, this probability is rather negligible (1017 is a typical value). In many situations, the new procedures are considerably more efficient and reliable than the classical (deterministic) ones; they also parallelize naturally. We present several numerical examples to illustrate the performance of the schemes. 10.1073/pnas.0709640104 Randomized algorithms for the low-rank approximation of matrices Edo Liberty Franco Woolfe Per-Gunnar Martinsson Vladimir Rokhlin Mark Tygert doi:10.1073/pnas.0709640104 Proceedings of the National Academy of Sciences (5 December 2007), 0709640104. 2007-12-06T23:55:18-00:00 2007 Proceedings of the National Academy of Sciences 0709640104 machine-learning http://www.citeulike.org/user/briordan/article/2134463 <i>(2007)</i> Seeking The Truly Correlated Topic Posterior: on tight approximate inference of logistic-normal admixture model Amr Ahmed Eric Xing (2007) 2007-12-17T02:03:17-00:00 2007 distributional-similarity machine-learning topics-model http://www.citeulike.org/user/briordan/article/1940101 <i>NeuroImage, Vol. 38, No. 3. (15 November 2007), pp. 478-487.</i><br /><br />Dynamic causal modelling (DCM) is a modelling framework used to describe causal interactions in dynamical systems. It was developed to infer the causal architecture of networks of neuronal populations in the brain [Friston, K.J., Harrison, L, Penny, W., 2003. Dynamic causal modelling. NeuroImage. Aug; 19 (4): 1273-302]. In current formulations of DCM, the mean structure of the likelihood is a nonlinear and numerical function of the parameters, which precludes exact or analytic Bayesian inversion. To date, approximations to the posterior depend on the assumption of normality (i.e., the Laplace assumption). In particular, two arguments have been used to motivate normality of the prior and posterior distributions. First, Gaussian priors on the parameters are specified carefully to ensure that activity in the dynamic system of neuronal populations converges to a steady state (i.e., the dynamic system is dissipative). Secondly, normality of the posterior is an approximation based on general asymptotic results, regarding the form of the posterior under infinite data [Friston, K.J., Harrison, L, Penny, W., 2003. Dynamic causal modelling. NeuroImage. Aug; 19 (4): 1273-302]. Here, we provide a critique of these assumptions and evaluate them numerically. We use a Bayesian inversion scheme (the Metropolis-Hastings algorithm) that eschews both assumptions. This affords an independent route to the posterior and an external means to assess the performance of conventional schemes for DCM. It also allows us to assess the sensitivity of the posterior to different priors. First, we retain the conventional priors and compare the ensuing approximate posterior (Laplace) to the exact posterior (MCMC). Our analyses show that the Laplace approximation is appropriate for practical purposes. In a second, independent set of analyses, we compare the exact posterior under conventional priors with an exact posterior under newly defined uninformative priors. Reassuringly, we observe that the posterior is, for all practical purposes, insensitive of the choice of prior. A Metropolis-Hastings algorithm for dynamic causal models Justin Chumbley Karl Friston Tom Fearn Stefan Kiebel doi:10.1016/j.neuroimage.2007.07.028 NeuroImage, Vol. 38, No. 3. (15 November 2007), pp. 478-487. 2007-11-19T22:34:31-00:00 2007 NeuroImage 38 3 478 487 bayesian machine-learning http://www.citeulike.org/user/briordan/article/2126067 <i>Information Processing & Management, Vol. 44, No. 1. (January 2008), pp. 163-180.</i><br /><br />Rocchio relevance feedback and latent semantic indexing (LSI) are well-known extensions of the vector space model for information retrieval (IR). This paper analyzes the statistical relationship between these extensions. The analysis focuses on each method's basis in least-squares optimization. Noting that LSI and Rocchio relevance feedback both alter the vector space model in a way that is in some sense least-squares optimal, we ask: what is the relationship between LSI's and Rocchio's notions of optimality? What does this relationship imply for IR? Using an analytical approach, we argue that Rocchio relevance feedback is optimal if we understand retrieval as a simplified classification problem. On the other hand, LSI's motivation comes to the fore if we understand it as a biased regression technique, where projection onto a low-dimensional orthogonal subspace of the documents reduces model variance. Query expansion and dimensionality reduction: Notions of optimality in Rocchio relevance feedback and latent semantic indexing Miles Efron doi:10.1016/j.ipm.2006.12.008 Information Processing & Management, Vol. 44, No. 1. (January 2008), pp. 163-180. 2007-12-16T02:06:05-00:00 2008 Information Processing & Management 44 1 163 180 computational-linguistics distributional-similarity machine-learning http://www.citeulike.org/user/briordan/article/2113835 <i>pp. 1-24.</i><br /><br />Highly recurrent neural networks can learn reverberating circuits called Cell Assemblies (CAs). These networks can be used to categorize input, and this paper explores the ability of CAs to learn hierarchical categories. A simulator, based on spiking fatiguing leaky integrators, is presented with instances of base categories. Learning is done using a compensatory Hebbian learning rule. The model takes advantage of overlapping CAs where neurons may participate in more than one CA. Using the unsupervised compensatory learning rule, the networks learn a hierarchy of categories that correctly categorize 97% of the basic level presentations of the input in our test. It categorizes 100% of the super-categories correctly. A larger hierarchy is learned that correctly categorizes 100% of base categories, and 89% of super-categories. It is also shown how novel subcategories gain default information from their super-category. These simulations show that networks containing CAs can be used to learn hierarchical categories. The network then can successfully categorize novel inputs. Creating hierarchical categories using cell assemblies Christian Huyck pp. 1-24. 2007-12-14T14:12:06-00:00 1 24 machine-learning http://www.citeulike.org/user/briordan/article/2113349 <i>IEEE Transactions on Knowledge and Data Engineering (16 November 2007)</i> Clustering of Count Data Using Generalized Dirichlet Multinomial Distributions Nizar Bouguila IEEE Transactions on Knowledge and Data Engineering (16 November 2007) 2007-12-14T12:18:10-00:00 2007 IEEE Transactions on Knowledge and Data Engineering IEEE Computer Society Digital Library machine-learning http://www.citeulike.org/user/briordan/article/2113337 <i>IEEE Transactions on Knowledge and Data Engineering (7 December 2007)</i> Explaining Classifications For Individual Instances Marko Robnik-Sikon Igor Kononenko IEEE Transactions on Knowledge and Data Engineering (7 December 2007) 2007-12-14T12:13:22-00:00 2007 IEEE Transactions on Knowledge and Data Engineering IEEE Computer Society Digital Library machine-learning methods http://www.citeulike.org/user/briordan/article/2101496 <i>Neural Networks, Vol. 20, No. 8. (October 2007), pp. 893-903.</i><br /><br />An enhanced self-organizing incremental neural network (ESOINN) is proposed to accomplish online unsupervised learning tasks. It improves the self-organizing incremental neural network (SOINN) [Shen, F., Hasegawa, O. (2006a). An incremental network for on-line unsupervised classification and topology learning. Neural Networks, 19, 90-106] in the following respects: (1) it adopts a single-layer network to take the place of the two-layer network structure of SOINN; (2) it separates clusters with high-density overlap; (3) it uses fewer parameters than SOINN; and (4) it is more stable than SOINN. The experiments for both the artificial dataset and the real-world dataset also show that ESOINN works better than SOINN. An enhanced self-organizing incremental neural network for online unsupervised learning Shen Furao Tomotaka Ogura Osamu Hasegawa doi:10.1016/j.neunet.2007.07.008 Neural Networks, Vol. 20, No. 8. (October 2007), pp. 893-903. 2007-12-13T01:41:43-00:00 2007 Neural Networks 20 8 893 903 machine-learning http://www.citeulike.org/user/briordan/article/1770144 <i>Machine Learning, Vol. 69, No. 1. (4 October 2007), pp. 55-74.</i><br /><br />Abstract   We propose three methods for extending the Boosting family of classifiers motivated by the real-life problems we have encountered. First, we propose a semisupervised learning method for exploiting the unlabeled data in Boosting. We then present a novel classification model adaptation method. The goal of adaptation is optimizing an existing model for a new target application, which is similar to the previous one but may have different classes or class distributions. Finally, we present an efficient and effective cost-sensitive classification method that extends Boosting to allow for weighted classes. We evaluated these methods for call classification in the AT&T VoiceTone� spoken language understanding system. Our results indicate that it is possible to obtain the same classification performance by using 30% less labeled data when the unlabeled data is utilized through semisupervised learning. Using model adaptation we can achieve the same classification accuracy using less than half of the labeled data from the new application. Finally, we present significant improvements in the “important” (i.e., higher weighted) classes without a significant loss in overall performance using the proposed cost-sensitive classification method. Extending boosting for large scale spoken language understanding Gokhan Tur doi:10.1007/s10994-007-5023-9 Machine Learning, Vol. 69, No. 1. (4 October 2007), pp. 55-74. 2007-10-15T12:59:00-00:00 2007 Machine Learning 69 1 55 74 machine-learning http://www.citeulike.org/user/briordan/article/1711091 <i>(2007)</i><br /><br />Pervasive and networked computers have dramatically reduced the cost of collecting and distributing large datasets. In this context, machine learning algorithms that scale poorly could simply become irrelevant. We need learning algorithms that scale linearly with the volume of the data while maintaining enough statistical efficiency to outperform algorithms that simply process a random subset of the data. This volume offers researchers and engineers practical solutions for training kernel machines from large scale datasets, with detailed descriptions of algorithms and experiments carried out on realistically large datasets. After a detailed description of state-of-the-art kernel machine technology, an introduction of the essential concepts discussed in the volume, and a comparison of primal and dual optimization techniques, the book progresses from well-understood techniques to more novel and controversial approaches. Many contributors have made their code and data available online for further experimentation. Topics covered include fast implementations of known algorithms, approximations that are amenable to theoretical guarantees, and algorithms that perform well in practice but are difficult to analyze theoretically. Large-Scale Kernel Machines (2007) 2007-09-30T11:42:22-00:00 2007 MIT Press machine-learning