Thu, 24 Jul 2008 23:45:10 BST CiteULike: bpacker's library [134 articles] http://www.citeulike.org/user/bpacker CiteULike.org en-gb Copyright © 2004-2008 citeulike.org http://www.citeulike.org/user/bpacker/article/2638542 <i>Advances in Knowledge Discovery and Data Mining (2005), pp. 290-300.</i><br /><br />Hierarchical text classification concerning the relationship among categories has become an interesting problem recently. Most research has focused on tree-structured categories, but in reality directed acyclic graph (DAG) - structured categories, where a child category may have more than one parent category, appear more often. In this paper, we introduce three approaches, namely, flat, tree-based, and DAG-based, for solving the multi-label text classification problem in which categories are organized as a DAG, and documents are classified into both leaf and internal categories. We also present experimental results of the methods using SVMs as classifiers on the Reuters-21578 collection and our data set of research papers in Artificial Intelligence. Keywords: text classification, hierarchies, multi-labels, SVM. Text Classification for DAG-Structured Categories Cao Nguyen Tran Dung Tru Cao doi:10.1007/11430919_36 Advances in Knowledge Discovery and Data Mining (2005), pp. 290-300. 2008-04-07T19:09:52-00:00 2005 Advances in Knowledge Discovery and Data Mining 290 300 classification hierarchy nlp http://www.citeulike.org/user/bpacker/article/2363626 <i>(2006)</i><br /><br />We consider the problem of learning accurate models from multiple sources of “nearby” data. Given distinct samples from multiple data sources and estimates of the dissimilarities between these sources, we provide a general theory of which samples should be used to learn models for each source. This theory is applicable in a broad decision-theoretic learning framework, and yields results for classification and regression generally, and for density estimation within the exponential family. A key component of our approach is the development of approximate triangle inequalities for expected loss, which may be of independent interest. Learning from Multiple sources Koby Crammer Michael Kearns Jennifer Wortman (2006) 2008-02-11T19:32:59-00:00 2006 cotraining hierarchy http://www.citeulike.org/user/bpacker/article/2363617 <i>(2001)</i><br /><br />Many domains are naturally organized in an abstraction hierarchy or taxonomy, where the instances in "nearby" classes in the taxonomy are similar. In this paper, we provide a general probabilistic framework for clustering data into a set of classes organized as a taxonomy, where each class is associated with a probabilistic model from which the data was generated. The clustering algorithm simultaneously optimizes three things: the assignment of data instances to clusters, the models... Probabilistic abstraction hierarchies E Segal D Koller D Ormoneit (2001) 2008-02-11T19:24:32-00:00 2001 hierarchy probabilistic-models http://www.citeulike.org/user/bpacker/article/2355132 <i>(1995), pp. 402-411.</i><br /><br />Each parameter ` in an abstract parameter space Θ is associated with a different probability distribution on a set Y . A parameter ` is chosen at random from Θ according to some a priori distribution on Θ, and n conditionally independent random variables Y n = Y 1 ; : : : Yn are observed with common distribution determined by `. We obtain bounds on the mutual information between the random variable Θ, giving the choice of parameter, and the random variable Y n , giving ... General Bounds on the Mutual Information Between a Parameter and n Conditionally Independent Observations David Haussler Manfred Opper (1995), pp. 402-411. 2008-02-08T23:37:25-00:00 1995 402 411 probabilistic-models statistics http://www.citeulike.org/user/bpacker/article/2355122 <i>Machine Learning, Vol. 28, No. 1. (1 July 1997), pp. 7-39.</i><br /><br />A Bayesian model of learning to learn by sampling from multiple tasks is presented. The multiple tasks are themselves generated by sampling from a distribution over an environment of related tasks. Such an environment is shown to be naturally modelled within a Bayesian context by the concept of an objective prior distribution. It is argued that for many common machine learning problems, although in general we do not know the true (objective) prior for the problem, we do have some idea of a set of possible priors to which the true prior belongs. It is shown that under these circumstances a learner can use Bayesian inference to learn the true prior by learning sufficiently many tasks from the environment. In addition, bounds are given on the amount of information required to learn a task when it is simultaneously learnt with several other tasks. The bounds show that if the learner has little knowledge of the true prior, but the dimensionality of the true prior is small, then sampling multiple tasks is highly advantageous. The theory is applied to the problem of learning a common feature set or equivalently a low-dimensional-representation (LDR) for an environment of related tasks. A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling Jonathan Baxter doi:10.1023/A:1007327622663 Machine Learning, Vol. 28, No. 1. (1 July 1997), pp. 7-39. 2008-02-08T23:32:43-00:00 1997 Machine Learning 28 1 7 39 hierarchy learning-theory probabilistic-models http://www.citeulike.org/user/bpacker/article/2324817 <i>(1998), pp. 583-589.</i> A new composition theorem for learning algorithms Nader Bshouty doi:10.1145/276698.276873 (1998), pp. 583-589. 2008-02-03T04:38:43-00:00 1998 583 589 ACM learning-theory phase-learning http://www.citeulike.org/user/bpacker/article/2324813 <i>Neural Networks, Vol. 14, No. 4-5. (May 2001), pp. 439-458.</i><br /><br />In this paper, learning algorithms for radial basis function (RBF) networks are discussed. Whereas multilayer perceptrons (MLP) are typically trained with backpropagation algorithms, starting the training procedure with a random initialization of the MLP's parameters, an RBF network may be trained in many different ways. We categorize these RBF training methods into one-, two-, and three-phase learning schemes. Two-phase RBF learning is a very common learning scheme. The two layers of an RBF network are learnt separately; first the RBF layer is trained, including the adaptation of centers and scaling parameters, and then the weights of the output layer are adapted. RBF centers may be trained by clustering, vector quantization and classification tree algorithms, and the output layer by supervised learning (through gradient descent or pseudo inverse solution). Results from numerical experiments of RBF classifiers trained by two-phase learning are presented in three completely different pattern recognition applications: (a) the classification of 3D visual objects; (b) the recognition hand-written digits (2D objects); and (c) the categorization of high-resolution electrocardiograms given as a time series (1D objects) and as a set of features extracted from these time series. In these applications, it can be observed that the performance of RBF classifiers trained with two-phase learning can be improved through a third backpropagation-like training phase of the RBF network, adapting the whole set of parameters (RBF centers, scaling parameters, and output layer weights) simultaneously. This, we call three-phase learning in RBF networks. A practical advantage of two- and three-phase learning in RBF networks is the possibility to use unlabeled training data for the first training phase. Support vector (SV) learning in RBF networks is a different learning approach. SV learning can be considered, in this context of learning, as a special type of one-phase learning, where only the output layer weights of the RBF network are calculated, and the RBF centers are restricted to be a subset of the training data. Numerical experiments with several classifier schemes including k-nearest-neighbor, learning vector quantization and RBF classifiers trained through two-phase, three-phase and support vector learning are given. The performance of the RBF classifiers trained through SV learning and three-phase learning are superior to the results of two-phase learning, but SV learning often leads to complex network structures, since the number of support vectors is not a small fraction of the total number of data points. Three learning phases for radial-basis-function networks Friedhelm Schwenker Hans Kestler Gunther Palm doi:10.1016/S0893-6080(01)00027-2 Neural Networks, Vol. 14, No. 4-5. (May 2001), pp. 439-458. 2008-02-03T04:36:06-00:00 2001 Neural Networks 14 4-5 439 458 phase-learning sparse-coding http://www.citeulike.org/user/bpacker/article/2324810 <i>Learning Theory (2006), pp. 199-213.</i><br /><br />The Composition Lemma is one of the strongest tools for learning complex classes. It shows that if a class is learnable then composing the class with a class of polynomial number of concepts gives a learnable class. In this paper we extend the Composition Lemma as follows: we show that composing an attribute efficient learnable class with a learnable class with polynomial shatter coefficient gives a learnable class. This result extends many results in the literature and gives polynomial learning algorithms for new classes. Exact Learning Composed Classes with a Small Number of Mistakes Nader Bshouty Hanna Mazzawi doi:10.1007/11776420_17 Learning Theory (2006), pp. 199-213. 2008-02-03T04:33:34-00:00 2006 Learning Theory 199 213 learning-theory phase-learning http://www.citeulike.org/user/bpacker/article/2306828 <i>(1995), pp. 369-376.</i><br /><br />We consider efficient agnostic learning of linear combinations of basis functions when the sum of absolute values of the weights of the linear combinations is bounded. With the quadratic loss function, we show that the class of linear combinations of a set of basis functions is efficiently agnostically learnable if and only if the class of basis functions is efficiently agnostically learnable. We also show that the sample complexity for learning the linear combinations grows polynomially if and ... On Efficient Agnostic Learning of Linear Combinations of Basis Functions Wee Lee Peter Bartlett Robert Williamson (1995), pp. 369-376. 2008-01-30T10:15:04-00:00 1995 369 376 sparse-coding http://www.citeulike.org/user/bpacker/article/2300741 <i>National Bureau of Economic Research Working Paper Series (July 2003), 9873.</i><br /><br />Author contact info: Marianne Bertrand Graduate School of Business University of Chicago 5807 South Woodlawn Avenue Chicago, IL 60637 Tel: 773/834-5943 Fax: 773/702-0458 E-Mail: marianne.bertrand@gsb.uchicago.edu Sendhil Mullainathan Department of Economics Littauer 208 Harvard University Cambridge, MA 02138 Tel: 617/496-2720 Fax: 617/495-7730 E-Mail: mullain@fas.harvard.edu We perform a field experiment to measure racial discrimination in the labor market. We respond with fictitious resumes to help-wanted ads in Boston and Chicago newspapers. To manipulate perception of race, each resume is assigned either a very African American sounding name or a very White sounding name. The results show significant discrimination against African-American names: White names receive 50 percent more callbacks for interviews. We also find that race affects the benefits of a better resume. For White names, a higher quality resume elicits 30 percent more callbacks whereas for African Americans, it elicits a far smaller increase. Applicants living in better neighborhoods receive more callbacks but, interestingly, this effect does not differ by race. The amount of discrimination is uniform across occupations and industries. Federal contractors and employers who list Equal Opportunity Employer' in their ad discriminate as much as other employers. We find little evidence that our results are driven by employers inferring something other than race, such as social class, from the names. These results suggest that racial discrimination is still a prominent feature of the labor market. Are Emily and Greg More Employable than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination Marianne Bertrand Sendhil Mullainathan National Bureau of Economic Research Working Paper Series (July 2003), 9873. 2008-01-29T04:35:09-00:00 2003 National Bureau of Economic Research Working Paper Series 9873 racism social-science http://www.citeulike.org/user/bpacker/article/2299577 <i>(5 October 2001)</i> Elements of Information Theory Thomas Cover Joy Thomas (5 October 2001) 2008-01-28T21:56:55-00:00 2001 statistics http://www.citeulike.org/user/bpacker/article/142759 <i>(2000), pp. 86-93.</i><br /><br />Recently there has been significant interest in supervised learning algorithms that combine labeled and unlabeled data for text learning tasks. The co-training setting [1] applies to datasets that have a natural separation of their features into two disjoint sets. We demonstrate that when learning from labeled and unlabeled data, algorithms explicitly leveraging a natural independent split of the features outperform algorithms that do not. When a natural split does not exist, co-training... Analyzing the Effectiveness and Applicability of Co-training Kamal Nigam Rayid Ghani (2000), pp. 86-93. 2005-03-29T20:46:17-00:00 2000 86 93 cotraining probabilistic-models http://www.citeulike.org/user/bpacker/article/2278466 <i>Intelligent Data Engineering and Automated Learning – IDEAL 2004 (2004), pp. 765-773.</i><br /><br />We study classification when the majority of data is unlabeled, and only a small fraction is labeled: the so-called semi-supervised learning situation. Blum and Mitchell’s co-training is a popular semi-supervised algorithm [1] to use when we have multiple independent views of the entities to classify. An example of a multi-view situation is classifying web pages: one view may describe the pages by the words that occur on them, another view describes the pages by the words in the hyperlinks that point to them. In co-training two learners each form a model from the labeled data and then incrementally label small subsets of the unlabeled data for each other. The learners then re-estimate their model from the labeled data and the psuedo-labels provided by the learners. Though some analysis of the algorithm’s performance exists [1] the computation performed is still not well understood. We propose that each view in co-training is effectively performing incremental EM as postulated by Neal and Hinton [3], combined with a Bayesian classifier. This analysis suggests improvements over the core co-training algorithm. We introduce variations, which result in faster convergence to the maximum possible accuracy of classification than the core co-training algorithm, and therefore increase the learning efficiency. We empirically verify our claim for a number of data sets in the context of belief network learning. Co-training from an Incremental EM Perspective Minoo Aminian Intelligent Data Engineering and Automated Learning – IDEAL 2004 (2004), pp. 765-773. 2008-01-23T01:32:34-00:00 2004 Intelligent Data Engineering and Automated Learning – IDEAL 2004 765 773 cotraining probabilistic-models http://www.citeulike.org/user/bpacker/article/2232917 <i>(1999)</i> Learning Mixtures of Gaussians Sanjoy Dasgupta (1999) 2008-01-15T02:34:44-00:00 1999 IEEE Computer Society probabilistic-models http://www.citeulike.org/user/bpacker/article/2211878 <i>(2002)</i><br /><br />A number of modern learning tasks involve estimation from heterogeneous information sources. Continuation methods for mixing heterogeneous sources A Corduneanu T Jaakkola (2002) 2008-01-09T19:12:50-00:00 2002 continuation-methods semisupervised-learning http://www.citeulike.org/user/bpacker/article/1866475 <i>(2003)</i><br /><br />Learning with hidden variables is a central challenge in probabilistic graphical models that has important implications for many real-life problems. The classical approach is using the Expectation Maximization (EM) algorithm. This algorithm, however, can get trapped in local maxima. In this paper we explore a new approach that is based on the Information Bottleneck principle. The Information Bottleneck EM algorithm G Elidan N Friedman (2003) 2007-11-05T04:43:43-00:00 2003 continuation-methods probabilistic-models statistics http://www.citeulike.org/user/bpacker/article/2086477 <i>Bernoulli, Vol. 10, No. 6. (2004), pp. 989-1010.</i><br /><br />We show that the `naive Bayes' classifier which assumes independent covariates greatly outperforms the Fisher linear discriminant rule under broad conditions when the number of variables grows faster than the number of observations, in the classical problem of discriminating between two normal populations. We also introduce a class of rules spanning the range between independence and arbitrary dependence. These rules are shown to achieve Bayes consistency for the Gaussian `coloured noise' model and to adapt to a spectrum of convergence rates, which we conjecture to be minimax. Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations Peter Bickel Elizaveta Levina Bernoulli, Vol. 10, No. 6. (2004), pp. 989-1010. 2007-12-10T18:47:42-00:00 2004 Bernoulli 10 6 989 1010 regularization statistics http://www.citeulike.org/user/bpacker/article/2086469 <i>TEST, Vol. 15, No. 2. (2006), pp. 271-344.</i><br /><br />Abstract  This paper is a selective review of the regularization methods scattered in statistics literature. We introduce a general conceptual approach to regularization and fit most existing methods into it. We have tried to focus on the importance of regularization when dealing with today's high-dimensional objects: data and models. A wide range of examples are discussed, including nonparametric regression, boosting, covariance matrix estimation, principal component estimation, subsampling. Regularization in statistics Peter Bickel Bo Li Alexandre Tsybakov Sara van de Geer Bin Yu Teófilo Valdés Carlos Rivero Jianqing Fan Aad van der Vaart doi:10.1007/BF02607055 TEST, Vol. 15, No. 2. (2006), pp. 271-344. 2007-12-10T18:43:06-00:00 2006 TEST 15 2 271 344 boosting regularization statistics http://www.citeulike.org/user/bpacker/article/1808736 <i>Psychological Bulletin, Vol. 71, No. 2. (February 1969), pp. 108-109.</i><br /><br />Considers the problem of the calculation of the bias of the maximum likelihood information estimate H, based on independent choices among k events. The expectation EH is calculated exactly as a function of the probabilities p1, p2, . . . , pkk. The bias H - EH is approximated by using a convergent expansion for a logarithm and using the 1st 2 terms of a finite expansion for the jth moment of a random variable. The resulting approximation is more generally valid, although less concise and simple, than the classical Miller-Madow approximation On the bias of information estimates AG Carlton Psychological Bulletin, Vol. 71, No. 2. (February 1969), pp. 108-109. 2007-10-23T03:12:06-00:00 1969 Psychological Bulletin 71 2 108 109 statistics http://www.citeulike.org/user/bpacker/article/1771591 <i>Network: Computation in Neural Systems, Vol. 7, No. 1. (1996), pp. 87-107.</i><br /><br />Measuring the information carried by neuronal activity is made difficult, particularly when recording from mammalian cells, by the limited amount of data usually available, which results in a systematic error. While empirical ad hoc procedures have been used to correct for such error, we have recently proposed a direct procedure consisting of the analytical calculation of the average error, its estimation (up to subleading terms) from the data, and its subtraction from raw information measures to yield unbiased measures. We calculate here the leading correction terms for both the average transmitted information and the conditional information and, since usually one must first regularize the data, we specify the expressions appropriate to different regularizations. Computer simulations indicate a broad range of validity of the analytical results, suggest the effectiveness of regularizing by simple binning and illustrate the advantage of this over the previously used `bootstrap' procedure. Analytical estimates of limited sampling biases in different information measures Stefano Panzeri Alessandro Treves doi:10.1088/0954-898X/7/1/006 Network: Computation in Neural Systems, Vol. 7, No. 1. (1996), pp. 87-107. 2007-10-15T23:11:18-00:00 1996 Network: Computation in Neural Systems 7 1 87 107 statistics http://www.citeulike.org/user/bpacker/article/1766905 <i></i><br /><br />The construction of appearance-based object detection systems is time-consuming and difficult because a large number of training examples must be collected and manually labeled in order to capture variations in object appearance. Semi-supervised training is a means for reducing the effort needed to prepare the training set by training the model with a small number of fully labeled examples and an additional set of unlabeled or weakly labeled examples. In this work we present a semi-supervised... Semi-Supervised Self-Training of Object Detection Models Chuck Rosenberg Martial Hebert Henry Schneiderman 2007-10-14T14:54:24-00:00 object-detection semisupervised-learning http://www.citeulike.org/user/bpacker/article/1766894 <i>(2000), pp. 86-93.</i> Analyzing the effectiveness and applicability of co-training Kamal Nigam Rayid Ghani doi:10.1145/354756.354805 (2000), pp. 86-93. 2007-10-14T14:50:46-00:00 2000 86 93 ACM Press cotraining semisupervised-learning http://www.citeulike.org/user/bpacker/article/1766875 <i>(2004)</i><br /><br />Co-training is a method for combining labeled and unlabeled data when examples can be thought of as containing two distinct sets of features. It has had a number of practical successes, yet previous theoretical analyses have needed very strong assumptions on the data that are unlikely to be satisfied in practice. Co-training and expansion: Towards bridging theory and practice N Balcan A Bluem K Yang (2004) 2007-10-14T14:45:26-00:00 2004 cotraining http://www.citeulike.org/user/bpacker/article/140030 <i>(1998)</i><br /><br />We consider the problem of using a large unlabeled sample to boost performance of a learning algorithm when only a small set of labeled examples is available. In particular, we consider a problem setting motivated by the task of learning to classify web pages, in which the description of each example can be partitioned into two distinct views. For example, the description of a web page can be partitioned into the words occurring on that page, and the words occurring in hyperlinks that point to... Combining Labeled and Unlabeled Data with Co-training Avrim Blum Tom Mitchell (1998) 2005-03-25T18:29:37-00:00 1998 cotraining semisupervised-learning http://www.citeulike.org/user/bpacker/article/1766865 <i>(2005)</i><br /><br />that these numbers depend on. We provide examples of sample-complexity bounds both for uniform convergence and #-cover based algorithms, as well as several algorithmic results. 21.1 Introduction As we have already seen in the previous chapters, there has been growing interest in using unlabeled data together with labeled data in machine learning, and a number of di#erent approaches have been developed. However, the assumptions these methods are based on are often quite distinct and not... An augmented PAC model for semi-supervised learning M Balcan A Blum (2005) 2007-10-14T14:41:46-00:00 2005 cotraining semisupervised-learning http://www.citeulike.org/user/bpacker/article/1766855 <i>(2003)</i><br /><br />One significant challenge in the construction of visual detection systems is the acquisition of sufficient labeled data. This paper describes a new technique for training visual detectors which requires only a small quantity of labeled data, and then uses unlabeled data to improve performance over time. Unsupervised improvement is based on the cotraining framework of Blum and Mitchell, in which two disparate classifiers are trained simultaneously. Unlabeled examples which are confidently... Unsupervised improvement of visual detectors using cotraining A Levin P Viola Y Freund (2003) 2007-10-14T14:38:57-00:00 2003 cotraining object-detection semisupervised-learning vision http://www.citeulike.org/user/bpacker/article/1617758 <i>No. UCSC-CRL-94-16. (1994)</i><br /><br />We consider two algorithm for on-line prediction based on a linear model. The algorithms are the well-known gradient descent (GD) algorithm and a new algorithm, which we call EG Σ . They both maintain a weight vector using simple updates. For the GD algorithm, the update is based on subtracting the gradient of the squared error made on a prediction. The EG Σ algorithm uses the components of the gradient in the exponents of factors that are used in updating the weight vector... Exponentiated Gradient Versus Gradient Descent for Linear Predictors Jyrki Kivinen Manfred Warmuth No. UCSC-CRL-94-16. (1994) 2007-09-04T08:15:47-00:00 1994 UCSC-CRL-94-16 algorithms optimization http://www.citeulike.org/user/bpacker/article/989397 <i>(2006)</i><br /><br />Image understanding requires not only individually estimating elements of the visual world but also capturing the interplay among them. In this paper, we provide a framework for placing local object detection in the context of the overall 3D scene by modeling the interdependence of objects, surface orientations, and camera viewpoint. Putting Objects in Perspective Derek Hoiem Alexei Efros Martial Hebert (2006) 2006-12-12T08:11:42-00:00 2006 recognition vision http://www.citeulike.org/user/bpacker/article/1577170 <i>(2007), pp. 17-24.</i> Uncovering shared structures in multiclass classification Yonatan Amit Michael Fink Nathan Srebro Shimon Ullman doi:10.1145/1273496.1273499 (2007), pp. 17-24. 2007-08-20T17:35:22-00:00 2007 17 24 ACM Press classification hierarchy product-of-experts vision http://www.citeulike.org/user/bpacker/article/1559169 <i>Journal of Chemometrics, Vol. 18, No. 1. (2004), pp. 37-42.</i><br /><br />Recently the idea of using generalized Procrustes analysis for aligning sets of molecules was introduced using standard algorithms. In this paper it is shown that, by tailoring the algorithm to this specific problem, a great gain in computational speed and memory efficiency can be obtained, but even more importantly, by using rotations without reflection, changes in chirality of molecules can be prevented, which was not previously possible. Copyright © 2004 John Wiley & Sons, Ltd. A dedicated generalized Procrustes algorithm for consensus molecular alignment Jacques Commandeur Pieter Kroonenberg William Dunn doi:10.1002/cem.842 Journal of Chemometrics, Vol. 18, No. 1. (2004), pp. 37-42. 2007-08-14T00:09:55-00:00 2004 Journal of Chemometrics 18 1 37 42 correspondence http://www.citeulike.org/user/bpacker/article/1558368 <i>(2005)</i><br /><br />We describe an algorithm for nonlinear dimensionality reduction based on semidefinite programming and kernel matrix factorization. Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization W Weinberger B Packer L Saul (2005) 2007-08-13T17:29:02-00:00 2005 dimensionality-reduction http://www.citeulike.org/user/bpacker/article/909787 <i>(2004)</i> Learning a kernel matrix for nonlinear dimensionality reduction Kilian Weinberger Fei Sha Lawrence Saul doi:10.1145/1015330.1015345 (2004) 2006-10-22T16:10:42-00:00 2004 ACM Press dimensionality-reduction http://www.citeulike.org/user/bpacker/article/1558325 <i>Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, Vol. 2 (2004), pp. II-988-II-995 Vol.2.</i><br /><br />Can we detect low dimensional structure in high dimensional data sets of images and video? The problem of dimensionality reduction arises often in computer vision and pattern recognition. In this paper, we propose a new solution to this problem based on semidefinite programming. Our algorithm can be used to analyze high dimensional data that lies on or near a low dimensional manifold. It overcomes certain limitations of previous work in manifold learning, such as Isomap and locally linear embedding. We illustrate the algorithm on easily visualized examples of curves and surfaces, as well as on actual images of faces, handwritten digits, and solid objects. Unsupervised learning of image manifolds by semidefinite programming KQ Weinberger LK Saul Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, Vol. 2 (2004), pp. II-988-II-995 Vol.2. 2007-08-13T17:14:31-00:00 2004 Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on 2 II-988 II-995 Vol.2 dimensionality-reduction http://www.citeulike.org/user/bpacker/article/1545446 <i>(2005)</i> Spectral Methods for Dimensionality Reduction Lawrence Saul (2005) 2007-08-09T08:19:11-00:00 2005 dimensionality-reduction http://www.citeulike.org/user/bpacker/article/1538694 <i>(2006), pp. 313-320.</i> Online multiclass learning by interclass hypothesis sharing Michael Fink Shai Shalev-Shwartz Yoram Singer Shimon Ullman doi:10.1145/1143844.1143884 (2006), pp. 313-320. 2007-08-06T20:35:05-00:00 2006 313 320 ACM Press boosting http://www.citeulike.org/user/bpacker/article/686689 <i>(1998)</i><br /><br />. The EM algorithm performs maximum likelihood estimation for data in which some variables are unobserved. We present a function that resembles negative free energy and show that the M step maximizes this function with respect to the model parameters and the E step maximizes it with respect to the distribution over the unobserved variables. From this perspective, it is easy to justify an incremental variant of the EM algorithm in which the distribution for only one of the unobserved variables... A View of the EM Algorithm that Justifies Incremental, Sparse, and other Variants R Neal G Hinton (1998) 2006-06-06T14:32:04-00:00 1998 Kluwer cotraining probabilistic-models http://www.citeulike.org/user/bpacker/article/201537 <i>(2001)</i><br /><br />This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the "Integral Image" which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features ... Rapid object detection using a boosted cascade of simple features P Viola M Jones (2001) 2005-05-16T15:51:34-00:00 2001 boosting vision http://www.citeulike.org/user/bpacker/article/1423550 <i>Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 26, No. 11. (2004), pp. 1475-1490.</i><br /><br />We study the problem of detecting objects in still, gray-scale images. Our primary focus is the development of a learning-based approach to the problem that makes use of a sparse, part-based representation. A vocabulary of distinctive object parts is automatically constructed from a set of sample images of the object class of interest; images are then represented using parts from this vocabulary, together with spatial relations observed among the parts. Based on this representation, a learning algorithm is used to automatically learn to detect instances of the object class in new images. The approach can be applied to any object with distinguishable parts in a relatively fixed spatial configuration; it is evaluated here on difficult sets of real-world images containing side views of cars, and is seen to successfully detect objects in varying conditions amidst background clutter and mild occlusion. In evaluating object detection approaches, several important methodological issues arise that have not been satisfactorily addressed in the previous work. A secondary focus of this paper is to highlight these issues, and to develop rigorous evaluation standards for the object detection problem. A critical evaluation of our approach under the proposed standards is presented. Learning to detect objects in images via a sparse, part-based representation S Agarwal A Awan D Roth Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 26, No. 11. (2004), pp. 1475-1490. 2007-06-29T18:34:16-00:00 2004 Pattern Analysis and Machine Intelligence, IEEE Transactions on 26 11 1475 1490 recognition vision http://www.citeulike.org/user/bpacker/article/899368 <i>(1997)</i><br /><br />This paper contains a large number of matrix identities which cannot be absorbed by mere reading. The reader is encouraged to take time and check each equation by hand and work out the examples. This is advanced material; see Searle (1982) for basic results. 1 Derivatives Old and new matrix algebra useful for statistics T Minka (1997) 2006-10-16T14:25:09-00:00 1997 math http://www.citeulike.org/user/bpacker/article/1229748 <i>(1997)</i><br /><br />We describe a directed acyclic graphical model that contains a hierarchy of linear units and a mechanism for dynamically selecting an appropriate subset of these units to model each observation. The non-linear selection mechanism is a hierarchy of binary units each of which gates the output of one of the linear units. There are no connections from linear units to binary units, so the generative model can be viewed as a logistic belief net (Neal 1992) which selects a skeleton linear model from... A Hierarchical Community of Experts Geoffrey Hinton Brian Sallans Zoubin Ghahramani (1997) 2007-04-16T12:21:21-00:00 1997 Kluwer Academic hierarchy probabilistic-models product-of-experts http://www.citeulike.org/user/bpacker/article/1402724 <i>(1999)</i><br /><br />It is possible to combine multiple probabilistic models of the same data by multiplying the probabilities together and then renormalizing. This is a very ecient way to model highdimensional data which simultaneously satises many dierent low-dimensional constraints because each individual expert model can focus on giving high probability to data vectors that satisfy just one of the constraints. Data vectors that satisfy this one constraint but violate other constraints will be ruled out... Training Products of Experts by Maximizing Contrastive Likelihood G Hinton (1999) 2007-06-21T16:13:55-00:00 1999 probabilistic-models product-of-experts http://www.citeulike.org/user/bpacker/article/1364780 <i>(2000), pp. 158-169.</i><br /><br />. We give a unified account of boosting and logistic regression in which each learning problem is cast in terms of optimization of Bregman distances. The striking similarity of the two problems in this framework allows us to design and analyze algorithms for both simultaneously, and to easily adapt algorithms designed for one problem to the other. For both problems, we give new algorithms and explain their potential advantages over existing methods. These algorithms can be divided into two... Logistic Regression, AdaBoost and Bregman Distances Michael Collins Robert Schapire Yoram Singer (2000), pp. 158-169. 2007-06-04T22:12:37-00:00 2000 158 169 boosting probabilistic-models http://www.citeulike.org/user/bpacker/article/1300309 <i>(1999)</i><br /><br />It is possible to combine multiple probabilistic models of the same data by multiplying the probabilities together and then renormalizing. This is a very ecient way to model high-dimensional data which simultaneously satises many dierent lowdimensional constraints. Each individual expert model can focus on giving high probability to data vectors that satisfy just one of the constraints. Data vectors that satisfy this one constraint but violate other constraints will be ruled out by their low... Products of experts G Hinton (1999) 2007-05-16T15:18:17-00:00 1999 probabilistic-models product-of-experts http://www.citeulike.org/user/bpacker/article/1361536 <i>(2002)</i><br /><br />We describe a modification to the AdaBoost algorithm that permits the incorporation of prior human knowledge as a means of compensating for a shortage of training data. We give a convergence result for the algorithm. Incorporating prior knowledge into boosting R Schapire M Rochery M Rahim N Gupta (2002) 2007-06-04T05:35:26-00:00 2002 boosting priors probabilistic-models http://www.citeulike.org/user/bpacker/article/513211 <i>(1997)</i><br /><br />Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace... Probabilistic principal component analysis M Tipping C Bishop (1997) 2006-02-20T12:43:23-00:00 1997 probabilistic-models http://www.citeulike.org/user/bpacker/article/1353717 <i>Neural Comp., Vol. 14, No. 8. (1 August 2002), pp. 1771-1800.</i><br /><br />It is possible to combine multiple latent-variable models of the same data by multiplying their probability distributions together and then renormalizing. This way of combining individual "expert" models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert, because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data. A product of experts (PoE) is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary. Training a PoE by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule. Fortunately, a PoE can be trained using a different objective function called "contrastive divergence" whose derivatives with regard to the parameters can be approximated accurately and efficiently. Examples are presented of contrastive divergence learning using several types of expert on several types of data. Training Products of Experts by Minimizing Contrastive Divergence Geoffrey Hinton Neural Comp., Vol. 14, No. 8. (1 August 2002), pp. 1771-1800. 2007-06-01T01:43:58-00:00 2002 Neural Comp. 14 8 1771 1800 probabilistic-models product-of-experts http://www.citeulike.org/user/bpacker/article/453442 <i>Lecture Notes in Computer Science, Vol. 1864 (2000), pp. 322-??.</i><br /><br />Most real-world data is stored in relational form. In contrast, most statistical learning methods, e.g., Bayesian network learning, work only with "flat" data representations, forcing us to convert our data into a form that loses much of the relational structure. The recently introduced framework of probabilistic relational models (PRMs) allow us to represent much richer dependency structures, involving multiple entities and the relations between them; they allow the properties of an... Learning Probabilistic Relational Models Lise Getoor Lecture Notes in Computer Science, Vol. 1864 (2000), pp. 322-??. 2005-12-30T17:22:38-00:00 2000 Lecture Notes in Computer Science 1864 322 ?? probabilistic-models http://www.citeulike.org/user/bpacker/article/899534 <i>(1998)</i><br /><br />When documents are organized in a large number of topic categories, the categories are often arranged in a hierarchy. The U.S. patent database and Yahoo are two examples. Improving text classification by shrinkage in a hierarchy of classes A Mccallum R Rosenfeld T Mitchell A Ng (1998) 2006-10-16T16:19:28-00:00 1998 classification hierarchy nlp statistics topic http://www.citeulike.org/user/bpacker/article/1067266 <i>(2002)</i><br /><br />In this paper, the task of determining expected values of sample moments, where the sample members have been selected based on noisy information, is considered. Exact expressions for expected values of sums of products of concomitants of selected order statistics are derived. Then, using Edgeworth and Cornish-Fisher approximations, explicit results that depend on coefficients that can be determined numerically are obtained. While the results are exact only for normal populations, it is... Expected sample moments of concomitants of selected order statistics D Arnold H Beyer (2002) 2007-01-25T17:23:41-00:00 2002 statistics http://www.citeulike.org/user/bpacker/article/1031416 <i>J Neurosci Methods, Vol. 153, No. 2. (15 June 2006), pp. 250-260.</i><br /><br />Synapses play a crucial role in information processing in the brain. Amplitude fluctuations of synaptic responses can be used to extract information about the mechanisms underlying synaptic transmission and its modulation. In particular, multiple-probability fluctuation analysis can be used to estimate the number of functional release sites, the mean probability of release and the amplitude of the mean quantal response from fits of the relationship between the variance and mean amplitude of postsynaptic responses, recorded at different probabilities. To determine these quantal parameters, calculate their uncertainties and the goodness-of-fit of the model, it is important to weight the contribution of each data point in the fitting procedure. We therefore investigated the errors associated with measuring the variance by determining the best estimators of the variance of the variance and have used simulations of synaptic transmission to test their accuracy and reliability under different experimental conditions. For central synapses, which generally have a low number of release sites, the amplitude distribution of synaptic responses is not normal, thus the use of a theoretical variance of the variance based on the normal assumption is not a good approximation. However, appropriate estimators can be derived for the population and for limited sample sizes using a more general expression that involves higher moments and introducing unbiased estimators based on the h-statistics. Our results are likely to be relevant for various applications of fluctuation analysis when few channels or release sites are present. Errors in the estimation of the variance: implications for multiple-probability fluctuation analysis. C Saviane RA Silver doi:10.1016/j.jneumeth.2005.11.003 J Neurosci Methods, Vol. 153, No. 2. (15 June 2006), pp. 250-260. 2007-01-09T11:51:38-00:00 2006 J Neurosci Methods 0165-0270 153 2 250 260 statistics