Distance Metric Learning for Large Margin Nearest Neighbor Classification Export

@article{citeulike:6064490, abstract = {The accuracy of k -nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k -nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner.}, author = {Weinberger, Kilian Q. and Saul, Lawrence K.}, citeulike-article-id = {6064490}, citeulike-linkout-0 = {http://portal.acm.org/citation.cfm?id=1577069.1577078}, issn = {1532-4435}, journal = {J. Mach. Learn. Res.}, keywords = {classification, distance, knn, large, learning, lmnn, mahalanobis, margin, maximization, metric, nn}, pages = {207--244}, posted-at = {2009-11-03 20:39:58}, priority = {0}, publisher = {JMLR.org}, title = {Distance Metric Learning for Large Margin Nearest Neighbor Classification}, url = {http://portal.acm.org/citation.cfm?id=1577069.1577078}, volume = {10}, year = {2009} }

TY - JOUR ID - citeulike:6064490 L3 - citeulike-article-id:6064490 N2 - The accuracy of k -nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k -nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner. JF - J. Mach. Learn. Res. SN - 1532-4435 EP - 244 TI - Distance Metric Learning for Large Margin Nearest Neighbor Classification PB - JMLR.org VL - 10 SP - 207 KW - classification KW - distance KW - knn KW - large KW - learning KW - lmnn KW - mahalanobis KW - margin KW - maximization KW - metric KW - nn AU - Weinberger, Kilian AU - Saul, Lawrence PY - 2009/// UR - http://portal.acm.org/citation.cfm?id=1577069.1577078 ER -