Are loss functions all the same? Export

@article{citeulike:1392538, abstract = {In this letter, we investigate the impact of choosing different loss functions from the viewpoint of statistical learning theory. We introduce a convexity assumption, which is met by all loss functions commonly used in the literature, and study how the bound on the estimation error changes with the loss. We also derive a general result on the minimizer of the expected risk for a convex loss function in the case of classification. The main outcome of our analysis is that for classification, the hinge loss appears to be the loss of choice. Other things being equal, the hinge loss leads to a convergence rate practically indistinguishable from the logistic loss rate and much better than the square loss rate. Furthermore, if the hypothesis space is sufficiently rich, the bounds obtained for the hinge loss are not loosened by the thresholding stage.}, address = {INFM-DISI, Universit\`{a} di Genova, 16146 Genoa, Italy. rosasco@disi.unige.it}, author = {Rosasco, L. and De Vito, E. and Caponnetto, A. and Piana, M. and Verri, A.}, citeulike-article-id = {1392538}, citeulike-linkout-0 = {http://dx.doi.org/10.1162/089976604773135104}, citeulike-linkout-1 = {http://view.ncbi.nlm.nih.gov/pubmed/15070510}, citeulike-linkout-2 = {http://www.hubmed.org/display.cgi?uids=15070510}, comment = {hinge/logistic loss better than squared loss}, doi = {10.1162/089976604773135104}, issn = {0899-7667}, journal = {Neural Comput}, keywords = {statistical-learning-theory}, month = {May}, number = {5}, pages = {1063--1076}, posted-at = {2007-06-15 20:19:04}, priority = {2}, title = {Are loss functions all the same?}, url = {http://dx.doi.org/10.1162/089976604773135104}, volume = {16}, year = {2004} }

TY - JOUR ID - citeulike:1392538 L3 - citeulike-article-id:1392538 N2 - In this letter, we investigate the impact of choosing different loss functions from the viewpoint of statistical learning theory. We introduce a convexity assumption, which is met by all loss functions commonly used in the literature, and study how the bound on the estimation error changes with the loss. We also derive a general result on the minimizer of the expected risk for a convex loss function in the case of classification. The main outcome of our analysis is that for classification, the hinge loss appears to be the loss of choice. Other things being equal, the hinge loss leads to a convergence rate practically indistinguishable from the logistic loss rate and much better than the square loss rate. Furthermore, if the hypothesis space is sufficiently rich, the bounds obtained for the hinge loss are not loosened by the thresholding stage. IS - 5 JF - Neural Comput SN - 0899-7667 AD - INFM-DISI, Università di Genova, 16146 Genoa, Italy. rosasco@disi.unige.it EP - 1076 TI - Are loss functions all the same? VL - 16 SP - 1063 KW - statistical-learning-theory N1 - hinge/logistic loss better than squared loss AU - Rosasco, L AU - De Vito, E AU - Caponnetto, A AU - Piana, M AU - Verri, A PY - 2004/05// UR - http://dx.doi.org/10.1162/089976604773135104 ER -