How to Compare Different Loss Functions and Their Risks Export

@article{Steinwart2007, abstract = {Many learning problems are described by a risk functional which in turn is defined by a loss function, and a straightforward and widely known approach to learn such problems is to minimize a (modified) empirical version of this risk functional. However, in many cases this approach suffers from substantial problems such as computational requirements in classification or robustness concerns in regression. In order to resolve these issues many successful learning algorithms try to minimize a (modified) empirical risk of a surrogate loss function, instead. Of course, such a surrogate loss must be "reasonably related" to the original loss function since otherwise this approach cannot work well. For classification good surrogate loss functions have been recently identified, and the relationship between the excess classification risk and the excess risk of these surrogate loss functions has been exactly described. However, beyond the classification problem little is known on good surrogate loss functions up to now. In this work we establish a general theory that provides powerful tools for comparing excess risks of different loss functions. We then apply this theory to several learning problems including (cost-sensitive) classification, regression, density estimation, and density level detection.}, author = {Steinwart, Ingo}, citeulike-article-id = {1646921}, citeulike-linkout-0 = {http://dx.doi.org/10.1007/s00365-006-0662-3}, citeulike-linkout-1 = {http://www.ingentaconnect.com/content/klu/365/2007/00000026/00000002/00000662}, doi = {10.1007/s00365-006-0662-3}, issn = {0176-4276}, journal = {Constructive Approximation}, keywords = {calibration, class\_probability\_estimation, classification, density\_estimation, loss, surrogate\_loss, theory}, month = {August}, number = {2}, pages = {225--287}, posted-at = {2008-01-15 03:21:15}, priority = {5}, publisher = {Springer}, title = {How to Compare Different Loss Functions and Their Risks}, url = {http://dx.doi.org/10.1007/s00365-006-0662-3}, volume = {26}, year = {2007} }

TY - JOUR ID - Steinwart2007 L3 - citeulike-article-id:1646921 N2 - Many learning problems are described by a risk functional which in turn is defined by a loss function, and a straightforward and widely known approach to learn such problems is to minimize a (modified) empirical version of this risk functional. However, in many cases this approach suffers from substantial problems such as computational requirements in classification or robustness concerns in regression. In order to resolve these issues many successful learning algorithms try to minimize a (modified) empirical risk of a surrogate loss function, instead. Of course, such a surrogate loss must be "reasonably related" to the original loss function since otherwise this approach cannot work well. For classification good surrogate loss functions have been recently identified, and the relationship between the excess classification risk and the excess risk of these surrogate loss functions has been exactly described. However, beyond the classification problem little is known on good surrogate loss functions up to now. In this work we establish a general theory that provides powerful tools for comparing excess risks of different loss functions. We then apply this theory to several learning problems including (cost-sensitive) classification, regression, density estimation, and density level detection. IS - 2 JF - Constructive Approximation SN - 0176-4276 EP - 287 TI - How to Compare Different Loss Functions and Their Risks PB - Springer VL - 26 SP - 225 KW - calibration KW - class_probability_estimation KW - classification KW - density_estimation KW - loss KW - surrogate_loss KW - theory AU - Steinwart, Ingo PY - 2007/08// UR - http://dx.doi.org/10.1007/s00365-006-0662-3 ER -