Statistics: Handle with Care, Detecting Multiple Model Components with the Likelihood Ratio Test TeX Export

@article{citeulike:4311945, abstract = {The likelihood ratio test (LRT) and the related \$F\$ test, do not (even asymptotically) adhere to their nominal \$\chi^2\$ and \$F\$ distributions in many statistical tests common in astrophysics, thereby casting many marginal line or source detections and non-detections into doubt. Although there are many legitimate uses of these statistics, in some important cases it can be impossible to compute the correct false positive rate. For example, it has become common practice to use the LRT or the \$F\$ test for detecting a line in a spectral model or a source above background despite the lack of certain required regularity conditions. In these and other settings that involve testing a hypothesis that is on the boundary of the parameter space, {\it contrary to common practice, the nominal \$\chi^2\$ distribution for the LRT or the \$F\$ distribution for the \$F\$ test should not be used}. In this paper, we characterize an important class of problems where the LRT and the \$F\$ test fail and illustrate this non-standard behavior. We briefly sketch several possible acceptable alternatives, focusing on Bayesian posterior predictive probability-values. We present this method in some detail, as it is a simple, robust, and intuitive approach. This alternative method is illustrated using the gamma-ray burst of May 8, 1997 (GRB 970508) to investigate the presence of an Fe K emission line during the initial phase of the observation.}, archivePrefix = {arXiv}, author = {Protassov, Rostislav and van Dyk, David A. and Connors, Alanna and Kashyap, Vinay L. and Siemiginowska, Aneta}, citeulike-article-id = {4311945}, citeulike-linkout-0 = {http://arxiv.org/abs/astro-ph/0201547}, citeulike-linkout-1 = {http://arxiv.org/pdf/astro-ph/0201547}, citeulike-linkout-2 = {http://dx.doi.org/10.1086/339856}, citeulike-linkout-3 = {http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=2002ApJ...571..545P}, day = {31}, doi = {10.1086/339856}, eprint = {astro-ph/0201547}, journal = {Astrophys. J.}, keywords = {bayes, detection, statistics}, month = {Jan}, pages = {545--559}, posted-at = {2009-04-15 09:54:55}, priority = {2}, title = {Statistics: Handle with Care, Detecting Multiple Model Components with the Likelihood Ratio Test}, url = {http://dx.doi.org/10.1086/339856}, volume = {571}, year = {2002} }

TY - JOUR ID - citeulike:4311945 L3 - citeulike-article-id:4311945 N2 - The likelihood ratio test (LRT) and the related $F$ test, do not (even asymptotically) adhere to their nominal $\chi^2$ and $F$ distributions in many statistical tests common in astrophysics, thereby casting many marginal line or source detections and non-detections into doubt. Although there are many legitimate uses of these statistics, in some important cases it can be impossible to compute the correct false positive rate. For example, it has become common practice to use the LRT or the $F$ test for detecting a line in a spectral model or a source above background despite the lack of certain required regularity conditions. In these and other settings that involve testing a hypothesis that is on the boundary of the parameter space, {\it contrary to common practice, the nominal $\chi^2$ distribution for the LRT or the $F$ distribution for the $F$ test should not be used}. In this paper, we characterize an important class of problems where the LRT and the $F$ test fail and illustrate this non-standard behavior. We briefly sketch several possible acceptable alternatives, focusing on Bayesian posterior predictive probability-values. We present this method in some detail, as it is a simple, robust, and intuitive approach. This alternative method is illustrated using the gamma-ray burst of May 8, 1997 (GRB 970508) to investigate the presence of an Fe K emission line during the initial phase of the observation. JF - Astrophys. J. EP - 559 TI - Statistics: Handle with Care, Detecting Multiple Model Components with the Likelihood Ratio Test VL - 571 SP - 545 KW - bayes KW - detection KW - statistics AU - Protassov, Rostislav AU - van Dyk, David AU - Connors, Alanna AU - Kashyap, Vinay AU - Siemiginowska, Aneta PY - 2002/Jan/31/ UR - http://dx.doi.org/10.1086/339856 ER -