Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Export

@article{9746022, abstract = {BACKGROUND: Few meta-analyses of randomised trials assess the quality of the studies included. Yet there is increasing evidence that trial quality can affect estimates of intervention efficacy. We investigated whether different methods of quality assessment provide different estimates of intervention efficacy evaluated in randomised controlled trials (RCTs). METHODS: We randomly selected 11 meta-analyses that involved 127 RCTs on the efficacy of interventions used for circulatory and digestive diseases, mental health, and pregnancy and childbirth. We replicated all the meta-analyses using published data from the primary studies. The quality of reporting of all 127 clinical trials was assessed by means of component and scale approaches. To explore the effects of quality on the quantitative results, we examined the effects of different methods of incorporating quality scores (sensitivity analysis and quality weights) on the results of the meta-analyses. FINDINGS: The quality of trials was low. Masked assessments provided significantly higher scores than unmasked assessments (mean 2.74 [SD 1.10] vs 2.55 [1.20]). Low-quality trials (score < or = 2), compared with high-quality trials (score > 2), were associated with an increased estimate of benefit of 34\% (ratio of odds ratios [ROR] 0.66 [95\% CI 0.52-0.83]). Trials that used inadequate allocation concealment, compared with those that used adequate methods, were also associated with an increased estimate of benefit (37\%; ROR=0.63 [0.45-0.88]). The average treatment benefit was 39\% (odds ratio [OR] 0.61 [0.57-0.65]) for all trials, 52\% (OR 0.48 [0.43-0.54]) for low-quality trials, and 29\% (OR 0.71 [0.65-0.77]) for high-quality trials. Use of all the trial scores as quality weights reduced the effects to 35\% (OR 0.65 [0.59-0.71]) and resulted in the least statistical heterogeneity. INTERPRETATION: Studies of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality.}, address = {Children's Hospital of Eastern Ontario Research Institute, and Department of Epidemiology and Community Medicine, University of Ottawa, Canada. 107656.3375@compuserve.com}, author = {Moher, D. and Pham, B. and Jones, A. and Cook, D. J. and Jadad, A. R. and Moher, M. and Tugwell, P. and Klassen, T. P.}, citeulike-article-id = {4416075}, citeulike-linkout-0 = {http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&#38;db=PubMed&#38;dopt=Citation&#38;list_uids=9746022%20}, comment = {0140-6736 (Print) Journal Article}, day = {22}, journal = {Lancet}, keywords = {bias, controlled, designstandards, double-blind, epidemiology, file-import-09-04-28, govt, humans, meta-analysis, method, non-us, publishingstandards, randomized, research, support, trialsstandards}, month = {August}, number = {9128}, pages = {609--13}, posted-at = {2009-04-28 16:01:55}, priority = {2}, title = {Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?}, url = {http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&#38;db=PubMed&#38;dopt=Citation&#38;list_uids=9746022%20}, volume = {352}, year = {1998} }

TY - JOUR ID - 9746022 L3 - citeulike-article-id:4416075 N2 - BACKGROUND: Few meta-analyses of randomised trials assess the quality of the studies included. Yet there is increasing evidence that trial quality can affect estimates of intervention efficacy. We investigated whether different methods of quality assessment provide different estimates of intervention efficacy evaluated in randomised controlled trials (RCTs). METHODS: We randomly selected 11 meta-analyses that involved 127 RCTs on the efficacy of interventions used for circulatory and digestive diseases, mental health, and pregnancy and childbirth. We replicated all the meta-analyses using published data from the primary studies. The quality of reporting of all 127 clinical trials was assessed by means of component and scale approaches. To explore the effects of quality on the quantitative results, we examined the effects of different methods of incorporating quality scores (sensitivity analysis and quality weights) on the results of the meta-analyses. FINDINGS: The quality of trials was low. Masked assessments provided significantly higher scores than unmasked assessments (mean 2.74 [SD 1.10] vs 2.55 [1.20]). Low-quality trials (score < or = 2), compared with high-quality trials (score > 2), were associated with an increased estimate of benefit of 34% (ratio of odds ratios [ROR] 0.66 [95% CI 0.52-0.83]). Trials that used inadequate allocation concealment, compared with those that used adequate methods, were also associated with an increased estimate of benefit (37%; ROR=0.63 [0.45-0.88]). The average treatment benefit was 39% (odds ratio [OR] 0.61 [0.57-0.65]) for all trials, 52% (OR 0.48 [0.43-0.54]) for low-quality trials, and 29% (OR 0.71 [0.65-0.77]) for high-quality trials. Use of all the trial scores as quality weights reduced the effects to 35% (OR 0.65 [0.59-0.71]) and resulted in the least statistical heterogeneity. INTERPRETATION: Studies of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality. IS - 9128 JF - Lancet AD - Children's Hospital of Eastern Ontario Research Institute, and Department of Epidemiology and Community Medicine, University of Ottawa, Canada. 107656.3375@compuserve.com EP - 13 TI - Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? VL - 352 SP - 609 KW - bias KW - controlled KW - designstandards KW - double-blind KW - epidemiology KW - file-import-09-04-28 KW - govt KW - humans KW - meta-analysis KW - method KW - non-us KW - publishingstandards KW - randomized KW - research KW - support KW - trialsstandards N1 - 0140-6736 (Print) Journal Article AU - Moher, D AU - Pham, B AU - Jones, A AU - Cook, DJ AU - Jadad, AR AU - Moher, M AU - Tugwell, P AU - Klassen, TP PY - 1998/08/22/ UR - http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&#38;db=PubMed&#38;dopt=Citation&#38;list_uids=9746022%20 ER -