The quality of reporting might not reflect the quality of the study: implications for undertaking and appraising a systematic review

The systematic review has become an increasingly popular method of synthesizing findings on a topic in order to inform clinical practice, commissioning of care, and future research. A central component of the systematic review is an assessment of study quality or risk of bias, i.e. an assessment of how near to the ‘truth’ the findings of the study are. While undertaking a recent systematic review, it became apparent that the outcomes of the quality appraisal process were somewhat different across systematic reviews where the same randomized controlled trials had been included. The quality of the report writing of the randomized controlled trials included was identified as one possible reason for this discrepancy. This had implications upon the conclusions drawn by the review. It is suggested that reasonable attempts to contact study authors should be made in order to inform the quality appraisal process while undertaking systematic reviews of randomized controlled trials and that the presence or absence of this process should be considered by research consumers when appraising the quality of a systematic review. This process enables a full assessment of study quality rather than simply an assessment of the quality of report writing.