Appraisal of: Golder S, Loke YK, Bland M. Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med 2011;8(5):e1001026.

Reviewer(s): 
Short description: 
This methodological overview investigates the impact of study design in meta-analyses of adverse effects data. The authors look at the estimates of the risk of adverse events derived from meta-analyses of randomized controlled trials (RCTs) and from observational studies. The evidence indicates that there is no difference on average between estimates of harm in meta-analyses of RCTs as compared to observational studies. The study suggests that systematic reviews of harms should not be restricted to specific study types.
Limitations stated by the author(s): 
Confounding factors may account for any differences identified when comparing pooled results from various study designs. Due to time constraints, data was not extracted from individual papers; rather the data reported by the authors of the meta-analyses were used. Because the search strategy was designed to identify methodological papers whose primary goal was to assess the influence of study design on adverse effects, systematic reviews with embedded data comparing study designs may have been missed. The considerable heterogeneity between the comparisons of different studies suggests that any differences could be specific to particular types of interventions or certain adverse events. For example, some types of adverse events may be more easily identified by RCTs as opposed to observational studies.
Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewer
Study Type: 
Review
Related Chapters: 

Comments from the authors:

This publication is related to Su Golder’s PhD Thesis “Evaluating and Optimising the Retrieval of Research Evidence for Systematic Reviews of Adverse Drug Effects and Adverse Drug Reactions” from 2013. The thesis is available from http://etheses.whiterose.ac.uk/4749/