Appraisal of: Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6(33)
This paper explores the relative recall technique to develop a gold standard for use in evaluating search filters. Filters or ‘hedges’ are search strategies used in conjunction with a topic search to focus on those records which are the focus of the filter. Validating a filter requires the existence of a ‘gold standard’ of records known to be of the study type required. However, developing a gold standard by handsearching can be time consuming. The authors identified 105 Cochrane Reviews that had used the Cochrane Highly Sensitive Search Strategy to identify studies. The included studies in those reviews were gathered to form a gold standard. Three variants of the Highly Sensitive Search Strategy (HSSS) filter were tested in terms of their ability to retrieve gold standard records in MEDLINE. Sensitivity ranging from 0.98 to 0.91 was achieved.