MEDLINE

Appraisal of: "Sampson M. Complementary approaches to searching MEDLINE may be sufficient for updating existing systematic reviews. J Clin Epidemiol 2016; 78: 108-115."

Short description: 

The aim of the article was to test whether the combined approach of a focused Boolean search paired with a second search using the similar articles feature of PubMed or support vector machine (SVM) can yield high recall with reasonable precision.

The general approach of a Boolean plus a ranking search is effective in MEDLINE retrieval for systematic reviews. Very high levels of identification of relevant MEDLINE records, with adequate precision, are possible using a focused Boolean search complemented by a document similarity or ranking method.

Limitations stated by the author(s): 

There are two limitations to our proposed strategy. Other databases should be searched in the unusual event that numerous studies, representing more than a small proportion of the total N, are not included in MEDLINE. Second, when it is important to find articles too new to be indexed by MEDLINE, systematic reviewers may wish to conduct a simple PubMed search limited to the nonindexed subsets.

Limitations stated by the reviewer(s): 
It is not possible to reproduce or use the support vector machine functionality.
Study Type: 
Single study

Appraisal of: "Boeker M., Vach W, Motschall E.. Time-dependent migration of citations through PubMed and OvidSP subsets: a study on a series of simultaneous PubMed and OvidSP searches. Stud Health Technol Inform. 2013;192:1196."

Reviewer(s): 
Short description: 

The aim of this study was to compare PubMed and OvidSP in terms of the timeliness of the different interfaces. The authors conclude OvidSP is updated within a period of two days.

Limitations stated by the author(s): 

No limits were discussed by the authors.

Limitations stated by the reviewer(s): 
This is a conference abstract so not enough detail is presented to ascertain the limitations.
Study Type: 
Single study

Appraisal of: "Fortier KJ, Kiss N, Tongbram V. What is the optimal search engine for results from Embase and MEDLINE: Ovid or Embase.com? Value Health 2013:16(3):A25."

Reviewer(s): 
Short description: 

The aim of this study was to compare OVID and Embase.com in terms of the results of searches and ease of searching. Using several searches the authors investigated differences in results and why any dispartieis exist. Ease of use was rates on a scale of 0 to 5. Although differences were found there were few if any relevant articles missed by either search interface. The authors conclude that there is no notable difference between OVID and Embase.com but Embase.com was rated easier to use overalll.

Limitations stated by the author(s): 

No limits were discussed by the authors.

Limitations stated by the reviewer(s): 
This is a conference abstract so not enough detail is presented to ascertain the limitations.

Appraisal of: Preston, L., et al. "Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking." Syst Rev 2015 4: 82.

Reviewer(s): 
Short description: 

A convenience sample of nine Health Technology Assessment (HTA) systematic reviews of diagnostic test accuracy, with 302 included citations, was analysed to determine the number and proportion of included citations that were indexed in and retrieved from MEDLINE and EMBASE. An assessment was also made of the number and proportion of citations not retrieved from these databases but that could have been identified from the reference lists of included citations.

287/302 (95 %) of the included citations were indexed across MEDLINE and EMBASE. The reviews’ searches of MEDLINE and EMBASE accounted for 85 % of the included citations (256/302). Of the forty-six (15 %) included citations not retrieved by the published searches, 24 (8 %) could be found in the reference lists of included citations. Only 22/302 (7 %) of the included citations were not found. 

Limitations stated by the author(s): 

This study used a small, non-random sample of diagnostic test accuracy systematic reviews. This was done for reasons of pragmatism: first, because the authors had full access to the search strategies and reference databases of these reviews and, second, because of the exploratory nature of this project. We also assumed that the vast majority of the included citations in the reviews were located through screening of titles, abstracts and full papers.

We have also assumed, because the number of studies missed by operating the proposed MEDLINE, EMBASE and reference tracking strategy is so small that the findings of the systematic reviews would not have been greatly affected by their omission. However, this is uncertain and can only be assessed statistically by excluding those particular studies from the many analyses reported in the reviews, although, as noted above, most of these reviews conducted narrative synthesis. Such an analysis is a major task to undertake retrospectively and has therefore not been completed in this exploratory study. Future work should test the findings of this small study in a larger, preferably prospective sample of systematic reviews from multiple institutions. If possible, statistical analysis should also be undertaken to quantify fully the impact of omitting any data from studies that might otherwise be missed.

Limitations stated by the reviewer(s): 
No further limitations were identified.
Study Type: 
Single study

Appraisal of: "Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E. Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking."

Short description: 

This exploratory study analysed a convenience sample of nine Health Technology Assessments (HTA) reviews of diagnostic test accuracy (DTA) to identify which of the included citations were indexed on MEDLINE or EMBASE; to identify the number and proportion of citations that were retrieved by the search strategies with or without the addition of checking the reference lists of included studies.                     

Of the 302 included citations included in the nine reviews, 287 (95%) were indexed in MEDLINE and EMBASE. The reviews searches accounted for 256 (85%) of the included citations and a further 24 (8%) could be identified from the reference lists of included citations. 7% of the citations (22/302) were not found by searching or reference checking.

The proportion of citations identified using both searches of MEDLINE, and EMBASE and checking the reference lists resulted in the identification of 280/302 (93%) included studies. The authors suggest that there might be a case for restricting systematic review searches  of DTA studies to MEDLINE, EMBASE and the reference lists of included studies. 

Limitations stated by the author(s): 

The study’s pragmatic, convenience sample of a small number of published DTA reviews was intended as an exploratory analysis only. The  authors did not conduct an analysis to assess the impact of excluding the unidentified studies on the conclusions of the HTAs, so the results of the study are uncertain. 

Limitations stated by the reviewer(s): 
Six of the nine studies in the sample used a published filter for diagnostic studies, which is not recommended by Cochrane. This may have impacted the sensitivity of the searches and the sample not be representative of other DTA reviews.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Pohl, S. Boolean and Ranked Information Retrieval for Biomedical Systematic Reviewing [thesis]. Department of Computer Science and Software Engineering: University of Melbourne; 2012.

Short description: 

The aim of the study was to devise and examine ranked retrieval methods with regard to their applicability to the search tasks involved in the preparation and update of systematic reviews. A ranked retrieval alternative in the form of the p-norm extended Boolean retrieval (EBR) model was shown to provide advantages over the current Boolean approach, but to also satisfy important task specific requirements. Using this retrieval method, a greater number of relevant studies than with the current Boolean approach can be identified for inclusion in systematic reviews, reducing the dependence on potentially biased secondary search methods. The method is close to current practice and hence requires minimal changes to the current process, which should facilitate its adoption.

Limitations stated by the author(s): 

"One caveat with this experimental setup is that the queries could possibly be biased by knowledge about the documents in the test set. Hence, absolute performance on the test set might not be reliable, but this setup still allows for relative comparison of the approach because both the optimized queries as well as the baseline would have this advantage." (p. 154)

Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study
Supplemental publications to the study: 

Appraisal of: Karimi S, Pohl S, Scholer F, Cavedon L, Zobel J. Boolean versus ranked querying for biomedical systematic reviews. BMC Med Inform Decis Mak. 2010; 10(1):58.

Short description: 

Boolean searching of biomedical databases is fundamental to the systematic review process but has the disadvantage of requiring the examination of all retrieved references, while the formulation of effective Boolean queries is complex, can require multiple iterations before a search strategy is finalised and can still fail to retrieve all relevant records. This study compared the performance of Boolean searching with ranked retrieval (in which results are ranked in order of relevance) when searching for a systematic review by a series of experiments. Investigations regarding ranked retrieval included: the effectiveness of different ranked query schemes; the effectiveness of exploiting metadata in MEDLINE records; the effectiveness of query expansion using MeSH; defining a stopping criterion for determining when to stop examining ranked documents; and combining Boolean and ranked retrieval in a hybrid search method. Results showed that ranked retrieval by itself was not viable for a search task requiring high recall such as for a systematic review, but that a refinement of the standard Boolean search process involving ranking within a Boolean result set can improve the overall search performance by providing early indication of the quality of the results, thereby speeding up the iterative query-refinement process. The authors conclude that an interactive query-development process using a hybrid ranked and Boolean retrieval system has the potential to contribute significant time-savings to the search process for a systematic review.

Limitations stated by the author(s): 

No limitations were stated by the authors.

Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study
Tags:

Appraisal of: Wright K, Golder S, Rodriguez-Lopez R. Citation searching: a systematic review case study of multiple risk behaviour interventions. BMC Med Res Methodol. 2014; 14:73.

Short description: 

Study about the effectiveness of using the citation sources Google Scholar, Scopus, Web of Science and OVIDSP MEDLINE to identify records for inclusion in a systematic review about multiple risk behaviour interventions.

40 included studies were used identified by traditional database searches from one systematic review. Each of the included studies was searched for in the four citation sources to retrieve the details of all papers that have cited these studies.

The highest number of citations was retrieved from Google Scholar (1680), followed by Scopus (1173), then Web of Science (1095) and lastly OVIDSP (213). To retrieve all the records identified by the citation tracking searching all four resources was required. Google Scholar identified the highest number of unique citations. The citation tracking identified 9 studies that met the review’s inclusion criteria (recall 22,5%).

Conclusion: Citation searching as a supplementary search method for systematic reviews may not be the best use of valuable time and resources.

Limitations stated by the author(s): 

In terms of searching, the scoping review had very broad coverage. Its aim was to identify any intervention promoting change in at least two risk behaviours and the search strategy incorporated terms for all of these (smoking, alcohol misuse, physical inactivity, unhealthy diet, illicit drug use, sexual risk behaviour, lack of seat belt use, lack of motorcycle/bicycle helmet use, lack of sunscreen use, gambling, poor oral hygiene and drink driving) in various set combinations. The resulting complexity will almost certainly have had an impact upon the overall performance of the database search strategies. As with any case study, there is uncertainty about how far the results of this study can be generalised, especially to other reviews with a more restrictive focus.

Limitations stated by the reviewer(s): 
No additional limitations were detected by the reviewers.
Study Type: 
Single study

Appraisal of: O'Leary N, Tiernan E, Walsh D, Lucey N, Kirkova J, Davis MP. The pitfalls of a systematic MEDLINE review in palliative medicine: symptom assessment instruments. Am J Hosp Palliat Care. 2007; 24(3): 181-184.

Short description: 

The authors describe the usefulness of the related articles link in PubMed as part of a systematic MEDLINE review. Of 1181 citations identified, through a PubMed search, 10 articles met the indusion criteria. as did a further 21 articles identified through hand-searching the references of the 10 articles. The PubMed related articles link of the 10 articles yielded 15% of all reports finally included in the systematic review. The authors suggest that it is a useful tool in PubMed for revietrieving complex evidence.

Limitations stated by the author(s): 

No limitations stated by the authors.

Limitations stated by the reviewer(s): 
Different numbers are described in the result section: references that met the inclusion criteria and references finally included in the systematic review. This selection process is not clearly documented and it is not clear how many references finally included in the systematic review are identified through which source.
Study Type: 
Single study

Appraisal of: Katchamart, W., Faulkner A, Feldman B,Tomlinson G, Bombardier C.. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol 2011;64: 805-807

Reviewer(s): 
Short description: 

The aim of this study was to compare the performance of Ovid-MEDLINE vs. PubMed for identifying randomized controlled trials of methotrexate (MTX) in patients with rheumatoid arthritis (RA). They measured the performance of each strategies for Ovid-MEDLINE and PubMed for a systematic review of MTX in RA. They found that PubMed retrieved more citations overall than Ovid-MEDLINE. Of 20 citations that met eligibility criteria for the review, Ovid-MEDLINE retrieved 17 (85%) and PubMed 18 (90%). The precision was 0.881% (NNR: 114) for Ovid-MEDLINE vs. 0.884% (NNR:113)  for PubMed.

Limitations stated by the author(s): 

The study was based on the results of searches conducted for one systematic review of one specific topic. The generalisability of the results needs to be tested in other reviews.

Limitations stated by the reviewer(s): 
The authors did not include the Ovid ‘MEDLINE In-Process & Other Non-Indexed Citations’.
Study Type: 
Single study
Syndicate content