Embase

Appraisal of: "Fortier KJ, Kiss N, Tongbram V. What is the optimal search engine for results from Embase and MEDLINE: Ovid or Embase.com? Value Health 2013:16(3):A25."

Reviewer(s): 
Short description: 

The aim of this study was to compare OVID and Embase.com in terms of the results of searches and ease of searching. Using several searches the authors investigated differences in results and why any dispartieis exist. Ease of use was rates on a scale of 0 to 5. Although differences were found there were few if any relevant articles missed by either search interface. The authors conclude that there is no notable difference between OVID and Embase.com but Embase.com was rated easier to use overalll.

Limitations stated by the author(s): 

No limits were discussed by the authors.

Limitations stated by the reviewer(s): 
This is a conference abstract so not enough detail is presented to ascertain the limitations.

Appraisal of: Preston, L., et al. "Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking." Syst Rev 2015 4: 82.

Reviewer(s): 
Short description: 

A convenience sample of nine Health Technology Assessment (HTA) systematic reviews of diagnostic test accuracy, with 302 included citations, was analysed to determine the number and proportion of included citations that were indexed in and retrieved from MEDLINE and EMBASE. An assessment was also made of the number and proportion of citations not retrieved from these databases but that could have been identified from the reference lists of included citations.

287/302 (95 %) of the included citations were indexed across MEDLINE and EMBASE. The reviews’ searches of MEDLINE and EMBASE accounted for 85 % of the included citations (256/302). Of the forty-six (15 %) included citations not retrieved by the published searches, 24 (8 %) could be found in the reference lists of included citations. Only 22/302 (7 %) of the included citations were not found. 

Limitations stated by the author(s): 

This study used a small, non-random sample of diagnostic test accuracy systematic reviews. This was done for reasons of pragmatism: first, because the authors had full access to the search strategies and reference databases of these reviews and, second, because of the exploratory nature of this project. We also assumed that the vast majority of the included citations in the reviews were located through screening of titles, abstracts and full papers.

We have also assumed, because the number of studies missed by operating the proposed MEDLINE, EMBASE and reference tracking strategy is so small that the findings of the systematic reviews would not have been greatly affected by their omission. However, this is uncertain and can only be assessed statistically by excluding those particular studies from the many analyses reported in the reviews, although, as noted above, most of these reviews conducted narrative synthesis. Such an analysis is a major task to undertake retrospectively and has therefore not been completed in this exploratory study. Future work should test the findings of this small study in a larger, preferably prospective sample of systematic reviews from multiple institutions. If possible, statistical analysis should also be undertaken to quantify fully the impact of omitting any data from studies that might otherwise be missed.

Limitations stated by the reviewer(s): 
No further limitations were identified.
Study Type: 
Single study

Appraisal of: "Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E. Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking."

Short description: 

This exploratory study analysed a convenience sample of nine Health Technology Assessments (HTA) reviews of diagnostic test accuracy (DTA) to identify which of the included citations were indexed on MEDLINE or EMBASE; to identify the number and proportion of citations that were retrieved by the search strategies with or without the addition of checking the reference lists of included studies.                     

Of the 302 included citations included in the nine reviews, 287 (95%) were indexed in MEDLINE and EMBASE. The reviews searches accounted for 256 (85%) of the included citations and a further 24 (8%) could be identified from the reference lists of included citations. 7% of the citations (22/302) were not found by searching or reference checking.

The proportion of citations identified using both searches of MEDLINE, and EMBASE and checking the reference lists resulted in the identification of 280/302 (93%) included studies. The authors suggest that there might be a case for restricting systematic review searches  of DTA studies to MEDLINE, EMBASE and the reference lists of included studies. 

Limitations stated by the author(s): 

The study’s pragmatic, convenience sample of a small number of published DTA reviews was intended as an exploratory analysis only. The  authors did not conduct an analysis to assess the impact of excluding the unidentified studies on the conclusions of the HTAs, so the results of the study are uncertain. 

Limitations stated by the reviewer(s): 
Six of the nine studies in the sample used a published filter for diagnostic studies, which is not recommended by Cochrane. This may have impacted the sensitivity of the searches and the sample not be representative of other DTA reviews.
Study Type: 
Single study
Related Chapters: 

Appraisal of: Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, Glanville JM. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev 2013, Issue 9.

Reviewer(s): 
Short description: 

This review assessed the performance (sensitivity, specificity and precision) of 70 filters (described in 19 studies) designed to identify diagnostic test accuracy studies for use with either the MEDLINE or Embase database. The filters were appraised for potential bias in terms of use of a diagnostic test accuracy search strategy; choice of gold standard; method of validation. The ISSG Search Filter Appraisal Checklist was used to assess quality. None of the filters tested met the pre-specified levels of sensitivity > 90% and precision >10%. Using methodological search filters as the sole method of identifying studies for systematic reviews of diagnostic test accuracy is not recommended.

Limitations stated by the author(s): 

A search for unpublished search filters was not conducted. The authors acknowledged the possibility of bias introduced by the studies that evaluated filters but did not replicate them in the paper. 

Limitations stated by the reviewer(s): 
No limitations have been identified by the reviewers.
Study Type: 
Review
Related Chapters: 

Appraisal of: Golder S, Wright K, Rodgers M. Failure or success of search strategies to identify adverse effects of medical devices: a feasibility study using a systematic review. Syst Rev 2014;3:113.

Short description: 

A systematic review of the safety of recombinant human bone morphogenetic protein-2 (rhBMP-2) for spinal fusion was used as a case study to investigate whether adverse effects terms were prevalent in the bibliographic records of the included studies. The aim of the study was to assess whether the development of a search filter for medical devices would be feasible.

Out of the 82 publications (49 studies) included in the systematic review, 51 were indexed in MEDLINE and 55 in Embase. The results showed that 94% of the MEDLINE records and 95% of the Embase records contained at least one adverse effects related term. However, the records contained a wide range of adverse effects terms, and no single term was able to retrieve the majority of the included publications. In addition, the most successful search terms were different from search terms shown to be most successful when searching for adverse drug effects. The study concluded that the development of adverse effects search filters for medical devices is feasible, but that such filters should be developed using a large set of relevant records in order to identify the optimal combination of search terms.

Limitations stated by the author(s): 

The main limitation to this study is that because only one systematic review was used as a case study, the generalisability of the study results is limited. This case study is also of a medical device with a pharmaceutical component, while a case study of a different kind of a medical device might have given different results.

In addition, unlike many other HTAs and systematic review authors, the study authors were able to obtain unpublished data directly from the manufacturer. They also included an unusually high number of conference abstracts and multiple publications for the same study in their review. This might have influenced the findings of this study.

Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewers.
Study Type: 
Single study

Comments from the authors:

Appraisal of: Golder S, Wright K, Rodgers M. The contribution of different information sources to identify adverse effects of a medical device: a case study using a systematic review of spinal fusion. Int J Technol Assess Health Care. 2014;30(14):1-7.

Short description: 

The included studies from a systematic review of the safety of recombinant human bone morphogenetic protein-2 (rhBMP-2) for spinal fusion were used for analysis. For each study, it was investigated in whichc sources they were available and where they were identified. If a study was available on a database but not retrieved by the original search strategy, the bibliographic record was examined to determine why it was not retrieved. The sensitivity, precision, and numbers needed to read for searches in each of the databases was calculated, as well as sensitivity*precision. The minimum combination of sources required to identify all the included publications using the original search strategies used was recorded.

The minimum combination of sources to identify all the publications was Science Citation Index (SCI), Embase, CENTRAL and either MEDLINE or PubMed, in addition to reference checking, contacting authors and using automated current awareness service.

The highest precision was achieved in CENTRAL, followed by ToxFile and MEDLINE. The highest sensitivity*precision was achieved by MEDLINE, followed by CENTRAL and Science Citation Index.

Limitations stated by the author(s): 

The main limitation of this study is that relatively few sources were searched in the original systematic review, meaning that only a few resources could be compared for their relative value in providing relevant data.

Also, as the analysis is only based on one case study systematic review, the generalisability of the results is limited.

In addition, the authors were able to obtain unpublished data directly from the manufacturer. They also included an unusually high number of conference abstracts and multiple publications for the same study in their review. This might have influenced the findings of this study.

This case study is also of a medical device with a pharmaceutical component, while a case study of a different kind of a medical device might have given different results.

 

Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewers.
Study Type: 
Single study

Appraisal of: Golder S. Optimising the retrieval of information on adverse drug effects. Health Info Libr J. 2013 Dec;30(4):327-331.

Short description: 

The article reports on the author’s PhD study and summarises it to emphasise the implications for practice. The research questions were: 1) which study designs provide the best evidence on adverse effects? 2) which sources of information efficiently provide the most relevant data on adverse drug effects? and 3) which search strategies are the most effective in retrieving relevant data on adverse drug effects from database sources? The questions were answered by using a mixed-method approach.

The results show that searchers should not rely solely on MEDLINE but search multiple sources, as well as retrieve unpublished studies and industry funded data. Searches in MEDLINE and Embase may now include adverse effects terms or search filters, although searchers need to apply them with some caution. In systematic reviews, there is still room for improvement in reporting of search strategies for adverse effects and in reporting more reproducible search strategies.

Limitations stated by the author(s): 

No limitations stated by the author.

Limitations stated by the reviewer(s): 
No additional limitations detected by the reviewers
Study Type: 
Single study

Comments from the authors:

This publication is related to Su Golder’s PhD Thesis “Evaluating and Optimising the Retrieval of Research Evidence for Systematic Reviews of Adverse Drug Effects and Adverse Drug Reactions” from 2013. The thesis is available from http://etheses.whiterose.ac.uk/4749/

 

 

Appraisal of: Fangerau HC. Finding European bioethical literature: An evaluation of the leading abstracting and indexing services. J Med Ethics 2004; 30 (3): 299–303.

Short description: 

The aim of the article is to ascertain indexing of medical ethics publications in commonly used bibliographic databases. The purpose of the author is to introduce a set of databases which provides the highest possible quantity of bioethical journal publications to researchers as well as the added value of each database considered.

The article provides information on the methods of collecting the existing Medical Ethics journals, the searching for coverage of these by bibliographic abstract and indexing services, the evaluation results, and finally the conclusions for doing literature searches on medical ethics journal publications.

The search for Medical Ethics journals yielded in 290 periodicals. Thereof 173 journals (60%) were identified in at least one of the considered databases; the remaining journals were not indexed. The highest coverage rates were found in Current Contents (23%) and Medline (22%). A maximum quantity of Medical Ethics journals (45%) was obtained when combining the 10 databases with the highest coverage: Current Contents, MEDLINE, Research Alert, Social Sciences Citation Index, EMBASE, AgeLine, CINAHL, E-psyche, Sociological Abstracts, and Family Index.

A tendency to index more North American than European journals was observed (=bias). EMBASE seems to cover the highest number of European journals.

However, the author recommends that “a medical ethics literature search has to be carried out in several databases in order to reach an adequate collection of literature.”

Limitations stated by the author(s): 
  • The analyzed set of journals is a representative sample, not the whole range of journals possibly comprising articles on medical ethics.
  • Current Contents cites literature from the last five years.
Limitations stated by the reviewer(s): 
No national / supranational services (as European databases) were included in the study. Due to the high turnover in existing relevant journals as well as indexing services (e.g. Current Contents now is part of the ISI Web of Science) the presented results couldn’t be reproduced in detail today. Nevertheless the main results and the recommendations are still valid.
Study Type: 
Single study

Appraisal of: Droste S, Dintsios CM, Gerber A. Information on ethical issues in health technology assessment: How and where to find them. Int J Technol Assess Health Care 2010; 26(4): 441–449.

Short description: 

The aim of the article is to present a procedure to identify information on ethical implications of the use and value-related decision-making in the assessment of health technologies. The purpose of the project is to develop, propose and test a step-by step workflow adapted to the specific issues of information on ethical issues (searching for arguments and qualitative data).

The article provides information on the background of developing the methodological approach as well as on the 8 working steps - Step 1 “Translation of the search question using the PICO scheme and additional components” to Step 8 “Final quality check and calculation of precision and recall”. The article is supplemented by the application results of the example “Ethical issues of autologous stem cell transplantation in metastatic breast cancer”.

Limitations stated by the author(s): 
  • Not all ethical issues related to health technologies can be identified by means of the approach described.
  • Some ethical issues cannot be found with systematic approaches - additional non-systematic searching may be useful.
  • The ethical issues relevant to the health technology investigated should be known in more detail before starting the systematic information retrieval.
  • Alleged innovations cannot be identified as such by the methodological approach.
Limitations stated by the reviewer(s): 
- Reference(s) to the validation work could have been provided. - The authors claim in the objective that if choosing the proper methodological approach: the central ethical dimensions then should be answerable. This seems like a too strong claim, what the authors do is to provide a brilliant methodological approach to address central ethical dimensions, to the extent there is anything published concerning these. Whether that answers ethical concerns is to some extent another matter and might call for a separate ethics analysis. Both since the sources might give conflicting answers, or give no answers at all. - Lack references to the database Philosopher’s index which in some cases might provide sources for a more general but relevant ethical discussion.
Study Type: 
Single study

Appraisal of: Coyle KB, Trochlil K, Iversen P. MEDLINE and EMBASE for health economic literature reviews [abstract]. Value Health 2012;15(4):A162.

Reviewer(s): 
Short description: 
The authors compared search results in MEDLINE and Embase for the indication chronic pulmonary obstructive disease (COPD) with regard to cost of illness (COI), burden of illness (BOI) and cost benefit analysis (CBA). MEDLINE retrieved a higher number of references for CBA, while Embase retrieved higher results in the areas of COI and BOI. Conference abstracts accounted for the higher Embase totals. The inclusion of conference abstracts in Embase support the practice of conducting economic literature reviews in both MEDLINE and Embase.
Limitations stated by the author(s): 
None stated - this is an abstract.
Limitations stated by the reviewer(s): 
Since the results were published only in abstract form, very limited information was presented on methods, results and conclusions. Specific search strategies were not provided, nor was it stated if the citations retrieved were evaluated for relevance. Only one subject area (COPD) was searched for citations published during 2010-11, resulting in low retrieval results. Differences in retrieval may therefore not have statistical significance or be generalizable to other topic areas.
Study Type: 
Single study
Related Chapters: 
Syndicate content