Costs and economic evaluation

Last revised: 
2017-03-31

Introduction

This domain focuses on the importance of obtaining information about costs and outcomes as well as efficacy and effectiveness when evaluating new technologies. Economic evaluation is an important part of health technology assessment because it assists with priority-setting between different health technologies. An economic evaluation identifies, measures, values and compares the costs and outcomes of a technology with its relevant comparator.

This domain overlaps with the effectiveness domain and the organizational domain (1).

Guidance on conducting searching as part of systematic reviews of economic evaluations has recently been published (2).

Sources to search

There are some databases which identify and collect economic evaluations and health economics studies (3,4,5,6) to promote efficient retrieval. These databases are built largely from MEDLINE and Embase, but offer a variety of value added information such as critical appraisals, results, categorisations and indexing. These databases can save time in identifying economic evaluations, but may not be comprehensive because of publication lags or geographical focus (e.g. the Cost-Effectiveness Analysis (CEA) registry). NHS EED ceased updating at the end of 2014 and is available only as a closed database. HEED is no longer available.This means that sensitive searches should also include searches of general medical databases such as MEDLINE and Embase (3,4,7,8,9). Searching Science Citation Index and conference abstracts (via websites as well as Embase) may also increase retrieval (7). Pitt et al. conducted a bibliometric analysis of full economic evaluations of health interventions published in 2012-14, comparing, among other things, the sensitivity and specificity of searches in 14 databases (10). This study confirms that Econlit is not a high yield resource for economic evaluations and suggests that Scopus may be a useful resource to search, which may merit investigation.

Searching non-database sources is likely to identify further studies outside of commercial journal publications (7).

The majority of recent reviews of economic evaluations have not followed published searching approaches in detail and are also currently poorly reported (11). Reviews should report the searches explicitly and search a range of resources (2,11). The following information sources should be considered when searching for economic evaluations and utility studies:

●      Specialist economic databases (CEA Registry is a live database; Paediatric Economic Database Evaluation           (PEDE) is a database of pediatric economic evaluations; NHS EED closed at the end of 2014 (12,13,14)
●      Technology assessment databases (the Health Technology Assessment (HTA) database)
●      General medical literature databases (MEDLINE, Embase) (12)
●      Websites of HTA agencies
●      Grey literature (conferences such as ISPOR and HTAi; the RePEC economic working papers collection) (3,4)
●      Collections of utility studies including ScHARRHUD and instrument websites (15), as well as utility mapping         collections (http://www.herc.ox.ac.uk/downloads/herc-database-of-mapping-studies).

Searches to identify information to populate economic models may involve a range of resources ranging from statistical sources to bibliographic databases (3,4,16,17,18,19). Guidance on suggested minimum searching levels for model parameters is available, although the author notes that much of the guidance has not been empirically tested (20).

Designing search strategies

Principles of systematic review methodology should be followed for the design of search strategies to identify economic evaluations. The development of sensitive subject searches within the specific economic evaluation databases is recommended to capture the population and the intervention of interest (3,4,21). An overview of methods for systematic reviews of health economic interventions suggests that a systematic search should use relevant elements of PICO combined with an economic search filter (14). Shemilt is more cautious still, suggesting that only intervention search terms may be required and focus can be achieved by adding the population concept (12). However, there is no requirement to add an economic evaluation search filter to searches within economic evaluation databases because they are pre-filtered (3,6). Search filters for economic studies can be considered (in combination with concepts capturing the population and/or intervention) in general bibliographic databases such as MEDLINE or Embase (14). Published search filters, which can be identified from the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Resource tend to have high sensitivity but poor precision (22,24). CADTH offers a more precision maximizing search filter for rapid reviews (25). Search strategies to identify cost-effectiveness information may need to be adapted from those developed for searching for effectiveness studies (26). Searching for particular economic methods may require the use of several techniques (27).

Searches to inform specific parameters of decision models may not be required to be as extensive and systematic as those to identify economic evaluations, as decision models are developed in an organic way, some parameters do not require the identification of comprehensive evidence and also it may not be feasible to conduct extensive searches for all parameters of a model (3,17, 20). Health state utility values (HSUVs) are important parameters in decision models and searching for them requires specific techniques (16) as well as the use of a search filter (28). Searching for cost of illness/burden of illness can make use of population search terms (perhaps taken from an accompanying effects review) (12).

Reference list

●(1) EUnetHTA Joint Action 2 Work Package 8. HTA Core Model version® 2.0; 2013 (pdf). [Further reference details] [Publication appraisal]  [Free full text]
●(2) Thielen FW, Van Mastrigt G, Burgers LT, Bramer WM, Majoie H, Evers S, et al. How to prepare a systematic review of economic evaluations for clinical practice guidelines: database selection and search strategy development (part 2/3). Expert Rev Pharmacoecon Outcomes Res. 2016; 1–17. [Further reference details] [Publication appraisal] [Free full text]
●(3) Glanville J, Paisley S. Searching for evidence on resource use, costs, effects and cost-effectiveness. In: Shemilt I et al (eds). Evidence based economics. Oxford:Wiley-Blackwell;2010. [Further reference details] [Publication appraisal] [Free Full text]
●(4) Glanville J, Paisley S. Identifying economic evaluations for health technology assessment. Int J Technol Assess Health Care 2010;26(4):436-440. [Further reference details] [Publication appraisal] [Free Full text]
●(5) Alton V, Eckerlund I, Norlund A. Health economic evaluations: how to find them. Int J Technol Assess Health Care 2006;22(4):512-517. [Further reference details] [Publication appraisal] [Free Full text]
●(6) Nixon J, Duffy S, Armstrong N, Craig D, Glanville J, Christie J, Drummond M, Kleijnen J. The usefulness of the NHS Economic Evaluation Database to researchers undertaking technology assessment reviews. Int J Technol Assess Health Care 2004;20(3):249-257. [Further reference details] [Publication appraisal] [Free Full text]
●(7) Royle P, Waugh N. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technol Assess 2003;7(34). [Further reference details] [Publication appraisal] [Free Full text]
●(8) Waffenschmidt S, Hausner E, Engel L, Volz F, Kaiser T. Benefit of searching different databases to identify health economic evaluations in German HTA-reports. Abstract presented at: Health Technology Assessment International (HTAi) 7th Annual Meeting; 2010 June 6-9; Dublin, Ireland. Abstract T4-29. [Further reference details] [Publication appraisal] [Free Full text]
●(9) Coyle KB, Trochlil K, Iversen P. MEDLINE and EMBASE for health economic literature reviews [abstract]. Value Health 2012;15(4):A162. [Further reference details] [Publication appraisal] [Free Full text]
●(10) Pitt C, Goodman C, Hanson K. Economic evaluation in global perspective: a bibliometric analysis of the recent literature. Health Econ. 2016 Feb;25 Suppl 1:9-28.  [Further reference details] [Publication appraisal] [Free full text]
●(11) Wood H, Arber M, Glanville JM. Systematic reviews of economic evaluations: How extensive are their searches? Int J Technol Assess Health Care. 2017; 27:1-7.[Further reference details] [Publication appraisal] [Free Full text]
●(12) Shemilt I, Mugford M, Vale L, Craig D. Searching NHS EED and HEED to inform development of economic commentary for Cochrane intervention reviews. Oxford: Cochrane Collaboration; 2011. [Further reference details] [Publication appraisal] [Free Full text]
●(13) Sullivan S M, Tsiplova K, Ungar W J. A scoping review of pediatric economic evaluation 1980-2014: do trends over time reflect changing priorities in evaluation methods and childhood disease?, Expert Rev Pharmacoecon Outcomes Res. 2016;16(5): 599-607. [Further reference details] [Publication appraisal] [Free full text]
●(14) Mathes T, Walgenbach M, Antoine SL, Pieper D, Eikermann M. Methods for systematic reviews of health economic evaluations: a systematic review, comparison, and synthesis of method literature. Med Decis Making. 2014;34(7):826-840. [Further reference details] [Publication appraisal] [Free full text]
●(15) Dakin H. Review of studies mapping from quality of life or clinical measures to EQ-5D: an online database. Health and Quality of Life Outcomes 2013;11(1):151. [Further reference details] [Publication appraisal] [Free full text]
●(16) Golder S, Glanville J, Ginnelly L. Populating decision-analytic models: the feasibility and efficiency of database searching for individual parameters. Int J Technol Assess Health Care 2005;21(3):305-311. [Further reference details] [Publication appraisal] [Free Full text]
●(17) Papaioannou D, Brazier J, Paisley S. Systematic searching and selection of health state utility values from the literature. Value Health 2013;16(4):686-695. [Further reference details] [Publication appraisal] [Free Full text]
●(18) Zechmeister-Koss I, Schnell-Inderst P, Zauner G. Appropriate evidence sources for populating decision analytic models within health technology assessment (HTA): a systematic review of HTA manuals and health economic guidelines. Med Decis Making 2014;34(3):288-299. [Further reference details] [Publication appraisal] [Free Full text]
●(19) De Cock E, Cosmatos I, Kirsch E. Use of databases for health resource utilizaton and cost analyses in EU-5: Results from a focused literature review [abstract]. Value in Health Conference: ISPOR 21st Annual International Meeting Research Washington, DC.  Value in Health 2016;19(3):A80-A8). [Further reference details] [Publication appraisal] [Free full text]
●(20) Paisley S. Identification of evidence for key parameters in decision-analytic models of cost effectiveness: a description of sources and a recommended minimum search requirement.  PharmacoEconomics. 2016 Jun;34(6):597-608. [Further reference details] [Publication appraisal] [Free full text]
●(21) Rosen AB, Greenberg D, Stone PW, Olchanski NV, Neumann PJ. Quality of abstracts of papers reporting original cost-effectiveness analyses. Med Decis Making 2005;25:424-428. [Further reference details] [Publication appraisal] [Free Full text]
●(22) Glanville J, Fleetwood K, Yellowlees A, Kaunelis D, Mensinkai S. Development and testing of search filters to identify economic evaluations in MEDLINE and EMBASE. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2009. [Further reference details][Publication appraisal] [Free Full text]
●(23) Glanville J, Kaunelis D, Mensinkai S. How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE. Int J Technol Assess Health Care 2009;25(4):522-529. [Further reference details] [Publication appraisal] [Free Full text]
●(24) McKinlay RJ, Wilczynski NL, Haynes RB, Hedges Team. Optimal search strategies for detecting cost and economic studies in EMBASE. BMC Health Serv Res 2006;6:67. [Further reference details] [Publication appraisal] [Free Full text]
●(25) Kaunelis D. When everything is too much: development of a CADTH narrow economic search filter. Poster presented at: 2011 CADTH Symposium. 2011 Apr 3-5; Vancouver, BC. [Further reference details] [Publication appraisal] [Free Full text]
●(26) Droste S, Dintsios C-M. Informationsgewinnung für gesundheitsökonomische Evaluationen im Rahmen von HTA-Berichten. Gesundheitsökonomie & Qualitätsmanagement 2011;16(1):35-57. [Further reference details][Publication appraisal] [Free Full text]
●(27) Hinde S, Spackman E, Claxton K, Sculpher MJ. The cost-effectiveness threshold: the results of a novel literature review method. Value Health 2011;14:A354. [Further reference details] [Publication appraisal] [Free Full text]
●(28) Arber M, Garcia S, Veale T, Edwards M, Shaw A, Glanville J. Sensitivity of a search filter designed to identify studies reporting health state utility values [poster]. Presented at the Cochrane Colloquium, Vienna, 3-7 October 2015. [Further reference details] [Publication appraisal] [Free full text]