Issues related to the frequency of exploratory analyses by evidence review groups in the NICE single technology appraisal process

Background: Evidence Review Groups (ERGs) critically appraise company submissions as part of the National Institute for Health and Care Excellence (NICE) Single Technology Appraisal (STA) process. As part of their critique of the evidence submitted by companies, the ERGs undertake exploratory analyses to explore uncertainties in the company’s model. The aim of this study was to explore pre-defined factors that might influence or predict the extent of ERG exploratory analyses. Objective: The aim of this study was to explore predefined factors that might influence or predict the extent of ERG exploratory analyses. Methods: We undertook content analysis of over 400 documents, including ERG reports and related documentation for the 100 most recent STAs (2009–2014) for which guidance has been published. Relevant data were extracted from the documents and narrative synthesis was used to summarise the extracted data. All data were extracted and checked by two researchers. Results: Forty different companies submitted documents as part of the NICE STA process. The most common disease area covered by the STAs was cancer (44%), and most ERG reports (n = 93) contained at least one exploratory analysis. The incidence and frequency of ERG exploratory analyses does not appear to be related to any developments in the appraisal process, the disease area covered by the STA, or the company’s base-case incremental cost-effectiveness ratio (ICER). However, there does appear to be a pattern in the mean number of analyses conducted by particular ERGs, but the reasons for this are unclear and potentially complex. Conclusions: No clear patterns were identified regarding the presence or frequency of exploratory analyses, apart from the mean number conducted by individual ERGs. More research is needed to understand this relationship.