Guest Blog Post by Natasha L. Bloodworth and Dale Weston, Public Health England
Public health interventions tend to be complex, context-dependent, and potentially costly (Rychetnik, Frommer, Hawe, & Shiell, 2002), thus it follows that there is increasing attention on the evaluation of such interventions, including how cost-effective they are (Weatherly et al., 2009). As public health benefits tend to: a) generate very broad costs and benefits, and: b) are generally targeted at a community or population level it is not as easy to assess their efficacy as it is for a clinical intervention for which a straightforward randomised controlled trial (RCT) is usually appropriate. Furthermore, an intervention may not work as well as intended due to weaknesses in design or because it was not implemented as intended (Steckler & Linnan, 2002), whilst positive outcomes might also be achieved even if not properly implemented (Moore et al., 2013). It is, therefore, important to focus both on what was delivered alongside the way it was delivered (Caroll et al., 2007; Montgomery et al., 2013). In addition to this, the exploration of the mechanisms through which interventions work is vital in discovering how the intervention works and how they might be replicated by future interventions (Grant et al., 2013).
It is also common for a public health intervention to have so-called “knock-on” effects, whereby a particular outcome not envisaged by the intervention designers is observed (Lorenc & Oliver, 2014) – for example, an intervention targeting obesity through diet and exercise could conceivably lead to a reduction in mood-related symptoms in some individuals. However, an area that has garnered less attention is the idea of an intervention resulting in adverse consequences (Macintyre & Petticrew, 2000), whether that be direct physical harm, psychological harm, or group/social harm. Examples of this might include the psychological distress following a false positive as a result of cancer screening (e.g. Bell et al., 1998), or by the stigmatisation of certain groups such as those targeted by obesity interventions (e.g. Puhl & Heuer, 2010). There is little assessment of this, with the majority of systematic reviews (a vital tool in assessing the efficacy of a healthcare intervention; Liberati et al., 2009) failing to extract data on potential negative effects (both direct and indirect). Indeed, those that have considered this have found little or no evidence (e.g. Ogilvie et al., 2007). Considering the focus on adverse incidents and patient safety in the medical literature, it would follow, however, that potential adverse effects should be give due consideration when evaluating the efficacy of an intervention.
A key benefit of the currently under-development MIDAS platform is its potential utility for evaluating public health interventions or decision-making as these are implemented. The use of these analyses could be invaluable for informing policy makers as to the efficacy or lack thereof regarding specific interventions/ decisions. Specifically, the ability of the MIDAS platform to integrate real-time population- or community-level data may enable both: a) the provision of insights regarding the efficacy of a given implemented intervention, with a view to informing the development of future public health interventions, and; b) the use of data-driven analytics and simulation that could predict the potential impacts of intended interventions before they are deployed (thereby negating the potential cost of implementing such interventions).
In summary, given the MIDAS platform’s anticipated role in supporting the planning and evaluation of interventions, it has the potential to be a powerful tool for ensuring a balanced evaluation of healthcare interventions and public health decision-making. The MIDAS platform could, therefore, go some way towards addressing some of the issues described above that are inherent in the evaluation of large-scale public health interventions.
References
Bell, R., Petticrew, M., Luengo, S., & Sheldon, T. A. (1998). Screening for ovarian cancer: a systematic review. National Coordinating Centre for Health Technology Assessment.
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation science, 2(1), 40.
Grant, A., Treweek, S., Dreischulte, T., Foy, R., & Guthrie, B. (2013). Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials, 14(1), 15.
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P., … & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS medicine, 6(7), e1000100.
Lorenc, T., & Oliver, K. (2013). Adverse effects of public health interventions: a conceptual framework. J Epidemiol Community Health, jech-2013.
Macintyre, S., & Petticrew, M. (2000). Good intentions and received wisdom are not enough. J Epidemiol Community Health, 54, 802-803.
Montgomery, P., Underhill, K., Gardner, F., Operario, D., & Mayo-Wilson, E. (2013). The Oxford Implementation Index: a new tool for incorporating implementation data into systematic reviews and meta-analyses. Journal of clinical epidemiology, 66(8), 874-882.
Moore, G., Raisanen, L., Moore, L., Ud Din, N., & Murphy, S. (2013). Mixed-method process evaluation of the welsh national exercise referral scheme. Health Education, 113(6), 476-501.
Ogilvie, D., Foster, C. E., Rothnie, H., Cavill, N., Hamilton, V., Fitzsimons, C. F., & Mutrie, N. (2007). Interventions to promote walking: systematic review. BMJ, bmj.39198.722720.BEv1.
Puhl, R. M., & Heuer, C. A. (2010). Obesity stigma: important considerations for public health. American journal of public health, 100(6), 1019-1028.
Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A. (2002). Criteria for evaluating evidence on public health interventions. Journal of Epidemiology & Community Health, 56(2), 119-127.
Steckler, A. B., Linnan, L., & Israel, B. A. (2002). Process evaluation for public health interventions and research (pp. 1-23). San Francisco, CA: Jossey-Bass.