Benefits of realist evaluation for rapidly changing health service delivery

Journal article


Jagosh, J., Stott, H., Halls, S., Thomas, R., Liddiard, C., Cupples, M., Cramp, F., Kersten, P., Foster, D. and Walsh, N.E. 2022. Benefits of realist evaluation for rapidly changing health service delivery. BMJ Open. 12 (e060347), pp. 1-5. https://doi.org/10.1136/bmjopen-2021-060347
AuthorsJagosh, J., Stott, H., Halls, S., Thomas, R., Liddiard, C., Cupples, M., Cramp, F., Kersten, P., Foster, D. and Walsh, N.E.
Abstract

Realist evaluation is a methodology that addresses the questions: ‘what works, for whom, in which circumstances,
and how?’. In this approach, programme theories are
developed and tested against available evidence. However,
when complex interventions are implemented in rapidly
changing environments, there are many unpredictable forces that determine the programme’s scope and architecture, as well as resultant outcome. These forces can be theorised, in real time, and included in realist evaluation outputs for current and future optimisation of programmes. Reflecting on a realist evaluation of first-contact physiotherapy in primary care (the FRONTIER Study), five important considerations are described for improving the quality of realist evaluation outputs when studying rapidly changing health service delivery. These are: (1) ensuring that initial programme theories are developed through creative thinking sessions, empirical and non-empirical literature, and stakeholder consultation; (2) testing the causal impact of formal and informal (eg, emergent) components of service delivery models; (3) contrasting initial programme theories with rival theory statements; (4) envisioning broad system impacts beyond the immediate implementation setting; and (5) incorporating rapidly evolving service developments and context changes into the theory testing process in real-time (eg, Additional Role Reimbursement Scheme, COVID-19). Through the reflections presented, the aim is to clarify the benefit of realist evaluation to assess emerging models of care and rapidly changing health service delivery.

KeywordsPhysiotherapy; First contact; Realist evaluation
Year2022
JournalBMJ Open
Journal citation12 (e060347), pp. 1-5
PublisherBMJ Publishing Group
ISSN2044-6055
Digital Object Identifier (DOI)https://doi.org/10.1136/bmjopen-2021-060347
Official URLhttps://bmjopen.bmj.com/content/bmjopen/12/7/e060347.full.pdf
FunderNational Institute for Health Research (NIHR) under its Research for Patient Benefit
Publication dates
Online29 Jul 2022
Publication process dates
Accepted05 Jul 2022
Deposited01 Aug 2022
Publisher's version
License
File Access Level
Open
Output statusPublished
Permalink -

https://repository.canterbury.ac.uk/item/91y9w/benefits-of-realist-evaluation-for-rapidly-changing-health-service-delivery

Download files


Publisher's version
Jagosh et al_FRONTIER_2022.pdf
License: CC BY 4.0
File access level: Open

  • 43
    total views
  • 25
    total downloads
  • 0
    views this month
  • 1
    downloads this month

Export as

Related outputs

Randomised pragmatic waitlist trial with process evaluation investigating the effectiveness of peer support after brain injury: protocol.
Kayes, N., Cummins, Christine, Weatherall, Mark, Smith, Greta, Te Ao, Braden, Elder, Hinemoa, Fadyl, J., Howard-Brown, Christine, Foster, Allison and Kersten, Paula 2023. Randomised pragmatic waitlist trial with process evaluation investigating the effectiveness of peer support after brain injury: protocol. BMJ Open. 13 (2), p. e069167. https://doi.org/bmjopen-2022-069167
Collegial surface acting emotional labour, burnout, and intention to leave in novice and pre-retirement nurses in the United Kingdom: a cross sectional study.
Theodosius, C., Koulougliotti, C., Kersten, P., Rosten, C., Clarke, A. and Warren, Z. 2021. Collegial surface acting emotional labour, burnout, and intention to leave in novice and pre-retirement nurses in the United Kingdom: a cross sectional study. Nursing Open. 8 (1), pp. 463-472. https://doi.org/10.1002/nop2.649
Design considerations for a Multiple Sclerosis Fatigue mobile app MS Energize: A pragmatic iterative approach using usability testing and resonance checks.
Van Kessel, K., Babbage, D., Kersten, P., Drown, J., Sezier, A., Thomas, P. and Thomas, S. 2021. Design considerations for a Multiple Sclerosis Fatigue mobile app MS Energize: A pragmatic iterative approach using usability testing and resonance checks. Internet Interventions. 24, p. 100371. https://doi.org/10.1016/j.invent.2021.100371
The immediate and subsequent impact of a first-time traumatic anterior shoulder dislocation, results from a national cohort study
Olds, M., Ellis, R., Parmar, P. and Kersten, P. 2020. The immediate and subsequent impact of a first-time traumatic anterior shoulder dislocation, results from a national cohort study. Shoulder and Elbow. 13 (2), pp. 223-232. https://doi.org/10.1177/1758573220921484
Predicting Recurrent Instability of the Shoulder (PRIS): A valid prediction tool of patients who will not have repeat shoulder instability after first-Time traumatic anterior dislocation
Olds, M., Ellis, R. and Kersten, P. 2020. Predicting Recurrent Instability of the Shoulder (PRIS): A valid prediction tool of patients who will not have repeat shoulder instability after first-Time traumatic anterior dislocation. Journal of Orthopaedic & Sports Physical Therapy. 50 (8), pp. 431-469. https://doi.org/10.2519/jospt.2020.9284
Provision of first contact physiotherapy in primary care across the UK: A survey of the service
Halls, S., Thomas. R., Stott, H., Cupples, M. E., Kersten, P., Cramp, F., Foster, D. and Walsh, N. 2020. Provision of first contact physiotherapy in primary care across the UK: A survey of the service. Physiotherapy. 108, pp. 2-9. https://doi.org/10.1016/j.physio.2020.04.005
Developing clinical expertise in musculoskeletal physiotherapy; Using observed practice to create a valued practice-based collaborative learning cycle
Carr, M., Morris, J. and Kersten, P. 2020. Developing clinical expertise in musculoskeletal physiotherapy; Using observed practice to create a valued practice-based collaborative learning cycle. Musculoskeletal Science and Practice. 50, p. 102278. https://doi.org/10.1016/j.msksp.2020.102278
Who will redislocate his/her shoulder? Predicting recurrent instability following a first traumatic anterior shoulder dislocation
Olds, M. K., Ellis, R., Parmar, P. and Kersten, P. 2019. Who will redislocate his/her shoulder? Predicting recurrent instability following a first traumatic anterior shoulder dislocation. BMJ Open Sport and Exercise Medicine. 5 (1). https://doi.org/10.1136/bmjsem-2018-000447
Digitizing a face-to-face group fatigue management program: exploring the views of people with multiple sclerosis and health care professionals via consultation groups and interviews
Thomas, S., Pulman, A., Thomas, P., Collard, S., Jiang, N., Dogan, H., Davies Smith, A., Hourihan, S., Roberts, F., Kersten, P., Pretty, K., Miller, J. K., Stanley, K. and Gay, M. 2019. Digitizing a face-to-face group fatigue management program: exploring the views of people with multiple sclerosis and health care professionals via consultation groups and interviews. JMIR Formative Research. 3 (2). https://doi.org/10.2196/10951
Incidence and outcomes of major trauma in New Zealand: findings from a feasibility study of New Zealand’s first national trauma registry
Czuba, K. J., Kersten, P., Anstiss, D., Kayes, N. M., Gabbe. B. J., Civil, I., Kool, B., Terry, G., Smith, G. A., Rohan, M. and Siegert, R. J. 2019. Incidence and outcomes of major trauma in New Zealand: findings from a feasibility study of New Zealand’s first national trauma registry. The New Zealand Medical Journal. 132 (1494), pp. 26-40.
Optimising the real-world impact of rehabilitation reviews: increasing the relevance and usability of systematic reviews in rehabilitation
Kayes, N., Martin, R. A., Bright, F. A. S., Kersten, P. and Pollock, A. 2019. Optimising the real-world impact of rehabilitation reviews: increasing the relevance and usability of systematic reviews in rehabilitation. European Journal of Physical and Rehabilitation Medicine. 55 (3), pp. 331-341. https://doi.org/10.23736/S1973-9087.19.05793-9
MS Energize: Field trial of an app for self-management of fatigue for people with multiple sclerosis
Babbage, D., Van Kessel, K., Drown, J., Thomas, S., Sezier, A., Thomas, P. and Kersten, P. 2019. MS Energize: Field trial of an app for self-management of fatigue for people with multiple sclerosis. Internet Interventions. https://doi.org/10.1016/j.invent.2019.100291
The Nijmegen Questionnaire: a valid measure for hyperventilation
Li Ogilvie, V., Kayes, N. and Kersten, P. 2019. The Nijmegen Questionnaire: a valid measure for hyperventilation. New Zealand Journal of Physiotherapy. 47 (3), pp. 160-171. https://doi.org/10.15619/NZJP/47.3.04
Cluster-randomised controlled trial of an occupational therapy intervention for children aged 11-13 years, designed to increase participation in order to prevent symptoms of mental illness
Tokolahi E., Vandal A., Kersten, P., Pearson J. and Hocking C. 2018. Cluster-randomised controlled trial of an occupational therapy intervention for children aged 11-13 years, designed to increase participation in order to prevent symptoms of mental illness. Child and Adolescent Mental Health. 23 (4), pp. 313-327. https://doi.org/10.1111/camh.12270
Developing a comprehensive framework of community integration for people with ABI: A conceptual analysis.
Shaikh, N., Kersten, P., Siegert, R. J. and Theodom, A. 2018. Developing a comprehensive framework of community integration for people with ABI: A conceptual analysis. Disability and Rehabilitation. 41 (4). https://doi.org/10.1080/09638288.2018.1443163
Strengths and Difficulties Questionnaire: internal validity and reliability for New Zealand pre-schoolers
Kersten, P., Vandal, A. C., Elder, H. and McPherson, K. M. 2018. Strengths and Difficulties Questionnaire: internal validity and reliability for New Zealand pre-schoolers. BMJ Open. 8 (4). https://doi.org/10.1136/bmjopen-2018-021551
Development of a toolkit to enhance care processes for people with a long-term neurological condition: a qualitative descriptive study
Sezier, A., Mudge, S., Kayes, N., Kersten, P., Payne, D., Harwood, M., Potter, E., Smith, G. and McPherson, K. M. 2018. Development of a toolkit to enhance care processes for people with a long-term neurological condition: a qualitative descriptive study. BMJ Open. 8 (6). https://doi.org/10.1136/bmjopen-2018-022038
Making sense of recovery after traumatic brain injury through a peer mentoring intervention: a qualitative exploration
Kersten, P., Cummins, C., Kayes, N., Babbage, D., Elder, H., Foster, A., Weatherall, M., Siegert, R. J., Smith, G. and McPherson, K. 2018. Making sense of recovery after traumatic brain injury through a peer mentoring intervention: a qualitative exploration. BMJ Open. 8 (10). https://doi.org/10.1136/bmjopen-2017-020672
Mapping the rehabilitation interventions of a community stroke team to the Extended International Classification of Functioning, Disability and Health Core Set for Stroke
Evans, M., Hocking, C. and Kersten, P. 2017. Mapping the rehabilitation interventions of a community stroke team to the Extended International Classification of Functioning, Disability and Health Core Set for Stroke. Disability and Rehabilitation. 39 (25). https://doi.org/10.1080/09638288.2016.1239763
Improving precision of the Five Facet Mindfulness Questionnaire using a Rasch approach.
Oleg N. Medvedev, Richard J. Siegert, Paula Kersten and Christian U. Krägeloh 2017. Improving precision of the Five Facet Mindfulness Questionnaire using a Rasch approach. Mindfulness. 8 (4), pp. 995-1008. https://doi.org/10.1007/s12671-016-0676-8
Concurrent validity of the Strengths and Difficulties Questionnaire in an indigenous pre-school population
Paula Kersten, Alain C. Vandal, Hinemoa Elder, Robyn Tauroa and Kathryn M. McPherson 2017. Concurrent validity of the Strengths and Difficulties Questionnaire in an indigenous pre-school population. Journal of Child and Family Studies. 26, p. 2126–2135. https://doi.org/10.1007/s10826-017-0725-5
Mobile technology use by people experiencing multiple sclerosis fatigue: survey methodology
Van Kessel, K., Babbage, D. R., Reay, N., Miner-Williams, W. M. and Kersten, P. 2017. Mobile technology use by people experiencing multiple sclerosis fatigue: survey methodology. JMIR mHealth and uHealth. 5 (2). https://doi.org/10.2196/mhealth.6192
Te Waka Kuaka, Rasch analysis of a cultural assessment tool in traumatic brain injury in Māori
Elder, H., Czuba, K., Kersten, P., Caracuel, A. and McPherson, K. 2017. Te Waka Kuaka, Rasch analysis of a cultural assessment tool in traumatic brain injury in Māori. F1000Research. 6 (1034). https://doi.org/10.12688/f1000research.11500.1
Measuring neurobehavioral functioning in people with traumatic brain injury: Rasch analysis of Neurobehavioral Functioning Inventory.
Czuba KJ, Kersten, P, Kayes NM, Smith G, Barker-Collo SL, Taylor WJ and McPherson KM 2016. Measuring neurobehavioral functioning in people with traumatic brain injury: Rasch analysis of Neurobehavioral Functioning Inventory. Journal of Head Trauma Rehabilitation. 31 (4), pp. E59-E68. https://doi.org/10.1097/HTR.0000000000000170
In children aged 18 years and under, what promotes recurrent shoulder instability after traumatic anterior shoulder dislocation? A systematic review and meta-analysis of risk factors.
Olds, M., Donaldson, K., Ellis, R. and Kersten, P. 2016. In children aged 18 years and under, what promotes recurrent shoulder instability after traumatic anterior shoulder dislocation? A systematic review and meta-analysis of risk factors. British Journal of Sports Medicine. https://doi.org/10.1136/bjsports-2015-095149
Development and content of a school-based occupational therapy intervention for promoting emotional wellbeing in children
Ema Tokolahi, Clare Hocking and Paula Kersten 2016. Development and content of a school-based occupational therapy intervention for promoting emotional wellbeing in children. Occupational Therapy in Mental Health. 32 (3), pp. 245-258. https://doi.org/10.1080/0164212X.2015.1129522
Making time; deeper connection, fuller stories, best practice. Experiences of Māori and non-Māori nurse-researchers
Hinemoa Elder, Paula Kersten, Kathryn McPherson, Lyn Dawson, Harry Harrison, Jaquie Harrison and Adrienne Poulter 2016. Making time; deeper connection, fuller stories, best practice. Experiences of Māori and non-Māori nurse-researchers. Annals of Psychiatry and Mental Health. 4 (6).
What influences the implementation of the New Zealand Stroke Guidelines for Physiotherapists and Occupational Therapists?
Suzie Mudge, Sankaran Murugan, Anna Hart and Paula Kersten 2016. What influences the implementation of the New Zealand Stroke Guidelines for Physiotherapists and Occupational Therapists? Disability and Rehabilitation. 39 (5), pp. 511-518. https://doi.org/10.3109/09638288.2016.1146361
Cross-cultural acceptability and utility of the Strengths and Difficulties Questionnaire: views of families
Kersten, P., Dudley, M., Nayar, S., Elder, H., Robertson, H., Tauroa, R. and McPherson, K. M. 2016. Cross-cultural acceptability and utility of the Strengths and Difficulties Questionnaire: views of families. BMC Psychiatry. 16 (347). https://doi.org/doi.org/10.1186/s12888-016-1063-7
Ordinal-to-interval scale conversion tables and national items for the New Zealand version of the WHOQOL-BREF
Krägeloh, C. U., Billington, D. R., Hsien-Chuan Hsu, P., Feng, X. J., Medvedev, O. N., Kersten, P., Landon, J. and Siegert, R. J. 2016. Ordinal-to-interval scale conversion tables and national items for the New Zealand version of the WHOQOL-BREF. PLoS ONE. 11 (11). https://doi.org/10.1371/journal.pone.0166065
A critical review of the psychometric properties of the Nijmegen Questionnaire for hyperventilation syndrome.
Ogilvie, V. L. and Kersten, P. 2015. A critical review of the psychometric properties of the Nijmegen Questionnaire for hyperventilation syndrome. New Zealand Journal of Physiotherapy. 43 (1). https://doi.org/10.15619/NZJP/43.1.01
Exploring researchers’ experiences of working with people with acquired brain injury
Greenwood, A., Theadom, A., Kersten, P. and McPherson, K. 2015. Exploring researchers’ experiences of working with people with acquired brain injury. Brain Injury. 29 (5). https://doi.org/10.3109/02699052.2014.1002422
Risk factors which predispose first-time anterior traumatic shoulder dislocations to recurrent instability in adults: A systematic review and meta-analysis
Olds, M., Ellis, R., Donaldson, K., Parmar, P. and Kersten, P. 2015. Risk factors which predispose first-time anterior traumatic shoulder dislocations to recurrent instability in adults: A systematic review and meta-analysis. British Journal of Sports Medicine. 49 (14). https://doi.org/10.1136/bjsports-2015-094584
Whakawhiti korero, a method for the development of a cultural assessment tool, Te Waka Kuaka in Maori traumatic brain injury
Elder, H. and Kersten, P. 2015. Whakawhiti korero, a method for the development of a cultural assessment tool, Te Waka Kuaka in Maori traumatic brain injury. Behavioural Neurology. https://doi.org/10.1155/2015/137402
Exploring strategies used following a group-based fatigue management programme for people with multiple sclerosis (FACETS) via the Fatigue Management Strategies Questionnaire (FMSQ)
S Thomas, P Kersten, P W Thomas, V Slingsby, A Nock, R Jones, A Davies Smith, K T Galvin, R Baker and C Hillier 2015. Exploring strategies used following a group-based fatigue management programme for people with multiple sclerosis (FACETS) via the Fatigue Management Strategies Questionnaire (FMSQ). BMJ Open. 2015 (5), p. e008274.. https://doi.org/10.1136/bmjopen-2015-008274.
A systematic review of evidence for the psychometric properties of the Strengths and Difficulties Questionnaire
Paula Kersten, Karol Czuba, Kathryn McPherson, Margaret Dudley, Hinemoa Elder, Robyn Tauroa and Alain Vandal 2015. A systematic review of evidence for the psychometric properties of the Strengths and Difficulties Questionnaire. International Journal of Behavioral Development. https://doi.org/10.1177/0165025415570647
Quality and reporting of cluster randomised controlled trials evaluating occupational therapy interventions: A systematic review
Ema Tokolahi, Clare Hocking, Paula Kersten and Alain C. Vandal 2015. Quality and reporting of cluster randomised controlled trials evaluating occupational therapy interventions: A systematic review. OTJR: Occupation, Participation and Health. 36 (1), pp. 14-24. https://doi.org/10.1177/1539449215618625
One year follow-up of a pragmatic multi-centre randomised controlled trial of a group-based fatigue management programme (FACETS) for people with MS.
Thomas P., Thomas S., Kersten P., Jones R., Slingsby V., Nock A., Davies Smith A., Baker R., Galvin K. and Hillier C. 2014. One year follow-up of a pragmatic multi-centre randomised controlled trial of a group-based fatigue management programme (FACETS) for people with MS. BMC Neurology. 14. https://doi.org/10.1186/1471-2377-14-109
Is the pain Visual Analogue Scale linear and responsive to change? An exploration using Rasch analysis
Kersten, P., White, P. J. and Tennant, A. 2014. Is the pain Visual Analogue Scale linear and responsive to change? An exploration using Rasch analysis. PLoS ONE. 9 (6), p. e99485. https://doi.org/10.1371/journal.pone.0099485
Cluster-randomised controlled trial of an occupational therapy group intervention for children designed to promote emotional wellbeing: study protocol
Tokolahi, E. M., Hocking, C., Kersten, P. and Vandal, A. C. 2014. Cluster-randomised controlled trial of an occupational therapy group intervention for children designed to promote emotional wellbeing: study protocol. BMC Psychology. 2. https://doi.org/10.1186/2050-7283-2-16
The Patient-Rated Wrist and Hand Evaluation: a systematic review of its validity and reliability
Taylor, J. and Kersten, P. 2014. The Patient-Rated Wrist and Hand Evaluation: a systematic review of its validity and reliability. New Zealand Journal of Physiotherapy. 42 (2), pp. 139-145. https://doi.org/10.15619/nzjp/42.3.01
Older adults’ experiences of community integration following traumatic brain injury.
Kent, J., Kersten, P. and Wright-St Clair, V. 2014. Older adults’ experiences of community integration following traumatic brain injury. New Zealand Journal of Physiotherapy. 42 (3), pp. 148-153.
The Multiple Sclerosis Fatigue Self-Efficacy Scale (MS-FSE): initial validation
Thomas, S., Kersten, P. and Thomas, P. 2014. The Multiple Sclerosis Fatigue Self-Efficacy Scale (MS-FSE): initial validation. Clinical Rehabilitation. 29 (4), pp. 376-387. https://doi.org/10.1177/0269215514543702
Bridging the gap between goal intentions and actions: a systematic review in patient populations
Kersten, P., McCambridge, A., McPherson, K., Kayes, N. and Theadom, A. 2014. Bridging the gap between goal intentions and actions: a systematic review in patient populations. Disability and Rehabilitation. 37 (7), p. 563=570. https://doi.org/10.3109/09638288.2014.935874
Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology.
Krägeloh CU, Czuba K, Billington R, Kersten, P and Siegert R 2014. Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology. Psychiatric Services. 66 (3), pp. 224-241. https://doi.org/10.1176/appi.ps.201400141
Bridging the goal intention-action gap in rehabilitation: a study of if-then implementation intentions in neurorehabilitation
Kersten, P., McPherson, K. M., Kayes, N. M., Theadom, A. and McCambridge, A. 2014. Bridging the goal intention-action gap in rehabilitation: a study of if-then implementation intentions in neurorehabilitation. Disability & Society. 37 (12), pp. 1073-1081. https://doi.org/10.3109/09638288.2014.955137