A co-designed patient reported experience measure for understanding the patient's and public experience of receiving X-ray results

Introduction: The National Health Service England (NHSE) General Practitioner (GP) contract mandates the implementation of electronic health records (EHR) to improve patient access to medical results. However, access to complex X-ray reports can lead to patient anxiety and misunderstanding, necessitating the requirement for clear communication. This paper reports an example of a good practice approach to research co-design and co-production with Patient and Public Involvement (PPI) experts to develop a Patient Reported Experience Measure (PREM) to explore accessing X-ray reports through EHR.

Method: A participatory method of a modified e-Delphi (iterative process) utilising PPI focus groups followed by two rounds of reviewing online survey questions to gain consensus. The PREM used Likert scoring, data analysis of each subscale score, and distributions, including face, construct, and content validity, reliability coefficient, principal component and factor analysis, and dimension reduction.

Results: The phase one PPI focus groups (n=28 participants) co-produced n=16 themes related to the experience of receiving X-ray results. Key themes emphasised the need for patient-friendly language to reduce anxiety and misunderstanding of complex language and terminology. The PREM tool was refined through two rounds of iterative feedback. The phase two survey included n=57 questions, while the phase three survey pared down questions to n=27.

Conclusion: It is critical to involve patients in the co-design and co-production of PREM tools to ensure they appropriately capture patient and public lived experiences. Implementing clear, patient-friendly communication within EHR X-ray reports could reduce anxiety and empower patients’ decision-making.

Implications for Practice: National testing of the PREM involving a large sample of diverse participant demographics is recommended. Future PREM findings will assist in recommending and developing strategies to improve EHR communication of X-ray report wording, structure, and content.

1. NHS England. Changes to the GP Contract in 2023/24 PRN00257. Changes to the GP Contract in 2023/24 PRN00257 2023:1–10.
2. Oh SC., Cook TS., Kahn CE. PORTER: a Prototype System for Patient-Oriented Radiology Reporting. J Digit Imaging 2016;29(4):450–4. Doi: 10.1007/s10278-016-9864-2.
3. Blease C. Open AI meets open notes: surveillance capitalism, patient privacy and online record access. J Med Ethics 2023:jme-2023-109574. Doi: 10.1136/jme-2023-109574.
4. Blease C., Fernandez L., Bell SK., Delbanco T., DesRoches C. Empowering patients and reducing inequities: is there potential in sharing clinical notes? BMJ Qual Saf 2020;29(10):1.8-2. Doi: 10.1136/bmjqs-2019-010490.
5. McCarthy DM., Waite KR., Curtis LM., Engel KG., Baker DW., Wolf MS. What Did the Doctor Say? Health Literacy and Recall of Medical Instructions. Med Care 2012;50(4):277–82. Doi: 10.1097/MLR.0b013e318241e8e1.
6. Kadom N., Tamasi S., Vey BL., Safdar N., Applegate KE., Sadigh G., et al. Info-RADS: Adding a Message for Patients in Radiology Reports. Journal of the American College of Radiology 2021;18(1):128–32. Doi: 10.1016/j.jacr.2020.09.049.
7. Alarifi M., Patrick T., Jabour A., Wu M., Luo J. Designing a Consumer-Friendly Radiology Report using a Patient-Centered Approach. J Digit Imaging 2021;34(3):705–16. Doi: 10.1007/s10278-021-00448-z.
8. Lockwood P., Burton C., Woznitza N., Shaw T. Assessing the barriers and enablers to the implementation of the diagnostic radiographer musculoskeletal X-ray reporting service within the NHS in England: a systematic literature review. BMC Health Serv Res 2023;23(1):1270. Doi: 10.1186/s12913-023-10161-y.
9. Health Education England. HEE Business Plan 2022/23, London; 2022.
10. Department of Health. Radiography Skills Mix , London; 2003.
11. Royal College of Radiologists. RCR Clinical radiology census report 2021, London; 2022.
12. NHS England. Diagnostic imaging network workforce guidance, London; 2022.
13. NHS England. Cancer Workforce Plan Phase 1: Delivering the cancer strategy to 2021, London; 2017.
14. NHS England. Five Year Forward View, London; 2014.
15. NHS England. Diagnostics: Recovery and Renewal, London; 2020.
16. NHS England. We Are The NHS: People Plan 2020/21-action for us all, London; 2020.
17. NHS England. Diagnostic imaging network workforce guidance, London; 2022.
18. Royal National Orthopaedic Hospital NHS Trust., NHS England., NHS Improvement. Getting it right first time: Radiology, GIRFT Programme National Speciality Report., London; 2020.
19. Orel SG., Kay N., Reynolds C., Sullivan DC. BI-RADS Categorization As a Predictor of Malignancy. Radiology 1999;211(3):845–50. Doi: 10.1148/radiology.211.3.r99jn31845.
20. Mityul MI., Gilcrease-Garcia B., Mangano MD., Demertzis JL., Gunn AJ. Radiology Reporting: Current Practices and an Introduction to Patient-Centered Opportunities for Improvement. American Journal of Roentgenology 2018;210(2):376–85. Doi: 10.2214/AJR.17.18721.
21. Goldberg-Stein S., Chernyak V. Adding Value in Radiology Reporting. Journal of the American College of Radiology 2019;16(9):1292–8. Doi: 10.1016/j.jacr.2019.05.042.
22. Schwartz LH., Panicek DM., Berk AR., Li Y., Hricak H. Improving Communication of Diagnostic Radiology Findings through Structured Reporting. Radiology 2011;260(1):174–81. Doi: 10.1148/radiol.11101913.
23. Hartung MP., Bickle IC., Gaillard F., Kanne JP. How to Create a Great Radiology Report. RadioGraphics 2020;40(6):1658–70. Doi: 10.1148/rg.2020200020.
24. Brady AP. Radiology reporting—from Hemingway to HAL? Insights Imaging 2018;9(2):237–46. Doi: 10.1007/s13244-018-0596-3.
25. Grieve FM., Plumb AA., Khan SH. Radiology reporting: a general practitioner’s perspective. Br J Radiol 2010;83(985):17–22. Doi: 10.1259/bjr/16360063.
26. Lourenco AP., Baird GL. Optimizing Radiology Reports for Patients and Referring Physicians: Mitigating the Curse of Knowledge. Acad Radiol 2020;27(3):436–9. Doi: 10.1016/j.acra.2019.03.026.
27. Naik SS., Hanbidge A., Wilson SR. Radiology Reports. American Journal of Roentgenology 2001;176(3):591–8. Doi: 10.2214/ajr.176.3.1760591.
28. Plumb AAO., Grieve FM., Khan SH. Survey of hospital clinicians’ preferences regarding the format of radiology reports. Clin Radiol 2009;64(4):386–94. Doi: 10.1016/j.crad.2008.11.009.
29. Jeblick K., Schachtner B., Dexl J., Mittermeier A., Stüber AT., Topalis J., et al. ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports. Eur Radiol 2023. Doi: 10.1007/s00330-023-10213-1.
30. Lyu Q., Tan J., Zapadka ME., Ponnatapura J., Niu C., Myers KJ., et al. Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Vis Comput Ind Biomed Art 2023;6(1):9. Doi: 10.1186/s42492-023-00136-5.
31. Cook TS., Oh SC., Kahn CE. Patients’ Use and Evaluation of an Online System to Annotate Radiology Reports with Lay Language Definitions. Acad Radiol 2017;24(9):1169–74. Doi: 10.1016/j.acra.2017.03.005.
32. Hutton D., Booker P., Shuttleworth S., Hall T., Cain M., Murphy T., et al. The Patient’s perspective: A review of the results from a radiotherapy patient experience survey in the North-West of England. Radiography 2023;29:S59–67. Doi: 10.1016/j.radi.2023.02.019.
33. Temple S., Rowbottom C., Simpson J. Patient views on the implementation of artificial intelligence in radiotherapy. Radiography 2023;29:S112–6. Doi: 10.1016/j.radi.2023.03.006.
34. Flood T., O Neill A., Oliveira CM., Barbosa B., Soares AL., Muscat K., et al. Patients’ perspectives of the skills and competencies of therapy radiographers/radiation therapists (TRs/RTTs) in the UK, Portugal and Malta; a qualitative study from the SAFE Europe project. Radiography 2023;29:S117–27. Doi: 10.1016/j.radi.2023.03.002.
35. O’Neill A., Hughes C., McClure P., Rainey C., McLaughlin L., McFadden S. Patient engagement with radiation therapists: Patient perspectives, challenges, and opportunities. A systematic review. Radiography 2023;29:S128–36. Doi: 10.1016/j.radi.2023.02.022.
36. Gordon L., Dickinson A., Offredy M., Smiddy J. A research note on the benefit of patient and public involvement in research: The experience of prostate cancer patients regarding information in radiotherapy. Radiography 2017;23(2):167–70. Doi: 10.1016/j.radi.2017.02.004.
37. Newton-Hughes A., Strudwick R. An exploration of values-based radiography from the perspective of the service user. Radiography 2023;29:S40–5. Doi: 10.1016/j.radi.2023.01.012.
38. Society and College of Radiographers. Patient Public and Practitioner Partnerships within Imaging and Radiotherapy: Guiding Principles, London; 2018.
39. Hancock A., Bleiker J. But what does it mean to us? Radiographic patients and carer perceptions of compassion. Radiography 2023;29:S74–80. Doi: 10.1016/j.radi.2023.02.005.
40. Kingsley C., Patel S. Patient-reported outcome measures and patient-reported experience measures. BJA Educ 2017;17(4):137–44. Doi: 10.1093/bjaed/mkw060.
41. Fontaine G., Poitras M-E., Sasseville M., Pomey M-P., Ouellet J., Brahim LO., et al. Barriers and enablers to the implementation of patient-reported outcome and experience measures (PROMs/PREMs): protocol for an umbrella review. Syst Rev 2024;13(1):96. Doi: 10.1186/s13643-024-02512-5.
42. Barrett D., Heale R. What are Delphi studies? Evid Based Nurs 2020;23(3):68–9.
43. NHS Health Research Authority. Defining Research. HRA Decision Tools.
44. National Institute for Health and Care Research. Briefing note five: how to involve members of the public in research UK Standards for Public Involvement, London; 2021.
45. GCP ICH. ICH Harmonised Guideline Integrated Addendum to ICH E6 (R1): Guideline for Good Clinical Practice ICH E6 (R2) ICH Consensus Guideline. London: ICH GCP 2020.
46. Medicines and Healthcare products Regulatory Agency (MHRA). Good Clinical Practice Guide, London; 2012.
47. National Institute for Health Research. NIHR Co-production in Action – Number One , 2021.
48. National Institute for Health Research. A brief guide to patient and public involvement and qualitative methods within health and social care research, 2021.
49. Garcia-Iglesias J., Beange I., Davidson D., Goopy S., Huang H., Murray F., et al. Ethical considerations in public engagement: developing tools for assessing the boundaries of research and involvement. Res Involv Engagem 2024;10(1):83. Doi: 10.1186/s40900-024-00617-8.
50. Burke NN., Stewart D., Tierney T., Worrall A., Smith M., Elliott J., et al. Sharing space at the research table: exploring public and patient involvement in a methodology priority setting partnership. Res Involv Engagem 2023;9(1):29. Doi: 10.1186/s40900-023-00438-1.
51. Blair C., Best P., Burns P., Campbell A., Davidson G., Duffy J., et al. ‘Getting involved in research’: a co-created, co-delivered and co-analysed course for those with lived experience of health and social care services. Res Involv Engagem 2022;8(1):20. Doi: 10.1186/s40900-022-00353-x.
52. Coldham T. Guidance on co-producing a research project, 2021.
53. Turk A., Boylan A., Locock L. A Researcher’s Guide to Patient and Public Involvement, Oxford; 2017.
54. Smits D-W., van Meeteren K., Klem M., Alsem M., Ketelaar M. Designing a tool to support patient and public involvement in research projects: the Involvement Matrix. Res Involv Engagem 2020;6(1):30. Doi: 10.1186/s40900-020-00188-4.
55. Braun V and C V. Using thematic analysis in psychology . Qualitative Research in Psychology 2006;3(2):77–101.
56. Braun V., Clarke V. Toward good practice in thematic analysis: Avoiding common problems and be(com)ing a knowing researcher. Int J Transgend Health 2023;24(1):1–6. Doi: 10.1080/26895269.2022.2129597.
57. Information Commissioner’s Office. What is personal data? . Guide to the General Data Protection Regulation (GDPR). Available from: https://ico.org.uk/for-organisations/guide-to-data-protection/guide-... [accessed October 17, 2022].
58. Information Commissioner’s Office. Special category data . Guide to the General Data Protection Regulation (GDPR). Available from: https://ico.org.uk/for-organisations/guide-to-data-protection/guide-... [accessed October 17, 2022].
59. Hatem G., Zeidan J., Goossens M., Moreira C. Normality testing methods and the importance of Skewness and Kurtosis in statistical analysis. BAU Journal - Science and Technology 2022;3(2). Doi: 10.54729/KTPE9512.
60. Taherdoost H. Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. SSRN Electronic Journal 2016. Doi: 10.2139/ssrn.3205040.
61. Grogan S. Validation of a questionnaire measuring patient satisfaction with general practitioner services. Quality in Health Care 2000;9(4):210–5. Doi: 10.1136/qhc.9.4.210.
62. Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975;28(4):563–75.
63. Aithal A., Aithal PS. Development and Validation of Survey Questionnaire & Experimental Data – A Systematical Review-based Statistical Approach. SSRN Electronic Journal 2020. Doi: 10.2139/ssrn.3724105.
64. Kaiser HF., Rice J. Little jiffy, mark IV. Educ Psychol Meas 1974;34(1):111–7.
65. Kaiser HF. An index of factorial simplicity. Psychometrika 1974;39(1):31–6.
66. Bartlett MS. The effect of standardization on a χ 2 approximation in factor analysis. Biometrika 1951;38(3/4):337–44.
67. Shrestha N. Factor Analysis as a Tool for Survey Analysis. Am J Appl Math Stat 2021;9(1):4–11. Doi: 10.12691/ajams-9-1-2.
68. Kaiser HF. The varimax criterion for analytic rotation in factor analysis. Psychometrika 1958;23(3):187–200.
69. Braun V., Clarke C., Weate P. Using thematic analysis in sport and exercise research. In: Smith B, and Sparkes AC, editors. Routledge Handbook of Qualitative Research in Sport and Exercise. 1st ed., Routledge; 2017, p. 191–205.
70. Office for National Statistics. Census 2021 Custom area profile Kent. Census 2021 Custom Area Profile Kent. Available from: https://www.ons.gov.uk/visualisations/customprofiles/build/ [accessed June 25, 2024].
71. Office for National Statistics. Census 2021 Custom area profile Medway. Census 2021 Custom Area Profile. Available from: https://www.ons.gov.uk/visualisations/customprofiles/build/#E0600003... [accessed June 25, 2024].
72. Crocker JC., Boylan A., Bostock J., Locock L. Is it worth it? Patient and public views on the impact of their involvement in health research and its assessment: a UK‐based qualitative interview study. Health Expectations 2017;20(3):519–28. Doi: 10.1111/hex.12479.
73. Perlis N., Finelli A., Lovas M., Berlin A., Papadakos J., Ghai S., et al. Creating patient-centered radiology reports to empower patients undergoing prostate magnetic resonance imaging. Canadian Urological Association Journal 2020;15(4). Doi: 10.5489/cuaj.6585.
74. Kuckelman IJ., Wetley K., Yi PH., Ross AB. Translating musculoskeletal radiology reports into patient-friendly summaries using ChatGPT-4. Skeletal Radiol 2024;53(8):1621–4. Doi: 10.1007/s00256-024-04599-2.
75. Darzi A. Independent investigation of the National Health Service in England , London; 2024.
76. Lee CI., Langlotz CP., Elmore JG. Implications of direct patient online access to radiology reports through patient web portals. Journal of the American College of Radiology 2016;13(12):1608–14.
77. Walker J., Darer JD., Elmore JG., Delbanco T. The road toward fully transparent medical records. New England Journal of Medicine 2014;370(1):6–8.
78. Wang J., Goldberg JE., Block T., Ostrow D., Carbone D., Recht M., et al. Patterns of Access to Radiology Reports and Images Through a Patient Portal. Journal of Imaging Informatics in Medicine 2024;37(2):504–9. Doi: 10.1007/s10278-024-00996-0.
79. Johnson AJ., Easterling D., Williams LS., Glover S., Frankel RM. Insight From Patients for Radiologists: Improving Our Reporting Systems. Journal of the American College of Radiology 2009;6(11):786–94. Doi: 10.1016/j.jacr.2009.07.010.
80. Alarifi M., Patrick T., Jabour A., Wu M., Luo J. Full Radiology Report through Patient Web Portal: A Literature Review. Int J Environ Res Public Health 2020;17(10):3673. Doi: 10.3390/ijerph17103673.
81. Vitzthum von Eckstaedt H., Kitts AB., Swanson C., Hanley M., Krishnaraj A. Patient-centered Radiology Reporting for Lung Cancer Screening. J Thorac Imaging 2020;35(2):85–90. Doi: 10.1097/RTI.0000000000000469.