AI assisted reader evaluation in acute CT head interpretation (AI-REACT): protocol for a multireader multicase study
Journal article
Howell Fu, Alex Novak, Dennis Robert, Shamie Kumar, Swetha Tanamala, Jason Oke, Kanika Bhatia, Ruchir Shah, Andrea Romsauerova, Tilak Das, Abdalá Espinosa, Mariusz Tadeusz Grzeda, Mariapaola Narbone, Rahul Dharmadhikari, Mark Harrison, Kavitha Vimalesvaran, Jane Gooch, Nicholas Woznitza, Nabeeha Salik, Alan Campbell, Farhaan Khan, David J Lowe, Haris Shuaib and Sarim Ather 2024. AI assisted reader evaluation in acute CT head interpretation (AI-REACT): protocol for a multireader multicase study. BMJ Open. 14 (2), p. e079824. https://doi.org/10.1136/bmjopen-2023-079824
Authors | Howell Fu, Alex Novak, Dennis Robert, Shamie Kumar, Swetha Tanamala, Jason Oke, Kanika Bhatia, Ruchir Shah, Andrea Romsauerova, Tilak Das, Abdalá Espinosa, Mariusz Tadeusz Grzeda, Mariapaola Narbone, Rahul Dharmadhikari, Mark Harrison, Kavitha Vimalesvaran, Jane Gooch, Nicholas Woznitza, Nabeeha Salik, Alan Campbell, Farhaan Khan, David J Lowe, Haris Shuaib and Sarim Ather |
---|---|
Abstract | Introduction A non-contrast CT head scan (NCCTH) is the most common cross-sectional imaging investigation requested in the emergency department. Advances in computer vision have led to development of several artificial intelligence (AI) tools to detect abnormalities on NCCTH. These tools are intended to provide clinical decision support for clinicians, rather than stand-alone diagnostic devices. However, validation studies mostly compare AI performance against radiologists, and there is relative paucity of evidence on the impact of AI assistance on other healthcare staff who review NCCTH in their daily clinical practice. Methods and analysis A retrospective data set of 150 NCCTH will be compiled, to include 60 control cases and 90 cases with intracranial haemorrhage, hypodensities suggestive of infarct, midline shift, mass effect or skull fracture. The intracranial haemorrhage cases will be subclassified into extradural, subdural, subarachnoid, intraparenchymal and intraventricular. 30 readers will be recruited across four National Health Service (NHS) trusts including 10 general radiologists, 15 emergency medicine clinicians and 5 CT radiographers of varying experience. Readers will interpret each scan first without, then with, the assistance of the qER EU 2.0 AI tool, with an intervening 2-week washout period. Using a panel of neuroradiologists as ground truth, the stand-alone performance of qER will be assessed, and its impact on the readers’ performance will be analysed as change in accuracy (area under the curve), median review time per scan and self-reported diagnostic confidence. Subgroup analyses will be performed by reader professional group, reader seniority, pathological finding, and neuroradiologist-rated difficulty. Ethics and dissemination The study has been approved by the UK Healthcare Research Authority (IRAS 310995, approved 13 December 2022). The use of anonymised retrospective NCCTH has been authorised by Oxford University Hospitals. The results will be presented at relevant conferences and published in a peer-reviewed journal. |
Keywords | AI; Artificial intelligence; CT scans; CT head scans; Radiology and imaging |
Year | 2024 |
Journal | BMJ Open |
Journal citation | 14 (2), p. e079824 |
Publisher | BMJ Publishing Group |
ISSN | 2044-6055 |
Digital Object Identifier (DOI) | https://doi.org/10.1136/bmjopen-2023-079824 |
Official URL | https://bmjopen.bmj.com/content/14/2/e079824 |
Funder | NIHR |
Publication dates | |
12 Feb 2024 | |
Publication process dates | |
Accepted | 28 Jan 2024 |
Deposited | 21 Feb 2024 |
Publisher's version | License File Access Level Open |
Output status | Published |
Additional information | Trial registration number NCT06018545. |
https://repository.canterbury.ac.uk/item/97237/ai-assisted-reader-evaluation-in-acute-ct-head-interpretation-ai-react-protocol-for-a-multireader-multicase-study
Download files
65
total views31
total downloads6
views this month1
downloads this month
Export as
Related outputs
An implementation facilitation intervention to improve the musculoskeletal X‑ray reporting by radiographers across London
Lockwood, P., Burton, C., Shaw, T., Woznitza, N., Compton, E., Groombridge, H., Hayes, N., Mane, U., O'Brien, A. and Patterson, S. 2025. An implementation facilitation intervention to improve the musculoskeletal X‑ray reporting by radiographers across London. BMC Health Services Research. 25 (248), p. 1. https://doi.org/10.1186/s12913-025-12356-xAccuracy of interpretation of nasogastric tube position on chest radiographs by diagnostic radiographers: A multi-case, multi-reader study
Creeden, A., McFadden, S., Ather, S. and Woznitza, N. 2025. Accuracy of interpretation of nasogastric tube position on chest radiographs by diagnostic radiographers: A multi-case, multi-reader study. Radiography. 31 (1), pp. 83-88. https://doi.org/10.1016/j.radi.2024.10.022Achieving earlier diagnosis of symptomatic lung cancer
Bradley, S., Baldwin, D., Bhartia, B., Black, G., Callister, Matthew Ej, Clayton, Karen, Eccles, Sinan R, Evison, Matthew, Fox, Jesme, Hamilton, W., Konya, J., Lee, Richard W, Bradley, S., Navani, Neal, Noble, Ben, Quaife, Samantha L, Randle, Amelia, Rawlinson, Janette, Richards, Michael, Woznitza, Nick and O'Dowd, Emma 2024. Achieving earlier diagnosis of symptomatic lung cancer. The British Journal of General Practice : The Journal of the Royal College of General Practitioners. 75 (750), pp. 40-43. https://doi.org/10.3399/bjgp25X740493Artificial intelligence (AI) for paediatric fracture detection: a multireader multicase (MRMC) study protocol.
Shelmerdine, S., Pauling, Cato, Allan, Emma, Langan, Dean, Ashworth, Emily, Yung, Ka-Wai, Barber, Joy, Haque, Saira, Rosewarne, David, Woznitza, N., Ather, S., Novak, A., Theivendran, Kanthan and Arthurs, O. 2024. Artificial intelligence (AI) for paediatric fracture detection: a multireader multicase (MRMC) study protocol. BMJ Open. 14 (12), p. e084448. https://doi.org/10.1136/bmjopen-2024-084448Evaluating the impact of artificial intelligence-assisted image analysis on the diagnostic accuracy of front-line clinicians in detecting fractures on plain X-rays (FRACT-AI): protocol for a prospective observational study.
Novak, A., Hollowday, Max, Espinosa Morgado, A., Oke, Jason, Shelmerdine, S., Woznitza, N., Metcalfe, David, Costa, Matthew L, Wilson, S., Kiam, Jian Shen, Vaz, J., Limphaibool, N., Ventre, Jeanne, Jones, Daniel, Greenhalgh, Lois, Gleeson, Fergus, Welch, Nick, Mistry, Alpesh, Devic, Natasa, Teh, James and Ather, S. 2024. Evaluating the impact of artificial intelligence-assisted image analysis on the diagnostic accuracy of front-line clinicians in detecting fractures on plain X-rays (FRACT-AI): protocol for a prospective observational study. BMJ Open. 14 (9), p. e086061. https://doi.org/10.1136/bmjopen-2024-086061A survey of the NHS reporting radiographer workforce in England
Lockwood, P., Burton, C., Shaw, T. and Woznitza, N. 2024. A survey of the NHS reporting radiographer workforce in England. Radiography Open. 10 (1), pp. 1-18. https://doi.org/10.7577/radopen.5635Reporting radiographers within the European Federation of Radiographer Society (EFRS) member countries - motivation for becoming a reporting radiographer.
Jensen, J, Blackburn, P A, Gale, N, Senior, C, Woznitza, N, Heales, C J and Pedersen, M R V 2024. Reporting radiographers within the European Federation of Radiographer Society (EFRS) member countries - motivation for becoming a reporting radiographer. Radiography. 30 (3), pp. 731-736. https://doi.org/S1078-8174(24)00055-5Reporting radiographers in Europe survey: An overview of the role within the European Federation of Radiographer Society (EFRS) member countries
Pedersen, M.R. V., Jensen, J., Senior, C., Gale, N., Heales, C. J. and Woznitza, N. 2023. Reporting radiographers in Europe survey: An overview of the role within the European Federation of Radiographer Society (EFRS) member countries. Radiography. 29 (6), pp. 1100-1107. https://doi.org/10.1016/j.radi.2023.09.005Assessing the barriers and enablers to the implementation of the diagnostic radiographer musculoskeletal X‑ray reporting service within the NHS in England: a systematic literature review
Lockwood, P., Burton, C., Woznitza, N. and Shaw, T. 2023. Assessing the barriers and enablers to the implementation of the diagnostic radiographer musculoskeletal X‑ray reporting service within the NHS in England: a systematic literature review. BMC Health Services Research. 23 (1270), pp. 1-41. https://doi.org/10.1186/s12913-023-10161-y