Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions

Journal article

Seetohul, J,, Shafiee, M. and Sirlantzis, K. 2023. Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions. Sensors. 23 (13), p. 6202. https://doi.org/10.3390/s23136202
AuthorsSeetohul, J,, Shafiee, M. and Sirlantzis, K.
AbstractDespite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future.
KeywordsAugmented reality (AR); Navigation; Robotic and autonomous systems (RAS); Surgery; Planning; Machine learning (ML); Robotic Surgical Procedures - methods
Year2023
JournalSensors
Journal citation23 (13), p. 6202
PublisherMDPI
ISSN1424-8220
Digital Object Identifier (DOI)https://doi.org/10.3390/s23136202
https://doi.org/s23136202
Official URLhttps://www.mdpi.com/1424-8220/23/13/6202
FunderEngineering and Physical Sciences Research Council
Publication dates
Online06 Jul 2023
Publication process dates
Accepted03 Jul 2023
Deposited04 Dec 2023
Publisher's version
License
File Access Level
Open
Output statusPublished
Permalink -

https://repository.canterbury.ac.uk/item/95v17/augmented-reality-ar-for-surgical-robotic-and-autonomous-systems-state-of-the-art-challenges-and-solutions

Download files

Publisher's version
sensors-23-06202-v2.pdf
License: CC BY 4.0
File access level: Open

  • 55
    total views
  • 25
    total downloads
  • 4
    views this month
  • 2
    downloads this month

Export as

Related outputs

Light field imaging technology for virtual reality content creation: A review
Khan, A., Hossain, M., Covaci, A., Sirlantzis, K. and Xu, C. 2024. Light field imaging technology for virtual reality content creation: A review. IET Image Processing. 18 (11), pp. 2817-2837. https://doi.org/10.1049/ipr2.13144
Cancer: Investigating the impact of the implementation platform on machine learning models
Olowolayemo, A. S., Souag, A. and Sirlantzis, K. 2024. Cancer: Investigating the impact of the implementation platform on machine learning models. in: Mengoni, M. and Souag, A. (ed.) AIHealth 2024, The First International Conference on AI-Health ThinkMind.
Effect of Gait Speed on Trajectory Prediction Using Deep Learning Models for Exoskeleton Applications
Kolaghassi, R., Marcelli, G. and Sirlantzis, K. 2023. Effect of Gait Speed on Trajectory Prediction Using Deep Learning Models for Exoskeleton Applications. Sensors. 23 (12), p. 5687. https://doi.org/10.3390/s23125687
Depth estimation and validation of plenoptic light field camera
Khan, Ali, Hossain, Md. Moinul, Sirlantzis, Konstantinos, Covaci, Alexandra and Chowdhury, Wasif Shafaet 2023. Depth estimation and validation of plenoptic light field camera. in: 2023 IEEE International Conference on Imaging Systems and Techniques (IST) IEEE.
Train vs. play: Evaluating the effects of gamified and non-gamified wheelchair skills training using virtual reality
Zorzi, C., Tabbaa, Luma, Covaci, Alexandra, Sirlantzis, Konstantinos and Marcelli, Gianluca 2023. Train vs. play: Evaluating the effects of gamified and non-gamified wheelchair skills training using virtual reality. Bioengineering. 10 (11), p. 1269. https://doi.org/10.3390/bioengineering10111269
Community areas of sustainable care and dementia excellence in Europe (CASCADE) - Final report: A process and technology evaluation of the CASCADE programme in the United Kingdom implementation sites
Martin, A., Hatzidimitriadou, E., Sangeorzan, I., Smith, R., Wright, T., Price, N., Sirlantzis, K. and Rajapakse, S. 2023. Community areas of sustainable care and dementia excellence in Europe (CASCADE) - Final report: A process and technology evaluation of the CASCADE programme in the United Kingdom implementation sites. Canterbury Canterbury Christ Church University.
Design and control of a single-leg exoskeleton with gravity compensation for children with unilateral cerebral palsy
Sarajchi, M. and Sirlantzis, K. 2023. Design and control of a single-leg exoskeleton with gravity compensation for children with unilateral cerebral palsy. Sensors. 23 (13), p. 6103. https://doi.org/1424-8220/23/13/6103
An annotated water-filled, and dry potholes dataset for deep learning applications
Dib, J., Sirlantzis, K. and Howells, G. 2023. An annotated water-filled, and dry potholes dataset for deep learning applications. Data in Brief. 48, p. 109206. https://doi.org/10.1016/j.dib.2023.109206
Virtual and augmented reality in healthcare education (VARE) project report
Hatzidimitriadou, E., Taylor, J., Clark, N., Field, S., Reader, J., Buttery, A., Singh, B., Arealis, G. and Sirlantzis, K. 2022. Virtual and augmented reality in healthcare education (VARE) project report. Canterbury Canterbury Christ Church University.