Automatic Re-orientation of 3D Echocardiographic Images in Virtual Reality Using Deep Learning

Lindsay Munroe*, Gina Sajith, Ei Lin, Surjava Bhattacharya, Kuberan Pushparajah, John Simpson, Julia A. Schnabel, Gavin Wheeler, Alberto Gomez, Shujie Deng

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


In 3D echocardiography (3D echo), the image orientation varies depending on the position and direction of the transducer during examination. As a result, when reviewing images the user must initially identify anatomical landmarks to understand image orientation – a potentially challenging and time-consuming task. We automated this initial step by training a deep residual neural network (ResNet) to predict the rotation required to re-orient an image to the standard apical four-chamber view). Three data pre-processing strategies were explored: 2D, 2.5D and 3D. Three different loss function strategies were investigated: classification of discrete integer angles, regression with mean absolute angle error loss, and regression with geodesic loss. We then integrated the model into a virtual reality application and aligned the re-oriented 3D echo images with a standard anatomical heart model. The deep learning strategy with the highest accuracy – 2.5D classification of discrete integer angles – achieved a mean absolute angle error on the test set of 9.0. This work demonstrates the potential of artificial intelligence to support visualisation and interaction in virtual reality.

Original languageEnglish
Title of host publicationMedical Image Understanding and Analysis - 25th Annual Conference, MIUA 2021, Proceedings
EditorsBartłomiej W. Papież, Mohammad Yaqub, Jianbo Jiao, Ana I. Namburete, J. Alison Noble
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages12
ISBN (Print)9783030804312
Publication statusPublished - 2021
Externally publishedYes
Event25th Annual Conference on Medical Image Understanding and Analysis, MIUA 2021 - Virtual, Online
Duration: 12 Jul 202114 Jul 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12722 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference25th Annual Conference on Medical Image Understanding and Analysis, MIUA 2021
CityVirtual, Online

Bibliographical note

Funding Information:
L. Munroe—This work is independent research funded by the National Institute for Health Research (NIHRi4i, 3D Heart Project, II-LA-0716-20001, https://www.3dheart. This work was also supported by the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z). Lindsay Munroe and Suryava Bhattacharya would like to acknowledge funding from the EPSRC Centre for Doctoral Training in Smart Medical Imaging (EP/S022104/1). Authors also acknowledge financial support from the Department of Health via the National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s and St Thomas’ NHS Foundation Trust in partnership with King’s College London and King’s College Hospital NHS Foundation Trust.

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.


  • 3D echocardiography
  • Deep learning
  • Virtual reality


Dive into the research topics of 'Automatic Re-orientation of 3D Echocardiographic Images in Virtual Reality Using Deep Learning'. Together they form a unique fingerprint.

Cite this