AI and Virtual Reality for Visualization and Segmentation of Intracardiac Anatomy and Great Vessels from Cardiovascular Magnetic Resonance Images

Background: Accurate visualization of intracardiac anatomy and great vessels is critical for diagnosis, surgical planning, and education in congenital and acquired heart disease. Cardiovascular magnetic resonance (CMR) provides high-resolution, 3D anatomical and functional data, but manual segmentation is time-consuming and subject to interobserver variability. Artificial intelligence (AI)-assisted segmentation algorithms offer automated, reproducible delineation of cardiac structures, while virtual reality (VR) platforms enable interactive, immersive visualization of complex anatomies and proofreading of AI segmentation results. Integrating AI segmentation with VR visualization has the potential to improve clinical decision-making, preoperative planning, and education for cardiologists, cardiothoracic surgeons, and patients with heart disease.

Methods: Five patients (2 male, 3 female, median age 15 y/o, range [10 – 16]) with congenital heart disease who underwent gadolinium-based contrast-enhanced CMR exams for evaluation of cardiac anatomy, function, and fibrosis were retrospectively selected with IRB approval. Post contrast static 3D whole-heart SSFP images were manually segmented in syGlass software into eight labels: aorta (AO), superior vena cava (SVC), inferior vena cava (IVC), pulmonary artery (PA), right atrium (RA), right ventricle (RV), left ventricle (LV), and left atrium (LA, including pulmonary veins). Manual segmentation time was recorded.


An AI-assisted algorithm using nnU-Net was trained on 48 HVSMR datasets and integrated into syGlass to immediately segment the heart and great vessels into the same eight labels, generating VR visualizations of 3D models. AI-assisted segmentation time was recorded. Two independent cardiologists qualitatively evaluated these models using a 5-point Likert scale (Table 1). Dice scores and Hausdorff distance metrics provided quantitative comparisons of the segmentations.

Results: Figure 1 compares 3D heart models generated from manual and AI-assisted segmentation in the same patient. Figure 2 shows an AI-assisted 3D heart model visualized in VR. Table 1 presents quantitative and qualitative comparisons of manual and AI-assisted segmentation across five patients with congenital heart disease. AI-assisted segmentation significantly reduced delineation time (in minutes) compared to manual segmentation (19.80±10.73 vs. 112.60±26.93, p = 0.01*). Overall, qualitative evaluation of 3D heart models by two independent cardiologists showed no significant difference between manual and AI-assisted segmentation (1.80±0.27 vs. 1.40±0.42, p = 0.07).

Conclusion: AI-assisted segmentation of CMR images enables rapid, accurate generation of 3D heart models for patients with congenital heart disease, with comparable quality to manual segmentation. Integration with VR platforms offers an efficient and immersive tool for visualizing complex intracardiac anatomy and great vessels, potentially enhancing pre-surgical planning and interdisciplinary communication.

(A) Quantitative comparison of AI-assisted versus manual heart segmentation using Dice score and Hausdorff distance. Higher Dice scores (closer to 1) indicate greater overlap between methods, while lower Hausdorff distances (closer to 0) indicate better boundary agreement. (B) Qualitative assessment of 3D heart models in VR by two independent clinicians using a 5-point Likert scale [1 = Excellent, 2 = Good, 3 = Acceptable, 4 = Moderate, 5 = Poor]. p-values obtained through a non-parametric Wilcoxon signed rank test
(A) - Quantitative

	Labels
	AO	SVC	IVC	LA & PV	LV	RA	PA	RV	Overall
Quantitative:
Dice score (%)	0.89 ± 0.03	0.86 ± 0.08	0.87 ± 0.08	0.70 ± 0.31	0.94 ± 0.03	0.88 ± 0.04	0.83 ± 0.08	0.93 ± 0.03	0.86 ± 0.06
Hausdorff distance (mm)	1.36 ± 0.77	0.92 ± 0.40	1.23 ± 0.70	3.42 ± 3.79	1.07 ± 0.38	2.18 ± 1.60	1.29 ± 1.60	1.09 ± 0.46	1.57 ± 0.68

(B) - Qualitative

	Labels
	AO	SVC	IVC	LA & PV	LV	RA	PA	RV	Overall
Clinicians 1 & 2
Manual	1.80 ± 0.27	1.40 ± 0.22	1.30 ± 0.27	1.40 ± 0.22	1.30 ± 0.27	1.20 ± 0.45	1.30 ± 0.45	1.40 ± 0.22	1.80 ± 0.27
AI-Assisted	1.30 ± 0.27	1.10 ± 0.22	1.40 ± 0.22	1.40 ± 0.42	1.20 ± 0.27	1.10 ± 0.22	1.20 ± 0.45	1.30 ± 0.27	1.40 ± 0.42
p-value	0.03*	0.05*	0.30	0.41	0.32	0.50	0.35	0.30	0.07

* Results are statistically significantFigure 1: An example of 3D heart models generated from manual (left) and AI-assisted (right) segmentation in a 16-year-old male patient with Ebstein's anomaly; AO (Orange), SVC (Royal Blue), IVC (Cyan), LA & PV (Gold), LV (Red), RA (Purple), PA (Light Blue), and RV (Pink)
Figure 2: An example of the AI-assisted 3D heart model from a 10-year-old female patient with Tetralogy of Fallot and left pulmonary artery stenosis visualized in VR; AO (Orange), SVC (Royal Blue), IVC (Cyan), LA & PV (Gold), LV (Red), RA (Purple), PA (Light Blue), and RV (Pink).