Generative Multitasking: bridging real-time and gated imaging

Background: Cardiovascular magnetic resonance (CMR) is a crucial but challenging image modality. Two major strategies for CMR imaging are available, gated imaging and real-time imaging, which have long been seen as mutually exclusive^1,2.


Gated imaging resolves cardiac phase and achieves high spatiotemporal resolution, but it averages across heartbeats and struggles with irregular rhythms. Real-time imaging captures single beats and tolerates arrhythmia, but it lowers spatiotemporal resolution; the lack of a shared phase reference also hinders quantitative mapping.


Here we propose a bridge between the two approaches: to augment cardiac phase (timing) with a latent "amplitude" to capture beat-to-beat functional variation, thus defining cardiac cycles as complex harmonics. This approach may enable irregularity-robust beat-to-beat imaging that is furthermore compatible with quantitative mapping. We investigate this complex-harmonic approach within the context of a new reconstruction framework, generative MR multitasking.

Methods: We tested the approach on MR multitasking data from n=8 healthy volunteers: first demonstrating the complex-harmonic concept on free-breathing GRE cine data, then evaluating T1–T2 mapping performance in n=7 volunteers using a free-breathing multitasking sequence (3 simultaneous slices, 1.7x1.7x8 mm resolution)³. A scan-specific conditional variational autoencoder (CVAE) comprises an encoder to learn latent motion states from self-gating data and a decoder which can generate a multidimensional temporal basis at arbitrary timings (replacing multitasking’s low-rank tensor model). We enforce the latent space to be interpretable as motion states by using a frequency cutoff to separate cardiac and respiratory motion and a Hilbert transform to split cardiac motion into phase (timing) and latent amplitude (functional variability).


We evaluated the effect on T1 and T2 mapping by calculating the voxelwise coefficient of variation (CoV) within the septum for each slice. CoVs for conventional vs generative multitasking were compared by ANOVA.

Results: The cine demonstration (Fig. 1) shows that complex-harmonic reconstruction retains the choice of either visualizing time-resolved beat-to-beat variation or post-hoc extraction of a typical (e.g., median-amplitude) cardiac cycle.

Generative multitasking with complex-harmonic reconstruction (Figs. 2 and 3) significantly reduced the voxelwise CoV of septal T1 by a factor of 2.3 (p< 0.001) and T2 by a factor of 2.6 (p< 0.001), reflecting substantially enhanced signal-to-noise ratio.

Conclusion: Adding a latent “amplitude” to complement cardiac phase provides a link between gated and real-time imaging, enabling quantitative CMR while capturing beat-to-beat variability. When integrated into MR multitasking, SNR of T1 and T2 maps improved by >2x. This approach holds significant promise for imaging in the presence of irregular cardiac rhythms and warrants further investigation in an arrhythmia cohort.

A: Conceptual illustration of a cardiac cycle, mapping ECG to cardiac phase; B. cyclic illustration of cardiac phase gating, which captures only the timing of the heart; C. complex-harmonic illustration of using latent “amplitude” to separate cardiac beats with different function; Results of cardiac phase gated reconstruction of an averaged cardiac cycle; E. Complex-harmonic reconstruction of the same data in D allows visualization of both a typical (e.g., median-amplitude) cardiac cycle and of time-resolved beat-to-beat variability (orange arrows point out different blood pool behaviors across beats), bridging gating and real-time imaging.
T1 and T2 maps by cardiac-phase-gated multitasking and complex-harmonic generative multitasking. Both are without the usual wavelet regularization in order to measure the intrinsic impact on SNR.
Voxelwise coefficient of variation (CoV) of T1 and T2 values in the septum across subjects and slices. P-values are from ANOVA analyses.