Joint model-based registration and segmentation strategies for integrated quantitative analysis of cardiac MRI sequences

Cardiac magnetic resonance imaging (MRI) is a state-of-the-art method for non-invasive visualization and functional analysis of the heart and enables not only qualitative but also quantitative analysis. Examples are cine MRI for assessment of cardiac contraction or T1 mapping for quantitative characterization of myocardial tissue. The different images result in a large amount of data and provide complementary information to assess cardiac structure and function, but are typically analyzed separately which makes manual analysis cumbersome. However, combined analysis of such image sequences is complicated by motion between the images, differences in anatomical view and differences in cardiac phase. In this work, we aim to improve (semi-)automated, joint analysis of multi-parametric cardiac MR sequences by developing model-based approaches that make use of application specific prior knowledge about MR acquisition and cardiac shape. This strategy will eventually improve the accuracy of different quantitative cardiac measures and will thereby contribute to a better and more efficient diagnosis. Specific contributions are: (1) model-based motion correction in cardiac T1 and extracellular volume (ECV) mapping, (2) T1 quantification using convolutional neural networks (CNNs), (3) integration of 2D short axis (SA) and long axis (LA) images for 3D left ventricular (LV) segmentation and (4) shape constrained segmentation by predicting myocardial shape and pose parameters. 

A first application is dedicated to motion correction in cardiac T1 and ECV mapping. We propose a new non-rigid registration framework combining a data-driven initialization with a model-based registration approach. Our method uses a model for T1 relaxation to avoid direct registration of images with highly varying contrast. Information from T1 model-fitting is also used to perform registration between native and contrast enhanced T1 maps to obtain a motion free ECV map. We showed the robustness of our approach regarding data-driven initialization, the choice of T1 relaxation model and multi-parametric registration for ECV mapping.

Second, CNNs are used for automatic myocardial T1 quantification. We train networks for both T1 map regression and segmentation. Our experiments demonstrated that the proposed CNN for T1 map prediction, opposite to the conventional pixelwise curve fitting methods, includes spatial information of neighboring pixels. Furthermore, we found that training on full images instead of only minimizing the loss over the myocardium improved myocardial T1 accuracy. For myocardial segmentation, we showed that simultaneous segmentation of all T1-weighted images achieves better result than segmenting one T1-weighted image or the T1 map itself. Additionally, we propose a multi-class segmentation approach for direct regional quantification in which radially separated myocardial segments are predicted and showed that this did not deteriorate global myocardial segmentation.

A third application integrates 2D SA and LA cine images to obtain a 3D segmentation of LV. We developed a 2D to 3D CNN accepting an undetermined number of 2D images. Instead of the standard horizontal and vertical LA views, we use an extended set of 12 equiangularly acquired LA images to better capture the shape of the mitral valve which represents the basal end of LV. Additionally, the CNN uses 3D image coordinates as a weak shape prior. We found that combining SA and LA views improved 3D LV segmentation and that the addition of image coordinates allowed to substantially decrease the required receptive field while maintaining similar segmentation performance. 

In the fourth application, we developed a CNN to directly predict shape and pose parameters of an underlying statistical model of the myocardium in 2D cine SA images. Compared to conventional semantic segmentation, the predicted shape coefficients are directly linked to an oriented landmark-based representation and as such allows straightforward calculation of regional shape properties. Similar to the third application, this application addresses a main limitation of a standard CNN which misses explicit shape constraints. Experiments showed an increase in performance when semantic segmentation was used to guide robust parameter prediction. Furthermore, we introduced two new loss functions to explicitly enforce consistency between parameter and segmentation outputs.

The developed approaches integrate different images and models. This has the benefit of a more efficient use of complementary information in the available data. Furthermore, the described hybrid representations allow to integrate shape, texture and context. Our experiments showed that this complementary information is able to significantly improve registration and segmentation performance.