Contrastive language image pretraining for a cardiac magnetic resonance image embedding with zero-shot capabilities

Why teaching computers to read heart scans matters

Heart MRI scans can reveal subtle signs of disease long before symptoms become obvious, but each scan includes hundreds of images that take specialists a long time to read. This study explores whether an artificial intelligence system can learn to "understand" these complex scans and their written reports so it can help doctors sort cases, recognize disease patterns, and even draft reports, all without being explicitly told what each picture shows.

A new way to pair pictures and words

The researchers built a system called CMR-CLIP that connects cardiac MRI images with the short summary section of the doctor’s report. Instead of treating each image on its own, they treat an entire exam as if it were a short video made of many standard heart views and imaging techniques. At the same time, the system reads the written impression that describes key findings and diagnoses. By training on more than 14,000 past exams and their reports from one health system, the model gradually learns a shared "language" that links visual patterns in the images with phrases in the text, without needing hand-drawn outlines or manual labels for every frame.

Learning to recognize disease with almost no teaching

Once trained, CMR-CLIP was tested on classic tasks that cardiologists face every day, such as spotting weak heart pumping, enlarged chambers, or thickened heart muscle. In a zero-shot setting, the model was only given short, human-readable prompts such as "left ventricle is dilated" and asked to decide whether they applied to a new exam. Even under these conditions, it reached solid accuracy across seven common findings and several major diseases, including hypertrophic cardiomyopathy and cardiac amyloidosis. It clearly beat general-purpose image–text systems, showing that heart MRI has unique patterns that generic models do not capture well.

Getting better with just a few examples

The team also tried few-shot learning, where the model sees only a handful of labeled examples for each condition before being asked to classify new cases. Using tiny training sets as small as one, two, or four exams per category, CMR-CLIP still improved steadily and often matched or surpassed other models that had seen many more examples. For instance, in judging left-sided heart dysfunction, performance rose from fair with one example to very high with 32 examples, and comparable results were seen for chamber enlargement and muscle thickening. This suggests that once the shared image–text space is learned, the system can adapt to new clinical tasks with far less labeled data than usual.

Finding matching scans and drafting reports

Because CMR-CLIP links pictures and words in a common space, it can retrieve the most relevant exam or report when given either a scan or a text query. In tests, it was far more likely than comparison models to rank the true matching report or scan near the top of the results, even when data came from different hospitals or MRI scanners. The authors then used the learned image features in two ways to help with reporting. One method simply finds the most similar past case and reuses its impression. A second method, called CMR-TARGET, feeds the image features into a text generator that writes a new impression sentence by sentence. This generative approach produced summaries that more closely matched real clinical reports according to standard language metrics.

Robust across scanners and imaging details

The researchers examined how design choices affected performance. Including both moving "cine" images and special contrast images that highlight scar tissue, as well as multiple viewing angles of the heart, clearly improved the system’s ability to retrieve and classify cases. Using more frames per exam helped capture subtle changes over the heartbeat, although it also required more computing power. The team also stressed the importance of stability: CMR-CLIP’s internal representation changed little when frames were shuffled or partially removed, indicating it focuses on disease-relevant signals rather than fragile details. Tests across different scanner brands and magnetic strengths showed that accuracy stayed relatively stable, hinting that the model can generalize beyond the center where it was trained.

What this could mean for heart care

To a non-specialist, the main message is that computers can now learn rich, reusable concepts from heart MRI exams and their written interpretations, even without detailed labels on each image. CMR-CLIP acts as a foundation model tailored to cardiac MRI: it can support diagnosis of several important heart diseases, help retrieve similar past cases, and draft structured or free-text reports that doctors can edit. While it does not replace expert readers and still depends on the quality and variety of its training data, this approach could reduce reading time, make results more consistent between hospitals, and eventually help extend advanced MRI-based heart care to more patients.

Citation: Nakashima, M., Qiu, J., Huang, P. et al. Contrastive language image pretraining for a cardiac magnetic resonance image embedding with zero-shot capabilities. Nat Commun 17, 4416 (2026). https://doi.org/10.1038/s41467-026-73022-2

Keywords: cardiac MRI, medical AI, vision-language model, cardiomyopathy, clinical decision support