Automating cardiac ultrasounds

We use machine learning to enhance cardiac ultrasound scanners used for medical diagnostics.

Medical ultrasound is a non-invasive diagnostic imaging technique that uses high-frequency sound waves to generate images of the inside of the body. Echocardiography is a crucial application of medical ultrasound, and serves as a primary imaging tool for cardiologists in the assessment of cardiac function. This real-time imaging tool can be used to visualise and quantify the overall structure and function of the heart, including cardiac muscle, valves, pathologies, and blood flow. During ultrasound examinations, a significant portion of the physicians’ time is dedicated to making necessary adjustments and measurements. Consequently, cardiologists have had a longstanding need for simplified, largely automated processes.

Collaboration with GE Vingmed Ultrasound

GE Vingmed Ultrasound develops cardiac ultrasound scanners and software tools, and is dedicated to the ongoing development of new and increasingly intelligent software tools. Their objective is to consistently advance and expand the capabilities of the scanners in order to boost productivity and accelerate decision-making processes.

Over the course of several years, NR has developed methods and tools grounded in machine learning that contribute to GE Vingmed Ultrasound’s work and objectives. Our methods are based on deep neural networks and trained to recognise objects and phenomena through annotations of ultrasound images provided by doctors.

Recognition requires intelligent solutions

There are numerous challenges associated with recognition which necessitate smart solutions. Ultrasound images are often noisy, diffuse and demand expertise for accurate annotation. At the same time, annotation is time-consuming and it can be challenging to acquire enough training data for machine learning. In addition, there is a need for relatively fast and compact algorithms that can be implemented on ultrasound scanners and deliver quick responses.

Streamlining the identification process with automation

When using the scanner, a sequence of ultrasound images is typically taken from a specific view of the heart. Streamlining this process for radiologists involves automatic identification and logging of the view. Deep learning models have been developed to address this, with important consideration of the scanner’s capabilities and the trade-off between accuracy, speed and memory usage. Another important focus area has been placed on enabling outlier detection and confidence estimation within the models.

Landmark detection and measurements

In patient care, measurements of the left ventricle (LV) are routinely performed, providing a quick quantitative assessment of heart health. However, this task is challenging due to the high variability arising from the difficulty of precisely detecting relevant structures. To address this, we have developed tools for landmark detection, employing both heatmap-based methods and graph convolutional networks. Our methods show performance levels comparable to manual identification, providing valuable support in heart health assessments.

Automatic prediction of cardiac events

An important task in echocardiography is to identify frames where key cardiac events occur in the ultrasound sequences, such as events related to when the heart is relaxed and when the heart contracts. These are important to cardiologists as many measurements are performed on the ultrasound frames that correspond to these events. We are currently developing deep learning models, like CNN-LSTM and transformer models, that can automatically predict cardiac events from temporal image data.