A Multimodal Deep Learning Framework for Real-Time Cardiac Function Assessment Automatic Estimation of Ejection Fraction and Endocardial Longitudinal Strain from Echocardiography and Electrocardiography

Cardiovascular disease is the leading cause of death worldwide, responsible for roughly one third of all deaths, which underlines the clinical importance of assessing left ventricular (LV) function through ejection fraction (EF) and longitudinal strain. Current echocardiographic software relies on manual intervention and retrospective analysis rather than real-time assessment. We propose an automatic, multimodal deep-learning pipeline that integrates echocardiographic imaging with a single-lead ECG to estimate EF and endocardial longitudinal strain (ELS)---a single-view geometric proxy for global longitudinal strain (GLS). A Siamese CNN extracts per-frame echo features for a BiLSTM that detects end-diastolic (ED) and end-systolic (ES) frames; these are combined with timing from a 1D ResNet ECG model through a Product-of-Experts (PoE) fusion. The detected frames drive a deformable, cycle-consistent segmentation network (Deform U-Net) from which EF and ELS are computed. On an external test set ($N{=}12$), the multimodal approach obtained an EF mean absolute error (MAE) of $5.30%$ (Pearson $r{=}0.801$) against the matched single-plane EchoPAC reference and an ELS MAE of $1.90%$, an improvement over the echo-only baseline (EF MAE $6.28%$, ELS MAE $2.57%$). Separately, the ECG was unsuited to continuous EF regression but discriminated reduced systolic function (EF $leq40%$) with an AUC of 0.845. With a per-cycle compute cost comfortably within a single cardiac cycle, the framework is feasible for real-time deployment.