
==== Front
J Med Ultrasound
J Med Ultrasound
JMU
J Med Ultrasound
Journal of Medical Ultrasound
0929-6441
2212-1552
Wolters Kluwer - Medknow India

JMU-32-215
10.4103/jmu.jmu_52_23
Original Article
Comparison between Handheld Echocardiography and Cardiac Magnetic Resonance for Stroke Volume and Left Ventricular Ejection Fraction Quantification
de Raat Frederique Maria 1*
Bouwmeester Sjoerd 2
Bouwman R. Arthur 13
Houthuizen Patrick 2
1 Department of Electrical Engineering, Technical University of Eindhoven, Eindhoven, The Netherlands
2 Department of Cardiology, Catharina Hospital, Eindhoven, The Netherlands
3 Department of Anesthesiology, Catharina Hospital, Eindhoven, The Netherlands
Address for correspondence: Dr. Frederique Maria de Raat, Technical University of Eindhoven, De Zaale, Eindhoven, The Netherlands. E-mail: f.m.d.raat@tue.nl
Jul-Sep 2024
28 8 2024
32 3 215220
02 5 2023
09 8 2023
11 9 2023
Copyright: © 2024 Journal of Medical Ultrasound
2024
https://creativecommons.org/licenses/by-nc-sa/4.0/ This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.
Background:

Reliable quantification of stroke volume (SV) and left ventricular ejection fraction (LVEF) is essential for point-of-care assessment in hemodynamically compromised patients. Handheld echocardiography (HHE) equipment has entered the market a few years ago and is now available for clinical use. However, the performance of HHE for SV and LVEF quantification in comparison to cardiac magnetic resonance (CMR) imaging as golden standard is yet unknown.

Methods:

Twenty volunteers were scanned with HHE, standard echocardiography (SE), and CMR. LVEF and SV were measured with each modality, and their accuracy and precision were evaluated.

Results:

Bias and limits of agreement (LOA) between HHE and CMR were −0.21% (−2.89: 2.48) and 11.24% (−15.79: 15.59) for LVEF and 29.85 ml (22.13: 37.57) and 32.34 ml (−15.01: 44.86) for SV, respectively. Bias and LOA between SE and CMR were −0.60% (−3.74:2.55) and 13.16% (−18.85:18.26) for LVEF and 32.08 ml (24.61:39.54) and 31.34 ml (−11.29:43.37) for SV, respectively.

Conclusion:

HHE versus CMR showed comparable accuracy and precision compared to SE versus CMR.

Handheld echocardiography
hemodynamic monitoring
left ventricle ejection fraction
point-of-care ultrasound
stroke volume
==== Body
pmcINTRODUCTION

Reliable point-of-care monitoring of the hemodynamic status is paramount to guide hemodynamic management for critically ill patients. To this end, echocardiography is increasingly used in clinical care. With the use of ultrasound, the physician is able to noninvasively quantify cardiac functional parameters such as left ventricular ejection fraction (LVEF) and stroke volume (SV),[123] which are frequently used to guide hemodynamic management of critically ill patients, such as fluid resuscitation, vasopressor, and inotropic therapy.[4567] However, standard ultrasound devices are not suitable for point-of-care echocardiography in critical care due to their limited availability at the patient’s bedside.

In recent years, the availability of handheld ultrasound devices in hospitals has increased, particularly for handheld echocardiography (HHE). Several studies have investigated the diagnostic accuracy of HHE for eyeballing LVEF, evaluating pericardial effusion or valve abnormalities.[8910] However, the literature lacks a comprehensive analysis of the validity of HHE for objective LVEF and SV quantification. In this observational study, the validity of a clinically certified HHE device (Lumify, Philips) is evaluated for quantification of LVEF and SV in healthy volunteers with cardiac magnetic resonance (CMR) as a golden reference technique. It is hypothesized that the agreement between HHE and CMR for SV and LVEF quantification is comparable to the agreement between standard echocardiography (SE) and CMR. With this study, we intended to contribute toward more clinical insights into the baseline performance of HHE for future point-of-care applicability.

PATIENTS AND METHODS

Study design

The present study is a prospective, nonrandomized, observational, single-center study. This study (R20.055) was approved by the National Review Board of the Medical Ethical Centre of Utrecht, The Netherlands, on March 5, 2021, and the Institutional Board. The study adhered to the principles of the Declaration of Helsinki (64th WMA General Assembly, Fortaleza, Brazil, October 2013) and was in accordance with the Medical Research Involving Human Subjects Act (WMO). Informed consent was obtained from all volunteers in the study.

Population

The study population consisted of healthy volunteers who did not have any known significant medical conditions or diseases. This study was conducted at the Radiology and Cardiology Department of the Catharina Hospital (Eindhoven, The Netherlands). Healthy volunteers above 18 years were approached by the main investigator and were included if they agreed to participate. In case during the acquisition, it became clear to the investigator that the volunteer was suffering from arrhythmias, valvular diseases, or pulmonary hypertension, the volunteer was excluded from the study. Demographic characteristics, including age, gender, and BMI, were collected.

Data acquisition

Data acquisition was done sequentially in the following order: standard echo (SE), handheld echo (HHE), and finally, CMR imaging. For this study purpose, ultrasound and CMR acquisition times were approximately 5 and 20 min, respectively.

Echocardiographic imaging and analysis

All acquisitions with an SE and HHE device were conducted by an EACVI-certified cardiologist (SB) blinded to the postprocessing results. The volunteers were in the left lateral position at the time of examination. During image acquisition, volunteers were asked to perform an expiratory hold maneuver. Gain, focus, and depth settings were adjusted to maximize endocardial visualization.

For the SE dataset, two-dimensional, single-beat images of the apical four- (A4CH) and two-chamber (A2CH) views were acquired with an EPIQ ultrasound system equipped with an X5-1 phased array transducer (1–5 MHz, Philips Ultrasound, Inc., Bothell, WA). The harmonic function was used to optimize image resolution. Storage and looping of cardiac cycles were ECG triggered.

For the HHE dataset, two-dimensional A4CH and A2CH views were acquired with the Lumify S4-1 phased array transducer (1–4 MHz, Philips Ultrasound, Inc., Bothell, WA). The HHE device did not allow ECG-triggered storage and looping of a single cardiac cycle. Instead, 8-s recordings of both the A2CH and A4CH views were acquired. From this dataset, only the second heartbeat was used for analysis.

Both SE and HHE imaging were performed in triplicate without changing the probe position. Acquisitions were saved as Digital Imaging and Communications in Medicine files and exported to the hospital server for offline postprocessing. Offline postprocessing was done by an independent blinded analyst trained in performing echocardiographic measurements.

Auto Strain (Auto Strain TOMTEC - ARENA lot 50, TOMTEC Imaging Systems GmbH, Germany) was used for offline postprocessing quantification of the SE and HHE images. This tool automatically identifies end-diastolic and end-systolic frames using the ECG signal. As mentioned before, the SE data included an ECG signal, but the HHE data did not. Therefore, end-diastolic and end-systolic frames from the HHE images were manually based on the A4CH and A2CH view and by using the included motion-mode (M-mode) tool. As such, using this M-mode tracing through the mitral annulus, end-diastole was defined as one frame before mitral valve closure. End-systole was defined as one frame before mitral valve opening or when end-systolic volume was deemed smallest by the operator [Figure 1]. Next, based on the end-diastolic and end-systolic frames, the software automatically traced the endocardial border and calculated the left ventricle end-diastolic volume (EDV), left ventricle end-systolic volume (ESV), SV, and LVEF according to the Simpson’s biplane method [Figure 2]. No manual adjustments were made for the endocardial border tracings.

Figure 1 An A4CH view from the handheld echocardiography dataset (a) with an M-mode tracing through the mitral annulus with the red lines indicating the start (solid) and end (dashed) of one cardiac cycle (b)

Figure 2 The delineation of the left ventricle (green line) in an A4CH view throughout the heart cycle: (a) Handheld echocardiography (HHE) image in diastole, (b) HHE image in systole, (c) standard echocardiography (SE) image in diastole, (d) SE image in systole

Cardiac magnetic resonance imaging and analysis

Study participants were imaged on a Philips 1.5-Tesla scanner (Philips Medical Systems, Best, The Netherlands). CMR images were acquired during repeated end-expiratory breath holds. Cine images were acquired using a retrospectively gated balanced steady-state free precession sequence with 25–30 cardiac phases per cardiac cycle and a slice thickness of 8 mm without inter-slice gap. Sequences include left ventricle (LV) four-chamber and LV two-chamber cine imaging, on which a multi-slice cine short axis was planned to include the entire LV. EDV, ESV, SV, and LVEF were measured using automated commercially available software (CVI42 version 5.13, Circle Cardiovascular Imaging, Calgary, Canada). All slices with at least 50% of the LV cavity circumference surrounded by myocardial tissue were included for LV analysis [Figure 3]. Papillary muscles were included in the blood volumes.

Figure 3 Multi-phase multi-slice steady-state free precession short-axis cine stack showing endocardial (red) and epicardial (green) contours at end-diastole in study subject MPUS001

Statistical analysis

Statistical analysis and data visualization were performed using IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp. and MATLAB version: 9.11.0 (R2021b), Natick, Massachusetts: The MathWorks Inc.; 2021. Data are shown as mean ± standard deviation, and the assumption of normality was tested using the Shapiro–Wilk normality test. The statistical analysis compared EDV, ESV, SV, and LVEF values of HHE versus CMR and SE versus CMR. In this article, validity is based on the assessment of correlation, accuracy, and precision. Correlation calculations were performed using linear regression with Pearson correlation coefficients for normally distributed data and using Spearman correlation for nonnormally distributed data. Correlation coefficients were considered poor (<0.4), moderate (0.4–0.7), strong (0.7–0.9), or very strong (>0.9).[11] The Bland–Altman method is used to evaluate the agreement between HHE versus CMR and SE versus CMR. As a result, this method assesses interchangeability between the experimental techniques (either HHE or SE) and the reference technique (CMR), instead of validating the experimental technique against a perfect reference. With the Bland–Altman method, the 95% confidence interval (CI) bias is determined as a measure of accuracy. As a measure of precision, the 95% CI of the limits of agreement (LOA) was used. The presence of proportional bias in the Bland–Altman plot was checked with regression analysis. For SV, a bias of up to 10% with respect to the mean of the reference method and a mean error of up to 30% with respect to the mean of the reference method were considered clinically acceptable. For LVEF, the clinically acceptable bias was set to 10% and the clinically acceptable mean error to 15%.[12] To verify the significance of the biases, a paired sample t-test was performed or a Mann–Whitney U, based on normality. Values of P < 0.05 were considered statistically significant.

RESULTS

The baseline characteristics of the 20 volunteers are presented in Table 1. Overall, CMR EDV ranged from 144 to 237 ml, ESV from 53 to 98 ml, LVEF from 53% to 66%, and SV from 79 to 151 ml. All endocardial borders of both SE and HHE were categorized as correct by the Auto Strain tool. The mean value of either the triple HHE or SE measurements per study subject was used in the analysis. Datapoints of each individual parameter were normally distributed.

Table 1 Baseline demographic characteristics

Variable	mean± SD	
Total number of participants (n)	20	
Male, n (%)	20	
Age (years)	30.1±6.9	
Body length (cm)	182.4±6.1	
Body weight (kg)	76.5±7.0	
BMI (kg/m2)	23.0±2.3	
BSA (m2)	2.0±0.1	
CMR EDV	191.9±26.4	
CMR ESV	74.2±11.9	
CMR LVEF	61.0±4.1	
Values are presented as mean±SD. BMI: Body mass index, BSA: Body surface area, CMR: Cardiac magnetic resonance; EDV: Left ventricle end-diastolic volume, ESV: Left ventricle end-systolic volume, LVEF: Left ventricle ejection fraction, SD: standard deviation

Technique comparison of handheld echocardiography versus cardiac magnetic resonance

The 95% CI of the bias and LOA were outside the clinically acceptable boundaries. The 95% CI for the bias and LOA were overall all lower than the comparison between SE and CMR [Tables 2, 3 and Figure 4]. The correlation between the HHE and CMR datasets for SV was 0.53 (0.11:0.79, P < 0.001), for EDV 0.82 (0.58:0.92, P < 0.001), for ESV 0.73 (0.43:0.89, P < 0.001), and for LVEF 0.00 (−0.45:0.44, P < 0.001) [Table 2 and Figure 4].

Table 2 Technique comparison of handheld echocardiography versus cardiac magnetic resonance

	n	Averaged HHE (I)	Averaged CMR (J)	Corr	95% CI of corr	Bias (J–I)	95% CI of Bias	LOA	95% CI of LOA	
LVEF (%)	20	61.35±3.85	61.15±4.23	0.00	(−0.45:0.44)	−0.21*	(−2.89:2.48)	11.24	(−15.79:15.59)	
EDV (mL)	20	144.66±18.73	193.75±27.05	0.82	(0.58:0.92)	49.09*	(41.60:56.57)	31.36	(5.59:43.50)	
ESV (mL)	20	55.86±9.02	75.10±12.17	0.73	(0.43:0.89)	19.24*	(15.37:23.11)	16.19	(−3.23:22.47)	
SV (mL)	20	88.80±13.00	118.65±19.14	0.53	(0.11:0.79)	29.85*	(22.13:37.57)	32.34	(−15.01:44.86)	
*P<0.05. Values are presented as mean±SD. Corr: Correlation coefficient, CI: Confidence interval, CMR: Cardiac magnetic resonance, EDV: Left ventricle end-diastolic volume, ESV: Left ventricle end-systolic volume, LVEF: Left ventricle ejection fraction, LOA: Limits of agreement, SV: Stroke volume, HHE: Handheld echocardiography, SD: Standard deviation

Table 3 Technique comparison of standard echocardiography versus cardiac magnetic resonance

	n	Averaged SE (I)	Averaged CMR (J)	Corr	95% CI of corr	Bias (J-I)	95% CI of bias	LOA	95% CI of LOA	
LVEF (%)	36	61.75±5.05	61.15±4.23	0.29	(−0.45:0.44)	−0.60*	(−3.74:2.55)	13.16	(−18.85:18.26)	
EDV (mL)	36	140.19±20.42	193.75±27.05	0.78	(0.51:0.91)	53.56*	(45.63:61.50)	33.23	(7.45:46.11)	
ESV (mL)	36	53.56±10.12	75.10±12.17	0.51	(0.09:0.78)	22.53*	(16.31:26.76)	21.89	(−8.83:30.36)	
SV (mL)	36	86.57±14.60	118.65±19.14	0.58	(0.19:0.81)	32.08*	(24.61:39.54)	31.34	(−11.29:43.37)	
*P<0.05. Values are presented as mean±SD. Corr: Correlation coefficient, CI: Confidence interval, CMR: Cardiac magnetic resonance, EDV: Left ventricle end-diastolic volume, ESV: Left ventricle end-systolic volume, LVEF: Left ventricle ejection fraction, LOA: Limits of agreement, SV: Stroke volume, SD: Standard deviation, SE: Standard echocardiography

Figure 4 Bland–Altman plots of the handheld echocardiography and cardiac magnetic resonance data with the limits of agreement (red dashed lines), bias (black solid line), and the regression (blue solid line): (a) End-diastolic volume, (b) end-systolic volume, (c) stroke volume, (d) left ventricle ejection fraction. EDV = End-diastolic volume; ESV = End-systolic volume; LVEF = Left ventricle ejection fraction; SV = Stroke volume; R = Correlation coefficient

Technique comparison of standard echocardiography versus cardiac magnetic resonance

The 95% CI of the bias and LOA were outside the clinically acceptable boundaries [Table 3 and Figure 5]. The correlation between the SE and CMR datasets quantified for SV was 0.58 (0.19:0.81, P < 0.0001), for EDV 0.78 (0.51:0.91, P < 0.001), for ESV 0.51 (0.09:0.78, P < 0.001), and for LVEF 0.29 (−0.45:0.44, P < 0.001) [Table 3 and Figure 5].

Figure 5 Bland–Altman plots of the standard echocardiography and cardiac magnetic resonance data with the limits of agreement (red dashed lines), bias (black solid line), and the regression (blue solid line): (a) End-diastolic volume, (b) end-systolic volume, (c) stroke volume, (d) left ventricle ejection fraction. EDV = End-diastolic volume; ESV = end-systolic volume; LVEF = left ventricle ejection fraction; SV = stroke volume; R = correlation coefficient

DISCUSSION

This study investigated the correlation, accuracy, and precision between an HHE device versus CMR in comparison to SE versus CMR. HHE versus CMR showed comparable correlation, accuracy, and precision compared to SE versus CMR. Furthermore, LVEF quantification with HHE is interchangeable with CMR based on accuracy and precision. However, HHE is not interchangeable with CMR for absolute volumetric measurements.

Several studies have extensively described the diagnostic accuracy of HHE for eyeballing LVEF or for the evaluation of pericardial effusion or valve abnormalities.[8910] However, the literature lacks a comprehensive analysis of the validity of HHE for objective LVEF and SV quantification in comparison to CMR. According to our results, HHE is not interchangeable with CMR for quantifying SV, EDV, and ESV. This is likely caused by the inferiority of ultrasound as a technique in general compared to magnetic resonance imaging since SE is neither interchangeable with CMR.[1314]

In line with earlier reports, our study showed no interchangeability between two-dimensional ultrasound and CMR for absolute volumetric measurements.[15] However, the study of Hoffman et al. reported biases of −0.8 ± 10.6%, −72.3 ± 39.8 ml, and −35.7 ± 32.5 ml for LVEF, EDV, and ESV, respectively, which are higher compared to our results. Hoffman et al. also reported wider limits of agreement (LOA) for various parameters: −21.6%:20.0% for LVEF, −5.7ml:150.5 ml for EDV, and −21.6ml:12.4 ml for ESV. This could be explained by technological improvements in the last couple of years of both the ultrasound techniques (SE and HHE) and the ultrasound quantification software (Auto Strain). Furthermore, Hoffman et al. reported a moderate correlation for LVEF between two-dimensional echocardiography and CMR, while our study shows a poor correlation for LVEF. This can be due to the relatively small study population and small LVEF range captured in this study of 53% till 66%.[16] However, LVEF quantification with HHE and SE compared to CMR does show interchangeability based on accuracy and precision.

This study provides a starting point for future research evaluating the clinical applicability of HHE devices for point-of-care assessment in a critical care setting. Although absolute volumes are underestimated with HHE compared to CMR, the temporal change in SV can be more interesting, as dynamic parameters in response to hemodynamic challenges have shown to be more valuable in fluid management and hemodynamic support in critically ill patients. Therefore, future research should focus on the evaluation of the trending ability of hemodynamic parameters derived from handheld devices. Future work should also focus on incorporating an operator-independent quantification tool in the HHE devices. The absence of an ECG tracing, as seen in the HHE device used in this study, could potentially impact the precision of automated quantification, since manual selection of end-diastolic and end-systolic frames is still required.[17]

With this study, it was aimed to provide baseline knowledge of HHE for future point-of-care applications by evaluating the clinical applicability of HHE devices in healthy study subjects. This study has several limitations. First, our study population consisted of only healthy volunteers which were in stable sinus rhythm had a low BMI and good acoustic windows. For clinical use, it would be valuable to evaluate the performance of HHE in a study population which represents the average population better. Image quality may be less good in obese patients due to suboptimal echo windows. Second, expert sonographers conducted all examinations, giving rise to a high reproducibility. Reproducibility may decrease with less experienced sonographers. Third, only optimally positioned test subjects were included. Therefore, the results from this study are not generalizable to bedridden patients. Fourth, automated quantification of HHE-derived images is M-mode dependent, which is yet a limitation for point-of-care assessment.

CONCLUSION

HHE versus CMR showed comparable accuracy and precision compared to SE versus CMR. Additional studies investigating the application of HHE at different hemodynamic conditions are needed to qualify HHE as a potential point-of-care hemodynamic monitoring device.

Financial support and sponsorship

Nil.

Conflict of interest

The authors F.M. de Raat, S. Bouwmeester, R. Bouwman, and P. Houthuizen have declared to have no conflict of interest. R. Bouwman is a clinical consultant for Philips Research in Eindhoven, The Netherlands, for which he receives consultant honoraria.

Acknowledgments

The authors are grateful to the ultrasound technicians of the Cardiology Department for their assistance and support in the acquisition of ultrasound data during this study.
==== Refs
REFERENCES

1 Kimura BJ Amundson SA Willis CL Gilpin EA DeMaria AN Usefulness of a hand-held ultrasound device for bedside examination of left ventricular function Am J Cardiol 2002 90 1038 9 12398984
2 Mark DG Hayden GE Ky B Paszczuk A Pugh M Matthews S Hand-carried echocardiography for assessment of left ventricular filling and ejection fraction in the surgical intensive care unit J Crit Care 2009 24 470.e1 7
3 Chamsi-Pasha MA Sengupta PP Zoghbi WA Handheld echocardiography: Current state and future perspectives Circulation 2017 136 2178 88 29180495
4 Bruss ZS Raja A Physiology, stroke volume StatPearls Treasure Island (FL) StatPearls Publishing 2021
5 Kosaraju A Goyal A Grigorova Y Makaryus AN Left ventricular ejection fraction StatPearls Treasure Island (FL) StatPearls Publishing 2022
6 Hajouli S Ludhwani D Heart failure and ejection fraction StatPearls Treasure Island (FL) StatPearls Publishing 2021
7 Paonessa JR Brennan T Pimentel M Steinhaus D Feng M Celi LA Hyperdynamic left ventricular ejection fraction in the intensive care unit Crit Care 2015 19 288 26250903
8 Thomas F Flint N Setareh-Shenas S Rader F Kobal SL Siegel RJ Accuracy and efficacy of hand-held echocardiography in diagnosing valve disease: A systematic review Am J Med 2018 131 1155 60 29856962
9 Ali S Bushari T Validation of the accuracy of handheld echocardiography for diagnosis of congenital heart disease Ann Pediatr Cardiol 2018 11 250 4 30271013
10 Giusca S Jurcut R Ticulescu R Dumitru D Vladaia A Savu O Accuracy of handheld echocardiography for bedside diagnostic evaluation in a tertiary cardiology center: Comparison with standard echocardiography Echocardiography 2011 28 136 41 21073515
11 Schober P Boer C Schwarte LA Correlation coefficients: Appropriate use and interpretation Anesth Analg 2018 126 1763 8 29481436
12 Montenij LJ Buhre WF Jansen JR Kruitwagen CL de Waal EE Methodology of method comparison studies evaluating the validity of cardiac output monitors: A stepwise approach and checklist Br J Anaesth 2016 116 750 8 27199309
13 Mor-Avi V Yodwut C Jenkins C Kühl H Nesser HJ Marwick TH Real-time 3D echocardiographic quantification of left atrial volume: Multicenter study for validation with CMR JACC Cardiovasc Imaging 2012 5 769 77 22897989
14 Badano LP Boccalini F Muraru D Bianco LD Peluso D Bellu R Current clinical applications of transthoracic three-dimensional echocardiography J Cardiovasc Ultrasound 2012 20 1 22 22509433
15 Hoffmann R von Bardeleben S ten Cate F Borges AC Kasprzak J Firschke C Assessment of systolic left ventricular function: A multi-Centre comparison of cineventriculography, cardiac magnetic resonance imaging, unenhanced and contrast-enhanced echocardiography Eur Heart J 2005 26 607 16 15618026
16 Bland JM Altman DG Correlation in restricted ranges of data BMJ 2011 342 d556 21398359
17 Mada RO Lysyansky P Daraban AM Duchenne J Voigt JU How to define end-diastole and end-systole?Impact of timing on strain measurements JACC Cardiovasc Imaging 2015 8 148 57 25577447
