
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39227373
71530
10.1038/s41598-024-71530-z
Article
A multi-task deep learning approach for real-time view classification and quality assessment of echocardiographic images
Li Xinyu 1
Zhang Hongmei 23
Yue Jing 1
Yin Lixue 23
Li Wenhua 23
Ding Geqi 23
Peng Bo 1
Xie Shenghua xieshenghua@med.uestc.edu.cn

23
1 https://ror.org/03h17x602 grid.437806.e 0000 0004 0644 5828 School of Computer Science and Software Engineering, Southwest Petroleum University, Chengdu, 610500 China
2 grid.54549.39 0000 0004 0369 4060 Ultrasound in Cardiac Electrophysiology and Biomechanics Key Laboratory of Sichuan Province, Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, 32# W. Sec 2, 1st Ring Rd., Chengdu, 610072 China
3 grid.54549.39 0000 0004 0369 4060 Department of Cardiovascular Ultrasound & Noninvasive Cardiology, Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, 32# W. Sec 2, 1st Ring Rd., Chengdu, 610072 China
3 9 2024
3 9 2024
2024
14 2048412 7 2024
28 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
High-quality standard views in two-dimensional echocardiography are essential for accurate cardiovascular disease diagnosis and treatment decisions. However, the quality of echocardiographic images is highly dependent on the practitioner’s experience. Ensuring timely quality control of echocardiographic images in the clinical setting remains a significant challenge. In this study, we aimed to propose new quality assessment criteria and develop a multi-task deep learning model for real-time multi-view classification and image quality assessment (six standard views and “others”). A total of 170,311 echocardiographic images collected between 2015 and 2022 were utilized to develop and evaluate the model. On the test set, the model achieved an overall classification accuracy of 97.8% (95%CI 97.7–98.0) and a mean absolute error of 6.54 (95%CI 6.43–6.66). A single-frame inference time of 2.8 ms was achieved, meeting real-time requirements. We also analyzed pre-stored images from three distinct groups of echocardiographers (junior, senior, and expert) to evaluate the clinical feasibility of the model. Our multi-task model can provide objective, reproducible, and clinically significant view quality assessment results for echocardiographic images, potentially optimizing the clinical image acquisition process and improving AI-assisted diagnosis accuracy.

Subject terms

Biomedical engineering
Image processing
Ultrasound
Medical imaging
the Sichuan Science and Technology Project2023YFQ0006 issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

Two-dimensional transthoracic echocardiography is widely used as a non-invasive, radiation-free, low-cost, and real-time imaging modality to assess cardiac function and cardiovascular disease diagnosis1. Cardiac images acquired based on specific probe positions and angles are referred to as standard views, which combined provide comprehensive information on cardiac structure and function2. High-quality standard views are the basis for reliable cardiac parameter measurements and accurate diagnosis3,4. However, compared with other imaging modalities, the acquisition process for clinical echocardiographic images is less automated, with echocardiographers manually adjusting probe positions and parameters, subjectively recognizing individual standard views, and selecting high-quality image frames5. The entire process is cumbersome, time-consuming, highly dependent on the echocardiographer's experience and maneuvers, and prone to inter- and intra-observer variability6–8. When the acquired views are of poor quality or key views are missing, the interpretation by human experts or artificial intelligence (AI) models is compromised. Current automated diagnostic frameworks for cardiac diseases do not yet incorporate image quality control (QC) into the analysis process, necessitating manual pre-filtering of low-quality images and thereby limiting clinical applicability9. Ideally, real-time QC should be implemented during patient image acquisition to ensure that the optimal images are captured within the same examination. Therefore, performing automatic view recognition and quality assessment during image acquisition or before downstream image analysis tasks is crucial. The former directly generates a high-quality image base, while the latter screens the optimal images for subsequent analysis.

Recently, deep learning has been widely used in echocardiographic image analysis, enabling automated clinical workflows for view classification, cardiac structure extraction, cardiac function quantification, and cardiac disease diagnosis10–13. View classification is the first step in echocardiographic analysis. Previous studies have proposed view classification models based on convolutional neural networks (CNNs), such as VGGNet and ResNet, achieving good recognition performance14–17. On-going research on image quality assessment mainly targets natural images and focuses on the various distortions of natural images during acquisition, compression, storage, and transmission18,19. Noise and artifacts are commonly found in ultrasound images due to the coherent interference of scattered waves20. However, unlike natural images, noisy ultrasound images are not always of low quality. The quality assessment of ultrasound images must also consider clinical practice requirements, emphasizing the visibility and integrity of specific anatomical structures21. Several studies have implemented quality assessment based on echocardiographic images, which are broadly categorized into three forms: categorical confidence, quality level classification, and quality score regression. Huang et al.22 and Zhang et al.23 used classification confidence for standard view recognition as the image quality score. The view classification confidence level represents the model's confidence in the predicted results but does not directly represent the image quality level from a clinical practice perspective. Zamzmi et al.24 proposed a MobileNetV2-s-based encoder-decoder network to recognize four standard views and classify them into two quality levels (good or poor). However, providing a continuous numerical score feedback to the operator during the image-acquisition process is more helpful than providing discrete quality-level feedback. Abdi et al.25 categorized the quality of end-systolic apical 4-chamber (A4C) view frames into six levels (0 to 5) based on the visibility and clarity of anatomical structures and proposed a nine-layer CNN for score regression prediction. Some studies were conducted based on echocardiography videos. Luong et al.26 set four quality score levels (0.25, 0.5, 0.75, and 1.0) based on the visibility of anatomical structures, and combined the DenseNet and LSTM networks to simultaneously achieve view recognition and quality assessment for nine echocardiographic videos. Labs et al.27 combined four convolutional layers and an LSTM network to assign scores ranging from zero to ten to four quality attributes in A4C and PLAX videos, respectively. These studies provide effective standard view quality assessment methods; however, several limitations remain. First, the current image quality assessment criteria are limited in scope and inadequate for clinically meaningful assessment of complex and diverse standard views. Second, most studies focus on a single task, with little research on a comprehensive multi-view classification and quality assessment pipeline. Third, traditional CNN architectures progressively abstract image information with deeper network layers, potentially losing spatial details at lower layers28.

In this study, we proposed four image quality attributes based on clinical practice needs and established evaluation criteria for six standard view categories accordingly. We developed a multi-task model that integrates view classification and image quality assessment into a unified framework. By sharing feature representations, multi-task learning enables the simultaneous learning of multiple related tasks in a single training step, effectively facilitating the exchange of information between tasks, thereby improving the overall performance and efficiency of the model 29,30. Furthermore, we introduced the Feature Pyramid Network (FPN)31 into echocardiographic image quality assessment for the first time to achieve the fusion and utilization of multi-scale features.

Methods

An overview of the study is provided in Fig. 1. A multi-task deep learning model was trained on a dataset consisting of 170,311 echocardiographic images to automatically generate view categories and quality scores for clinical quality control workflows. This study conformed to the principles outlined in the Declaration of Helsinki and was approved by the Ethics Board of our institution (No. 2023–407).Fig. 1 Overview of the study design. (a) Seven types of echocardiographic standard views were collected, including apical 4-chamber (A4C), parasternal view of the pulmonary artery (PSPA), parasternal long axis (PLAX), parasternal short axis at the mitral valve level (PSAX-MV), parasternal short axis at the papillary muscle level (PSAX-PM), parasternal short axis at the apical level (PSAX-AP), and other views(Others). (b) Four quality attributes were summarized, including overall contour, key anatomical structural details, standard view display, and image display parameter adjustments. (c) Model development workflow for data collection, data labeling, data preprocessing, and model training. (d) Two clinical application workflows: the left side shows real-time quality control during image acquisition, and the right side shows pre-stored image screening prior to AI-assisted diagnosis. Artwork attribution in (c) and (d): www.flaticon.com.

Data

The study is a retrospective study. A large number of echocardiographic studies were randomly extracted from the picture archiving and communication system (PACS) of the Sichuan Provincial People's Hospital between 2015 and 2022 to establish the experimental dataset, with all subjects aged 18 and above. Images showing severe cardiac malformations that prevented recognition of anatomical structures were excluded. The dataset consists of 107,311 echocardiographic images and includes six standard views commonly used in clinical practice: A4C view, parasternal view of the pulmonary artery (PSPA), parasternal long axis (PLAX), parasternal short axis at the mitral valve level (PSAX-MV), parasternal short axis at the papillary muscle level (PSAX-PM), and parasternal short axis at the apical level (PSAX-AP). Except for these six views, all other views are classified as “others”. For standard views with unevenly distributed quality levels, we performed undersampling to balance the data distribution. All images were acquired using ultrasound machines from different manufacturers such as Philips, GE, Siemens, and Mindray. The dataset was randomly divided into training (70%), validation (10%), and test (20%) sets through stratified sampling (Table 1). The distribution of quality scores for three subsets can be found as Supplementary Figure S1 online. Table 1 Distribution of experimental dataset.

View	Training	Validation	Test	Total	
A4C	14,109	2016	4,032	20,157	
PSPA	12,317	1760	3,520	17,597	
PLAX	19,493	2785	5,570	27,848	
PSAX-MV	7,851	1122	2,244	11,217	
PSAX-PM	11,226	1604	3,208	16,038	
PSAX-AP	3,885	556	1,111	5,552	
Others	50,330	7191	14,381	71,902	
Total	119,211	17,034	34,066	170,311	

Quality scoring method

We established percentage scoring criteria for different standard views based on four attributes: overall contour, key anatomical structural details, standard view display (see Supplementary Fig. S2 online for an example), and image display parameter adjustments. Each attribute contributed to the score in a ratio of 3:4:2:1. Table 2 presents the scoring criteria for the PLAX view. Two accredited echocardiographers with at least five years of experience individually annotated all images in the dataset. The average of their annotations was used as the final expert score label. A third experienced cardiology expert, with over ten years of experience, conducted a review assessment of images with score differences of > 10. The “others” view was set to zero points for training purposes. Table 2 PLAX view scoring definition.

Attributes	Optimum quality	Good quality	Average quality	Low quality	
Overall contour

(The mitral valve, aortic valve, and peri-valvular contours; the right ventricle anterior wall and interventricular septum, left ventricular inferolateral wall, left atrium, and ascending aorta contours)

	Displayed completely	Very few contours are missing, not affecting the judgment of the cardiac chambers or valve contours	Some contours are missing, slightly affecting the judgment of cardiac chambers and valve contours	Most contours are missing, affecting the judgment of cardiac chambers and valve contours	
Score	[27, 30]	[24,27)	[18,24)	[0,18)	
Key anatomical structural details

(The mitral valve, aortic valve, and peri-valvular tissues; the endocardium of the right ventricle and septum, the basal and mid inferolateral wall of the left ventricle; the left atrial anterior and posterior walls)

	Displayed clearly	Very Few structures appear blurry, but not affecting the measurement of the anterior and posterior diameters of the left ventricle and left atrium	Some structures appear blurry, slightly affecting the measurement of the anterior and posterior diameters of the left ventricle and left atrium	Most structures appear unclear, affecting the measurement of anterior and posterior diameters of the left ventricle and left atrium	
Score	[36, 40]	[32,36)	[24,32)	[0,24)	
Standard view display

(deformations such as compression, tension, and offset)

	Standard, no visible deformation	Slight deformation, but not affecting structural and functional observations	Slight deformation, slightly affecting structural and functional observations	High deformation, affecting structural and functional observations	
Score	[18, 20]	[16,18)	[12,16)	[0,12)	
Image display parameter adjustments

(parameters such as gain, sector angle, and depth)

	Appropriate	Some parameter adjustments are inappropriate, but not affecting structural and functional observations	Some parameter adjustments are inappropriate, slightly affecting structural and functional observations	Most parameter adjustments are poor, affecting structural and functional observations	
Score	[9, 10]	[8,9)	[6,8)	[0,6)	
Sum	[90,100]	[80,90)	[60,80)	[0,60)	

Model development

The model architecture is shown in Fig. 2 and mainly consists of a backbone network, neck network, and two branch modules for view classification and quality assessment. The backbone network is used to learn and extract the multi-scale image features. We choose the output feature maps S2,S3,S4,S5 (with output sizes of 1/4, 1/8, 1/16, and 1/32 of the original resolution, respectively) from the last four stages of the backbone network as the input for the neck network. To obtain the best backbone network, we compared six different deep CNN architectures, namely, MobileNetV332, DenseNet12133, VGG1634, EfficientNet35, ResNet5036, and ConvNeXt37, and selected VGG16.Fig. 2 Proposed multi-task model architecture. The model consists of a backbone network, a neck network, a view classification branch, and a quality assessment branch. A single-frame image is input into the backbone network to extract features. Subsequently, the neck network enhances and fuses these multi-scale features. The highest-level feature from the neck network is fed into the view classification branch to get a view class, while the fused multi-scale feature is input into the quality assessment branch to generate a quality score.

The neck network serves as an intermediate feature layer for further processing and fusing the features extracted from the backbone network for the two subsequent tasks. The highest-layer feature,S5, is a more discriminative high-level semantic feature that reflects the network's understanding of the overall context of the image and is suitable for classification tasks. Lv et al.38 proposed that conducting the self-attention mechanism on high-level features with richer semantic concepts could capture the connections between conceptual entities in an image. Therefore, to further enhance the expressiveness of the features, we input S5 into a Vision Transformer Block (VTB)39 that unites a multi-head attention layer and a feedforward layer to facilitate intra-scale feature interaction. The feature map after this step is denoted as S5′, which is applied to view classification. Subsequently, FPN is applied to fuse the features of the four scales S5′,S4,S3,S2 in a layer-by-layer manner from the top down for cross-scale feature interactions. We denote the set of feature maps output from the FPN as P5,P4,P3,P2, and each feature map has strong semantic information. Next, we fuse all scale feature maps using an Adaptive Feature Fusion Block (AFFB) to better model image quality perception. As shown in Fig. 3, the AFFB module first upsamples the feature maps at different scales to the size of P2 and then concatenates them. Subsequently, the channel attention is calculated using the Squeeze-and-Excitation Block40 to adaptively adjust the importance of each channel feature. Finally, element-wise addition is performed on the features from each scale to generate the final fused feature map F, which is used to perform the quality assessment task.Fig. 3 Adaptive Feature Fusion Block. The block integrates channel attention mechanisms to adaptively fuse feature outputs from the feature pyramid network at four scales, generating final quality-aware features for quality assessment.

For the view classification branch (VCB), a linear classifier is used to generate the view classification results. Simultaneously, a projection head is utilized to map the feature dimensions to a specified size to compute the Supervised Contrastive Loss41. The goal of supervised contrastive learning is to pull features of the same class closer together in the feature vector space while pushing the features of different classes apart. By applying supervised contrastive loss, we aimed to overcome the problem of small inter-class differences in echocardiographic images. For the quality assessment branch (QAB), a global average pooling is performed on feature map F to generate a K-dimensional feature vector, which is then fed to a multilayer perceptron (MLP) to fit and generate the final image quality score.

Model training

We jointly trained the model using the cross-entropy loss (for the view classification task), supervised contrastive loss, and mean squared error loss (for the quality assessment task). Additionally, to address the imbalance problem in multi-task training, an auto-tuning strategy42 was applied to learn the relative loss weights for each task.

The model was implemented in Python v3.8.12 using PyTorch v1.12.0 and was iteratively trained on two NVIDIA GeForce RTX 3090 GPUs, each with 24 GB of RAM. During training, the initial learning rate was set to 1e-5, and the batch size was set to 128. The Adam optimizer with a weight decay of 1e-5 was used. The input images were resized to 224 × 224, and pixel values were normalized to the range 0 to 1. No data augmentation was performed to prevent changes in image quality. An early stop strategy was used to stop training and reduce overfitting. The best model from the validation set was applied to the test set to evaluate the model performance.

Evaluation metrics

Five performance evaluation metrics, accuracy (ACC), precision (PRE), sensitivity (SEN), specificity (SPE), and F1 score (F1), were applied to validate the view classification performance. A confusion matrix was constructed to analyze the classification effect on different views. For quality assessment, Pearson’s linear correlation coefficient (PLCC), Spearman’s rank-order correlation coefficient (SROCC), mean absolute error (MAE), and root mean square error (RMSE) were used as evaluation indices. Indicators, such as the number of model parameters and inference time, were also considered to comprehensively evaluate the model performance. The Kruskal-Wallis test was employed to assess significant differences among the independent groups, with p < 0.05 considered statistically significant. For multiple comparisons, the Dunn-Bonferroni tests were applied. The Bootstrap analysis technique was utilized to calculate the 95% confidence intervals. Statistical analyses were conducted using SPSS v27.0 or Python v3.8.12.

Results

Evaluation of view classification task

The overall accuracy of the view classification task on the test set was 97.8% (95%CI, 97.7–98.0), with macro-average PRE, SEN, SPE, and F1 exceeding 94.8% (Table 3). The confusion matrix is shown in Fig. 4, which indicates that the model is prone to confusion when recognizing the three parasternal short axis views. The Grad-CAM maps in Fig. 5 reveal the image regions on which the model focuses when making classification decisions. It can be observed that the A4C view focuses on the mitral valve, tricuspid valve, ventricular septum, and atrial septum. The PLAX view focuses on the aortic and mitral valves, while the PSPA view focuses on the pulmonary artery wall and pulmonary valve. Additionally, the three similar parasternal short axis views effectively focus on key anatomical structural details, including the fish-mouth-like mitral valve orifice (mitral valve level), two sets of strong echo papillary muscles (papillary muscle level), and the annular left ventricular wall structure in three planes (apical level). To further show the robustness of the model, the Grad-CAM maps following data augmentation and under pathological conditions can be found in Supplementary Figures S3, S4 online, respectively. Table 3 Performance of the view classification task.

Views	PRE (%)	SEN (%)	SPE (%)	F1 (%)	
A4C	99.9 (99.7–100.0)	99.8 (99.6–99.9)	99.9 (99.9–100.0)	99.8 (99.7–99.9)	
PSPA	99.3 (99.0–99.6)	99.7 (99.5–99.9)	99.9 (99.9–99.9)	99.5 (99.3–99.7)	
PLAX	99.8 (99.7–99.9)	99.8 (99.7–99.9)	99.9 (99.9–100.0)	99.8 (99.7–99.9)	
PSAX-MV	90.7 (89.5–91.9)	90.5 (89.2–91.7)	99.3 (99.3–99.4)	90.6 (89.8–91.5)	
PSAX-PM	91.4 (90.5–92.4)	89.9 (88.8–90.8)	99.1 (99.0–99.2)	90.7 (89.9–91.4)	
PSAX-AP	83.1 (81.0–85.0)	88.1 (86.2–90.0)	99.4 (99.3–99.5)	85.5 (84.0–87.0)	
Others	99.8 (99.7–99.9)	99.7 (99.6–99.8)	99.9 (99.8–99.9)	99.8(99.7–99.8)	
Total	94.8 (94.5–95.2)	95.3 (95.0–95.7)	99.7 (99.6–99.7)	95.1 (94.7–95.5)	

Fig. 4 Confusion matrix between different views. The Y-axis of the confusion matrix shows the true labels and the X-axis shows the labels predicted by the model. Reading across true-label rows, the two numbers above and below each box indicate the sample size and its percentage.

Fig. 5 Grad-CAM maps for different views. Different colors indicate the activation level in different image areas. Red areas indicate high activation, while blue areas indicate low activation.

Evaluation of quality assessment task

The results of the quality assessment task on the test set are presented in Table 4. The average PLCC and SROCC values were 0.898 (95%CI, 0.893–0.902) and 0.893 (95%CI, 0.888–0.897), respectively, indicating a strong correlation between the model-predicted and expert subjective scores. The average MAE and RMSE values were 6.54 (95%CI, 6.43–6.66) and 9.42 (95%CI, 9.24–9.60), respectively, which are within the acceptable range relative to the label range of 0–100. The scoring effect of the samples for each view is shown in Fig. 6. It can be seen that there is a significant image quality improvement as the score increases. Table 4 Performance of the quality assessment task.

Views	PLCC	SPOCC	MAE	RMSE	
A4C	0.913 (0.902–0.921)	0.921 (0.916–0.926)	5.47 (5.29–5.64)	7.86 (7.50–8.27)	
PSPA	0.901 (0.893–0.909)	0.912 (0.906–0.918)	6.33 (6.08–6.56)	9.73 (9.33–10.11)	
PLAX	0.867 (0.859–0.876)	0.905 (0.900–0.910)	6.53 (6.34–6.71)	9.52 (9.19–9.83)	
PSAX-MV	0.905 (0.896–0.915)	0.868 (0.855–0.881)	6.94 (6.65–7.24)	9.81 (9.38–10.28)	
PSAX-PM	0.935 (0.930–0.940)	0.896 (0.887–0.904)	5.69 (5.48–5.91)	8.16 (7.87–8.49)	
PSAX-AP	0.865 (0.846–0.881)	0.853 (0.831–0.872)	8.26 (7.81–8.76)	11.41 (10.75–12.10)	
Total	0.898 (0.893–0.902)	0.893 (0.888–0.897)	6.54 (6.43–6.66)	9.42 (9.24–9.60)	

Fig. 6 Examples of the test results for the six standard views. The orange value in each panel represents the expert scores, while the green values are the prediction scores for the proposed method.

Effect of different backbones and additional modules on the proposed method

The performance of the proposed method when implemented on different backbone networks is presented in Table 5. Compared with other CNNs, VGG16 achieved the best trade-off between accuracy, number of parameters, and inference time. To analyze the effectiveness of each module in the proposed method, the ablation experiments were conducted using the VGG16-based quality assessment model (single task) as the baseline. As shown in Table 6, with the sequential addition of the neck network, view classification task, and supervised contrastive loss modules, the performance of our model was significantly improved. Furthermore, we conducted a comparison to assess the impact of including or excluding the "others" view on model performance. Table 5 Comparison of different backbone networks with our method.

Model	View Classification (%)	Quality Assessment	Param (M)	Time (ms)	
ACC	PRE	SEN	SPE	F1	PLCC	SPOCC	MAE	RMSE	
MobileNetV3-L	91.7

(91.5–92.0)

	83.8

(83.2–84.3)

	83.3

(82.8–83.8)

	98.7

(98.6–98.7)

	83.5

(83.0–84.0)

	0.755

(0.745–0.764)

	0.752

(0.743–0.761)

	9.75

(9.58–9.94)

	14.51

(14.24–14.80)

	6.03	2.84	
DenseNet121	96.9

(96.7–97.1)

	93.5

(93.1–93.9)

	93.3

(92.8–93.7)

	99.5

(99.5–99.5)

	93.3

(92.9–93.7)

	0.864

(0.857–0.871)

	0.858

(0.852–0.864)

	7.36

(7.22–7.49)

	10.69

(10.45–10.93)

	10.4	6.95	
VGG16	97.8

(97.7–98.0)

	94.8

(94.5–95.2)

	95.3

(95.0–95.7)

	99.7

(99.6–99.7)

	95.1

(94.7–95.5)

	0.898

(0.893–0.902)

	0.893

(0.888–0.897)

	6.54

(6.43–6.66)

	9.42

(9.24–9.60)

	17.8	2.80	
EfficientNet-B4	92.0

(91.7–92.3)

	84.3

(83.7–84.8)

	83.6

(83.1–84.2)

	98.7

(98.7–98.8)

	83.9

(83.4–84.5)

	0.779

(0.770–0.788)

	0.772

(0.764–0.781)

	9.29

(9.13–9.47)

	13.87

(13.61–14.12)

	19.5	5.61	
Resnet50	95.7

(95.5–95.9)

	90.9

(90.4–91.3)

	90.3

(89.8–90.7)

	99.3

(99.3–99.3)

	90.5

(90.1–91.0)

	0.830

(0.822–0.838)

	0.820

(0.813–0.827)

	8.35

(8.21–8.50)

	11.96

(11.73–12.22)

	27.4	3.42	
ConvNeXt-T	93.6

(93.4–93.9)

	87.7

(87.2–88.3)

	86.1

(85.6–86.7)

	99.0

(98.9–99.0)

	86.7

(86.2–87.3)

	0.809

(0.801–0.817)

	0.799

(0.792–0.807)

	8.96

(8.80–9.11)

	12.74

(12.48–12.96)

	30.9	3.35	
Inference time is measured based on the ONNXRuntime-1.13.1 framework with a NVIDIA GeForce GTX 3090 GPU. Bold values represent the highest performance for each column.

Table 6 Improvement of the baseline with additional modules.

Components	View Classification (%)	Quality Assessment	
ACC	PRE	SEN	SPE	F1	PLCC	SPOCC	MAE	RMSE	
Baseline	\	\	\	\	\	0.825

(0.818–0.831)

	0.815

(0.808–0.822)

	9.54

(9.40–9.67)

	12.44

(12.26–12.62)

	
Baseline + Neck	\	\	\	\	\	0.872

(0.867–0.877)

	0.862

(0.856–0.867)

	7.86

(7.74–7.99)

	10.64

(10.43–10.83)

	
Baseline + Neck + VCT	97.0

(96.8–97.1)

	93.6

(93.1–94.0)

	93.9

(93.5–94.3)

	99.5

(99.4–99.5)

	93.7

(93.3–94.1)

	0.894

(0.889–0.898)

	0.886

(0.881–0.890)

	6.61

(6.50–6.73)

	9.55

(9.38–9.74)

	
Baseline + Neck + VCT + LSCL (ours)	97.8

(97.7–98.0)

	94.8

(94.5–95.2)

	95.3

(95.0–95.7)

	99.7

(99.6–99.7)

	95.1

(94.7–95.5)

	0.898

(0.893–0.902)

	0.893

(0.888–0.897)

	6.54

(6.43–6.66)

	9.42

(9.24–9.60)

	
Baseline + Neck + VCT + LSCL (no “others” view)	95.9

(95.6–96.2)

	93.4

(92.9–93.8)

	93.7

(93.3–94.2)

	99.2

(99.2–99.3)

	93.5

(93.1–94.0)

	0.904

(0.900–0.908)

	0.895

(0.890–0.900)

	6.20

(6.08–6.33)

	8.91

(8.74–9.08)

	
Neck refers to our Neck Network, VCT refers to our View Classification Task and LSCL refers to Supervised Contrastive Loss. Bold values represent the highest performance for each column.

Application of the proposed method for echocardiographic image quality analysis

To verify the feasibility of our proposed method for standard view quality assessment, we compared the archived image quality among three groups of echocardiographers with different levels of experience (3 juniors, 3 seniors, and 3 experts). The junior group has 1–2 years of experience, the senior group has 4–5 years of experience, and the expert group has over 10 years of experience. The distribution of manufacturers among the three groups of echocardiographers was relatively balanced. We hypothesize that the image quality from the expert group is higher compared to the other groups. Images collected by nine echocardiographers between July and December 2023 were predicted using the proposed model. The subjects were males aged 18–40 years, without obvious cardiac structural or functional abnormalities. Based on the predictions, 6000 images were randomly selected from each echocardiographer, comprising 1000 images per view type. In total, 54,000 images from nine echocardiographers, covering six standard views, were used for statistical analysis.

The Kruskal-Wallis test indicated that there was a significant difference in quality scores across the three groups of echocardiographers on each view (p < 0.001). The box plot further illustrates the distribution of quality scores for the three groups across six standard views (Fig. 7). After adjusting for multiple comparisons, the median quality score of the expert group was higher than that of both the senior and junior groups for each view (p-adj < 0.001). Except for the PSPA view, the median quality score of the senior group was higher than that of the junior group (p-adj < 0.001).Fig. 7 Distribution of quality scores by view and group. The box plot provides a visual representation of the distribution differences between different groups (junior, senior, and expert), detailing statistical measures such as the minimum, first quartile, median, third quartile, and maximum.

Discussion

In this study, we developed and validated a multi-task model that simultaneously performs view recognition and percentile image quality assessment for seven types of views. The rationale for integrating these two tasks into a single model is that they are interrelated, as the view type determines the focus of the quality assessment. The results of the ablation experiments show that it is feasible to train a generic model to extract features from different echocardiographic views for quality assessment. Furthermore, introducing view classification as an auxiliary task can provide additional support for feature learning, which improves the quality assessment performance. The model performs well on both tasks. For the view classification task, misclassifications were mainly focused on three parasternal short axis views, which were mainly attributed to the high similarity between their anatomical structures. However, guided by the supervised contrastive learning loss to learn distinctive feature representations, relatively accurate recognition results can be obtained. For the quality assessment task, the results show that the proposed model incorporating quality-aware features at different scales effectively learned the judgment criteria used by human experts in echocardiographic image quality assessment. Even for the PSAX-AP view, our proposed model achieved acceptable results with a small sample size.

Compared with previous methods, our study has some strengths. For quality assessment, we summarized four clinically significant quality attributes that ensure image quality scores are closely aligned with diagnostic value. We applied the model to analyze archived images from echocardiographers of varying experience levels and confirmed that those with higher levels of experience produced higher-quality images. This demonstrates the model's ability to analyze image quality from a clinical diagnostic perspective effectively. Regarding model design, Prior methods focused solely on high-level single-scale features and overlooked low-level details. To address this issue, we added a hierarchical neck network to perform multi-scale perception modeling at a low computational cost, simulating the human visual system's hierarchical processing of visual stimuli at different scales. The results indicate that the quality assessment task significantly improved by adaptively integrating high-level semantic information with low-level detailed information through the neck network. From a clinical application perspective, our study utilizes echocardiographic images, rendering it more pertinent to real-world practice compared to video-based studies. The proposed model accurately captures static images at each moment, avoiding the complexity and computational costs incurred by using a 2D + t model to process dynamic video data. Additionally, our dataset encompasses six standard views and classifies all other views into an "others" category, enabling the model to directly differentiate between the six target views and other views. The results show that the introduction of the “others” view increases data diversity and improves the view categorization effect, with only a minor compromise on the quality assessment effect. In contrast, the model proposed by Luong et al.26 does not include an "others" category and relies solely on a confidence threshold to classify images: images with confidence below the threshold belong to the "others" category, while those with confidence above the threshold belong to one of the target views. However, since lower-quality target views may exhibit lower confidence, setting an effective threshold to distinguish them from "others" is quite difficult. Particularly in AI-assisted diagnosis, misclassifying other views as the required standard views can significantly affect diagnostic accuracy.

The proposed multi-task model effectively reduces the deployment pressure by merging the two tasks, achieving a good trade-off between accuracy and inference time. After deployment on a 3090 GPU, the model required no more than 2.8 ms to process a frame of 224 × 224 pixels. The model can be developed as part of a QC system that is applicable in several clinical scenarios. In echocardiography training, immediate feedback regarding the types of views and quality scores can help novice operators master the technical essentials of echocardiography more quickly and alleviate the shortage of faculty in underdeveloped areas43. During echocardiographic examinations, the system can assist operators in standardizing imaging and monitoring the progress of view acquisition, reducing measurement variability, and improving diagnostic quality44,45. Furthermore, the system can perform post-analysis on large-scale stored images or serve as a preprocessing step for AI-assisted diagnostic systems, selecting high-quality, interpretable cardiac ultrasound images from pre-stored data.

Our study has several limitations. First, the standard views covered are limited, and commonly used apical series views such as the apical 2-chamber and apical 3-chamber need to be further incorporated. Since our method does not impose specific constraints on view selection, it can theoretically accommodate additional standard echocardiographic views. Second, our method generates an overall quality score for echocardiographic images, and the individual scoring of different quality attributes should be explored in future research. Third, although the model was developed with a diverse dataset, its robustness and reliability still necessitate further validation in real-world clinical settings.

Supplementary Information

Supplementary Figures.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-71530-z.

Author contributions

X.L: Software, Formal analysis, Methodology, Writing-Original Draft. H.Z: Conceptualization, Data curation, Writing-Review & Editing. J.Y: Methodology, Writing-Review & Editing. L.Y, W.L, G.D and B.P: Data curation, Writing—Review & Editing. S.X: Conceptualization, Methodology, Writing—Original Draft, Supervision, Funding acquisition. All authors approved the manuscript.

Funding

This work was supported by the Sichuan Science and Technology Project (grant no. 2023YFQ0006) Project.

Data availability

The datasets generated during and analysed during the current study are available from the corresponding author on reasonable request.

Code availability

The code for this work is available upon request.

Competing interests

The authors declare no competing interests.

Ethical approval

This study was approved by the Ethics Board of Sichuan Provincial People's Hospital (No. 2023–407). The Ethics Board of Sichuan Provincial People's Hospital also approved the waiver of informed consent for this study.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Xinyu Li and Hongmei Zhang 
==== Refs
References

1. Lang RM Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American society of echocardiography and the european association of cardiovascular imaging Eur. Heart J. Cardiovasc. Imaging 2015 16 3 233 271 10.1093/ehjci/jev014 25712077
Lang, R. M. et al. Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American society of echocardiography and the european association of cardiovascular imaging. Eur. Heart J. Cardiovasc. Imaging 16(3), 233–271 (2015).25712077 10.1093/ehjci/jev014
2. Mitchell C Guidelines for performing a comprehensive transthoracic echocardiographic examination in adults: recommendations from the American Society of Echocardiography J. Am. Soc. Echocardiogr. 2019 32 1 1 64 10.1016/j.echo.2018.06.004 30282592
Mitchell, C. et al. Guidelines for performing a comprehensive transthoracic echocardiographic examination in adults: recommendations from the American Society of Echocardiography. J. Am. Soc. Echocardiogr. 32(1), 1–64 (2019).30282592 10.1016/j.echo.2018.06.004
3. Nagata Y Impact of image quality on reliability of the measurements of left ventricular systolic function and global longitudinal strain in 2D echocardiography Echo Res. Pract. 2018 5 1 28 39 10.1530/ERP-17-0047
Nagata, Y. et al. Impact of image quality on reliability of the measurements of left ventricular systolic function and global longitudinal strain in 2D echocardiography. Echo Res. Pract. 5(1), 28–39 (2018).10.1530/ERP-17-0047
4. Foley TA Measuring left ventricular ejection fraction-techniques and potential pitfalls Eur. Cardiol. 2012 8 2 108 114 10.15420/ecr.2012.8.2.108
Foley, T. A. et al. Measuring left ventricular ejection fraction-techniques and potential pitfalls. Eur. Cardiol. 8(2), 108–114 (2012).10.15420/ecr.2012.8.2.108
5. Zhou J Du M Chang S Chen Z Artificial intelligence in echocardiography: detection, functional evaluation, and disease diagnosis Cardiovasc. Ultrasound 2021 19 1 1 11 10.1186/s12947-021-00261-2 33388070
Zhou, J., Du, M., Chang, S. & Chen, Z. Artificial intelligence in echocardiography: detection, functional evaluation, and disease diagnosis. Cardiovasc. Ultrasound 19(1), 1–11 (2021).33388070 10.1186/s12947-021-00261-2
6. Letnes JM Variability of echocardiographic measures of left ventricular diastolic function The HUNT study. Echocardiography 2021 38 6 901 908 10.1111/echo.15073
Letnes, J. M. et al. Variability of echocardiographic measures of left ventricular diastolic function. The HUNT study. Echocardiography 38(6), 901–908 (2021).10.1111/echo.15073
7. Liao Z On modelling label uncertainty in deep neural networks: automatic estimation of intra-observer variability in 2d echocardiography quality assessment IEEE Trans. Med. Imaging 2019 39 6 1868 1883 10.1109/TMI.2019.2959209 31841401
Liao, Z. et al. On modelling label uncertainty in deep neural networks: automatic estimation of intra-observer variability in 2d echocardiography quality assessment. IEEE Trans. Med. Imaging 39(6), 1868–1883 (2019).31841401 10.1109/TMI.2019.2959209
8. Ouyang D Video-based AI for beat-to-beat assessment of cardiac function Nature 2020 580 7802 252 256 10.1038/s41586-020-2145-8 32269341
Ouyang, D. et al. Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020).32269341 10.1038/s41586-020-2145-8
9. Liu B A deep learning framework assisted echocardiography with diagnosis, lesion localization, phenogrouping heterogeneous disease, and anomaly detection Sci. Rep. 2023 13 1 3 10.1038/s41598-022-27211-w 36593284
Liu, B. et al. A deep learning framework assisted echocardiography with diagnosis, lesion localization, phenogrouping heterogeneous disease, and anomaly detection. Sci. Rep. 13(1), 3 (2023).36593284 10.1038/s41598-022-27211-w
10. Barry T The Role of Artificial Intelligence in Echocardiography J. Imaging 2023 9 2 50 10.3390/jimaging9020050 36826969
Barry, T. et al. The Role of Artificial Intelligence in Echocardiography. J. Imaging 9(2), 50 (2023).36826969 10.3390/jimaging9020050
11. Sehly A Artificial Intelligence in Echocardiography: The Time is Now Rev. Cardiovasc. Med. 2022 23 8 256 10.31083/j.rcm2308256 39076632
Sehly, A. et al. Artificial Intelligence in Echocardiography: The Time is Now. Rev. Cardiovasc. Med. 23(8), 256 (2022).39076632 10.31083/j.rcm2308256
12. Kusunose K Steps to use artificial intelligence in echocardiography J. Echocardiogr. 2021 19 1 21 27 10.1007/s12574-020-00496-4 33044715
Kusunose, K. Steps to use artificial intelligence in echocardiography. J. Echocardiogr. 19(1), 21–27 (2021).33044715 10.1007/s12574-020-00496-4
13. Wang, W. et al. An Automated Heart Shunt Recognition Pipeline Using Deep Neural Networks. J. Imaging Informatics Med. 1–16 (2024).
14. Madani A Arnaout R Mofrad M Arnaout R Fast and accurate view classification of echocardiograms using deep learning NPJ Digit. Med. 2018 1 1 6 10.1038/s41746-017-0013-1 30828647
Madani, A., Arnaout, R., Mofrad, M. & Arnaout, R. Fast and accurate view classification of echocardiograms using deep learning. NPJ Digit. Med. 1(1), 6 (2018).30828647 10.1038/s41746-017-0013-1
15. Santosh Kumar BP Haq MA Sreenivasulu P Siva D Alazzam MB Alassery F Karupusamy S Fine-tuned convolutional neural network for different cardiac view classification J. Supercomput. 2022 78 16 18318 18335 10.1007/s11227-022-04587-0
Santosh Kumar, B. P. et al. Fine-tuned convolutional neural network for different cardiac view classification. J. Supercomput. 78(16), 18318–18335 (2022).10.1007/s11227-022-04587-0
16. Belciug S Deep learning and Gaussian mixture modelling clustering mix a new approach for fetal morphology view plane differentiation J. Biomed. Inform. 2023 143 104402 10.1016/j.jbi.2023.104402 37217028
Belciug, S. Deep learning and Gaussian mixture modelling clustering mix a new approach for fetal morphology view plane differentiation. J. Biomed. Inform. 143, 104402 (2023).37217028 10.1016/j.jbi.2023.104402
17. Wu L Standard echocardiographic view recognition in diagnosis of congenital heart defects in children using deep learning based on knowledge distillation Front. Pediatr. 2022 9 770182 10.3389/fped.2021.770182 35118028
Wu, L. et al. Standard echocardiographic view recognition in diagnosis of congenital heart defects in children using deep learning based on knowledge distillation. Front. Pediatr. 9, 770182 (2022).35118028 10.3389/fped.2021.770182
18. Yang, S. et al. Maniqa: Multi-dimension attention network for no-reference image quality assessment. Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 1191–1200 (2022).
19. Zhang S Wang Y Jiang J Dong J Yi W Hou W CNN-based medical ultrasound image quality assessment Complexity 2021 2021 1 9938367 10.1155/2021/9938367
Zhang, S. et al. CNN-based medical ultrasound image quality assessment. Complexity 2021(1), 9938367 (2021).10.1155/2021/9938367
20. Zhang F Yoo YM Koh LM Kim Y Nonlinear diffusion in Laplacian pyramid domain for ultrasonic speckle reduction IEEE Trans. Med. Imaging 2007 26 2 200 211 10.1109/TMI.2006.889735 17304734
Zhang, F., Yoo, Y. M., Koh, L. M. & Kim, Y. Nonlinear diffusion in Laplacian pyramid domain for ultrasonic speckle reduction. IEEE Trans. Med. Imaging 26(2), 200–211 (2007).17304734 10.1109/TMI.2006.889735
21. Czajkowska J Juszczyk J Piejko L Glenc-Ambroży M High-frequency ultrasound dataset for deep learning-based image quality assessment Sensors 2022 22 4 1478 10.3390/s22041478 35214381
Czajkowska, J., Juszczyk, J., Piejko, L. & Glenc-Ambroży, M. High-frequency ultrasound dataset for deep learning-based image quality assessment. Sensors 22(4), 1478 (2022).35214381 10.3390/s22041478
22. Huang KC Artificial intelligence aids cardiac image quality assessment for improving precision in strain measurements Cardiovasc. Imaging 2021 14 2 335 345
Huang, K. C. et al. Artificial intelligence aids cardiac image quality assessment for improving precision in strain measurements. Cardiovasc. Imaging 14(2), 335–345 (2021).
23. Zhang J Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy Circulation 2018 138 16 1623 1635 10.1161/CIRCULATIONAHA.118.034338 30354459
Zhang, J. et al. Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy. Circulation 138(16), 1623–1635 (2018).30354459 10.1161/CIRCULATIONAHA.118.034338
24. Zamzmi G Rajaraman S Hsu LY Sachdev V Antani S Real-time echocardiography image analysis and quantification of cardiac indices Med. Image. Anal. 2022 80 102438 10.1016/j.media.2022.102438 35868819
Zamzmi, G., Rajaraman, S., Hsu, L. Y., Sachdev, V. & Antani, S. Real-time echocardiography image analysis and quantification of cardiac indices. Med. Image. Anal. 80, 102438 (2022).35868819 10.1016/j.media.2022.102438
25. Abdi AH Automatic quality assessment of echocardiograms using convolutional neural networks: feasibility on the apical four-chamber view IEEE Trans. Med. Imaging 2017 36 6 1221 1230 10.1109/TMI.2017.2690836 28391191
Abdi, A. H. et al. Automatic quality assessment of echocardiograms using convolutional neural networks: feasibility on the apical four-chamber view. IEEE Trans. Med. Imaging 36(6), 1221–1230 (2017).28391191 10.1109/TMI.2017.2690836
26. Luong C Automated estimation of echocardiogram image quality in hospitalized patients Int. J. Cardiovasc. Imaging 2021 37 229 239 10.1007/s10554-020-01981-8 33211237
Luong, C. et al. Automated estimation of echocardiogram image quality in hospitalized patients. Int. J. Cardiovasc. Imaging 37, 229–239 (2021).33211237 10.1007/s10554-020-01981-8
27. Labs RB Vrettos A Loo J Zolgharni M Automated assessment of transthoracic echocardiogram image quality using deep neural networks Intell. Med. 2023 3 03 191 199 10.1016/j.imed.2022.08.001
Labs, R. B., Vrettos, A., Loo, J. & Zolgharni, M. Automated assessment of transthoracic echocardiogram image quality using deep neural networks. Intell. Med. 3(03), 191–199 (2023).10.1016/j.imed.2022.08.001
28. Ding Y AP-CNN: Weakly supervised attention pyramid convolutional neural network for fine-grained visual classification IEEE Trans. Image Process. 2021 30 2826 2836 10.1109/TIP.2021.3055617 33556008
Ding, Y. et al. AP-CNN: Weakly supervised attention pyramid convolutional neural network for fine-grained visual classification. IEEE Trans. Image Process. 30, 2826–2836 (2021).33556008 10.1109/TIP.2021.3055617
29. Zhang Y Yang Q An overview of multi-task learning Natl. Sci. Rev. 2018 5 1 30 43 10.1093/nsr/nwx105
Zhang, Y. & Yang, Q. An overview of multi-task learning. Natl. Sci. Rev. 5(1), 30–43 (2018).10.1093/nsr/nwx105
30. Xu Z Zhang Q Li W Li M Yip PSF Individualized prediction of depressive disorder in the elderly: a multitask deep learning approach Int. J. Med. Inform. 2019 132 103973 10.1016/j.ijmedinf.2019.103973 31569007
Xu, Z., Zhang, Q., Li, W., Li, M. & Yip, P. S. F. Individualized prediction of depressive disorder in the elderly: a multitask deep learning approach. Int. J. Med. Inform. 132, 103973 (2019).31569007 10.1016/j.ijmedinf.2019.103973
31. Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B. & Belongie, S. Feature pyramid networks for object detection. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2117–2125 (2017).
32. Howard, A. et al. Searching for mobilenetv3. Proc. IEEE/CVF Int. Conf. Comput. Vis. 1314–1324 (2019).
33. Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 4700–4708 (2017).
34. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Int. Conf. Learn. Representations 1–14 (2015).
35. Tan, M. & Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. Int. Conf. Mach. Learn. 6105–6114 (2019).
36. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 770–778 (2016).
37. Liu, Z., Mao, H., Wu, C. Y., Feichtenhofer, C., Darrell, T. & Xie, S. A convnet for the 2020s. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 11976–11986 (2022).
38. Zhao, Y. et al. Detrs beat yolos on real-time object detection. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 16965–16974 (2024).
39. Dosovitskiy, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. Preprint at 10.48550/arXiv.2010.11929 (2021).
40. Hu, J., Shen, L. & Sun, G. Squeeze-and-excitation networks. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 7132–7141 (2018).
41. Khosla P Supervised contrastive learning Adv. Neural Inf. Process. Syst. 2020 33 18661 18673
Khosla, P. et al. Supervised contrastive learning. Adv. Neural Inf. Process. Syst. 33, 18661–18673 (2020).
42. Liebel, L. & Körner, M. Auxiliary tasks in multi-task learning. Preprint at 10.48550/arXiv.1805.06334 (2018).
43. Narang A Utility of a deep-learning algorithm to guide novices to acquire echocardiograms for limited diagnostic use JAMA Cardiol. 2021 6 6 624 632 10.1001/jamacardio.2021.0185 33599681
Narang, A. et al. Utility of a deep-learning algorithm to guide novices to acquire echocardiograms for limited diagnostic use. JAMA Cardiol. 6(6), 624–632 (2021).33599681 10.1001/jamacardio.2021.0185
44. Ferraz S Coimbra M Pedrosa J Assisted probe guidance in cardiac ultrasound: A review Front. Cardiovasc. Med. 2023 10 1056055 10.3389/fcvm.2023.1056055 36865885
Ferraz, S., Coimbra, M. & Pedrosa, J. Assisted probe guidance in cardiac ultrasound: A review. Front. Cardiovasc. Med. 10, 1056055 (2023).36865885 10.3389/fcvm.2023.1056055
45. Zhang Z Artificial intelligence-enhanced echocardiography for systolic function assessment J. Clin. Med. 2022 11 10 2893 10.3390/jcm11102893 35629019
Zhang, Z. et al. Artificial intelligence-enhanced echocardiography for systolic function assessment. J. Clin. Med. 11(10), 2893 (2022).35629019 10.3390/jcm11102893
