
==== Front
Transl Psychiatry
Transl Psychiatry
Translational Psychiatry
2158-3188
Nature Publishing Group UK London

2972
10.1038/s41398-024-02972-2
Article
DeepASD: a deep adversarial-regularized graph learning method for ASD diagnosis with multimodal data
http://orcid.org/0009-0002-4305-3412
Chen Wanyi 12
Yang Jianjun 3
Sun Zhongquan 1
http://orcid.org/0000-0001-5097-2113
Zhang Xiang 4
http://orcid.org/0000-0002-8823-9969
Tao Guangyu 5
Ding Yuan 1
http://orcid.org/0000-0001-5714-9189
Gu Jingjun 26
Bu Jiajun 2
Wang Haishuai haishuai.wang@zju.edu.cn

127
1 https://ror.org/059cjpv64 grid.412465.0 Department of Hepatobiliary and Pancreatic Surgery, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, Zhejiang China
2 https://ror.org/00a2xv884 grid.13402.34 0000 0004 1759 700X Zhejiang Provincial Key Laboratory of Service Robot, College of Computer Science, Zhejiang University, Hangzhou, China
3 grid.27255.37 0000 0004 1761 1174 Department of General Practice, Shandong Provincial Third Hospital, Shandong University, Jinan, Shandong China
4 https://ror.org/04dawnj30 grid.266859.6 0000 0000 8598 2218 Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC USA
5 grid.16821.3c 0000 0004 0368 8293 Department of Radiology, Shanghai Chest Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
6 Pujian Technology, Hangzhou, Zhejiang China
7 https://ror.org/03wkvpx79 0000 0005 0475 7227 Shanghai Artificial Intelligence Laboratory, Shanghai, China
14 9 2024
14 9 2024
2024
14 3753 9 2023
23 4 2024
31 5 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by/4.0/ Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Autism Spectrum Disorder (ASD) is a prevalent neurological condition with multiple co-occurring comorbidities that seriously affect mental health. Precisely diagnosis of ASD is crucial to intervention and rehabilitation. A single modality may not fully reflect the complex mechanisms underlying ASD, and combining multiple modalities enables a more comprehensive understanding. Here, we propose, DeepASD, an end-to-end trainable regularized graph learning method for ASD prediction, which incorporates heterogeneous multimodal data and latent inter-patient relationships to better understand the pathogenesis of ASD. DeepASD first learns cross-modal feature representations through a multimodal adversarial-regularized encoder, and then constructs adaptive patient similarity networks by leveraging the representations of each modality. DeepASD exploits inter-patient relationships to boost the ASD diagnosis that is implemented by a classifier compositing of graph neural networks. We apply DeepASD to the benchmarking Autism Brain Imaging Data Exchange (ABIDE) data with four modalities. Experimental results show that the proposed DeepASD outperforms eight state-of-the-art baselines on the benchmarking ABIDE data, showing an improvement of 13.25% in accuracy, 7.69% in AUC-ROC, and 17.10% in specificity. DeepASD holds promise for a more comprehensive insight of the complex mechanisms of ASD, leading to improved diagnosis performance.

Subject terms

Autism spectrum disorders
Predictive markers
This work is supported by the National Key R&D Program of China (2022ZD0160703), the National Natural Science Foundation of China (62202422), the Zhejiang Province Science and Technology Plan Project (2022C03134), and the Shanghai Key Laboratory Open Project (STCSM 22DZ2229005)issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

Autism spectrum disorder (ASD) is one of the most common neurodevelopmental disorders, mainly characterized by qualitative impairments in social functioning [1–6]. With 1 in 36 children suffering from ASD [7], ASD severely affects their quality of life such as social communication and interaction. Precise and timely diagnosis of ASD can trigger earlier intervention and bring positive long-term outcomes in communication skills, verbal and cognitive abilities, etc. [2] The mainstream of ASD clinical diagnosis depends on traditional behavioral criteria [8], which often results in delayed diagnosis or misdiagnosis due to being time-consuming and technically demanding. Beyond behavioral evaluations, a number of MRI-based computational approaches attempt to construct functional connectivity (FC) maps to learn abnormal connectivity between brain regions, and then employ deep neural networks to differentiate ASD and typical controls (TCs).

However, deep learning-based automatic diagnosis techniques have, in most cases, focused on single-modal MRI data. Compared to single brain atlas fMRI, different brain atlases [9–11] and heterogeneous medical data from different modalities [12–14] can provide complementary information to each other. Nevertheless, there have not been superb multimodal-based results obtained in the clinical evaluation of ASD. Due to the diversity of the underlying distribution and complicated associations across modalities [15], leveraging a variety of types of data to encode more detailed and comprehensive information for accurate ASD diagnosis is challenging. Pioneer methods have imposed a graph convolution network (GCN) [16–18] to aggregate knowledge and learn a patient similarity network. Although it is documented that patient similarities can provide complementary knowledge for ASD diagnosis [19], existing approaches have mainly emphasized the intra-modal knowledge from a single modality for patient similarity network construction while rarely considering both intra-modal and cross-modal information to enhance patient alignments.

To address these challenges, we propose an end-to-end adaptive graph learning framework, named DeepASD, to identify ASD by exploiting multimodal representations of functional Magnetic Resonance Imaging (fMRI) and non-imaging phenotypic data, aiming to construct a unified graph to represent knowledge and patient similarity for classification. Specifically, the distributions of different modalities are first aligned using the most informative modality (i.e., fMRI) by training an adversarial-regularized encoder (i.e., a feature transformer and a discriminator) for each modality. The intra-modal knowledge within each modality is then represented as a graph to reveal the connections among patients reflected within each modality. DeepASD further aggregates cross-modal knowledge to align the different modalities for patient representations, and adopts a graph convolution network to identify ASD diagnoses. We evaluate DeepASD on two datasets from the Autism Brain Imaging Data Exchange (ABIDE). DeepASD exhibited improved diagnostic ability for ASD compared to alternative approaches. The model achieved the following performance metrics, presented in mean ± standard deviation format, on the ABIDE A dataset: accuracy (ACC) of 87.38 ± 2.87%, area under curve (AUC) of 92.76 ± 4.00%, specificity (SEN) of 88.35 ± 6.83%, and sensitivity (SPE) of 86.51 ± 8.41%. Similarly, on the ABIDE B dataset, the results were ACC 88.09 ± 2.92%, AUC 93.59 ± 2.45%, SEN 87.58 ± 3.68%, and SPE 88.49 ± 4.73%. We visualized the features learned by our multimodal adversarial-regularized encoder, showing that the different modal features still maintain their specificity but are more distinguishable than the original features in the reduced-dimensional space. We also visualized and analyzed the learned patient similarity graphs, revealing that the learned patient similarity networks are highly correlated with age and sex. In addition, we found that the fMRI modality is significantly correlated with ASD diagnoses in the multimodal datasets. The findings are consistent with the demographic disparities recorded in autism diagnoses [20]. Therefore, we conclude that DeepASD is a promising tool for ASD diagnosis by leveraging multimodal clinical data.

Results

Given the multimodal medical data, DeepASD applies adversarial representation learning to align embeddings of different modalities, and exploits intra-modal correlations, aiming to construct a joint patient similarity network for ASD identification. DeepASD consists of three staged (Fig. 1b; Section “Methods”): an adversarial-regularized encoder, graph learning and fusion, and a GNN classifier. We first develop a multimodal adversarial-regularized encoder to cope with the feature heterogeneity of each modality and reduce the distributional divergence. Through the graph learning module, we then construct the patient similarity network for each modality independently and generate a global adjacency matrix by leveraging knowledge from all modalities. Finally, DeepASD identifies ASD by feeding the learned global adjacency matrix and aligned feature embeddings into a graph neural network (GNN) classifier.Fig. 1 The workflow of DeepASD.

a Multimodal data for ASD diagnosis. The left panel shows that the multimodal datasets consist of fMRI, automated anatomical quality assessment metrics (ANAT), automated functional quality assessment metrics (FUNC), and demographic information (PHENO). According to the information of the multimodal data, DeepASD constructs a multimodal patient similarity network as shown in the middle panel. Using the weighted fusion method (weights are automatically learned), we obtain a global patient similarity network from the multimodal patient similarity network. The right panel presents the global patient similarity matrix (taking six patients as an example). The darker the color, the more similar of the patient embeddings. b The proposed DeepASD framework. DeepASD first adopts an adversarial-regularized encoder module to align the embeddings from different modalities, thereby the learned embeddings will be aligned into the same latent space. We then construct a patient similarity graph for each modality, where each graph node denotes one patient and the edges denote the inter-patient connections. After that, we fuse the multiple constructed modality-specific graphs into a global graph that represents the patient similarity globally. Finally, we employ a graph neural network classifier for ASD diagnosis based on the inter-patient global graph and aligned embeddings. c Model validation. We compare the proposed DeepASD with six state-of-the-art baselines, including three traditional machine learning-based models and three deep learning-based models, on two benchmarking datasets. The proposed DeepASD outperforms all baselines significantly.

Datasets

DeepASD was evaluated on the benchmarking dataset of ABIDE [21], which consists of fMRI images and corresponding phenotypic observations. Specifically, there are two multimodal datasets in the ABIDE data: (i) ABIDE A, which contains four modalities, i.e., demographic information, automated anatomical quality assessment metrics, automated functional quality assessment metrics, and fMRI connection networks. The dataset includes 871 subjects (468 TC and 403 ASD subjects). We follow (Parisot, 2017) [19] for the dataset settings and preprocessing. (ii) ABIDE B, encompassing 949 subjects, adheres to the settings outlined by (Wang, 2020) [10]. This cohort includes 419 individuals diagnosed with ASD and 530 TC subjects. The dataset integrates three brain atlases (CC200, AAL - Automated Anatomical Labeling, and DOS - Dosenbach160) along with demographic information to create a multimodal dataset comprising four modalities. The reason why the dataset was split into ABIDE A and ABIDE B is because of different purpose and specific objectives for analysis [10, 19]. Specifically, ABIDE A divides phenotypic information into detailed demographic information, automated anatomical quality assessment metrics, and automated functional quality assessment metrics. This subdivision facilitates a comprehensive investigation of the interplay between various phenotypic information and a single atlas in fMRI diagnostics. On the other hand, ABIDE B categorizes the fMRI modality based on different atlases (i.e., CC200, AAL, and DOS) and integrates it using demographic information, aiming to explore the diagnostic implications of combining multiple fMRI types with demographic information. For a fair comparison, all data are preprocessed using the Configurable Pipeline for the Analysis of Connectomes (C-PAC) based on the preprocessed fMRI datasets [22].

DeepASD enables effective multi-modal data integration and accurate prediction of ASD

The proposed DeepASD model was compared with other shallow machine learning methods and recently reported multimodal deep learning models. To obtain an unbiased estimation and more robust performance for ASD diagnosis, we performed a 10-fold stratified cross-validation strategy on the two datasets. To eliminate the effects of randomness in the experiments, we employed an independent two-sample t-test to calculate P-value.

Figure 2a shows the AUC and ACC of multiple methods. On the ABIDE A dataset, DeepASD achieved an ACC of 87.38 ± 2.87%, which significantly outperforms all baselines (P-vacitelue < 0.05). On ABIDE B, the of our DeepASD is also significantly higher than that of baselines, as shown in Table 1. Additionally, Fig. 2b illustrates that DeepASD outperforms other methods in terms of SEN and SPE on both ABIDE A and ABIDE B datasets. In measuring SEN and SPE, we adhere to a cutoff of 0.5, as per our experimental methodology utilizing deep learning techniques. This choice is made due to our dataset presenting a balanced distribution of positive and negative samples, thus warranting a fixed threshold of 0.5. Moreover, as shown in Fig. 2c, DeepASD also achieves a relatively high area under the receiver operating characteristic (ROC) curve AUC values of 92.76 ± 4.00% on ABIDE A.Fig. 2 Performance of DeepASD on ABIDE A and ABIDE B.

Panels a and b illustrate the classification performance in a two-dimensional manner on the two datasets. Panel c shows the ROC curves of DeepASD compared with state-of-the-art baselines on the ABIDE A dataset. The mean ROC curve and standard deviation of 10-fold cross-validation results are shown as bold lines and shad regions, respectively. Panel d reports the performance margin (in terms of ACC, AUC, SEN, and SPE), showing DeepASD outperforms other baselines. Each dot denotes a fold among the 10-fold cross-validation.

Table 1 Quantitative comparisons on the two datasets.

Methods	ABIDE Aa	ABIDE Bb	
ACC (%)	AUC (%)	SEN (%)	SPE (%)	ACC (%)	AUC (%)	SEN (%)	SPE (%)	
Baseline-1	68.20 ± 5.13	73.02 ± 5.39	55.34 ± 10.86	79.30 ± 4.97	72.39 ± 3.04	81.92 ± 3.32	55.13 ± 6.30	86.04 ± 2.82	
Baseline-2	78.30 ± 8.43	86.64 ± 7.71	75.45 ± 9.61	80.76 ± 9.43	80.92 ± 3.14	88.63 ± 1.96	77.81 ± 3.34	83.40 ± 5.52	
ACERTA-ABIDE	64.07 ± 5.52	67.98 ± 7.32	40.52 ± 12.10	84.40 ± 2.69	82.41 ± 3.25	88.98 ± 2.65	75.19 ± 3.68	88.11 ± 4.31	
ASD-DiagNet	67.74 ± 5.41	73.81 ± 5.47	62.79 ± 8.24	72.02 ± 5.33	85.04 ± 2.93	92.14 ± 1.95	81.63 ± 7.13	87.74 ± 3.07	
population gcn	82.32 ± 7.33	89.34 ± 8.25	79.90 ± 10.80	84.39 ± 7.09	83.35 ± 3.79	91.32 ± 3.09	77.58 ± 8.17	87.92 ± 2.95	
AIMAFE	61.53 ± 5.85	63.10 ± 6.93	34.94 ± 10.50	84.40 ± 4.70	81.04 ± 3.38	88.67 ± 3.58	73.75 ± 7.61	86.79 ± 3.16	
MultiSurv	64.07 ± 5.92	67.82 ± 6.05	63.16 ± 18.50	64.98 ± 10.21	79.88 ± 5.04	91.23 ± 2.77	79.49 ± 12.73	80.19 ± 16.41	
deepManReg	69.22 ± 6.25	77.97 ± 5.42	64.25 ± 11.06	73.52 ± 7.44	78.40 ± 1.88	85.85 ± 1.05	71.36 ± 5.08	83.96 ± 3.89	
DeepASD	87.38 ± 2.87	92.76 ± 4.00	88.35 ± 6.83	86.51 ± 8.41	88.09 ± 2.92	93.59 ± 2.45	87.58 ± 3.68	88.49 ± 4.73	
Best results are marked in bold.

aABIDE A dataset consists of demographic information, automated anatomical quality assessment metrics, automated functional quality assessment metrics, and fMRI.

bABIDE B dataset consists of fMRIs based on AAL, CC200, DOH atlas.

The shallow baselines include Baseline-1 [23], and Baseline-2 [24], which are commonly used in the biomedical field. The deep learning-based baselines are ACERTA-ABIDE [25], ASD-DiagNet [26], population-gcn [19], AIMAFE [10], MultiSurv [27], and deepManReg [28]. Baseline-1 [23] extracts the top 10 features of two modalities, such as clinical characteristics and laboratory test results, and then uses Random Forest to achieve higher accuracy. Baseline-2 [24] develops a predictive model using L1-regularised logistic regression (lasso) on multimodal features extracted from retinal fundus images, clinical measurements, and genomic data. In the ACERTA-ABIDE [25] study, Pearson correlation coefficients were computed for fMRI data, which were then fed into two stacked denoising autoencoders. This process was part of the unsupervised pre-training stage, aimed at extracting a lower-dimensional representation from the ABIDE dataset. Subsequently, the weights of these encoders were applied to a multilayer perceptron (MLP) for classification purposes. In the ASD-DiagNet [26] study, Pearson correlation calculations were performed on fMRI data. The approach included using features fused from the five nearest neighbors based on EROS similarity as a method for data augmentation. These features were subsequently input into an autoencoder, designed with tied weights, to extract a lower-dimensional feature representation. Finally, these reduced features were fed into a single-layer perceptron (SLP) for diagnostic classification. In the population-gcn [19] study, the construction of patient-patient relational graphs was based on demographic information, specifically sex and site. This structured graph then served as a framework into which fMRI data were input as features for classification through a graph neural network. AIMAFE [10] applies a stacked denoising autoencoder to learn multi-atlases deep feature representation and an ensemble learning method to conduct the ASD identification task. MultiSurv [27] integrates multiple feature extractors and multiple feature fusion methods to develop a deep learning model for cancer survival prediction. deepManReg [28] adopts multiple deep neural networks for different modalities, and jointly trains them to align multimodal features into a common latent space and then uses the cross-modal manifolds to regularize the classification network to improve phenotype predictions. Every model is able to take multimodal data as inputs by design.

Qualitative performance of DeepASD

DeepASD learns a unified global graph where each node denotes a patient and the edge denotes the connections between patients (a.k.a. patient similarity networks). The node feature of the patient similarity network indicates patient representations learned from the multimodal data. Figure 3 visualizes the similarity matrix of patient representation to qualitatively evaluate the learned features. The heatmap shows the cosine similarity of different patient representations, indicating the features used for prediction have a high degree of sex and age specificity. From the first column of Fig. 3, we can see that the original features are not able to reveal the similarity related to sex and age, while the correlations between the learned patient similarity and demographic features (i.e., age and sex) tends to be clear by leveraging our proposed multimodal adversarial-regularized encoder and multi-graph fusion GNN module. The results demonstrate the bias of sex and age in the diagnosis process, which is consistent with the phenomenon described in (Wiggins, 2020) [20].Fig. 3 Visualization of cosine similarity heatmap across patients.

The left column shows the results of clustering by sex (a), while the middle column is the results of clustering by age in the ABIDE A dataset (b). The first row represents the similarity matrices on raw features. The second row shows the similarity matrices of multimodal fused features after the multimodal adversarial-regularized encoder (details in Fig. 1b; Section “Multimodal adversarial-regularized encoder”. The last row presents the similarity matrices of fused features through GNN. There is a significant age and sex bias in the diagnosis of ASD, and the results show that the contrast between the diagnostics and the presentation of sex and age clusters, indicating that our multimodal adversarial-regularized encoder and multi-graph fusion GNN module are capable of learning more representative features for diagnosis. c–e are visualizations (t-SNE) of the feature representations in ABIDE A. In the three panels, the red color denotes ASD while green colors denotes TC. c Visualization of the raw features for each modality. d Visualization of raw features and learned features. The learned features are taken after the multimodal adversarial-regularized encoder. It’s clearly observed that the learned features are more distinguishable than the raw features. e Visualization of the learned features for each modality. Compared to the raw feature (c), the learned feature space (e) demonstrate that the fMRI has obvious class separability after feature extraction. In addition, the PHENO modality is also separable while the modalities of ANAT and FUNC are less divisible. The observation validates our explanation on the importance of modalities (Section “Multiple modalities enable more accurate diagnosis”; Fig. 4).

The adversarial-regularized graph learning contributes to DeepASD

The proposed DeepASD mainly contains three components: the feature extractor, the adversarial modal discriminator, and the multi-graph fusion GNN. We first applied the feature extractor module to extract discriminative features from each modal, and then demonstrated the effectiveness of the remaining two components via an ablation study.

We used the ABIDE A data as an example for the ablation study. To verify the effectiveness of the multi-graph fusion GNN, we removed this module from DeepASD. As shown in Table 2, the AUC dropped from 92.76% to 82.99%, and there was an obvious decrease in terms of ACC, SEN, and SPE. Therefore, the learned patient similarity graphs from the multi-graph fusion GNN was able to improve the performance of ASD diagnosis. Similarly, the AUC dropped by 6.51% (from 82.99% to 77.59%) if the adversarial modal discriminator was removed. This is reasonable because the adversarial representation learning facilitates multi-modal feature alignment and eliminates the distribution gap between different modalities. Therefore, we can observe from the ablation study that the components in our DeepASD are effective to achieve a precise diagnosis of ASD.Table 2 Ablation study.

Dataset	Modules	Metrics	
Feature Extractor	Modal Discriminator	Multi-graph fusion GNN	ACC	AUC	SEN	SPE	
ABIDE A	✓			72.78 ± 7.05	77.59 ± 3.97	64.35 ± 12.35	80.83 ± 9.94	
		✓	72.22 ± 6.58	78.25 ± 6.06	65.79 ± 8.28	77.78 ± 0.07	
✓	✓		76.74 ± 3.79	82.99 ± 4.66	69.18 ± 12.36	83.38 ± 7.78	
✓		✓	83.47 ± 10.30	86.80 ± 10.14	80.12 ± 16.33	86.33 ± 11.03	
✓	✓	✓	87.38 ± 2.87	92.76 ± 4.00	88.35 ± 6.83	86.51 ± 8.41	
ABIDE B	✓			86.05 ± 5.30	92.88 ± 3.26	81.72 ± 14.25	88.84 ± 3.94	
		✓	86.29 ± 3.23	93.45 ± 1.70	82.21 ± 7.69	89.52 ± 3.66	
✓	✓		85.58 ± 4.29	91.78 ± 3.10	79.34 ± 8.07	89.92 ± 2.25	
✓		✓	86.51 ± 2.78	93.39 ± 2.38	83.53 ± 5.32	88.87 ± 4.12	
✓	✓	✓	88.09 ± 2.92	93.59 ± 2.45	87.58 ± 3.68	88.49 ± 4.73	
Best results are marked in bold.

For a fair comparison, all the baselines use the same splits of the dataset to carry out a 10-fold stratified cross-validation strategy. Bold font indicates the best results. We adopt a 2-layer MLP as a feature extractor, which produces features that can be directly fed into the classifier for diagnosis. The combination of the feature extractor and modal discriminator constitute our multimodal adversarial-regularized encoder. On the ABIDE A dataset, the encoder outperforms the base MLP. By incorporating a multi-graph fusion GNN module, our DeepASD model is improved to achieve optimal results.

Multiple modalities enable more accurate diagnosis

To assess the presence of confounding relationships between disease status and certain modalities of metadata, we first employed two-dimensional t-distributed stochastic neighbor embedding (t-SNE) [29] to visualize representations of each modality. It can be observed that there is no obvious clustering boundary of representations from raw features in the dataset (Fig. 3d left), while the representation generated by adversarial training of DeepASD is more discriminative, i.e., compact intra-class scatter and incompact inter-class scatter (Fig. 3d right). In addition, there is less overlapping across classes in each modality in Fig. 3c (top) than in Fig. 3e (bottom), indicating a better classification performance of the adversarial-regularized encoder in the DeepASD method. We also quantitatively compared the performance of DeepASD through the Silhouette score, and the score (0.5024) of DeepASD is much higher than that of in the raw data (0.4259), which validates the effectiveness of DeepASD in terms of representation learning.

Due to the complexity of the mechanism of ASD, multimodal data are able to provide more comprehensive information for diagnosis. Here, we compared the performance of DeepASD using each modal separately as inputs in the ABIDE A such as demographic information (PHENO), automated anatomical quality assessment metrics (ANAT), automated functional quality assessment metrics (FUNC), and FMRI. Notably, the model achieved the best performance when using the FMRI modal alone, while the performance was further improved if the other three modalities are combined as the input (Fig. 4a, b). Interestingly, this is consistent with the importance of each modality learned by DeepASD model during the modal fusion using weighted multimodal feature representations (Fig. 4d).Fig. 4 Visualization of modality importance.

a The performance of training DeepASD on the multi-modal data and single-modality (PHENO, ANAT, FUNC, and fMRI, respectively). b Classification performance using different combinations of the four modalities (PHENO, ANAT, FUNC, FMRI) on the ABIDE A. The classifiers that contain the fMRI modality significantly outperform those without the fMRI modality, highlighting the significance of fMRI in the diagnostic process. Additionally, we also observe that combining auxiliary modalities with fMRI is able to further enhance its performance in ASD diagnosis. c Classification performance using different combinations of the four modalities (PHENO, AAL, DOS, CC200) on the ABIDE B. Though the classification results using the three brain atlas fMRI (AAL, DOS, CC200) are promising, the results are further improved by adding the PHENO modality data. d, e Illustration of the weight of each modality (on the ABIDE A and ABIDE B, respectively) in multi-graph fusion. The results show that fMRI is the most significant contributor in both datasets, which aligns with the results shown in b and c.

We also conducted comparison experiments using each single modal and combinations of different modals in the ABIDE B as the inputs. It can be observed from Fig. 4c that DeepASD achieved a better performance for ASD diagnosis by combining the three brain atlases FMRI, and the DOS brain region is of importance for classifying ASD if only single modal was applied. Interestingly, it is also consistent with the learned importance of each modal in the fusion module (Fig. 4e) through DeepASD.

Discussion

DeepASD aims to integrate multimodal clinical data for precise ASD diagnosis. The multimodal data were first projected into an intermediate common dimensional subspace through a multimodal adversarial-regularized encoder, which aligned the distributions of minor modalities with that of the anchor modality (i.e., fMRI). Importantly, DeepASD allows characterizing the latent patient similarity network for depicting inter-patient relationships, which facilitates the ASD diagnosis.

DeepASD benefits from the multimodal adversarial-regularized encoder and graph learning modules (Section “The adversarial-regularized graph learning contributes to DeepASD”), and achieved the best classification accuracy in terms of the diagnostic task compared to six benchmark models. DeepASD showed improved performance when trained with multiple modalities compared to single modality, indicating that fMRI is significant for ASD diagnosis, in which the DOS brain atlas is the most important fMRI modality.

Although DeepASD achieved superb performance, we noted that the performance of DeepASD may be decreased using low dimensional data. In the ABIDE A data, the dimensions of PHENO, ANAT, and FUNC are less than 50, while the dimension of fMRI is more than 100. From Fig. 3e, we observed that the separability of the low dimensional data became worse when projecting the low dimensional data (i.e., PHENO, ANAT, and FUNC) into the common dimension together with the high-dimensional fMRI data. Therefore, a key challenge of multi-modality fusion is how to exploit the information in low-dimensional clinical data.

We observed that DeepASD could be more effective by mimicking the psychiatric diagnostic process and integrating clinical autism rating scales [30, 31] as a modality. The autism scales are the gold standard in clinical diagnosis based on the performance of the clinical examination such as psychiatric examination, physical examination, laboratory tests, etc.

Interestingly, it should be noted that the interpretability of DeepASD indicated the importance of each modality, i.e., we observed that fMRI is the most significant modality for ASD diagnosis. However, one of the limitations of DeepASD is generalized to present more fine-grained interpretations. Therefore, the future work may include the interpretability in terms of the contribution of each feature for ASD diagnosis as in GradCAM [32], Shapley values [33], and DeepLIFT [34].

Methods

DeepASD overview

DeepASD receives N patients data and each patient is associated with K modalities. Let X={xi}i=1N denotes the raw multimodal features of N patients, and Y={yi}i=1N denotes the corresponding labels of each patient. For patient i, the feature xi={xim1,xim2,⋯,ximk} is composed of K modalities, and we denote the modality by superscript mk. For example, ximk denotes the dmk-dimensional features of k-th modality of the i-th patient. As shown in Fig. 1, for each modality, we fed xmk into feature extractor fmk⋅ as input (Section “Feature extractor”). The fmk⋅ generates aligned feature representation in RN×dc through the multimodal adversarial-regularized encoder (Section “Multimodal adversarial-regularized encoder”), where dc represents the dimension of the aligned subspace. Then, in the multi-graph fusion GNN module (Section “Graph neural networks for multi-graph fusion”), we aggregate adjacency matrix {A}k=1K and node embeddings f⋅={fmkxmk}k=1K in the patient similarity network (generated by all modalities), in order to generate one fusion adjacency matrix A∈RN×N. Thus, a patient network G=V,E,X is built for ASD diagnosis, in which nodes represent patients V and Aij∈A denotes the edge weights of eij∈E. From the network, we employ Simple Spectral Graph Convolution (S2GC) [35] and a one-layer Multilayer Perceptron (MLP) to predict labels of nodes (i.e., the ASD prediction result of each patient).

Feature extractor

Since each modality has its specific characteristics and patterns, we design a modality-specific extractor for each modal that feds each modal data into the module and projects the samples in each modality into a dc-dimensional subspace, where we align the dimension of different modalities. Each feature extractor for each modal {fmk(⋅):Rdmk→Rdc}k=1K is consisted of two-layer fully connected networks with the Leaky ReLU activation function, following with a one-layer fully connected networks for classification.

Multimodal adversarial-regularized encoder

Adversarial networks [36, 37] have demonstrated effectiveness to align different data distributions. Since the features are heterogeneous across the modalities and each modality provides distinct information in terms of other modalities, we develop a multimodal adversarial-regularized encoder method to eliminate the feature heterogeneity and reduce the distributional divergence. As shown in Fig. 1b, we construct two competitive modules: the modal discriminator and the feature extractor. The modal discriminator d⋅ aims to distinguish the modality of features, while the feature extractor f⋅ attempts to against the former. By leveraging the adversarial learning manner, we are able to obtain aligned distributions from all modalities through training a competitive loss Ld,f (Eq. (1)) that minimizes over d⋅ but maximizes over f⋅.1 Ld,f=∑k=1K1N∑i=1NLsd(fmkximk,zimk.

where Ls⋅,⋅ is the squared loss, and zimk is the one-hot labels of ximk. In addition, there is a classification loss as shown in Eq. (2).2 Lf=∑k=1K1N∑i=1NLcfximk,yi+τ∥f∥2

where Lc⋅,⋅ is the cross-entropy loss and τ is a positive regularization parameter. The model tends to generate better discriminability via the feature extractor f⋅ by minimizing Lf. By combining two losses together, we have3 minfmaxdLf−βLd,f

where β is the trade-off parameter between the classification loss and the modal discriminator loss.

To alleviate the problem that the adversarial optimization on the Ld,f term may lead to vanishing gradients if f⋅ and g⋅ are not well synchronized, we adopt the invert label loss [38] as defined in Eq. (4):4 Lˆd,f=∑k=1K1N∑i=1NLsd(fmkximk,zˆimk.

where zˆimk is the one-hot inverted label of ximk in each modal. Thus, the objective in Eq. (5) can be reformulated as5 minfLf+βLˆd,f,mindLd,f

Graph neural networks for multi-graph fusion

For multimodal features learned from the multimodal adversarial-regularized encoder, we apply a learnable cosine similarity [39] method in Eq. (6) to learn an inductive patient similarity graph, as follows,6 Aij=WAfiTWAfj∥WAfi∥∥WAfj∥

where Aij is the learned similarity matrix between patient i and j, WA is the learned node embedding, fi and fj are the features from feature extractor {fmkximk}k=1K. We also employ a threshold θ to constrain the similarity strength between each node.

For each modality, we learn an adjacency matrix {A}k=1K to capture patient relationships (a.k.a., similarity) in different modality, and then we combine all patient graphs into one graph whose adjacency matrix is A by weighted sum operation such that A=∑k=1KwkAk. We obtain fused feature representations fˆ by concatenating the aligned features from {fmkximk}k=1K. Based on the learned graph structure A∈RN×N and fusion feature fˆ∈Rdc, we apply S2GC [35] and one-layer MLP for downstream ASD diagnosis task.

A spectral convolution of a graph signal x with a filter gθ is defined as gθ⋆x=UgθUTx where U is the matrix of eigenvectors of the normalized graph Laplacian L=I−D−12AD12 with respect to the diagonal degree matrix D. By the renormalization trick, we use a normalized version T~=D−12~AD12~=D+In−12A+InD+In−12 to replace the matrix I−D−12AD12 Motivated by Markov Diffusion Kernel [40], S2GC includes self-loops and its final output can be defined as follows,7 Yˆ=softmax1C∑c=1C1−αT~cX+αXW

where W represents the network parameter, and α is a trade-off between the self-information of a node and its consecutive neighborhoods. To this end, α is typically set to 0.05 with a range of values between 0 and 1 in the experiments. The term T~cX is computed as T~⋅(T~⋅(⋯(T~X)⋯)), where the multiplication is iteratively applied C times.

Then, we build a final classifier to identify ASD. The classifier is composed of two-layer graph convolution layers with ReLU activation, followed by a fully connected layer. The loss function for this graph learning classification is given by Eq. (8)8 Lg=LcgA,fˆ,y

where Lc⋅,⋅ is the cross-entropy loss, g⋅,⋅ is the GCN model, y is the one-hot label of each patient data.

By combining all the aforementioned losses, we adopt the following joint loss function to guide the optimization of all modules simultaneously:9 L=Lg+ηLf+βLˆd,f

where η and β are hyper-parameters to balance the loss terms.

Methodologies for training and testing

We first competitively train feature extractor f⋅ and modal discriminator d⋅ once, and then jointly train the graph construction embedding WA and GCN g⋅,⋅ once in one training epoch. Through this training strategy, we are able to simultaneously obtain significant patient representations and patient similarity graphs with high prediction accuracy.

After selecting hyper-parameters through preliminary experiments, we optimize f⋅, d⋅, WA, g⋅,⋅ via the Adam optimizers [41] with learning rates of 0.004, 0.001, 0.001, 0.001, respectively. We empirically set β=0.03, τ=0.004, and η=1, and the other hyper-parameters are fine-tuned according to the dataset size.

We implement the proposed DeepASD using PyTorch. All experiments were conducted with a 10-fold cross-validation to divide the dataset into training and test sets, with 10% of the training set randomly selected as the validation set. Ultimately, the training, validation, and test sets were non-overlapping. The training of the model for 500 epochs on ABIDE A and ABIDE B datasets, utilizing a single Tesla V100 GPU, took approximately 15 min and 45 min, respectively.

Supplementary information

Supplementary File

Figure D1

Supplementary information

The online version contains supplementary material available at 10.1038/s41398-024-02972-2.

Acknowledgements

This work is supported by the National Key R&D Program of China (2022ZD0160703), the National Natural Science Foundation of China (62202422), the Zhejiang Province Science and Technology Plan Project (2022C03134), the Shanghai Key Laboratory Open Project (STCSM 22DZ2229005), and Natural Science Foundation of Shandong Province (ZR2021MH227).

Author contributions

WC conceived and designed the study, performed data analysis, and wrote the manuscript. JY provided critical clinical insights and reviewed the manuscript. ZS contributed to data interpretation and critical revision of the manuscript. XZ contributed to study design and contributed to manuscript writing. GT provided clinical expertise and reviewed the manuscript. YD provided critical clinical insights and reviewed the manuscript. JB helped perform the analysis with constructive discussions. JG contributed computational resources (e.g., high-performance GPUs). HW conceived and designed the study, revised the manuscript critically for important intellectual content, and supervised the overall project.

Data availability

The ABIDE dataset is available from ref. [21], and the preprocessed dataset ABIDE A and ABIDE B are provided with the paper.

Code availability

An interactive code base for DeepASD implementation is available at https://codeocean.com/capsule/6043886/tree.

Competing interests

The authors declare no competing interests.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Wanyi Chen, Jianjun Yang, Zhongquan Sun.
==== Refs
References

1. Lever AG Geurts HM Psychiatric co-occurring symptoms and disorders in young, middle-aged, and older adults with autism spectrum disorder J Autism Dev Disord 2016 46 1916 30 10.1007/s10803-016-2722-8 26861713
Lever AG, Geurts HM. Psychiatric co-occurring symptoms and disorders in young, middle-aged, and older adults with autism spectrum disorder. J Autism Dev Disord. 2016;46:1916–30.26861713 10.1007/s10803-016-2722-8
2. Megerian JT Dey S Melmed RD Coury DL Lerner M Nicholls CJ Evaluation of an artificial intelligence-based medical device for diagnosis of autism spectrum disorder NPJ Digital Med 2022 5 1 11 10.1038/s41746-022-00598-6
Megerian JT, Dey S, Melmed RD, Coury DL, Lerner M, Nicholls CJ, et al. Evaluation of an artificial intelligence-based medical device for diagnosis of autism spectrum disorder. NPJ Digital Med. 2022;5:1–11.10.1038/s41746-022-00598-6
3. Wang H Avillach P Diagnostic classification and prognostic prediction using common genetic variants in autism spectrum disorder: Genotype-based deep learning JMIR Med Inform 2021 9 e24754 10.2196/24754 33714937
Wang H, Avillach P. Diagnostic classification and prognostic prediction using common genetic variants in autism spectrum disorder: Genotype-based deep learning. JMIR Med Inform. 2021;9:e24754.33714937 10.2196/24754
4. Wang H, Li L, Chi L, Zhao Z. Autism screening using deep embedding representation. In: Computational science–ICCS 2019: 19th international conference, faro, Portugal, June 12–14, 2019, proceedings, part II 19. Springer; 2019. p. 160–73.
5. Boxhoorn S Schütz M Mühlherr AM Mössinger H Luckhardt C Freitag CM The effect of perceptual expectation on processing gain, attention and the perceptual decision bias in children and adolescents with autism spectrum disorder (ASD) Sci Rep 2022 12 1 14 10.1038/s41598-022-25971-z 34992227
Boxhoorn S, Schütz M, Mühlherr AM, Mössinger H, Luckhardt C, Freitag CM. The effect of perceptual expectation on processing gain, attention and the perceptual decision bias in children and adolescents with autism spectrum disorder (ASD). Sci Rep. 2022;12:1–14.34992227 10.1038/s41598-022-25971-z
6. Wang H, Chi L, Su C, Zhao Z. ASDFace: Face-based autism diagnosis via heterogeneous domain adaptation. In: Proceedings of the 31st ACM international conference on information & knowledge management. 2022, pp 4999–5003.
7. Maenner MJ Warren Z Williams AR Amoakohene E Bakian AV Bilder DA Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2020 MMWR Surveill Summ 2023 72 1 10.15585/mmwr.ss7202a1 36952289
Maenner MJ, Warren Z, Williams AR, Amoakohene E, Bakian AV, Bilder DA, et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2020. MMWR Surveill Summ. 2023;72:1.36952289 10.15585/mmwr.ss7202a1
8. Sharma SR Gonda X Tarazi FI Autism spectrum disorder: classification, diagnosis and therapy Pharmacol Ther 2018 190 91 104 10.1016/j.pharmthera.2018.05.007 29763648
Sharma SR, Gonda X, Tarazi FI. Autism spectrum disorder: classification, diagnosis and therapy. Pharmacol Ther. 2018;190:91–104.29763648 10.1016/j.pharmthera.2018.05.007
9. Yao Z Hu B Xie Y Moore P Zheng J A review of structural and functional brain networks: small world and atlas Brain Inform 2015 2 45 52 10.1007/s40708-015-0009-z 27747502
Yao Z, Hu B, Xie Y, Moore P, Zheng J. A review of structural and functional brain networks: small world and atlas. Brain Inform. 2015;2:45–52.27747502 10.1007/s40708-015-0009-z
10. Wang Y Wang J Wu F-X Hayrat R Liu J AIMAFE: Autism spectrum disorder identification with multi-atlas deep feature representation and ensemble learning J Neurosci Methods 2020 343 108840 10.1016/j.jneumeth.2020.108840 32653384
Wang Y, Wang J, Wu F-X, Hayrat R, Liu J. AIMAFE: Autism spectrum disorder identification with multi-atlas deep feature representation and ensemble learning. J Neurosci Methods. 2020;343:108840.32653384 10.1016/j.jneumeth.2020.108840
11. Wang Y Liu J Xiang Y Wang J Chen Q Chong J MAGE: automatic diagnosis of autism spectrum disorders using multi-atlas graph convolutional networks and ensemble learning Neurocomputing 2022 469 346 53 10.1016/j.neucom.2020.06.152
Wang Y, Liu J, Xiang Y, Wang J, Chen Q, Chong J. MAGE: automatic diagnosis of autism spectrum disorders using multi-atlas graph convolutional networks and ensemble learning. Neurocomputing. 2022;469:346–53.10.1016/j.neucom.2020.06.152
12. Lynch CJ Liston C New machine-learning technologies for computer-aided diagnosis Nat Med 2018 24 1304 5 10.1038/s41591-018-0178-4 30177823
Lynch CJ, Liston C. New machine-learning technologies for computer-aided diagnosis. Nat Med. 2018;24:1304–5.30177823 10.1038/s41591-018-0178-4
13. Wang H, Chi L, Zhao Z. ASDPred: an end-to-end autism screening framework using few-shot learning. In: Proceedings of the 31st ACM international conference on information & knowledge management. 2022, p. 5004–8.
14. Song X Zhou F Frangi AF Cao J Xiao X Lei Y Multi-center and multi-channel pooling GCN for early AD diagnosis based on dual-modality fused brain network IEEE Trans Med Imaging 2022 42 354 67 10.1109/TMI.2022.3187141
Song X, Zhou F, Frangi AF, Cao J, Xiao X, Lei Y. et al. Multi-center and multi-channel pooling GCN for early AD diagnosis based on dual-modality fused brain network. IEEE Trans Med Imaging. 2022;42:354–67.10.1109/TMI.2022.3187141
15. Osorio D Interpretable multi-modal data integration Nat Comput Sci 2022 2 8 9 10.1038/s43588-021-00186-w 38177710
Osorio D. Interpretable multi-modal data integration. Nat Comput Sci. 2022;2:8–9.38177710 10.1038/s43588-021-00186-w
16. Kazi A, Shekarforoush S, Arvind Krishna S, Burwinkel H, Vivar G, Kortüm K, et al. InceptionGCN: receptive field aware graph convolutional network for disease prediction. In: International conference on information processing in medical imaging. Springer; 2019. p. 73–85.
17. Li X, Dvornek N, Zhuang J, Ventola P, Duncan J. Graph embedding using infomax for ASD classification and brain functional difference detection. In: Proceedings of spie–the international society for optical engineering. 2020. p. 1131702–1131702.
18. Yang T, Al-Duailij MA, Bozdag S, Saeed F. Classification of autism spectrum disorder using rs-fMRI data and graph convolutional networks. In: 2022 IEEE international conference on big data (big data). IEEE; 2022. p. 3131–8.
19. Parisot S, Ktena SI, Ferrante E, Lee M, Moreno RG, Glocker B, et al. Spectral graph convolutions for population-based disease prediction. In: International conference on medical image computing and computer-assisted intervention. Springer; 2017. p. 177–85.
20. Wiggins LD Durkin M Esler A Lee L-C Zahorodny W Rice C Disparities in documented diagnoses of autism spectrum disorder based on demographic, individual, and service factors Autism Res 2020 13 464 73 10.1002/aur.2255 31868321
Wiggins LD, Durkin M, Esler A, Lee L-C, Zahorodny W, Rice C, et al. Disparities in documented diagnoses of autism spectrum disorder based on demographic, individual, and service factors. Autism Res. 2020;13:464–73.31868321 10.1002/aur.2255
21. Di Martino A Yan C-G Li Q Denio E Castellanos FX Alaerts K The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism Mol Psychiatry 2014 19 659 67 10.1038/mp.2013.78 23774715
Di Martino A, Yan C-G, Li Q, Denio E, Castellanos FX, Alaerts K, et al. The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol Psychiatry. 2014;19:659–67.23774715 10.1038/mp.2013.78
22. Craddock C Benhajali Y Chu C Chouinard F Evans A Jakab A The Neuro Bureau preprocessing initiative: open sharing of preprocessed neuroimaging data and derivatives Front Neuroinform 2013 7 27 24319427
Craddock C, Benhajali Y, Chu C, Chouinard F, Evans A, Jakab A, et al. The Neuro Bureau preprocessing initiative: open sharing of preprocessed neuroimaging data and derivatives. Front Neuroinform. 2013;7:27.24319427
23. Chen Y Ouyang L Bao FS Li Q Han L Zhang H A multimodality machine learning approach to differentiate severe and nonsevere COVID-19: model development and validation J Med Internet Res 2021 23 e23948 10.2196/23948 33714935
Chen Y, Ouyang L, Bao FS, Li Q, Han L, Zhang H, et al. A multimodality machine learning approach to differentiate severe and nonsevere COVID-19: model development and validation. J Med Internet Res. 2021;23:e23948.33714935 10.2196/23948
24. Fetit AE Doney AS Hogg S Wang R MacGillivray T Wardlaw JM A multimodal approach to cardiovascular risk stratification in patients with type 2 diabetes incorporating retinal, genomic and clinical features Sci Rep 2019 9 3591 10.1038/s41598-019-40403-1 30837638
Fetit AE, Doney AS, Hogg S, Wang R, MacGillivray T, Wardlaw JM, et al. A multimodal approach to cardiovascular risk stratification in patients with type 2 diabetes incorporating retinal, genomic and clinical features. Sci Rep. 2019;9:3591.30837638 10.1038/s41598-019-40403-1
25. Heinsfeld AS Franco AR Craddock RC Buchweitz A Meneguzzi F Identification of autism spectrum disorder using deep learning and the ABIDE dataset NeuroImage Clin 2018 17 16 23 10.1016/j.nicl.2017.08.017 29034163
Heinsfeld AS, Franco AR, Craddock RC, Buchweitz A, Meneguzzi F. Identification of autism spectrum disorder using deep learning and the ABIDE dataset. NeuroImage Clin. 2018;17:16–23.29034163 10.1016/j.nicl.2017.08.017
26. Eslami T Mirjalili V Fong A Laird AR Saeed F ASD-DiagNet: a hybrid learning approach for detection of autism spectrum disorder using fMRI data Front Neuroinform 2019 13 70 10.3389/fninf.2019.00070 31827430
Eslami T, Mirjalili V, Fong A, Laird AR, Saeed F. ASD-DiagNet: a hybrid learning approach for detection of autism spectrum disorder using fMRI data. Front Neuroinform. 2019;13:70.31827430 10.3389/fninf.2019.00070
27. Vale-Silva LA Rohr K Long-term cancer survival prediction using multimodal deep learning Sci Rep 2021 11 13505 10.1038/s41598-021-92799-4 34188098
Vale-Silva LA, Rohr K. Long-term cancer survival prediction using multimodal deep learning. Sci Rep. 2021;11:13505.34188098 10.1038/s41598-021-92799-4
28. Nguyen ND Huang J Wang D A deep manifold-regularized learning model for improving phenotype prediction from multi-modal data Nat Comput Sci 2022 2 38 46 10.1038/s43588-021-00185-x 35480297
Nguyen ND, Huang J, Wang D. A deep manifold-regularized learning model for improving phenotype prediction from multi-modal data. Nat Comput Sci. 2022;2:38–46.35480297 10.1038/s43588-021-00185-x
29. Van der Maaten L Hinton G Visualizing data using t-SNE J Mach Learn Res 2008 9 2579 605
Van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9:2579–605.
30. Rellini E Tortolani D Trillo S Carbone S Montecchi F Childhood autism rating scale (CARS) and autism behavior checklist (ABC) correspondence and conflicts with DSM-IV criteria in diagnosis of autism J Autism Dev Disord 2004 34 703 8 10.1007/s10803-004-5290-2 15679189
Rellini E, Tortolani D, Trillo S, Carbone S, Montecchi F. Childhood autism rating scale (CARS) and autism behavior checklist (ABC) correspondence and conflicts with DSM-IV criteria in diagnosis of autism. J Autism Dev Disord. 2004;34:703–8.15679189 10.1007/s10803-004-5290-2
31. Rodgers J Wigham S McConachie H Freeston M Honey E Parr JR Development of the anxiety scale for children with autism spectrum disorder (ASC-ASD) Autism Res 2016 9 1205 15 10.1002/aur.1603 26887910
Rodgers J, Wigham S, McConachie H, Freeston M, Honey E, Parr JR. Development of the anxiety scale for children with autism spectrum disorder (ASC-ASD). Autism Res. 2016;9:1205–15.26887910 10.1002/aur.1603
32. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision. 2017. p. 618–26.
33. Sundararajan M, Najmi A. The many shapley values for model explanation. In: International conference on machine learning. PMLR, 2020. p. 9269–78.
34. Shrikumar A, Greenside P, Kundaje A. Learning important features through propagating activation differences. In: International conference on machine learning. PMLR, 2017. p. 3145–53.
35. Zhu H, Koniusz P. Simple spectral graph convolution. In: International conference on learning representations. 2020.
36. Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation. In: International conference on machine learning. PMLR, 2015. p. 1180–9.
37. Wang H Tao G Ma J Jia S Chi L Yang H Predicting the epidemics trend of COVID-19 using epidemiological-based generative adversarial networks IEEE J Sel Top Signal Process 2022 16 276 88 10.1109/JSTSP.2022.3152375
Wang H, Tao G, Ma J, Jia S, Chi L, Yang H, et al. Predicting the epidemics trend of COVID-19 using epidemiological-based generative adversarial networks. IEEE J Sel Top Signal Process. 2022;16:276–88.10.1109/JSTSP.2022.3152375
38. Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. p. 7167–76.
39. Nguyen HV, Bai L. Cosine similarity metric learning for face verification. In: Asian conference on computer vision. 2010. p. 709–20.
40. Fouss F Francoisse K Yen L Pirotte A Saerens M An experimental investigation of kernels on graphs for collaborative recommendation and semisupervised classification Neural Netw 2012 31 53 72 10.1016/j.neunet.2012.03.001 22497802
Fouss F, Francoisse K, Yen L, Pirotte A, Saerens M. An experimental investigation of kernels on graphs for collaborative recommendation and semisupervised classification. Neural Netw. 2012;31:53–72.22497802 10.1016/j.neunet.2012.03.001
41. Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv preprint arXiv:14126980 2014.
