==== Front BMC Bioinformatics BMC Bioinformatics BMC Bioinformatics 1471-2105 BioMed Central London 5373 10.1186/s12859-023-05373-2 Research Comparing methods for drug–gene interaction prediction on the biomedical literature knowledge graph: performance versus explainability Aisopos Fotis fotis.aisopos@iit.demokritos.gr Paliouras Georgios paliourg@iit.demokritos.gr grid.6083.d 0000 0004 0635 6999 Institute of Informatics and Telecommunications, National Centre for Scientific Research Demokritos, Athens, Greece 30 6 2023 30 6 2023 2023 24 2727 4 2023 1 6 2023 © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/ Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. This paper applies different link prediction methods on a knowledge graph generated from biomedical literature, with the aim to compare their ability to identify unknown drug-gene interactions and explain their predictions. Identifying novel drug–target interactions is a crucial step in drug discovery and repurposing. One approach to this problem is to predict missing links between drug and gene nodes, in a graph that contains relevant biomedical knowledge. Such a knowledge graph can be extracted from biomedical literature, using text mining tools. In this work, we compare state-of-the-art graph embedding approaches and contextual path analysis on the interaction prediction task. The comparison reveals a trade-off between predictive accuracy and explainability of predictions. Focusing on explainability, we train a decision tree on model predictions and show how it can aid the understanding of the prediction process. We further test the methods on a drug repurposing task and validate the predicted interactions against external databases, with very encouraging results. Keywords Drug–target interaction Knowledge graph Graph embeddings Deep learning issue-copyright-statement© BioMed Central Ltd., part of Springer Nature 2023 ==== Body pmcIntroduction Drug development is an extremely time-consuming and expensive process with low rates of new therapeutic discoveries [1]. However, knowing targets of potential clinical significance, can play a crucial role in the process of rational drug development, as candidate compounds targeting specific proteins may be employed to achieve intended therapeutic effects (target-based drug discovery) [2]. Hence, drug–target interaction (DTI) identification has become one of the hottest research topics in the medical and pharmaceutical industry. ‘Targets’ either refer to specific proteins or related genes that have been associated with a disease. By identifying DTIs, researchers can choose pharmaceutical substances to be tested on “target” genes of interest, in the context of a clinical trial. However, a drug rarely interacts with a single known target [3], a phenomenon that may lead to unwanted adverse effects, but also to opportunities for drug re-purposing. Therefore, the prediction of novel, unknown drug–target interactions emerges as a useful task in drug (re)discovery, as well as other related fields. Identified interactions for commercial drugs have been documented in various online databases and repositories, such as KEGG, DrugBank, ChEMBL, STITCH and others. Despite major overlaps, the DTIs existing in those repositories are neither identical, nor complete, as new, previously unknown DTIs are continuously being added. These additions may be the result of genetic [4] or proteomic research [5], or they can be generated by computational methods. While in-vivo experiments provide the ultimate validation of such interactions, genetic and proteomic validation methods are also considered adequate. However, only a limited number of DTIs can be tested in a ‘wet lab’ experiment, making such experiments impractical in terms of time and resources needed. Therefore, data analytics methods for DTI prediction can make the discovery process more efficient, by minimising the list of target candidates to be tested in a ‘wet lab’ experiment (Fig. 1).Fig. 1 Overview of the target-based drug discovery process facilitated by computational methods. Candidate molecular targets are identified by data analytics and validated before lead discovery starts; assays and screens are then used to find a lead Several recent publications (e.g. [2, 6, 7]) focus on the exploitation of Knowledge Graphs for the task of DTI prediction. In particular, this paper proposes the use of a disease-specific Knowledge Graph, generated automatically from the scientific literature, in order to identify highly probable, but yet undocumented DTIs. For this purpose, a benchmark if first created, aggregating disease-specific DTIs that are documented in existing databases. Then, a variety of machine learning methods are being tested on the benchmark, striving for a balance between predictive accuracy and explainability of predictions. The latter is particularly important, when justifying the need for a resource-intensive ‘wet lab’ experiment. Thus, the main contributions of this work are summarized in the following points:A methodology and software to cross-reference drug and target identifiers across different databases and open vocabularies. Using this methodology, a benchmark is created for Lung Cancer, providing drug-gene interactions, drawn from different online repositories (DrugBank, KEGG, TTD, DGIdb). The benchmark is made publicly available.1 A comparative analysis of state-of-the-art link prediction approaches for predicting DTIs on a disease-specific Knowledge Graph derived from the biomedical literature. In addition to existing methods, we introduce an explainable path analysis technique, and compare against various other methods. The top-50 predictions of unknown DTIs produced by the best methods are cross-validated against online repositories, with very encouraging results. The rest of the document is organised as follows:  “Related work” section presents recent work on link prediction in biomedical Knowledge Graphs, and explainability of such approaches. “Generating a knowledge graph from biomedical literature” section briefly describes the Biomedical Literature Knowledge Graph construction pipeline, while  “Methods” section analyses the various approaches being assessed, including a new path-based discovery method (BLGPA). Lastly, “Evaluation” section evaluates the performance of different approaches and discusses explainability of the predictions, while  “Drug rediscovery test” section presents a drug re-positioning test for Lung Cancer, identifying and cross-checking unknown disease-specific DTIs. Finally, “Conclusions” section summarizes the presented work, indicating open research issues. Related work There is rich literature related to computational methods for the prediction of drug–target interactions. Recent work mainly uses deep learning techniques on drug or protein descriptors (e.g. [8–10]), yielding encouraging results. Focusing on approaches that treat DTI prediction as a link prediction problem in Knowledge Graphs, we distinguish the following categories of methods. DTI prediction on biomedical knowledge graphs build from open databases SemEP [11] and esDSG [12] adopt similar approaches, extracting certain relations (i.e. drug–target interactions, drug-drug and target-target similarities) from open ontologies to produce a low-dimensional graph, and then apply edge-based community detection. Those approaches employ edge partitioning for community detection to identify missing drug–target protein interaction links. The proposed methods are validated either on benchmark datasets (Ding et al. [13]) or by using external DTI databases (STITCH, KEGG) as gold standards. On the other hand, Mohamed et al. [2] present an approach that constructs a KG from various chemical and protein-related databases (Drugbank, KEGG, InterPro, Uniprot). Using this KG, they propose a graph embedding model (TriModel) to learn vector representations for drugs and targets. Trimodel is a tensor factorization method, based on DistMult and ComplEx, and can be used to predict interaction links between drugs and proteins. The prediction process uses confidence scores that are calculated directly by the embedding models’ scoring function. Similarly, Ye et al. [7] propose a framework called KGE_NFM that combines Knowledge Graph embeddings and recommendation system techniques for DTI predictions. The authors emphasize the cold start problem of new protein targets being identified for a complicated disease. KGE_NFM captures the latent information from KGs using graph embeddings (KGE) and then applies a neural factorization machine (NFM) to extract features for the DTI prediction task. Graph convolutional networks Relational Graph Convolutional Networks (RGCNs) [14] extend Graph Convolutional Networks to deal with the data of heterogeneous Knowledge Graphs that contain various relation types [15]. Following such an approach, Zitnik et al. [16] present a method called Decagon that models polypharmacy side effects in a drug–protein Knowledge Graph. The use case examined in that paper focuses mainly on drug-drug interactions, but it can be extended to DTIs. An inductive version of RGCN (I-RGCN) is proposed in [17] and is used to discover potential treatments for COVID-19. The method is applied on a biological Knowledge Graph, termed Drug Repurposing Knowledge Graph (DRKG), including information from six existing databases (DrugBank, Hetionet, GNBR, String, IntAct and DGIdb). Link prediction on biomedical literature graphs The method that is closest to the work presented in this paper is SemaTyP [1], which analyses PubMed articles using SemRep [18] and extracts semantic features from the resulting Knowledge Graph paths. The goal of SemaTyP is to predict interactions between entity triplets (drug–protein-disease). Triples are generated and scored against TTD2 that is used as a gold standard. The best scoring triples are then selected for Drug Discovery. The successor of SemaTyP, GrEDeL [6], constructs a similar KG, and then calculates the TransE embeddings of all entities/relations along each path, using them to train an LSTM network model, in order to evaluate candidate DTI pairs. In our earlier work, we have also proposed a method, called DDI-BLKG[19], which uses simple path analysis (frequent relational chains), in order to identify previously unknown drug-drug interactions. In the experimental comparison of this paper we include an extended version of DDI-BLKG, called BLGPA, focusing on DTI prediction. Link prediction explainability As mentioned in “Introduction” section, for any prediction to find its way to the wet lab, trust by health science researchers is essential. Therefore, statistical and traditional machine learning methods based on rules [20] and using features that make sense to humans [21] have an advantage over complex black-box models. For the more complex approaches, explainability techniques such as LIME (Local Interpretable Model-Agnostic Explanation) [22] are needed, in order to gain an understanding of which features are used and how for making a decision. However, graph embeddings model the nodes and links of KGs with derived features that are hard to interpret, thus requiring special explainability approaches. Various methods have been proposed to explain the link predictions of an RGCN, with limited success, regarding the quality of explanations, i.e. how understandable they are in fact by the prediction consumers. The most prominent one is ExplaiNE [23], which quantifies how the predicted probability of a link changes when weakening or removing a link with a neighboring node. One the other hand, GNNExplainer [24] explains the predictions of any GNN by learning a mask over the adjacency matrix, in order to identify an informative subgraph, together with a small subset of node features, that are most influential for the prediction. Halliwell [25] recently attempted to evaluate such explainability techniques, proposing the use of a scoring metric on benchmark datasets, which allows comparison of their explanations. Methods Generating a knowledge graph from biomedical literature MEDLINE and PubMed are the most popular sources of biomedical knowledge, offering more than 28 million citations. PubMed supports semantic retrieval based on manually curated topic annotations using the MeSH vocabulary.3 Additionally, PubMed Central (PMC) provides the full text of about 4.7 million articles. In this work, we aim at generating disease-specific Knowledge Graphs from biomedical literature, exploiting publicly available tools to retrieve relevant article abstracts as well as their full text when available.Fig. 2 Overview of the process of harvesting and analysing biomedical literature text, using Natural Language Processing tools As shown in Fig. 2, all articles relevant to a particular disease (e.g. Lung Cancer) are harvested through appropriate semantic queries to PubMed and the title, abstract and topic annotations for each article are extracted [26]. In addition, PMC is queried and any available full text for the articles is also retrieved. The textual information collected in this manner is then mined to identify semantic entities and their relations. The basis for the conceptual modeling of information is the Unified Medical Language System (UMLS) vocabulary [27], and in particular the Metathesaurus [28], which links differing expressions of the same biomedical meaning under a single concept, and the Semantic Network [29], which enriches the concepts, grouping them into 133 semantic types, such as diseases and chemicals, and defines semantic relations between those types. In order to harness the power of this representation, SemRep  [18] is being employed, a UMLS-based tool that extracts biomedical predications, i.e. semantic triples in the form of subject-predicate-object, from unstructured text. The subject and object arguments of these predications are concepts from the UMLS and the predicate is one of the semantic relations of the Semantic Network, connecting the semantic types of the subject and object, in the context of the specific sentence (Literature mining process of Fig. 2). In addition to the relations defined by the Semantic Network, we create two new types, which we call “MENTIONED_IN” and “HAS_MESH”. The “MENTIONED_IN” relation expresses the occurrence of a concept in an article, as can be seen in Fig. 2. The motivation behind this new relation is that it allows us to take into account the co-occurrence of concepts in the documents. Co-occurrence provides a way of measuring the association between two terms and potentially helps in targeting interesting associations (for example between medications and disorders). Articles also appear in the knowledge graph as nodes connected with concepts through “MENTIONED_IN” edges. On the other hand, the “HAS_MESH” predicate allows us to capture the semantic information generated by the curators of biomedical articles. Each article in PubMed is associated with a set of MeSH tags and these associations are encoded in the graph in the form of “article-HAS MESH-concept” triples. These triples express the topics to which each article is connected, according to human curators, hence providing rather robust knowledge. Beyond literature, ontologies and databases are other important sources of domain knowledge in the field of biomedicine. In order to enrich the information in the disease-specific KG, we have developed a harvester to integrate taxonomic relations (is_a) from ontologies available in the Open Biomedical Ontologies (OBO) format [30] (e.g. Gene Ontology, Disease Ontology). The resulting disease-specific Knowledge Graph is stored in a Neo4j4 graph database, capturing naturally the connectivity between entities and facilitating efficient retrieval and analysis. Predicting drug target interaction links Using the multi-relational disease-specific Knowledge Graph (KG) created as explained in “Generating a knowledge graph from biomedical literature” section, we aim to infer probable interactions between drugs and potential targets. Specifically, given a pair of graph nodes, corresponding to a drug and a target gene, drug–target interaction prediction can be formulated as a link prediction problem. The prediction task is modeled as a supervised learning problem, where data samples are generated from node pairs in the KG and the goal is to determine which node pairs should be linked with an interaction relation. In this section, we present the different KG link prediction methods that we have included in our experiments. They range from simple, but explainable path classifiers to powerful, but complex Deep Learning (DL) techniques, such as Graph Convolutional Networks. Note that all methods make a closed world assumption (CWA), where a non-existing triple in the training data is interpreted as a negative interaction, i.e., the corresponding relationship is considered false [31]. Rule-based methods Rule-based approaches for Knowledge Graph link prediction have an important advantage, which is their ability to provide an explanation in terms of the rules used in a prediction. From this family of methods, we have chosen to use Anytime Bottom Up Rule Learning (AnyBURL) [20], an efficient approach to learning logical rules from large Knowledge Graphs. AnyBURL generalizes positive examples, which take the form of paths between drugs and their targets, to generate rules that are applicable to other cases and can infer missing graph links. The approach is similar to the path ranking algorithm (PRA) [32], learning path rules and ranking them according to an estimated level of confidence. In the context of this work, AnyBURL has been configured to provide the confidence score for each candidate interacting pair of nodes. Semantic path analysis As mentioned in “Related work” section, Semantic Type Path (SemaTyP) is another explainable technique for DTI prediction. SemaTyP analyses a biomedical Knowledge Graph, called SemKG, produced from all abstracts of PubMed posted before June 2013, in order to address the following tasks: Classify all paths from Drug nodes passing through Target nodes to end at Disease nodes, (DTD paths). The goal is to identify the paths with interacting/associated (drug–target-disease) triples. Rediscover drugs that are suitable for a disease, based on the DTD path scores. The DTD path scoring task entails the DTI task examined in the current work, under certain assumptions. Specifically, for the training data, SemaTyP focuses on paths of the form πl=ρ(drugi⇒diseasei;targeti,l); denoting paths of length l, reaching node diseasei from source node drugi, over node targeti. A set of features are extracted from each path, by aggregating the occurrences of subject/object node semantic types and the types of different relations along the path. A logistic regression classifier is then employed, learning to discriminate positive pair paths from the negative ones. A hybrid technique: BLGPA In our earlier work, we presented a semantic path encoding method akin to SemaTyP, called DDI-BLKG. Here, we extend that method with a simple Path Ranking approach, leading to a hybrid link prediction technique, which we call Biomedical Literature Graph Path Analysis (BLGPA). BLGPA is trained on data samples generated by aggregating all possible paths between a specific drug-gene pair of nodes. More formally, let V=v1,v2,...,vN be the set of nodes of the graph, consisting of the biomedical entities, mapped to UMLS concepts, along with the articles they were mentioned in. Additionally, let R=r1,r2,...,rM denote the set of relations in the graph and (d, t) a drug-gene pair under examination. Also, let πl be a single path of length l connecting d and t. The path πl consists of a sequence of nodes and relations starting from node d and ending at node t: (d,r0,v1,r1,v2,r2,...,rl-1,t). Let Πd,t={π1l,π2l,...,πN(d,t)l} be the set of all possible paths of length up to l between the examined pair of nodes, as illustrated in Fig. 3. Each path πil∈Πd,t is processed in order to extract two types of features:Semantic Encoding (SE) Path Ranking (PR) Semantic Encoding - SE. Given a path πil=(d,r0,v1,r1,v2,r2,...,rl-1,t) of length l between a drug d and a gene t, a straightforward feature representation is to encode the involved node and relation types. MetaMap uses 127 different semantic types5ST={st1,st2,...,st127}, in its entity recognition process, while from the UMLS Semantic Network6 we have used the 35 most relevant and expressive semantic relations SR={sr1,sr2,...,sr35}. In order to preserve the order in which node and relation types appear in a graph path, we generate 127+35=162 features for every possible position j in the path sequence πil: fvSE,j={nodjst1,nodjst2,...,nodjst127,reljsr1,reljsr2,...,reljsr35}, where nodjstx denotes one of the 127 possible node types and reljsry one of the 35 relation types. Each feature captures the frequency of a semantic type or a relation at a specific position in the path, i.e. the number of different articles from which the corresponding triplet with those types has been extracted. Note that the subject/object nodes of the same triplet may be attributed to different semantic types in different articles. Given a maximum path length l, the full SE feature vector of a given path will have a length of: |fvSE|=l×162, ignoring the semantic type of the final node, which is always ‘gngm’: Gene or Genome. Given the above, for a specific path πil the corresponding SE feature vector will be: fiSE(d,t)=‖n=ln=0[cst1,cst2,...,cst127,csr1,csr2,...,csr35]T where csti, csri denote the frequency count of the specific semantic type and semantic relation respectively, and ‖ denotes the concatenation of these occurrence vectors for each transition in the path. For paths π(d,t)n of length n