
==== Front
Brief Bioinform
Brief Bioinform
bib
Briefings in Bioinformatics
1467-5463
1477-4054
Oxford University Press

10.1093/bib/bbae438
bbae438
Problem Solving Protocol
AcademicSubjects/SCI01060
Adapting differential molecular representation with hierarchical prompts for multi-label property prediction
https://orcid.org/0009-0008-2971-7087
Kang Linjia College of Informatics, Huazhong Agricultural University, Wuhan, Hubei 430070, China

https://orcid.org/0009-0008-0276-2590
Zhou Songhua College of Informatics, Huazhong Agricultural University, Wuhan, Hubei 430070, China

https://orcid.org/0009-0003-1462-5124
Fang Shuyan College of Informatics, Huazhong Agricultural University, Wuhan, Hubei 430070, China

https://orcid.org/0000-0001-7217-4462
Liu Shichao College of Informatics, Huazhong Agricultural University, Wuhan, Hubei 430070, China

Corresponding author. E-mail: scliu@mail.hzau.edu.cn
The authors wish it to be known that, in their opinion, the first two authors should be regarded as Joint First Authors.

9 2024
09 9 2024
09 9 2024
25 5 bbae43828 5 2024
05 8 2024
21 8 2024
© The Author(s) 2024. Published by Oxford University Press.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com

Abstract

Accurate prediction of molecular properties is crucial in drug discovery. Traditional methods often overlook that real-world molecules typically exhibit multiple property labels with complex correlations. To this end, we propose a novel framework, HiPM, which stands for Hierarchical Prompted Molecular representation learning framework. HiPM leverages task-aware prompts to enhance the differential expression of tasks in molecular representations and mitigate negative transfer caused by conflicts in individual task information. Our framework comprises two core components: the Molecular Representation Encoder (MRE) and the Task-Aware Prompter (TAP). MRE employs a hierarchical message-passing network architecture to capture molecular features at both the atom and motif levels. Meanwhile, TAP utilizes agglomerative hierarchical clustering algorithm to construct a prompt tree that reflects task affinity and distinctiveness, enabling the model to consider multi-granular correlation information among tasks, thereby effectively handling the complexity of multi-label property prediction. Extensive experiments demonstrate that HiPM achieves state-of-the-art performance across various multi-label datasets, offering a novel perspective on multi-label molecular representation learning.

prompt learning
multi-label learning
molecular representation
molecular property prediction
National Natural Science Foundation of China 10.13039/501100001809 62102158 Huazhong Agricultural University Scientific & Technological Self-innovation Foundation Fundamental Research Funds for the Central Universities 10.13039/501100012226 2662021JC008 2662022JC004 BC2024108
==== Body
pmcIntroduction

Traditional drug discovery requires an average of 10–15 years and costs over 2 billion dollars [1, 2]. However, recent advancements in molecular representation learning have revolutionized this field, significantly reducing both the time and cost [3–5]. Previous studies [6–9] typically represent molecules as topological graphs and employ Graph Neural Networks (GNNs) to capture both molecular structure and chemical information. This strategy has been widely validated as effective for learning molecular representations. Nonetheless, real-world molecules usually exhibit multiple properties and existing research often overlook the specific problems in this scenario. This limitation can result in significant gaps in understanding the complete biological activity of molecules, ultimately hindering the efficiency of drug discovery.

Considering molecular property prediction as a multi-label learning task is promising in resolving the above challenge. However, several inherent issues in multi-label learning must be addressed. One primary issue is the exponential growth of the output space. For instance, 32 labels can lead to as many as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $2^{32}$\end{document} combinations. The other issue is gradient conflict [10, 11]. Since multi-label learning is a special case of multi-task learning [12, 13], the gradient directions of different labels may conflict, making it difficult for the model to optimize the performance of all labels simultaneously. An effective solution to these problems is to explore the correlations between labels [14–17]. The potential correlations among labels are intricate, which can be pairwise, involve triples of labels, or even be common across all labels. Prompt learning is an emerging paradigm in deep learning, offering significant flexibility in task adaptation [18]. Recent studies [19–22] have introduced prompt learning into the field of GNNs. These methods generally employ soft prompts to learn task information, revealing the potential of prompt learning in enhancing task information capture capabilities of GNNs. Therefore, using a graph prompt method to model complex task correlations appears beneficial.

Furthermore, another issue specific to molecular property prediction tasks is that certain motifs [23] cause molecules to exhibit multiple properties in many cases. For example, as shown in Fig. 1, the salicylic acid structure is a common motif in Non-steroidal Anti-inflammatory Drugs, indicating that drugs containing this structure have anti-inflammatory property. Meanwhile, the carboxyl group (-COOH) in the salicylic acid structure imparts hydrophilicity and acidity to these compounds. Thus, motif information is critical for multi-label molecular property prediction tasks.

Figure 1 The salicylic acid structures in aspirin, salsalate, and sodium salicylate are specially emphasized by dashed circles, which collectively imply that these compounds are anti-inflammatory, acidic, and hydrophilic.

In this paper, we propose Hierarchial Prompted Molecular (HiPM), an innovative framework to address the challenge of multi-label molecular property prediction. HiPM comprises two modules: the Molecular Representation Encoder (MRE) and the Task-Aware Prompter (TAP). MRE employs a hierarchical network architecture to learn molecular chemical structure features at both the atom and motif levels. TAP utilizes a hierarchical prompt tree to model multi-granular task correlations, employing learnable soft prompts and agglomerative hierarchical clustering to construct the tree. Ultimately, HiPM generates molecular representations that encapsulate differential task correlation information.

We conducted performance comparison experiments on six multi-label datasets from MoleculeNet [24], and the results indicate that our method achieves state-of-the-art performance across all datasets, attaining the best results on five of them. Additionally, we provide extensive supplementary experiments to offer a deeper understanding of our method.

In conclusion, our contributions can be summarized as follows:

To the best of our knowledge, HiPM is the first to apply a prompt-based method to model multi-label task correlations in the field of molecular representation learning.

We designed the TAP, which enables the model to adaptively learn multi-granular task correlation information.

We conducted comprehensive experiments to demonstrate that our framework achieves state-of-the-art performance in multi-label molecular property prediction scenarios and exhibits excellent interpretability.

Related work

Many graph-based methods have been proposed for molecular property prediction. For instance, Gilmer et al. [25] introduced the message passing neural networks (MPNNs) framework, which unifies existing models for graph data and enhances molecular property predictions. Wang et al. [26] developed a pre-training framework for 3D molecular graphs to obtain comprehensive representations. Lv et al. [27, 28] leveraged meta-learning with graph attention networks to capture local atomic group effects and their interactions. Jiang et al. [29] proposed a pharmacological constraint-based multi-view molecular representation graph to extract significant chemical information from functional substructures and reactions. Several prompt-based methods have also been introduced into the field of molecular property prediction. Guo et al. [30] used the language model as an agent to highlight task-relevant features by understanding natural language descriptions. Fang et al. [31] developed a molecular contrastive learning framework using functional prompts from a knowledge graph. Despite these advancements, they fall short in effectively addressing the challenges of multi-label molecular property prediction, without considering the correlations between tasks. In contrast, our method uses prompt learning to capture task correlations in multi-label scenarios. To the best of our knowledge, our work is the first to apply prompt learning to model multi-label task correlations in molecular representation learning.

Method

Overview of HiPM

In this section, we provide an overview of HiPM. As illustrated in Fig. 2, our framework consists of two primary components: the MRE and the TAP.

Figure 2 Overview of the HiPM Framework: (A) illustrates the architecture of our model, where the prompt matrix is directly fused with the molecular representations generated using Eq 10. (B) details the process of calculating task affinities using cosine similarities. (C) describes the methodology for constructing the hierarchical prompt tree structure, utilizing the agglomerative hierarchical clustering algorithm. (D) outlines the process of computing soft prompts for non-leaf nodes in a bottom-up manner. (E) explains how the prompt matrix is derived from the nodes corresponding to the prefix paths of tasks on the prompt tree. We use Equation 8 to perform fusion on all prompts along the path.

MRE is a hierarchical architecture that incorporates the GNN layer for each level and an information interaction module between different levels. Specifically, MRE captures molecular information using MPNN [25] at both the atom and motif levels. Following each message passing phase, MRE employs a Transformer-based Local Augmentation module [32] to integrate information from these two levels and generate the augmented motif-level message. Ultimately, MRE produces a molecular representation that encapsulates molecular chemical and structural information from both levels through a readout process (Fig. 2A).

TAP is designed to facilitate the joint learning of shared information across tasks at multiple granularities. In this module, each task is initialized with a learnable soft prompt, and their affinity is measured using cosine similarity. By calculating the affinity between all soft prompts, an affinity matrix is obtained (Fig. 2B). Under the guidance of task affinity, TAP applies agglomerative hierarchical clustering algorithm to construct a tree structure (Fig. 2C). Each leaf node in the tree corresponds to a specific task. Subsequently, TAP calculates the prompt for each node in a bottom-up manner (Fig. 2D). Finally, TAP constructs the prompt matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{P}$\end{document} using the prompts along the path from the root node to each leaf node (Fig. 2E).

In summary, HiPM integrates the prompt matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{P}$\end{document} with the molecular representations module to generate molecular representations that encapsulate task correlations.

Molecular Representation Encoder

The hierarchical message passing mechanism allows MRE to generate effective molecular representations, which have been extensively proved to be simple and effective for learning multi-level molecular structural information [32–35]. The following sections provide the details of MRE:

MPNNs for atom and motif levels

In both the atom and motif levels, a molecule can be abstracted as a graph \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $G = (V, E, F_{n}, F_{e})$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $|V| = N_{n}$\end{document} denotes a set of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $N_{n}$\end{document} nodes, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $|E| = N_{e}$\end{document} denotes a set of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $N_{e}$\end{document} edges, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $F_{n} \in \mathbb{R}^{N_{n} \times d_{n}}$\end{document} denotes the node features, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $F_{e} \in \mathbb{R}^{N_{e} \times d_{e}}$\end{document} denotes the edge features. Here, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ d_{n} $\end{document} represents the dimension of the node features, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $ d_{e} $\end{document} represents the dimension of the edge features. At the atom level, atoms are represented as nodes and chemical bonds as edges. At the motif level, motifs are treated as nodes, and the overlapping atoms between motifs are treated as edges. Molecular motifs are extracted using an established algorithm from the literature [36]. Each level utilizes a one-layer MPNN [25] to learn molecular features. Through the message passing process, MRE captures information at each level, enabling the fine-grained learning of molecular features.

Information interaction between different levels

Learning molecular representations at isolated levels can result in the loss of local information. To improve the interaction between motifs and atoms, MRE employs a Transformer-based Local Augmentation module, which facilitates the interaction of information across different levels.

Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $M_{a}^{(l)}$\end{document} denote the messages from the atom level generated by the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $l$\end{document}th step of message passing, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $M_{s}^{(l)}$\end{document} denote the messages from the motif level and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $F_{m}$\end{document} denote the features of motif nodes. Here, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $M_{a} \in \mathbb{R}^{N_{a} \times d_{a}}, M_{s} \in \mathbb{R}^{N_{m} \times d_{m}}, F_{m} \in \mathbb{R}^{N_{m} \times d_{f}}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $N_{a}$\end{document} denotes the atom number, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $N_{m}$\end{document} denotes the motif number, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $d_{a}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $d_{m}$\end{document} denote the dimension of the message from the atom and motif levels, while \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $d_{f}$\end{document} denotes the dimension of the feature of motif.

First, in the style of the Transformer [37], three separate linear layers are employed to learn \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $Q^{(l)}$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $K^{(l)}$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $V^{(l)}$\end{document} respectively, which can be formulated as follows:

(1) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*}& \left\{ \begin{aligned} Q^{(l)} &= F_{m}^{(l)}W_{q}\\ K^{(l)} &= (M_{a}^{(l)} \oplus M_{s}^{(l)})W_{k}\\ V^{(l)} &= (M_{a}^{(l)} \oplus M_{s}^{(l)})W_{v} \end{aligned} \right.\end{align*}\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\oplus $\end{document} denotes the concatenation, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $W_{q}$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $W_{k}$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $W_{v}$\end{document} are the projection matrix for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $Q$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $K$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $V$\end{document}, respectively.

Subsequently, the multi-head attention mechanism is employed to facilitate information interaction across the levels, and the corresponding formulas are as follows:

(2) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*} & head_{i}^{(l)}=softmax\left(\frac{Q^{(l)}{K^{(l)}}^{\top}}{\sqrt{d_{k}}}\right)V^{(l)}\qquad\qquad\quad \end{align*}\end{document}

(3) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*} & M_{m}^{(l+1)}=(head_{1}^{(l)} \oplus head_{2}^{(l)} \oplus...\oplus head_{n}^{(l)})W_{o}\end{align*}\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $head_{i}$\end{document} denotes the attention computed by the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th head, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $d_{k}$\end{document} denotes the dimension of features that the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $k$\end{document}th head processes, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $W_{o}$\end{document} is the projection matrix of multi-head feature concatenation to generate the new motif message \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $M_{m}^{(l+1)}$\end{document} with atom-level and motif level information.

Task-Aware Prompter

TAP is designed to generate prompt matrix that containing multi-granular task correlation information. At the end of each training epoch, it constructs a hierarchical prompt tree. Based on the hierarchical tree, a prompt matrix is built for the forward propagation in next training epoch. Further details are provided in the following sections:

Measure of inter-task affinity

To effectively reflect the conflict situations during overall training, we quantify task affinity using the cosine similarity of the cumulative prompt gradients over one training epoch.

Specifically, we use an affinity matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $A$\end{document} to describe the affinity among tasks, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $A_{ij}$\end{document} denotes the affinity between the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $j$\end{document}th tasks. The soft prompt corresponding to the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th task is defined as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{i}$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{i} \in \mathbb{R}^{d_{r}}$\end{document}, with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $d_{r}$\end{document} denoting the dimension of the molecular representation from MRE. Each soft prompt is initialized with a normal distribution, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{i} \sim \mathcal{N}(\mu , \sigma ^{2} I)$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mu $\end{document} is the mean vector and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\sigma ^{2} I$\end{document} is the covariance matrix. For the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $j$\end{document}th tasks, their affinity is calculated by Equation 4:

(4) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*}& A_{ij}=\frac{{\nabla p_{i}}^{\top} \nabla p_{j}}{\|\nabla p_{i}\| \|\nabla p_{j}\|}\end{align*}\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\nabla p_{i}$\end{document} represents the cumulative gradient of the soft prompt for the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th task, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\|\nabla p_{i}\|$\end{document} denotes the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\text{L}_{2}$\end{document} norm of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\nabla p_{i}$\end{document}.

Construction of hierarchical prompt tree

The construction of hierarchical prompt tree is decomposed into two steps: constructing the structure of the prompt tree and generating the prompts for each node.

Agglomerative hierarchical clustering algorithm naturally forms a binary tree structure, providing an effective method for constructing prompt trees. Since the clustering process is typically based on distance, we need to transform the affinity matrix (cosine similarity matrix) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $A$\end{document} to a cosine distance matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{A}$\end{document}. The transformation method is as follows:

(5) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*}& \tilde{A}=1-A\end{align*}\end{document}

Then, the structure of the prompt tree is obtained through the following process. Considering \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} tasks with the corresponding cumulative prompt gradients \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\nabla p_{1}, \nabla p_{2}, \ldots , \nabla p_{m}$\end{document} obtained over one training epoch, initially, each gradient vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\nabla p$\end{document} is treated as an individual cluster \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c$\end{document}, forming a set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathcal{C}$\end{document}. In each iteration, the algorithm identifies the two closest clusters in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathcal{C}$\end{document}, merges them into a new cluster, and updates \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathcal{C}$\end{document} accordingly. Specifically, if the clusters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{i}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{j}$\end{document} are merged into \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{k}$\end{document}, then \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{i}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{j}$\end{document} are removed from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathcal{C}$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $c_{k}$\end{document} is added. This merging process continues until only one cluster remains in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\mathcal{C}$\end{document}, at which point the algorithm terminates.

It is important to note that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{A}$\end{document} only provides the distances between initial clusters. We employ Equation 6 to calculate the distance between clusters not included in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{A}$\end{document}. In this equation, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $d(u,v)$\end{document} denotes the distance between two clusters, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $u$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $u_{i}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v_{j}$\end{document} refer to the points in clusters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $u$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v$\end{document}. Additionally, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $|u|$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $|v|$\end{document} denote the number of points of clusters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $u$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $v$\end{document}, respectively.

(6) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*}& d(u,v)=\sum_{{u_{i} \in u, v_{j}\in v}}\frac{d(u_{i},v_{j})}{|u||v|}\end{align*}\end{document}

Based on the tree structure, we adopted a recursive method for calculating the prompts for tree nodes. Let the prompt tree be denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T$\end{document}. First, for the leaf nodes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{i}$\end{document} of the binary tree, we assign a learnable vector as its corresponding soft prompt \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{i}$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $1 \le i \le m$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} is the total number of tasks. In the prompt tree, a non-leaf node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{f}$\end{document} must have a left child node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{l}$\end{document} and a right child node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{r}$\end{document} due to the nature of agglomerative hierarchical clustering. Then, the soft prompt of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{f}$\end{document} can be calculated by Equation 7, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{f}$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{l}$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{r}$\end{document} are the soft prompts of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{f}$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{l}$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{r}$\end{document}, respectively.

(7) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*}& p_{f} = p_{l} + p_{r}\end{align*}\end{document}

Through this recursive structure, we can calculate the soft prompt corresponding to each node of the tree from bottom to the top. For leaf nodes, the soft prompts contain individual information related to their corresponding tasks. For non-leaf nodes, the soft prompts contain shared information from their descendant nodes. The higher a node is positioned in the prompt tree, the more labels it encompasses. Therefore, the granularity of task information contained in the soft prompts of each tree node varies.

Calculation of prompt matrix

During model inference, it is essential to consider the information of a specific task at all levels of granularity. To address this problem, we generate a new soft prompt \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $p_{i}^{\prime}$\end{document} containing multi-granular task information for each task.

For any task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $t_{k}$\end{document}, with its corresponding leaf node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{k}$\end{document}, the prefix path of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{k}$\end{document} is defined as the set consisting of all nodes unidirectionally connected to it, which is denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $pre$\end{document}. We use the FindPrefix in Algorithm 1 to compute prefix path of each task. Then, the new soft prompt for task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $t_{k}$\end{document} is calculated by Equation 8, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $S = \{T_{k}\} \cup pre$\end{document}, a node set including both the leaf node \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $T_{k}$\end{document} and all nodes in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $pre$\end{document}.

(8) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*}& p_{k}^{\prime}=\frac{\sum_{j \in S}p_{j}}{|S|}\end{align*}\end{document}

For higher computational parallelism, we transform the set of soft prompts corresponding to all tasks \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $P=\{p_{1}^{\prime},p_{2}^{\prime},p_{3}^{\prime},...,p_{m}^{\prime}\}$\end{document} into a prompt matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{P}$\end{document} for forward propagation, where the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th column of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{P}$\end{document} represents the prompt for the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th label, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $m$\end{document} denotes the total number of labels. The process for calculating the prompt matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{P}$\end{document} is outlined in Algorithm 1 and more details can be found in supplementary materials.

Integration of prompts and molecular representation

We fuse the molecular representation produced by MRE with the prompts to generate the molecular representation including differential information across multiple granularities. Given the molecular representation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $r \in \mathbb{R}^{d_{r}}$\end{document} and the prompt matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{P} \in \mathbb{R}^{d_{r} \times m}$\end{document}, we use the Equations 9 and 10 to integrate the multi-label prompt information into the molecular representation.

(9) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*} & {H=r\tilde{P}}\qquad\qquad\qquad\qquad\qquad\quad \end{align*}\end{document}

(10) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*} & {r^{^{\prime}}=\sigma_{2}(\sigma_{1}(HW_{1}+b_{1})W_{2}+b_{2})}\end{align*}\end{document}

Here, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $r^{^{\prime}} \in \mathbb{R}^{m}$\end{document} represents the new molecular representation fused with multi-label prompt information. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\sigma _{1}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\sigma _{2}$\end{document} are non-linear activation functions. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $W_{1} \in \mathbb{R}^{m \times m}$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $b_{1} \in \mathbb{R}^{m}$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $W_{2} \in \mathbb{R}^{m \times m}$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $b_{2} \in \mathbb{R}^{m}$\end{document} are trainable parameters. Since the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th column of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\tilde{P}$\end{document} represents the prompt corresponding to the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th label, the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th element of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $r^{^{\prime}}$\end{document} is related only to the molecular representation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $r$\end{document} and the prompt corresponding to the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th label. This method effectively integrates task correlation prompt information into the molecular representation, enhancing the model’s capability to handle multi-label tasks. Finally, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $r^{^{\prime}}$\end{document} is directly fed into a classifier or regressor for prediction.

Updating the hierarchical prompt tree

Updating the hierarchical prompt tree is essentially a process of reconstruction, which requires careful consideration of both the basis and the timing of updates.

Impact of gradient descent on the optimization of the tree structure. Agglomerative hierarchical clustering operates on the principle of maximum similarity. We hypothesize that tasks with higher similarity are more likely to be grouped into the same cluster as model performance improves. Thus, our method for updating the prompt tree involves rerunning Algorithm 1 based on the new task affinity and prompt gradients.

Timing for updating the prompt tree. We reconstruct the prompt tree at the end of each epoch. This update frequency helps to keep the additional time overhead from clustering within an acceptable range and does not increase the instability of model performance. Through periodic reconstruction, the hierarchical prompt tree can be dynamically adjusted as the model learns, thereby better capturing the correlations between labels.

Prediction and loss

In this section, we provide relevant details of the prediction and loss function. For the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th label, HiPM yields predicted outcomes at different levels, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\hat{y}_{i,a}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\hat{y}_{i,m}$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\hat{y}_{i,a}$\end{document} is from the atom level and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\hat{y}_{i,m}$\end{document} is from the motif level. To effectively align the feature spaces for different levels, we introduce an additional contrastive loss as described in Equation 11. Then, the loss for the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th label is calculated using Equation 12.

(11) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*} & \text{c}_{i}=\frac{1}{N}\sum{(\hat{y}_{i,a}-\hat{y}_{i,m})^{2}} \end{align*}\end{document}

(12) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*} & l_{i} = l_{i,a} + l_{i,m} + \lambda \cdot c_{i}\quad\end{align*}\end{document}

Here, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $l_{i,a}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $l_{i,m}$\end{document} represent the losses computed at the atom and motif level, respectively. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\lambda $\end{document} denotes the penalty strength of the contrastive loss. For classification tasks, we use the Binary Cross-Entropy loss function, while for regression tasks, we use the Mean Squared Error loss function.

Table 1 Performance comparison on six multi-label datasets under scaffold splitting. Each model is run with three random seeds, and we report the average ROC-AUC (classification) or MAE (regression) scores, along with the corresponding standard deviations.

Type	Classification(ROC-AUC)	Regression(MAE)	
Dataset	Clintox(\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document})	SIDER(\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document})	Tox21(\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document})	ToxCast(\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document})	QM8(\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\downarrow $\end{document})	QM9(\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\downarrow $\end{document})	
MPNN	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.879\pm 0.054$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.595\pm 0.030$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.808\pm 0.024$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.691\pm 0.013$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0146\pm 0.0015$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $5.253\pm 0.465$\end{document}	
D-MPNN	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.879\pm 0.040$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.610\pm 0.027$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.808\pm 0.023$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.718\pm 0.011$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0124\pm 0.0013$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $4.797\pm 0.343$\end{document}	
ML-MPNN	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.865\pm 0.027$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.609\pm 0.017$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.796\pm 0.021$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.708\pm 0.003$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0225\pm 0.0005$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $5.734\pm 0.581$\end{document}	
FP-GNN	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.765\pm 0.038$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.598\pm 0.014$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.803\pm 0.024$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.694\pm 0.014$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0165\pm 0.0019$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $6.491\pm 0.367$\end{document}	
MVGNN	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.894\pm 0.035$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.647\pm 0.022$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.835\pm 0.010$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.736\pm 0.011$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0125\pm 0.0001$\end{document}	2.347\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\pm $\end{document}0.018	
HimGNN	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.922\pm 0.037$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.623\pm 0.033$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.803\pm 0.028$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.732\pm 0.021$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0211\pm 0.0065$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $6.434\pm 0.908$\end{document}	
GROVER	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.734 \pm 0.032$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.567 \pm 0.031$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.736 \pm 0.022$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.593 \pm 0.021$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0184 \pm 0.0009$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $6.497 \pm 1.356$\end{document}	
MolCLR	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.900\pm 0.025$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.597\pm 0.009$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.811\pm 0.026$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.691\pm 0.012$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0182\pm 0.0001$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $3.215\pm 0.106$\end{document}	
MGSSL	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.877\pm 0.021$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.591\pm 0.016$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.788\pm 0.022$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.573\pm 0.004$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0322\pm 0.0033$\end{document}	/	
HiMol	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.702\pm 0.092$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.593\pm 0.022$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.815\pm 0.010$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.692\pm 0.014$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.4232\pm 0.0033$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $3.355\pm 1.229$\end{document}	
KANO	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.903\pm 0.039$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.593\pm 0.026$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.797\pm 0.012$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.696\pm 0.020$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0148\pm 0.0025$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $2.544\pm 0.257$\end{document}	
HiPM (wo/pro)	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.948\pm 0.041$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.666\pm 0.006$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.770\pm 0.005$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.735\pm 0.006$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0171\pm 0.0003$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $5.344\pm 0.112$\end{document}	
HiPM (wo/cls)	0.969\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\pm $\end{document}0.009	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.653\pm 0.007$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.799\pm 0.007$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.751\pm 0.006$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.0336\pm 0.0004$\end{document}	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $8.352\pm 0.476$\end{document}	
HiPM (Ours)	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.928\pm 0.014$\end{document}	0.672\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\pm $\end{document}0.010	0.843\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\pm $\end{document}0.007	0.786\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\pm $\end{document}0.004	0.0117\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\pm $\end{document}0.0001	\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $5.238\pm 0.199$\end{document}	
Note: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\uparrow $\end{document} means that the higher result is better and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\downarrow $\end{document} means that the lower result is better. Note that the MGSSL model on QM9 is too time consuming to finish in time, and its results are not presented. The best score in each column is in bold and the second best score is underlined. ’HiPM(wo/pro)’ and ’HiPM(wo/cls)’ are variants of HiPM, as detailed in the Ablation Study section.

To address label imbalance in classification, we weight the loss for each label. Specifically, for the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th label, let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $N$\end{document} denote the total number of molecules and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $N_{i}$\end{document} denote the number of molecules possessing the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $i$\end{document}th property label. The weight \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $w_{i}$\end{document} is then calculated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $w_{i} = \frac{N_{i}}{N}$\end{document}. The final weighted loss is given by Equation 13.

(13) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{align*}& {\mathbb{L}(\cdot) = \sum_{i=1}^{m} w_{i} \cdot l_{i}}\end{align*}\end{document}

Experiment

In this section, we evaluate the performance of HiPM across various multi-label datasets. Specifically, we aim to answer the following questions:

How does HiPM compare to state-of-the-art models on the multi-label molecular property prediction tasks?

How does the model performance differ when considering task correlation versus not considering it?

What are the task correlations obtained through clustering by TAP?

How does TAP affect the attention weights of key molecular substructures?

How does TAP affect the predicted probabilities of HiPM for each task?

Experimental setup

Benchmark datasets

To evaluate the effectiveness of HiPM for multi-label property prediction, we conducted experiments on six multi-label datasets from MoleculeNet [24]. These datasets include four classification datasets and two regression datasets, covering two categories: physiology and quantum mechanics. The statistics of these datasets are shown in Table 2. For a molecule represented by a SMILES [38] string, we convert it into a 2D topological graph using RDKit (https://www.rdkit.org/) and DGL-LifeSci (https://lifesci.dgl.ai/) for further processing.

Table 2 Statistics of datasets.

Dataset	Tasks	Task Type	Molecules	
ClinTox	2	Classification	1478	
SIDER	27	Classification	1427	
Tox21	12	Classification	7831	
ToxCast	617	Classification	8575	
QM8	12	Regression	21 786	
QM9	12	Regression	133 885	

Baseline models

We compared HiPM with existing state-of-the-art methods, categorized into supervised and self-supervised methods. The supervised methods include MPNN [25], D-MPNN [39], ML-MPNN [40], FP-GNN [41], MVGNN [42], and HimGNN [32]. The self-supervised methods consist of GROVER [43], MolCLR [6], MGSSL [35], HiMol [33], and KANO [31]. Detailed information about these methods is available in the supplementary materials.

Implementation details

For all datasets, we employ scaffold splitting, which is considered superior to random splitting and helps to prevent information leakage [44]. The datasets are divided into training, validation, and test sets in an 8:1:1 ratio. We use the widely adopted Optuna [45] framework for hyper-parameter tuning, performing 20 trials per dataset to identify the optimal parameters based on validation set performance. Detailed hyper-parameter information is available in our code repository. Model training is executed on two NVIDIA GeForce 1080Ti GPUs, with 60 epochs for classification tasks and 200 epochs for regression tasks.

Performance comparison (RQ1)

In this section, we compare the performance of HiPM against various baseline models across six multi-label datasets. From Table 1, several key observations can be made:

(i) HiPM shows better performance in multi-label scenarios. In classification tasks, HiPM outperforms all baseline models across all datasets, achieving an average ROC-AUC improvement of 3.6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document}. In regression tasks, HiPM shows an average improvement of 6.4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document} over other baseline models on QM8.

(ii) HiPM performs relatively average on QM9 compared to the state-of-the-art models. QM9, which comprises quantum mechanical calculations for a large number of small organic molecules, is characterized by its intricate data distribution. This complexity likely renders HiPM less optimal for such scenarios.

(iii) HiPM is particularly effective in scenarios where there are significant correlations between labels (e.g. Tox21), or when dealing with a larger number of labels with more complex correlations (e.g. ToxCast). This effectiveness highlights HiPM’s capability to capture and leverage the intricate relationships between multiple labels.

Ablation study (RQ2)

In this section, we conducted ablation studies to further analyze the effectiveness of TAP. Specifically, we designed the following two variants of HiPM:

HiPM (wo/pro), which directly removes TAP. This variant reflects the effectiveness of MRE.

HiPM (wo/cls), which includes TAP without clustering. This variant shows the model performance when only individual task information is considered.

For each dataset and variant, we used three random seeds for experiments, and the experimental results are shown in Table 1 and Fig. 4.

Comparison between considering and not considering task association

The results show that HiPM achieves superior outcomes on five datasets compared to HiPM (wo/cls), reaching the optimal performance. The experimental results validate the significance of the clustering process in enhancing the model’s ability to capture shared information. Furthermore, these results illustrate that multi-granular shared information contributes to improving model performance.

Are prompts without task association information useful?

The results reveal that HiPM (wo/cls) underperforms compared to HiPM (wo/pro) on half of the datasets. This suggests that focusing solely on individual task information may lead to conflicts. Therefore, excluding task information from the molecular representation entirely might be more beneficial when task correlations are not considered.

Interpretability analysis (RQ3)

In this section, we visualized the affinity matrix and hierarchical prompt tree for 12 tasks of Tox21. The labels in Tox21 are primarily categorized into Nuclear Receptor (NR) and Stress Response (SR) groups. Figure 3 demonstrates that dark colors are concentrated in the upper left and lower right corners of the matrix, indicating a high degree of task affinity within the NR and SR categories. For instance, the tasks NR-AR, NR-AR-LBD, and NR-AhR exhibit strong correlations and all belong to the NR category, demonstrating that our model can effectively capture the correlations between tasks. Conversely, the affinity between different categories is relatively low, such as between NR-Aromatase and SR-ATAD5, suggesting that our model can also capture the differences between tasks.

Figure 3 Visualization results of the affinity matrix (A) and hierarchical clustering tree (B) for the 12 tasks of Tox21. In the affinity matrix, darker colors represent higher task affinities. In the hierarchical clustering tree, the leaf nodes are labeled, and potential reasons for the prioritized clustering of certain tasks are provided. Specifically, CS-DM stands for Cellular Stress and Defense Mechanisms, CSR stands for Cellular Stress Response, NR-HM stands for Nuclear Receptor and Hormone Metabolism, NR-HR stands for Nuclear Receptor and Hormone Regulation, and NRA stands for Nuclear Receptor Activity. The clustering of these leaf nodes reflects their functional similarities and associations within their respective toxicological mechanisms.

Figure 4 Results of ablation experiments. Each variant was run with three random seeds. We report the average ROC-AUC (classification) or MAE (regression) scores along with their standard deviations. Higher ROC-AUC values indicate superior performance, while lower MAE values are preferable. The error bars denote standard deviations.

As shown in Fig. 3, the clustering results align closely with the affinity, prioritizing tasks with high affinity to be clustered into the same group. We have provided possible reasons for these clusters, which match known facts. This indicates that our agglomerative hierarchical clustering method possesses good interpretability. Additional information on other datasets can be found in the supplementary materials.

Visualization analysis (RQ4 & RQ5)

In this section, we selected several molecules from Tox21 to observe changes in molecular motif weights before and after incorporating the TAP module. Figure 5 illustrates the changes in molecular motif weights and predicted probabilities. For simplicity and clarity, only motifs with weight differences greater than \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $0.1$\end{document} were visualized.

Figure 5 Visualization analysis results that highlight key motifs and the changes in the corresponding attention weights before and after integrating the TAP module. Bar charts illustrate the variations in predicted probabilities for each label generated by HiPM before and after the inclusion of TAP. The value of each bar is calculated as the predicted probability (after) minus the predicted probability (before). An increase (or decrease) in predicted probability after adding TAP, where the true label is 1 (or 0), indicates an increase in the model’s confidence in the correct answer.

Tox21 is a dataset focused on toxicity. After integrating the TAP module, the weights of certain toxicity-related motifs, such as aminomethyl and chlorophenyl, increased significantly, aligning the predicted probabilities more closely with the ground truths. This suggests that our model effectively captures key motif structures essential for these tasks.

Conclusion

In this paper, we present HiPM, a hierarchical prompted multi-label molecular representation learning framework developed to tackle the issues associated with multi-label molecular property prediction. HiPM leverages an innovative hierarchical prompt method, enabling the model to learn task-specific prompts and effectively capture correlation information across tasks. By constructing a hierarchical prompt tree based on task affinities, our method reveals latent multi-granular correlations among property labels. Overall, HiPM exhibits considerable potential in advancing multi-label molecular property prediction, offering a robust tool for drug discovery research. In future work, we aim to explore mechanisms for dynamically managing label associations across different datasets, which could enhance HiPM’s ability to adaptively generalize across a wide range of molecular property prediction tasks.

Key Points

We introduced HiPM, a hierarchical prompted molecular representation learning framework designed to address the complexity of multi-label molecular property prediction.

HiPM employs a hierarchical prompt tree to model multi-granular task correlations, generating molecular representations that incorporate differential task information.

By capturing multi-granular correlation information among labels, HiPM mitigates the negative transfer caused by conflicts of individual task information.

Extensive experimental results demonstrate that HiPM exhibits strong competitiveness compared to existing state-of-the-art models.

Supplementary Material

Supplementary_Materials-submit_bbae438

Author contributions

L.K. and S.Z. contributed equally to this work. L.K. and S.Z. conceived the idea and initiated the project. L.K. and S.Z. designed and implemented the HiPM framework. S.L. supervised the project. L.K., S.Z., and S.F. tested the performance of the HiPM framework and compared it with state-of-the-art methods. L.K., S.Z., S.F., and S.L. drafted the manuscript. All authors reviewed and approved the final manuscript.

Funding

This work was supported by the National Natural Science Foundation of China (62102158); Huazhong Agricultural University Scientific & Technological Self-innovation Foundation; Fundamental Research Funds for the Central Universities (2662021JC008, 2662022JC004, BC2024108). The funders have no role in study design, data collection, data analysis, data interpretation, or writing of the manuscript.

Data availability

Complete datasets and source code for HiPM are available for free on GitHub at https://github.com/zhousongh/HiPM.

Appendix

Hyper-parameter sensitivity analysis

In this section, we conducted hyper-parameter sensitivity analysis to study the impact of hyper-parameters on HiPM performance. Since TAP does not introduce additional hyper-parameters, we selected two key parameters for analysis: the number of attention heads (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $heads$\end{document}) and the hidden layer feature dimension (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $dim$\end{document}) in MRE. The range of hyper-parameter adjustments varied for different datasets and the results are shown in Fig. A1. Based on the experimental results, we can draw two main conclusions:

(1) Larger datasets do not necessarily benefit from larger feature dimensions. For example, HiPM performs best when the feature dimension is set to 192 on QM9 dataset, with over 100 000 molecules.

(2) Multi-head attention generally performs better than single-head attention, but more attention heads are not always better. In almost all datasets, multi-head attention outperforms single-head attention. However, too many attention heads might lead to overly sparse features being processed by each head, resulting in performance degradation.

Figure A1 Hyper-parameter sensitivity analysis of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $dim$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $heads$\end{document}. The first row shows the analysis results for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $dim$\end{document}. The second row shows the analysis results for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $heads$\end{document}. Higher ROC-AUC values and lower MAE values indicate better performance.

Statistical significance analysis

Following the literature [46], we conducted a Bland–Altman analysis to measure the consistency between the predicted value and ground truths. The results are shown in Fig. A2. In each plot, the second dashed line indicates the mean bias and the first and third dashed line indicates the 95\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document} confidence limits, together forming the statistical range of bias. Specifically, the mean biases for the three tasks in QM8 are 0.0474, 0.0337, and 0.0399, with over 95\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document} of the data points falling within the 95\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document} confidence interval. Similarly, for QM9, the mean biases for the three tasks are −0.0037, 0.0282, and 0.1522, with over 95\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document} of the data points falling within the 95\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document} confidence interval. These results demonstrate that within an acceptable range of error, the predicted values of HiPM can replace the experimentally measured ground truths.

Figure A2 High agreement between HiPM and the ground truths assessed by the Bland–Altman analysis. The more points that fall within the 95\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $\%$\end{document} confidence interval (indicated by the first and third dashed lines), the better the consistency.
==== Refs
References

1 Nurken B , MohamadA. An overview of drug discovery and development. Future Med Chem  2020;12 :939–47. 10.4155/fmc-2019-0307.32270704
2 Catacutan DB , AlexanderJ, ArnoldA. et al .  Machine learning in preclinical drug discovery. Nat Chem Biol  2024;20 :960–73. 10.1038/s41589-024-01679-1.39030362
3 Chen H , EngkvistO, WangY. et al .  The rise of deep learning in drug discovery. Drug Discov Today  2018;23 :1241–50. 10.1016/j.drudis.2018.01.039.29366762
4 Deng J , YangZ, OjimaI. et al .  Artificial intelligence in drug discovery: applications and techniques. Brief Bioinform  2022;23 :bbab430. 10.1093/bib/bbab430.34734228
5 Jessica V , DominicC, PaulC. et al .  Applications of machine learning in drug discovery and development. Nat Rev Drug Discov  2019;18 :463–77. 10.1038/s41573-019-0024-5.30976107
6 Wang Y , WangJ, CaoZ. et al .  Molclr: Molecular contrastive learning of representations via graph neural networks. Nat Mach Intell  2022;4 :279–87. 10.1038/s42256-022-00447-x.
7 Zhang R , WuC, YangQ. et al .  MolFeSCue: enhancing molecular property prediction in data-limited and imbalanced contexts using few-shot and contrastive learning. Bioinformatics  2024;40 :btae118. 10.1093/bioinformatics/btae118.
8 Cai L , HeY, FuX. et al .  AEGNN-M:a 3D graph-spatial co-representation model for molecular property prediction. IEEE J Biomed Health Inform  2024;1–9. 10.1109/JBHI.2024.3368608.
9 Zhang L , NiuD, ZhangB. et al .  Property-guided few-shot learning for molecular property prediction with dual-view encoder and relation graph learning network. IEEE J Biomed Health Inform  2024;1–12. 10.1109/JBHI.2024.3381896.
10 Yu T , KumarS, GuptaA. et al .  Gradient surgery for multi-task learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook, NY, USA: CurranAssociates Inc., 2020.
11 Liu Y , LuY, LiuH. et al .  Hierarchical prompt learning for multi-task learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, British Columbia, Canada: IEEE, 2023.
12 Read J . From multi-label learning to cross-domain transfer: a model-agnostic approach. Applied Intelligence  2023;53 :25135–53. 10.1007/s10489-023-04841-9.
13 Huang Y , WangW, WangL. et al .  Multi-task deep neural network for multi-label learning. In: 2013 IEEE International conference on image processing, pp. 2897–900. Melbourne, Australia: IEEE, 2013.
14 Zhang M , ZhouZ. A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng  2014;26 :1819–37. 10.1109/TKDE.2013.39.
15 Liu W , WangH, ShenX. et al .  The emerging trends of multi-label learning. IEEE Trans Pattern Anal Mach Intell  2022;44 :7955–74. 10.1109/TPAMI.2021.3119334.34637378
16 Yeh C , WuW, KoW. et al .  Learning deep latent space for multi-label classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31 , 2017. 10.1609/aaai.v31i1.10769.
17 Zhang M , ZhangK. Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. NY, USA: ACM, 2010.
18 Liu P , YuanW, FuJ. et al .  Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. ACM Comput Surv  2023;55 :1–35. 10.1145/3560815.
19 Sun M , ZhouK, HeX. et al .  GPPT: Graph Pre-training and Prompt Tuning to generalize graph neural networks. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. NY, USA: ACM, 2022.
20 Sun X , ChengH, LiJ. et al .  All in one: Multi-task prompting for graph neural networks. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. NY, USA: ACM, 2023.
21 Liu Z , YuX, FangY. et al .  GraphPrompt: unifying pre-training and downstream tasks for graph neural networks. In: Proceedings of the ACM Web Conference. NY, USA: ACM, 2023.
22 Fang T , ZhangY, YangY. et al .  Universal Prompt Tuning for Graph Neural Networks. Advances in Neural Information Processing Systems, NY, USA: Curran Associates, Inc., 2024.
23 Ron M , ShaiS, ShalevI. et al .  Network motifs: simple building blocks of complex networks. Science  2002;298 :824–7.12399590
24 Wu Z , RamsundarB, FeinbergE. et al .  MoleculeNet: A benchmark for molecular machine learning. Chem. Sci. London, UK: The Royal Society of Chemistry, 2018.
25 Gilmer J , SchoenholzS, RileyP. et al .  Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning. Sydney, NSW, Australia: JMLR, 2017.
26 Wang X , ZhaoH, TuW. et al .  Automated 3D pre-training for molecular property prediction. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2419–2430. Long Beach, CA, USA: ACM, 2023.
27 Lv Q , ChenG, YangZ. et al .  Meta learning with graph attention networks for low-data drug discovery. IEEE Trans Neural Netw Learn Syst  2023;35 :1–13
28 Lv Q , ChenG, YangZ. et al .  Meta-MolNet: a cross domain benchmark for few examples drug discovery. IEEE Trans Neural Netw Learn Syst  2024;1–15. 10.1109/TNNLS.2024.3359657.
29 Jiang Y , JinS, JinX. et al .  Pharmacophoric-constrained heterogeneous graph transformer model for molecular property prediction. Communications. Chemistry  2023;6 . 10.1038/s42004-023-00857-x.
30 Guo H , ZhaoS, WangH. et al .  MolTailor: tailoring chemical molecular representation to specific tasks via text prompts. In: AAAI Conference on Artificial Intelligence, Vol. 38, 2024, pp. 18144–52. 10.1609/aaai.v38i16.29772.
31 Fang Y , ZhangQ, ZhangN. et al .  Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nat Mach Intell  2023;5 :542–53. 10.1038/s42256-023-00654-0.
32 Han S , FuH, WuY. et al .  HimGNN: a novel hierarchical molecular graph representation learning framework for property prediction. Brief Bioinform  2023;24 :bbad305.37594313
33 Zang X , ZhaoX, TangB. Hierarchical molecular graph self-supervised learning for property prediction. Communications. Chemistry  2023;6 . 10.1038/s42004-023-00825-5.
34 Jin W , ReginaB, JaakkolaT. Hierarchical generation of molecular graphs using structural motifs. In: Proceedings of the 37th International Conference on Machine Learning, vol 119 , pp. 4839–4848. Sydney, NSW, Australia: JMLR, 2020.
35 Zhang Z , LiuQ, WangH. et al .  Motif-based graph self-supervised learning for molecular property prediction. In: Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2021.
36 Ji Z , ShiR, LuJ. et al .  ReLMole: molecular representation learning based on two-level graph similarities. J Chem Inf Model  2022;62 :5361–72. 10.1021/acs.jcim.2c00798.36302249
37 Vaswani A , ShazeerN, ParmarN. et al .  Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, CA, USA: Curran Associates Inc., 2017.
38 David W . Smiles, a chemical language and information system. 1. Introduction to methodology and encoding rules. J Chem Inf Comput Sci  1988;28 :31–6. 10.1021/ci00057a005.
39 Yang K , SwansonK, JinW. et al .  Analyzing learned molecular representations for property prediction. J Chem Inf Model  2019;59 :3370–88. 10.1021/acs.jcim.9b00237.31361484
40 Wang Z , LiuM, LuoY. et al .  Advanced graph and sequence neural networks for molecular property prediction and drug discovery. Bioinformatics  2022;38 :2579–86. 10.1093/bioinformatics/btac112.35179547
41 Cai H , ZhangH, ZhaoD. et al .  FP-GNN: a versatile deep learning architecture for enhanced molecular property prediction. Brief Bioinform  2022;23 :bbac408. 10.1093/bib/bbac408.36124766
42 Ma H , BianY, RongY. et al .  Cross-dependent graph neural networks for molecular property prediction. Bioinformatics  2022;38 :2003–9. 10.1093/bioinformatics/btac039.35094072
43 Rong Y , BianY, XuT. et al .  Self-supervised graph transformer on large-scale molecular data. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook, NY, USA: CurranAssociates Inc., 2020.
44 Lv Q , ZhouJ, YangZ. et al .  3D graph neural network with few-shot learning for predicting drug–drug interactions in scaffold-based cold start scenario. Neural Netw  2023;165 :94–105. 10.1016/j.neunet.2023.05.039.37276813
45 Akiba T , SanoS, YanaseT. et al .  Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Anchorage, AK, USA: ACM, 2019.
46 Lv Q , ChenG, ZhaoL. et al .  Mol2Context-vec: learning molecular representation from context awareness for drug discovery. Brief Bioinform  2021;22 :bbab317.34428290
