
==== Front
Brief Bioinform
Brief Bioinform
bib
Briefings in Bioinformatics
1467-5463
1477-4054
Oxford University Press

10.1093/bib/bbae446
bbae446
Review
AcademicSubjects/SCI01060
Deep learning approaches for non-coding genetic variant effect prediction: current progress and future prospects
https://orcid.org/0000-0003-4444-6197
Wang Xiaoyu Monash Biomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Melbourne, VIC 3800, Australia
Monash Data Futures Institute, Monash University, Melbourne, VIC 3800, Australia

Li Fuyi South Australian immunoGENomics Cancer Institute (SAiGENCI), Faculty of Health and Medical Sciences, The University of Adelaide, Adelaide, SA 5005, Australia

Zhang Yiwen School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC 3004, Australia

Imoto Seiya Genome Center, Institute of Medical Science, The University of Tokyo, Minato-ku, Tokyo 108-8639, Japan
Collaborative Research Institute for Innovative Microbiology, The University of Tokyo, Bunkyo-ku, Tokyo 113-8657, Japan

Shen Hsin-Hui Department of Materials Science and Engineering, Faculty of Engineering, Monash University, Clayton, VIC 3800, Australia

Li Shanshan School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC 3004, Australia

Guo Yuming School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC 3004, Australia

Yang Jian School of Life Sciences, Westlake University, Hangzhou, Zhejiang 310030, China
Westlake Laboratory of Life Sciences and Biomedicine, Hangzhou, Zhejiang 310024, China

https://orcid.org/0000-0001-8031-9086
Song Jiangning Monash Biomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Melbourne, VIC 3800, Australia
Monash Data Futures Institute, Monash University, Melbourne, VIC 3800, Australia

Corresponding authors. Jiangning Song, Biomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Victoria 3800, Australia. Fax: +61 3 9902 9400; E-mail: Jiangning.Song@monash.edu; Jian Yang, School of Life Sciences, Westlake University, Hangzhou, China. E-mail: jian.yang@westlake.edu.cn
9 2024
13 9 2024
13 9 2024
25 5 bbae44602 4 2024
08 8 2024
27 8 2024
© The Author(s) 2024. Published by Oxford University Press.
2024
https://creativecommons.org/licenses/by/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Recent advancements in high-throughput sequencing technologies have significantly enhanced our ability to unravel the intricacies of gene regulatory processes. A critical challenge in this endeavor is the identification of variant effects, a key factor in comprehending the mechanisms underlying gene regulation. Non-coding variants, constituting over 90% of all variants, have garnered increasing attention in recent years. The exploration of gene variant impacts and regulatory mechanisms has spurred the development of various deep learning approaches, providing new insights into the global regulatory landscape through the analysis of extensive genetic data. Here, we provide a comprehensive overview of the development of the non-coding variants models based on bulk and single-cell sequencing data and their model-based interpretation and downstream tasks. This review delineates the popular sequencing technologies for epigenetic profiling and deep learning approaches for discerning the effects of non-coding variants. Additionally, we summarize the limitations of current approaches in variant effect prediction research and outline opportunities for improvement. We anticipate that our study will offer a practical and useful guide for the bioinformatic community to further advance the unraveling of genetic variant effects.

non-coding variants
variant effect prediction
machine learning
deep learning
Major Inter-Disciplinary Research Project
==== Body
pmcIntroduction

Most heritable diseases, such as Alzheimer’s disease, psychiatric disorders, coronary heart disease, and Parkinson’s disease, stem from the intricate interplay of genetic and environmental factors [1]. Genetic factors play a pivotal role in influencing individual susceptibility to these diseases. Emerging studies underscore the significance of variant effects of various diseases, such as Frazer et al. (Evolutionary model of variant effect [EVE]) [2], MacArthur et al. [3], and Lappalainen et al. [4]. Leveraging genome-wide association studies (GWAS), previous investigations have revealed the existence of over 200,000 genetic variants, linked to more than 3000 human traits [5]. Deciphering the function impact of these variants and their contributions to phenotypic changes is crucial for advancing precision medicine.

In the past decades, there has been a growing appreciation of the significance of non-coding sequences that were once considered ‘junk’ DNAs due to their unknown functions [6]. In the human genome, more than 98% is non-coding [7], harboring the vast majority (>90%) of disease-associated single nucleotide polymorphisms (SNPs) [8]. In contrast to protein-coding variants that directly impact protein structure or function, non-coding variants exert their influence through the modulation of gene regulatory processes. Therefore, understanding the function of non-coding variants can help unravel the gene regulatory process and decode human diseases.

In this review, we explore state-of-the-art deep learning methods for identifying the effects of the non-coding variants and emphasizing their application in downstream tasks and model interpretation. Additionally, we critically point out the limitations of current methodologies, followed by proposing promising directions for future research.

Non-coding region, gene regulation, and epigenetic modification

Non-coding regions refer to the DNAs that do not encode proteins, comprising a large proportion of the genomes. Non-coding regions often contain regulatory elements that regulate gene transcription, such as enhancers and promoters. The gene regulatory process operates across multiple levels, encompassing transcriptional, post-transcriptional, translational, and post-translational mechanisms. Non-coding variants primarily impact gene regulatory process through regulatory elements and chromatin status at the transcriptional level. These regulatory elements can be broadly classified into proximal/cis-regulatory elements (e.g., the promoter, enhancer, insulator, and silencers) and distal/trans-regulatory elements (e.g., Transcription Factor [TF] and non-coding RNA) that located at a considerable distance from the transcription start site (TSS) [9]. Understanding the functional significance of regulatory elements and unravelling their contributions to disease pathogenesis is a complex task. This involves extensive epigenetic modifications, encompassing histone modification, transcription factor (TF) binding, DNA accessibility, and genome topology (Fig. 1). Therefore, previous computational methods considered multiple epigenetic modifications (mentioned earlier) to comprehensively quantify non-coding variant effects.

Figure 1 An overview of hierarchical organization of chromatin structure and the gene regulatory process.

Histones are a class of proteins rich in lysine and arginine residues, forming nucleosomes by involving DNA wrapped around them, constituting the basic structural units of chromatin. Histone modifications, such as acetylation of arginine or lysine residues and lysine methylation, play a crucial role in gene regulation and expression. Histone acetylation, one of the most common histone modifications, regulates gene expression by reshaping the structure of chromatin, and some studies suggest that acetylation may be associated with gene activation and local chromatin opening [10]. Histone methylation, another critical modification, encompasses diverse forms with primary modification sites located predominantly on histone H3. Histone modification has been revealed to be closely linked to early embryonic development [11]. The NIH Roadmap Epigenomics Consortium proposed five core types of histone modifications [12], including H3K4me1 or H3K27ac (enhancer), H3K4me3 (promoter), H3K36me3 (transcribed regions), H3K27me3 (Polycomb repression), and H3K9me3 (heterochromatin).

Transcription factors are proteins that play a crucial role in transcription and the regulation of gene expression by binding to TF binding sites in cis-regulatory regions like enhancers and promoters. These TFs can influence cell differentiation and mutations within the TF binding sites that have been linked to multiple human diseases, including heart disease [13], diabetes [14], and chronic kidney disease [15].

Moreover, chromatin accessibility serves as a crucial indicator for evaluating the chromosomal state at a genome-wide scale. It represents the extent to which chromatinized DNA is available for interaction with other regulatory elements, providing insights into over 90% of TF binding sites [16] and potential regulatory gene locations. Typically, chromosomes exhibit intricate folding, exposing DNA only during the transcription process and facilitating the binding of other regulatory elements.

Furthermore, 3D chromatin architecture is intricately linked to gene regulation, influencing gene expression across various scales, encompassing nucleosome-nucleosome interaction, chromatin loops, topologically associating domains (TADs), and compartments [17]. Regulatory elements can physically interact with distant elements through chromatin loops, despite the considerable element distance. For instance, the promoter–enhancer loop. A recently noteworthy revelation, known as TADs, is the partitioning of chromatin into sub-megabase-scale domains based on the frequency of chromatin interactions, within which enhancer-promoter interactions predominantly occur. Additionally, numerous DNA-binding proteins, including CCCTC-binding factor (CTCF), yin yang 1 (YY1), and zinc finger protein 143 (ZNF143), in conjunction with the mediator complex, play essential roles in shaping chromatin structure and governing gene regulation.

Experimental technologies for interpretation of non-coding variants

The advancement of next-generation sequencing technologies, coupled with the decreasing costs of sequencing techniques, has resulted in a substantial increase in the accumulation of genome and epigenetic data. This presents a remarkable opportunity to transform precision medicine, disease diagnosis, and evolutionary biology, by unraveling the latent potential of non-coding DNA. The continuous advancements in sequencing technology have greatly enriched our understanding of genomics. The effects of variants can be observed in plenty of aspects, including TF binding, histone modification, chromatin accessibility, and chromatin structure. Among these, gene expression stands out as the most intuitionistic measurement for assessing variant effects. In this section, we provide an overview of the fundamental sequencing technologies employed for genome and epigenetic profiling in Table 1.

Table 1 Sequencing technologies for investigating gene regulation and variant effects

Data types	Methods	Pros	Cons	
Gene expression	DNA microarray [18]	Relatively low cost, no length bias	Rely on the known sequences	
RNA-seq [19]	Higher resolution, lower limit of detection	Relatively high cost, length inherent bias	
CAGE [20]	Can identify the TSS activity, high reproducibility	Only for total mature RNA, detection bias for long-lived transcripts	
DNA accessibility	DNase-seq [21]	Can examine regulatory regions	Limited by the intrinsic bias for DNase I	
ATAC-seq [22]	High efficiency, low cellular input requirement	Tn5 transposase binding bias, relatively high cost	
MNase-seq [23]	Maximize the preservation of the original chromatin structure	Easily lose some weak binding sites	
FAIRE-seq [24]	Not rely on enzymes	Lower resolution and signal-to-noise ratio	
Histone modification and TF binding	Chip-seq [25]	Higher resolution, less noise, and greater coverage	Require a large number of cells, antibody requirement, potential for epitope masking	
CUT&RUN [26]	Lower background noise, low cellular input requirement	Require end polishing and adapter ligation	
CUT&Tag [27]	Lower background noise, low cellular input requirement, low cost	Not suitable for low abundance targets, TF, and chromatin-associated proteins	
	DAP-seq [28]	No sample-specific reagents requirement, retain tissue/cell-line specific secondary modification	Relatively higher failure rate, not suitable for low abundance targets, only for TF binding events	
3D genome	Hi-C [29]	Detects all intra- and inter-chromosomal interactions	Lower resolution	
Micro-C [30]	Higher resolution	Fewer long-range or inter-chromosomal interactions	

The most widely employed sequencing technologies for identifying transcript abundances encompass DNA microarray [18], RNA-seq [19], and Cap Analysis Gene Expression (CAGE) [20]. DNA microarray technology quantifies the abundance of known genes or RNA species through binding targets to specific oligonucleotide probes. In contrast, RNA-seq utilizes high-throughput next-generation sequencing technologies not only to quantify RNA abundance but also to discover novel sequences and identify alternative splicing events. CAGE is a crucial method for measuring transcript abundances and comprehensively identifying the TSS at the single-nucleotide resolution. In comparison to DNA microarray and RNA-seq, CAGE not only measures gene expression level but also defines the promoter landscape. CAGE measures RNA expression by capturing the 5’end of capped RNA, corresponding to the TSS. This method provides valuable insights into the transcription dynamics, thereby facilitating variant effect prediction.

Chromatin immunoprecipitation sequencing (ChIP-seq) [25] is a powerful approach for researchers studying the interaction between DNA and DNA-binding proteins, such as TFs and histone with specific modifications, in the pan-genomic regulatory landscape. ChIP-seq has also been widely adopted for TF binding and histone modification identification. Antibodies participated in the ChIP-seq process by binding to the protein of interest present in the DNA sequences. This specific binding enriches the target DNA sequence, which is then processed into a DNA library. Therefore, the accuracy of ChIP-seq highly depends on the quality and specificity of antibodies. The genome segments are sequenced using high-throughput sequencing technologies and the sequenced fragments are aligned to the reference genome to identify the location of binding protein. Moreover, Cleavage under targets and release using nuclease (CUT&RUN) [26] and Cleavage under targets and tagmentation (CUT&Tag) [27] are also powerful methods for the target protein, with a high signal-to-noise ratio. To address the limitations associated with specific reagent requirements, such as antibodies or primers, DNA Affinity Purification Sequencing (DAP-seq) [28] has been recently developed. It enables the identification of genome-wide TF binding events while preserving tissue- or cell line-specific chemical modifications. However, DAP-seq has a relatively low success rate and is not well-suited for quantifying low-abundance proteins.

In addition to the TF binding and histone modification, chromatin accessibility is also crucial for studying gene regulation. Principal methods for measuring chromatin accessibility include DNase I hypersensitive site sequencing (DNase-seq) [21], Assay for transposase-accessible chromatin with high-throughput sequencing (ATAC-seq) [22], Micrococcal nuclease digestion with high-throughput sequencing (MNase-seq) [23], and Formaldehyde-Assisted Isolation of Regulatory Elements sequencing (FAIRE-seq) [24]. To examine the chromatin state, most sequencing technologies aim to identify unprotected DNA sequences in the open chromatin region. Several enzymes are adopted in different approaches, such as DNA endonuclease DNase I (DNase-seq), micrococcal nuclease (MNase-seq), and hyperactive Tn5 transposase (ATAC-seq). Unlike other sequencing methods, FAIRE-seq targets accessible DNA regions based on their lack of histone cross-linking. Coupled with high-throughput sequencing technology, these approaches can effectively identify DNA accessibility.

Advancements in sequencing technology over the past decade have significantly improved our understanding of chromosome conformation and nuclear architecture. A major breakthrough was the establishment of chromosome conformation capture (3C) technologies, enabling the investigation of chromosome conformation [31]. Subsequently, various 3C derivatives were introduced, including the widely used highest-throughput 3C (Hi-C) [29] and the micrococcal nuclease chromosome conformation assay (Micro-C) [30]. Hi-C is the most extensively employed method for comprehending genome-wide chromatin folding and interactions. Micro-c utilizes micrococcal nuclease (MNase) instead of restriction enzymes, allowing for the capture of interactions at a finer resolution due to MNase’s ability to generate more uniform fragmentation. However, Micro-C may lose some long-range inter-chromosomal interactions.

Single-cell sequencing technologies

Advancements in single-cell isolation and barcoding technologies have propelled significant development in single-cell sequencing technologies in recent years. In contrast to bulk sequencing technologies, single-cell sequencing is able to capture diverse cell states and developmental trajectories. Single-cell sequencing technology excels in profiling multi-omics data, encompassing gene expression, DNA modification, histone modification, DNA accessibility, and 3D genome architecture as summarized in Table 2.

Table 2 Single-cell sequencing technologies

	Sequencing technologies	
Gene expression	scRNA-seq, Smart-seq, Quartz-seq, Drop-Seq, DroNc-seq, CEL-seq, C1-CAGE, RamDa-seq	
DNA modification	scBS-seq, scRRBS	
Histone modification	scCHIP-seq	
DNA accessibility	scNOME-seq, scATAC-seq, scDNase-seq	
3D genome	scHi-C	

To assess gene expression (transcript expression), various single-cell sequencing methods based on whole-transcriptome amplification (WTA) have emerged. These methods include Smart-seq, Quartz-seq, CEL-seq, and C1-CAGE. Additionally, RamDa-seq is designed for measuring non-poly(A) transcripts, including long non-coding RNAs and enhancer RNAs. Complementing the WTA methods, several microdroplet-based protocols such as Drop-Seq and DroNc-seq, along with microwell-based methods like Seq-Well, have been further proposed.

Beyond gene expression, single-cell sequencing technologies are versatile for epigenetic profiling from multiple aspects, including DNA modification (single-cell bisulfite sequencing [scBS-seq] [32], single-cell reduced-representation bisulfite sequencing [scRRBS] [33]), DNA accessibility (single-cell nucleosome occupancy and methylome sequencing [scNOME-seq] [34], Single-cell sequencing assay for transposase-accessible chromatin [scATAC-seq] [35], single-cell DNase sequencing [scDNase-seq] [36]), Histone modification (single-cell chromatin immunoprecipitation DNA sequencing [scChIP-seq] [37]) and 3D genome (single-cell high-resolution 3C methodologies [scHi-C] [38]).

Moreover, driven by advancements in high-throughput technologies, there has been an exponential increase in high-quality sequencing data and publicly accessible databases, facilitating the comprehensive assessment of variant effects on a larger scale. These databases are summarized in Table 3.

Table 3 Publicly accessible epigenetic profile databases

Database	Histone modification	DNA accessibility	Gene expression	3D genome	Species	URL	Reference	
B	S	B	S	B	S	B	S	
ENCODE	✓		✓	✓	✓	✓	✓		Human/Mouse	https://www.encodeproject.org/	[39]	
Roadmap	✓		✓						Human	https://www.ncbi.nlm.nih.gov/geo/roadmap/epigenomics/	[12]	
IHEC	✓				✓				Human	https://epigenomesportal.ca/ihec/index.html	[40]	
Cistrome DB	✓		✓						Human/Mouse	http://cistrome.org/db/	[41]	
FANTOM5					✓		✓		Human/Mouse	https://fantom.gsc.riken.jp/5/	[42]	
4DN	✓	✓	✓	✓	✓		✓	✓	Multiple	https://data.4dnucleome.org/	[43]	
GEO	✓	✓	✓	✓	✓	✓	✓	✓	Multiple	https://www.ncbi.nlm.nih.gov/geo/	[44]	
Single Cell Portal				✓		✓			Multiple	https://singlecell.broadinstitute.org/single_cell	[45]	
PanglaoDB						✓			Human/Mouse	https://panglaodb.se/index.html	[46]	
Single cell expression atlas						✓			Multiple	https://www.ebi.ac.uk/gxa/sc/home	[47]	
HCA data portal						✓			Human/Mouse	https://data.humancellatlas.org/	[48]	
Abbreviations: Roadmap: Roadmap epigenomics database. IHEC: International Human Epigenome Consortium. 4DN: 4D Nucleome Data Portal. HCA: Human cell atlas. B: Bulk sequencing technologies. S: Single-cell sequencing technologies.

Deep learning approaches for analyzing the effects of non-coding variants

Given the significance of non-coding regions, numerous studies have dedicated substantial efforts to unravel the effects of non-coding variants [49]. Experimental methods for identifying variant effects are low-throughput, time-consuming, and labor-intensive. In this regard, deep learning approaches prove highly effective in deciphering variant effects, considering the remarkable complexity of gene expression and transcriptional regulation. The impact of non-coding variants is often described as alterations in the regulatory process and corresponding gene expression. A comprehensive and robust model can offer novel insights into the mechanisms of variant effects and their associations with human diseases and traits. In recent years, an increasing number of studies have leveraged cutting-edge deep learning-based technologies for predicting variant effects, driven by the emergence of high-throughput sequencing, public epigenetic data, and artificial intelligence advancements [50–53]. We provide an overview of the current deep learning approaches for analyzing the effects of non-coding variants in Table 4. To further evaluate the usability of non-coding variant models, we tested the models’ running time to generate the prediction results based on the web server or NVIDIA GeForce RTX 3090 (Table S1). The sequence length and model complexity are considered as the primary factors that influence the processing time.

Table 4 Summary of deep learning methods for non-coding variants

Name	Year	Type	TF	Histone mark	DNA accessibility	CAGE	3D genome	DNA methylation	Input length	Genome	Algorithms	Species	Cell-type	Website	
DeepBind	2015	Peak	509	–	–	–	–	–	14-101 nt	GRCh37/hg19	CNN	human, mouse	multiple	http://tools.genes.toronto.edu/deepbind/
 (not available)	
Deepsea	2015	Peak	690	104	125	–	–	–	1000 bp	GRCh37/hg19	CNN	human	multiple	http://deepsea.princeton.edu/job/analysis/create/	
Basset	2016	Peak	–	–	164	–	–	–	500-1000	GRCh37/hg19	CNN	human	164	NA	
DanQ	2016	Peak	690	104	125	–	–	–	1000 bp	GRCh37/hg19	CNN, BLSTMs	human	multiple	NA	
CpGenie	2017	Peak	–	–	–	–	–	50	500 bp	GRCh37/hg19	CNN	human	GM12878	NA	
Basenji	2018	Genetic track	–	2307	949	973	–	–	131 kb	GRCh37/hg19	CNN, dilated CNN	human	multiple	NA	
DeFine	2018	Peak	148	–	–	–	–	–	300 bp	GRCh37/hg19	CNN	human	K562, GM12878	http://define.cbi.pku.edu.cn/	
DeepFIGV	2019	Peak	–	3 histones for 75 individuals	69	–	–	–	300-2000 bp	GRCh37/hg19	CNN	human	multiple	NA	
NCNet	2019	Peak	690	104	125	–	–	–	1000 bp	GRCh37/hg19	Bottleneck Residual then Recurrent Network Model	human	multiple	NA	
ExPecto	2019	Peak	690	978	334	–	–	–	2000 bp	GRCh37/hg19	CNN + spatial transformation module + Linear model for tissue-specific expression	human	multiple (218)	http://hb.flatironinstitute.org/expecto	
TBiNet	2020	Peak	690	–	–	–	–	–	1000 bp	GRCh37/hg19	CNN, Attention BiLSTM	human	multiple	http://tbinet.korea.ac.kr/  
(Not available)	
Basenji2	2020	Genetic track	2131	1860	684/228	638/357	–	–	131 kb	GRCh38/hg38	CNN, dilated CNN	human mouse	multiple	NA	
DeepFun	2021	Peak	4795	1536	1548	–	–	–	1000 bp	GRCh37/hg19	CNN (similar to Basset)	human	225	https://bioinfo.uth.edu/deepfun/	
Enformer	2021	Genetic track	2131	1860	684	638	–	–	196 kb	GRCh38/hg38	CNN, residual block, transformer	human mouse	multiple	NA	
Sei	2022	Peak	9471	10,064	2373	–	–	–	4000 bp	GRCh38/hg38	Residual dual linear and nonlinear paths; residual dilated convolutional network; spatial basis function	human	multiple	https://hb.flatironinstitute.org/sei	
GraphReg	2022	Genetic track	–	H3K4me3, H3K27ac	DNase-seq	–	Hi-C, HiChIP, Micro-C, HiCAR	–	6 M	GRCh37/hg19	CNN, dilated CNN, Graph attention network	human mouse	multiple	NA	
aYear: Year of Publication.

In this review, we focus on the sequence-based model, particularly those with DNA sequences as the sole input. Figure 2 illustrates the typical workflow for predicting non-coding variant effects and Fig. 3 showcases the evolution of the deep learning methods in this domain. To predict non-coding variant effects, the input data of these models, namely DNA sequences, typically range from 300 bp (base pairs) to 6 Mb (megabase pairs), targeting descriptors such as histone modifications, TF binding, DNA accessibility, and gene expression. Commonly employed architectures for these models include convolutional neural networks (CNNs), dilated CNNs, bidirectional long short-term memory (BLSTM), and Transformers.

Figure 2 A typical workflow illustrating the application of machine learning approaches in predicting variant effects.

Figure 3 Deep learning methods for non-coding variant effects.

CNNs are a type of deep neural network widely used in computer vision and sequence analysis [54]. A fundamental element of CNNs is the convolutional kernel, which slides over the input data to capture underlying patterns. Specifically, in DNA sequence analysis, this kernel can detect motifs within the DNA sequence. CNN layers are typically used as the initial layers to extract features from DNA sequences. Dilated CNNs, a variant of standard CNNs, incorporate dilated kernels that have spaces between kernel elements. Such design allows them to capture distant features, thereby enabling them to capture long-distance dependencies within the data. These networks are typically used following CNN layers to integrate the extracted information. Apart from convolutional layers, long short-term memory (LSTM) networks are another popular machine learning algorithm for processing DNA sequences [55, 56]. LSTMs have a set of gates that manage the flow of information across long sequences. The forget gate discards irrelevant information from the previous time step while allowing relevant information to persist within the network. Given the bidirectional nature of DNA, which consists of two polynucleotide chains, bidirectional LSTMs, incorporating both forward and backward LSTMs, are more commonly used than unidirectional LSTMs due to their excellent ability to effectively handle this two-way data flow.

Transformer is another type of deep learning method that is also widely used to model sequence data [57]. It includes the multi-head attention mechanism that uses multiple self-attention layers to learn the relationships between the tokens (i.e., learned motifs). The attention formula can be expressed as follows:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $$ Attention\left(Q,K,V\right)= softmax\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)V $$\end{document}

where the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $Q$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $K$\end{document}, V stand for query, key, and value, respectively, all of which are generated by the same feature vector. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${d}_k$\end{document}is the dimensionality of the key. To integrate the positional information of the sequence, Transformers use the positional encoding to weight each token.

Additionally, a multi-task learning strategy was employed to integrate the information from different epigenetic prediction tasks. In contrast to single-task learning, multi-task learning typically shares some common layers and parameters within a single model, thereby facilitating the simultaneous training of multiple tasks. Each task utilizes specific parameters to address the differences between tasks. In the context of epigenetic profiling, the shared parameters might include the common motifs learned across different tasks and potential relationships between these motifs in DNA sequences. Accordingly, this approach can enable the overall model to more comprehensively learn the gene regulatory process from multiple tasks.

For the non-coding variants model, we categorize deep learning methods into peak-based and genetic track-based methods based on the target formats.

Peak-based binary classification approaches

Peak-based methods center around identifying the presence or absence of peaks within designated regions. These methods leverage peak calling algorithms to define target peaks, ultimately addressing a binary classification problem. In 2015, Alipanahi et al. introduced, DeepBind, a deep learning framework using the position weight matrices (PWMs) of sequences for identifying sequence specificities of DNA- and RNA-binding proteins. Shortly after, Zhou et al. proposed DeepSEA [53], the first fully sequence-based deep learning model for non-coding variant effect prediction. Unlike previous approaches, DeepSEA takes the DNA sequence as input, instead of the known motifs or other handcrafted features. DeepSEA employs a deep convolutional network (CNN) with three convolution layers and one fully connected layer. DeepSEA is trained on the whole genome functional annotation on the single-nucleotide resolution with the input of 1000 bp DNA sequence. Multi-tasking learning is adopted for large-scale chromatin-profiling data, which involves the 690 TF binding profiles, 104 histone-mark profiles, and 125 DNase I hypersensitive site profiles, to share the parameters. The variant effect was measured by a log2 fold change of odds. Subsequently, to help understand gene accessibility and interpret the function of non-coding regions, Kelley et al. introduced Basset [58] to predict DNA accessibility in 164 cell types Using a CNN model with three convolutional layers and two fully connected layers. Basset is able to capture the DNA binding motifs by PWMs in their initial convolution layer. The emergence of new deep learning networks also facilitated the identification of non-coding variant effects. DanQ [59] achieved a better performance than DeepSEA by combining the CNN model and BLSTM. Specifically, in addition to CNN layers, DanQ utilized one convolutional layer to capture the motif and a BLSTM layer to learn the grammar of regulation. In 2019, Zhou et al. proposed ExPecto [50] (DeepSEA2.0), an expansion of the DeepSEA model, which contains three key components: a deep learning model for epigenetic profile prediction, a spatial transformation module, and tissue-specific expression models. Compared with DeepSEA, ExPecto increased training epigenetic profiles from 919 to 2002, incorporating 334 DNase-seq profiles, 978 histone modification profiles, and 690 TF binding profiles for 218 tissues and cell types. The input length expanded to 2000 bp and thus enhanced information capture substantially. The deep learning framework employed in ExPecto resembles DeepSEA, with two additional modules included for spatial and tissue-specific information. TBiNet [60] focuses on the TF binding sites prediction and leveraging a combination of CNN, attention mechanism, and BiLSTM. DeFine [61] is a CNN model designed for the TF binding intensity prediction using TFs Chip-seq data from K562 and GM12878 cell lines. DeepFIGV [62], sharing the framework of Basset, uses distinct training data, including three kinds of histone modification profiles and one DNA accessibility profile. Inspired by deep residual networks, NCNet [63] enhances identification performance by incorporating bottleneck residual and recurrent neural networks based on DanQ. Moreover, DeepFun [64] offers a user-friendly web server, employing a CNN model akin to the Basset [58] with three CNN layers and two fully connected layers.

A recent breakthrough is Sei proposed by Chen et al. [51], a deep-learning framework designed for predicting regulatory activity and elucidating the mechanisms underlying disease-related variants. Sei emerges as the most comprehensive model for the regulatory network to date. It leverages an extensive dataset encompassing 9471 TF binding profiles, 10 064 histone mark profiles, and 2373 chromatin accessibility profiles, encompassing a total of 21 907 cis-regulatory elements. The identification of sequence classes from the whole genome contributes to a broader understanding of global regulatory activities. Sei has successfully unveiled potential mechanisms for cell type-specific regulatory activities and the identification of disease-associated variants based on the sequence class.

Genetic tracks-based regression approaches

This section delves into regression approaches that utilize quantitative genomic profiles (genomic tracks) for model training, contrasting with binary targets (peak calls) employed in previous studies. The use of genomic tracks offers a richer source of information compared to peak calling. To expand the Basset model [58], Kelley et al. extended their research to encompass a broader array of epigenetic and expression profiles, including TF binding, chromatin accessibility, and histone modification. The training dataset encompasses 949 DNase-seq profiles, 2307 histone modification profiles, and 973 CAGE-seq profiles. They predicted genomic profiles at a finer resolution through introducing a deep learning model named Basenji [65]. Basenji incorporates four convolutional layers and seven dilated convolutional layers that capture up to 131 kb distal influences. Consequently, the test accuracy of this model improved with an increased receptive field width.

While earlier studies predominantly focused on human genomic information, incorporating data from multiple species in model training can enhance the transfer of annotations from other species to the human genome. Recognizing the potential of cross-species information, Kelley introduced Basenji2 [66], a model trained simultaneously on human and mouse data. Basenji2 utilized 638 CAGE profiles, 684 DNase-seq and ATAC-seq profiles, and 3991 Chip-seq profiles in human and 357 CAGE profiles. 228 DNase-seq and ATAC-seq profiles and 1058 Chip-seq profiles for mouse. The initial stage of Basenji2 employs a CNN model to extract motifs from the DNA sequence. Subsequently, residual networks are incorporated, featuring a dilated convolutional layer to capture information in long sequences and mitigate the vanishing gradient issue. Finally, a linear transform is involved to predict human and mouse regulatory activities.

Harnessing the potency of Transformer, Avsec et al. proposed Enformer, one of the most powerful deep learning algorithms renowed for its excellence in natural language processing, [52]. This transformer-based model utilized the same training dataset as Basenji2, marking a significant stride in DNA sequence analysis akin to language processing. Following a pattern similar to prior research, the initial module of Enformer comprises seven convolutional layers, complemented by an architecture featuring eleven transformer layers. A noteworthy enhancement in Enformer is the expanded receptive field, allowing for the integration of long-range information. In its final stage, Enformer incorporates organism-specific heads for human and mouse data, enabling adaptability across different species. In a related vein, Zrimec et al. used deep learning methods to unravel gene expression [67]. Karbalayghareh et al. introduced a novel non-coding variant effect prediction method, GraphReg, that breaks new ground in the field. This groundbreaking approach integrates 3D genome information, amalgamating CNN, dilated CNN, and graph attention network components to predict epigenetic profiles and gene expression.

In the realm of regulatory activity prediction, CNN stands out as the predominant architecture, particularly as the first layer. The initial convolution layer, equipped with various position-weight matrices, excels in motif detection. Empirical evidence suggests that convolutional layers combined with max-pooling are highly effective for extracting TFs motifs. An essential consideration in non-coding variants prediction is the choice between binary (peak call) and quantitative (genetic tracks) targets. One of the foremost challenges is managing long DNA sequence and their corresponding deep neural networks. To address this, two main strategies are adopted: refining the model structure and lowering the resolution of the target. This evolution has seen the transition from CNN to dilated CNN and then to the transformer, thereby enlarging the receptive field of the input in prediction model from 1 kb to 196 kb. Some models, like Enformer and Basenji2, have reduced resolution to better accommodate distal regulatory elements. Furthermore, multi-tasking learning is frequently employed in non-coding variant prediction to leverage shared parameters across different tasks, enhancing overall model performance.

3D genome structure prediction

In addition to epigenetic modification, the regulatory activities of the genome are notably influenced by its 3D structure. Certain mutations can alter chromatin contacts, thereby affecting the regulation of gene expression. To predict genome folding, recent deep learning methods like DeepC [68], Akita [69], and Orca [70] have been developed. Specifically, DeepC [68] utilizes transfer learning for chromatin folding prediction. It features a convolutional module with max pooling, followed by a dilated convolutional module, culminating in a fully connected layer that synthesizes the learned information for chromatin folding prediction. Fudenbery and Kelley, along with their colleagues, proposed Akita [69], which also predicts 3D genome folding. Inspired by Basenji, the first module of Akita, comprises 1D convolutional and dilated residual convolutional layers. To mimic chromatin interactions, the 1D output of this model is then converted into 2D format by averaging the outputs across genomic bin pairs. Subsequently, multiple symmetric dilated residual 2D convolutional layers are employed. Finally, the model compares its predicted map with the target map. Both Akita and DeepC process ~1 Mb DNA sequence as input but differ in their data processing strategies; DeepC encodes Hi-C data as a vector, whereas Akita converts 1D input into a 2D map. Another important contribution to this field is Orca [70], proposed by Zhou in 2022. Orca is a multi-scale deep learning model with a multi-resolution encoder and cascading decoder, designed to predict the chromatin interaction frequencies across scales from kilobase to whole chromosome.

Cis-regulatory element prediction

The identification of non-coding variant effects can also be approached through the prediction of cis-regulatory elements. We summarize the deep learning methods for the cis-regulatory elements in Table 5. Currently, most of the methods are sequence-based models that primarily concentrate on the identification of a single type of cis-regulatory element. For the promoter prediction, previous methods only take the DNA sequence with length normally less than 600 bp as the input, such as DeeProPre [75] and HMPI [70]. For the Enhancer identification and strength prediction, a state-of-the-art method is ADH-Enhancer [79, 81], which is an attention-based hybrid deep learning model that combines the CNN, LSTM and attention module. Compared with the promoter and enhancer, the identification of the silencer is more complex. However, the mechanism of the silencer is unclear. Therefore, various methods are proposed to incorporate more genetic information to facilitate the prediction. For instance, DeepICHS [84] improves the accuracy of prediction through incorporating epigenetic profiling data. DeepICHIS uniquely integrates DNA sequence information with multi-omics data, including histone modification, TF binding, and DNA accessibility, to identify cell-specific silencers. Another example is SilenceREIN [85], which is the first attempt to combine the DNA sequence and chromatin structure information for silencer identification. SilenceREIN leverages the structure information of the regulatory element interaction network to locate the silencer on anchors of chromatin loops. In summary, deep learning methods have been widely used for single cis-regulatory element identification. However, the impact of structure information and the epigenetic profile for cis-regulatory element identification has been overlooked.

Table 5 Summary of deep learning methods for cis-regulatory elements identification

Regulatory elements	Methods	Year	Algorithm	Input length	Inputs	Reference	
Promoter	CNNProm	2017	CNN	81 bp	DNA sequences	[71]	
DeeReCT-PromID	2019	CNN	600 bp	DNA sequences	[72]	
DeePromoter	2019	CNN, BiLSTM	300 bp	DNA sequences	[73]	
HM-Prom	2021	CNN	300 bp	DNA sequences	[74]	
HMPI	2022	CNN	251 bp	DNA sequence, DNA structure	[75]	
DeeProPre	2022	CNN, BiLSTM, Attention	300 bp	DNA sequence	[75]	
Enhancer	DeepEnhancer	2016	CNN	300 bp	DNA sequences	[76]	
BiRen	2017	CNN, GRU, BRNN	1000 bp	DNA sequences	[77]	
iEnhancer-GAN	2021	CNN, GAN	200 bp	DNA sequences	[78]	
SENIES	2022	CNN	200 bp	DNA sequences, DNA structure	[79]	
iEnhancer-BERT	2022	CNN, Bert, Transfer learning	200 bp	DNA sequence	[80]	
	ADH-Enhancer	2024	CNN, LSTM, Attention	200 bp	DNA sequence	[81]	
Silencer	Huang et al.	2019	SVM	–	DNA sequences, Histone modification, DNA accessibility	[82]	
DeepSilencer	2021	CNN	–	DNA sequences	[83]	
DeepICSH	2023	CNN, Attention, Autoencoder	200 bp	DNA sequence, Histone modification, DNA accessibility, TF binding	[84]	
SilenceREIN	2024	CNN, GraphSAGE, MLP	600 bp	DNA sequence, Histone modification, TF binding,	[85]	
aYear: Year of Publication.

Phenotypic variant effect prediction

In this review, we primarily focus on computational methods that predict the effect of non-coding variants at the transcription level. However, given the advancements in deep learning models for predicting the effects of phenotypic variants, this section also explores models that can be used to predict phenotypic variant effects. Phenotypic variants are more complex than non-coding counterparts, involving multiple processes including post-transcriptional, translational, and post-translational processes. There are increasing research efforts devoted to predicting phenotype-associated variant effects, incorporating diverse types of information such as individual variants, structural information, and coding sequences. For example, DeepPerVar [86] is a CNN-LSTM multi-modal model that integrates individual variants with epigenetic data to improve phenotype-specific predictions. GenNet is another deep learning approach that utilizes individual variants to predict phenotypes [87]. Sequence UNET leverages the protein sequences to predict the variant frequency, with the purpose of better understanding their impacts on phenotypes [88]. Additionally, another method Cue integrates structural information to predict multiple types of complex structural variants that have been linked to genetic diseases [89].

Predicting with the single-cell sequencing technologies

In the realm of single-cell sequencing, which provides a granular view of cellular states and development, developing effective deep-learning models is challenging due to data’s scarcity and relatively limited volume. Recent advancements have brought forward several deep-learning approaches for deciphering gene regulation at the single-cell level, and we outlined these approaches in Table 6. Distinct from methods for bulk variant analysis, these models often employ a crucial strategy: leveraging variant effect models trained on bulk sequencing data to refine single-cell prediction. For instance, Huatuo [90] employs the Expecto model to extract chromatin embeddings and uses an XGBoost model for predicting single-cell transcriptome profiles, particularly focusing on scRNA-seq based gene expression. Similarly, Seq2cell leverages the pre-trained Enformer for epigenetic embedding generation, enabling the integration of distal information up to 196 kb in its analysis.

Table 6 Summary of deep learning methods based on single-cell sequencing technologies

Name	Year	Targets	Algorithms	Inputs length	Species	Cell-type	
ScFAN	2020	TF binding	CNN	1000 bp	Human	3	
scBasset	2022	DNA accessibility	CNN	1344	Human/mouse	multiple	
nvwa	2022	Gene expression	CNN, RNN	13 kb	8	multiple	
scEpoLock	2022	DNA accessibility	CNN	1000 bp	Human	5	
Huatuo [90]	2023	Gene expression	Transfer learning, XGBoost	40 kb	Human	multiple	
Seq2cell [91]	2023	Gene expression	Transfer learning	196 kb	Human	multiple	
scXpresso	2023	Gene expression	CNN	10.5 kb	Human	multiple	
aYear: Year of Publication.

Interpretation and Down-stream tasks

The breakthrough advances in deep learning models for non-coding variants have transformed the comprehension of gene regulation and disease-associated variants. Due to the qualitative nature of variant effects, the interpretation and evaluation of non-coding variants are conducted through various tasks. These include motif identification, prioritization of functional SNPs, cell-specific gene expression analysis, identification of regulatory elements, and analysis of disease-associated loci, as detailed in Table 7. Drawing from existing research, we classify these tasks into two broad categories: model-based interpretation and downstream tasks.

Table 7 Interpretation and down-stream tasks of non-coding variants models

Name	Year	Motif analysis	In silico saturation mutagenesis	Functional SNP prioritization	Regulatory elements analysis	Disease-associated loci	
DeepBind	2015	✓					
Deepsea	2015		✓	✓			
Basset	2016		✓	✓			
DanQ	2016	✓	✓	✓			
CpGenie	2017	✓		✓			
Basenji	2018		✓	✓	✓	✓	
DeFine	2018	✓		✓			
DeepFIGV	2019	✓		✓			
NCNet	2019						
ExPecto	2019		✓				
TBiNet	2020	✓					
Basenji2	2020			✓	✓		
DeepFun	2021	✓	✓				
Enformer	2021		✓	✓	✓		
Sei	2022					✓	
GraphReg	2022	✓			✓	✓	
aYear: Year of Publication.

Model-based interpretation

Motif analysis

Motifs-based methods are a predominant strategy for uncovering hidden patterns in sequence-to-activity models, particularly in CNN models. As illustrated in Fig. 4A, the first layer neurons of the convolutional network are adept at detecting motifs by screening the DNA sequence with a PWM. Some studies have extracted these motifs and compared them with TF binding motifs from databases like JASPAR [92]—an open-source repository containing thousands of non-redundant TF binding profiles. These investigations confirm that motifs identified by the model align with experimentally validated motifs. In layers beyond the first convolutional layer, such as the second layer or deeper, the neurons can represent interactions between motifs or longer motif structures.

Figure 4 Interpretation and down-stream tasks.

In silico saturation mutagenesis

Another model-based interpretation method is implemented based on the in-silico saturation mutagenesis, which is among the most crucial approaches for variant effect models. Given the challenges of directly quantifying the variant effect, the prevailing strategy simulates variant effects by involving substantial alterations in multiple epigenetic profiles resulted from a base substitution. The effects of the variant can be defined in various formats. For instance, in DeepSEA, the impact of a based substitution is quantified by the log2 fold change:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $$ effect={\log}_2\left(\frac{P_0}{1-{P}_0}\right)-{\log}_2\left(\frac{P_1}{1-{P}_1}\right) $$\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${P}_0$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} ${P}_1$\end{document} are the probability of the original sequence and the predicted mutated sequence, respectively.

Down-stream tasks

Functional SNP prioritization

Genome-wide association studies (GWAS) serve as a cornerstone for discovering disease genes. GWAS have revealed thousands of genetic variants associated with human diseases and traits [93]. Traditional GWAS analyses assume linear relationships between genetic variants and phenotypes. However, the causal effects between variants and phenotypes are often complex and nonlinear. Additionally, some variants are in linkage disequilibrium, displaying strong correlations with other variants. This correlation can pose challenges in pinpointing disease-causing mutations. Artificial intelligence methods can enhance the discovery of disease susceptibility loci and the understanding of variant effects. Computational models can identify Expression quantitative trait loci (eQTL) [94] without measuring thousands of individual gene expression profiles. Functional SNP prioritization employs two strategies: (i) Computing the correlation between predicted variants and eQTL variant effects using methods such as Spearman correlation and the Signed Linkage Disequilibrium Profile regression score. (ii) Constructing a model that utilizes embeddings from the non-coding variant prediction model to identify eQTLs.

Gene regulation elements identification

Deep learning methods not only aid in predicting non-coding variant effects but also provide a global interpretation of the gene regulatory network. By analyzing predicted changes in each profile, potential regulatory mechanisms and their corresponding cis-regulatory elements can be inferred. This offers new perspectives on regulatory element identification, including enhancer-promoter interaction identification, cell type or tissue-specific enhancers identification, and distal regulatory element identification. Enhancer plays a crucial role in mammalian transcriptional control, influencing cell differences and the timing of the cell cycle. Several studies have demonstrated the ability of deep learning models on prioritizing enhancers with high accuracy [52], leveraging the attention mechanism or the gradient × input as contribution score. Apart from enhancers, insulator elements, usually located near TAD boundaries, are also essential for gene regulation.

Disease-associated loci analysis

Unraveling the mechanisms underlying diseases and pinpointing disease-associated genetic variants is a formidable challenge. Leveraging deep learning models trained on large-scale datasets, several studies have begun to shed light on disease mechanisms from new angles. Analyzing changes in rich chromatin profiles utilized by non-coding variant models can provide insights into the disease-associated pathways impacted by these variants. For example, Basset [58] defined SNP Accessibility Difference profiles to pinpoint causal variants associated with autoimmune diseases. Similarly, Basenji [65] demonstrated how certain loci associated with autoimmune diseases and blood cell traits could alter transcription or chromatin profiles of genes. Additionally, Sei offered a comprehensive map of mutation effects, classifying and quantifying the direction and magnitude of variant effects at the sequence class level, encompassing all 853 regulatory diseases.

Conclusions and future perspectives

In this review, we have centered on exploring sequence-to-activity models concerning the impact of non-coding variants at the transcriptional level. The past few years have witnessed a surge in efforts to unravel the complexities of gene regulation and the effects of genetic variants. In the face of the intricate nature of gene regulation and the rapidly expanding trove of genomic data, deep learning has risen as a pivotal tool. Its exceptional proficiency in detecting complex patterns has facilitated several groundbreaking discoveries, ranging from gene expression prediction to the identification of crucial regulatory elements. Despite these advancements, there are still opportunities within this field remains for further enhancement and development.

The evolution of deep learning methods in genomics is characterized by increasing reliance on longer DNA sequence inputs, cross-species training, and expansive training datasets. Analysis of recent models reveals that longer DNA sequences enable more comprehensive information capture. This is crucial because gene expression can be influenced by trans-regulatory elements located distantly from the TSS. However, challenges still lie in effectively managing and interpreting long-range information as input lengths increase. Therefore, it is essential to develop robust deep learning models capable of handling these long-range interactions with minimal information loss. Another promising area of development is the integration of 3D genome data, specifically chromatin interaction information. While traditional studies have focused primarily on sequence distances, incorporating the spatial gene positioning of genes—critical for understanding gene regulation—could significantly enhance model accuracy and insights.

Furthermore, current models for predicting the effects of individual genomic variants on gene expression may not always achieve the desired level of accuracy [95]. For instance, Sasse et al. [95] observed the discrepancies in gene expression prediction accuracy when applying Enformer to genomic data from 839 individuals, particularly in the predicted direction (i.e., increase or decrease) of gene expression changes. Some of the predictions displayed a negative correlation with observed gene expression, suggesting that the model performed poorly for predicting the expression of certain genes. Furthermore, the input sequence length used by some models exceeded 131 kb, leading to a considerably large potential sequence space. To improve accuracy in gene expression analysis, it is vital to incorporate individual variant information. This approach deviates from traditional methods that rely on a single reference genome, emphasizing the need to consider locus-specific variants that can influence gene expression. In addition to DNA-based sequence analysis, post-transcriptional processes, including RNA-protein and RNA–RNA interactions, also play a significant role in impacting the effects of variants. However, to our knowledge, there are currently no methods available that take into consideration the effect of such post-transcriptional processes. Therefore, a comprehensive approach that accounts for the RNA abundance in combination with DNA analysis is crucial and desirable to better model and interpret the impact of non-coding variants on gene expression processes.

Key Points

We reviewed the deep learning methods for the effect of non-coding variant prediction, including the methods based on bulk sequencing and single-cell sequencing. Additionally, we comprehensively introduced the sequencing methods and publicly available databases relevant to the non-coding variants prediction.

Current sequence-to-activity methods for non-coding variants focus on the incorporation of long-range information and large-scale epigenetic information across different cell types or species. The model-based interpretation and downstream tasks can facilitate the understanding of the variant effects.

Accurately predicting the effect of non-coding variants remains challenging. Future research that integrates more information, such as long-range interaction information, individual variants, and RNA interaction information, is crucial for fully understanding the gene regulatory process and variants effects.

Supplementary Material

Supplementary_document_bbae446

Acknowledgements

This work was financially supported by the National Health and Medical Research Council of Australia (APP1127948, APP1144652), Major and Seed Inter-Disciplinary Research projects awarded by Monash University. This work was also partially supported by the International Collaborative Research Program of Institute for Chemical Research, Kyoto University (2023-30 and 2024-26) and a Grant from the International Joint Usage/Research Center, Institute of Medical Science, The University of Tokyo (K23-2074 and K24-2125).

Funding

Financial support to JS from Monash University (Major Inter-Disciplinary Research Project) is gratefully acknowledged.

Conflict of interest: The authors declare that they have no conflict of interest.
==== Refs
References

1. Hunter  DJ . Gene–environment interactions in human diseases. Nat Rev Genet  2005;6 :287–98.15803198
2. Frazer  J, Notin  P, Dias  M. et al.  Disease variant prediction with deep generative models of evolutionary data. Nature  2021;599 :91–5.10.1038/s41586-021-04043-8.34707284
3. MacArthur  DG, Manolio  TA, Dimmock  DP. et al.  Guidelines for investigating causality of sequence variants in human disease. Nature  2014;508 :469–76.10.1038/nature13127.24759409
4. Lappalainen  T, MacArthur  DG. From variant to function in human disease genetics. Science  2021;373 :1464–8.34554789
5. Buniello  A, MacArthur  JAL, Cerezo  M. et al.  The NHGRI-EBI GWAS catalog of published genome-wide association studies, targeted arrays and summary statistics 2019. Nucleic Acids Res  2019;47 :D1005–12.10.1093/nar/gky1120.30445434
6. Consortium EP . An integrated encyclopedia of DNA elements in the human genome. Nature  2012;489 :57.22955616
7. Venter  JC, Adams  MD, Myers  EW. et al.  The sequence of the human genome. Science  2001;291 :1304–51.11181995
8. Schaub  MA, Boyle  AP, Kundaje  A. et al.  Linking disease associations with regulatory information in the human genome. Genome Res  2012;22 :1748–59.10.1101/gr.136127.111.22955986
9. Cheung  VG, Spielman  RS. Genetics of human gene expression: mapping DNA variants that influence gene expression. Nat Rev Genet  2009;10 :595–604.10.1038/nrg2630.19636342
10. Shvedunova  M, Akhtar  A. Modulation of cellular processes by histone and non-histone protein acetylation. Nat Rev Mol Cell Biol  2022;23 :329–49.35042977
11. Jambhekar  A, Dhall  A, Shi  Y. Roles and regulation of histone methylation in animal development. Nat Rev Mol Cell Biol  2019;20 :625–41.10.1038/s41580-019-0151-1.31267065
12. Kundaje  A, Meuleman  W, Ernst  J. et al.  Integrative analysis of 111 reference human epigenomes. Nature  2015;518 :317–30.10.1038/nature14248.25693563
13. Reschen  ME, Gaulton  KJ, Lin  D. et al.  Lipid-induced epigenomic changes in human macrophages identify a coronary artery disease-associated variant that regulates PPAP2B expression through altered C/EBP-beta binding. PLoS Genet  2015;11 :e1005061.25835000
14. Gaulton  KJ, Ferreira  T, Lee  Y. et al.  Genetic fine mapping and genomic annotation defines causal mechanisms at type 2 diabetes susceptibility loci. Nat Genet  2015;47 :1415–25.10.1038/ng.3437.26551672
15. Köttgen  A, Glazer  NL, Dehghan  A. et al.  Multiple loci associated with indices of renal function and chronic kidney disease. Nat Genet  2009;41 :712–7.10.1038/ng.377.19430482
16. Thurman  RE, Rynes  E, Humbert  R. et al.  The accessible chromatin landscape of the human genome. Nature  2012;489 :75–82.22955617
17. Bonev  B, Cavalli  G. Organization and function of the 3D genome. Nat Rev Genet  2016;17 :661–78.10.1038/nrg.2016.112.27739532
18. Heller  MJ . DNA microarray technology: devices, systems, and applications. Annu Rev Biomed Eng  2002;4 :129–53.12117754
19. Marguerat  S, Bähler  J. RNA-seq: from technology to biology. Cell Mol Life Sci  2010;67 :569–79.10.1007/s00018-009-0180-6.19859660
20. Shiraki  T, Kondo  S, Katayama  S. et al.  Cap analysis gene expression for high-throughput analysis of transcriptional starting point and identification of promoter usage. Proc Natl Acad Sci  2003;100 :15776–81.14663149
21. Song  L, Crawford  GE. DNase-seq: a high-resolution technique for mapping active gene regulatory elements across the genome from mammalian cells. Cold Spring Harb Protoc  2010;2010 :pdb.prot5384.20150147
22. Buenrostro  JD, Wu  B, Chang  HY. et al.  ATAC-seq: a method for assaying chromatin accessibility genome-wide. Curr Protoc Mol Biol  2015;109 :21.29.1–9.10.1002/0471142727.mb2129s109.
23. Cui  K, Zhao  K. Genome-wide approaches to determining nucleosome occupancy in metazoans using MNase-seq Chromatin Remodeling: Methods and Protocols. Methods Mol Biol  2012;833 :413–9.22183607
24. Giresi  PG, Kim  J, McDaniell  RM. et al.  FAIRE (formaldehyde-assisted isolation of regulatory elements) isolates active regulatory elements from human chromatin. Genome Res  2007;17 :877–85.17179217
25. Kaufmann  K, Muino  JM, Østerås  M. et al.  Chromatin immunoprecipitation (ChIP) of plant transcription factors followed by sequencing (ChIP-SEQ) or hybridization to whole genome arrays (ChIP-CHIP). Nat Protoc  2010;5 :457–72.10.1038/nprot.2009.244.20203663
26. Skene  PJ . Henikoff S, An efficient targeted nuclease strategy for high-resolution mapping of DNA binding sites. Elife  2017;6 :e21856.
27. Kaya-Okur  HS, Wu  SJ, Codomo  CA. et al.  CUT&tag for efficient epigenomic profiling of small samples and single cells. Nat Commun  2019;10 :1930.31036827
28. Bartlett  A, O'Malley  RC, S-sC  H. et al.  Mapping genome-wide transcription-factor binding sites using DAP-seq. Nat Protoc  2017;12 :1659–72.10.1038/nprot.2017.055.28726847
29. Van Berkum  NL, Lieberman-Aiden  E, Williams  L. et al.  Hi-C: a method to study the three-dimensional architecture of genomes. JoVE (J Vis Exp)  2010;e1869.
30. Hsieh  T-HS, Weiner  A, Lajoie  B. et al.  Mapping nucleosome resolution chromosome folding in yeast by micro-C. Cell  2015;162 :108–19.10.1016/j.cell.2015.05.048.26119342
31. Jerkovic´  I, Cavalli  G. Understanding 3D genome organization by multidisciplinary methods. Nat Rev Mol Cell Biol  2021;22 :511–28.33953379
32. Smallwood  SA, Lee  HJ, Angermueller  C. et al.  Single-cell genome-wide bisulfite sequencing for assessing epigenetic heterogeneity. Nat Methods  2014;11 :817–20.10.1038/nmeth.3035.25042786
33. Guo  H, Zhu  P, Wu  X. et al.  Single-cell methylome landscapes of mouse embryonic stem cells and early embryos analyzed using reduced representation bisulfite sequencing. Genome Res  2013;23 :2126–35.10.1101/gr.161679.113.24179143
34. Pott  S . Simultaneous measurement of chromatin accessibility, DNA methylation, and nucleosome phasing in single cells. Elife  2017;6 :e23203.28653622
35. Pott  S, Lieb  JD. Single-cell ATAC-seq: strength in numbers. Genome Biol  2015;16 :1–4.25583448
36. Jin  W, Tang  Q, Wan  M. et al.  Genome-wide detection of DNase I hypersensitive sites in single cells and FFPE tissue samples. Nature  2015;528 :142–6.10.1038/nature15740.26605532
37. Rotem  A, Ram  O, Shoresh  N. et al.  Single-cell ChIP-seq reveals cell subpopulations defined by chromatin state. Nat Biotechnol  2015;33 :1165–72.10.1038/nbt.3383.26458175
38. Nagano  T, Lubling  Y, Várnai  C. et al.  Cell-cycle dynamics of chromosomal organization at single-cell resolution. Nature  2017;547 :61–7.10.1038/nature23001.28682332
39. Luo  Y, Hitz  BC, Gabdank  I. et al.  New developments on the encyclopedia of DNA elements (ENCODE) data portal. Nucleic Acids Res  2020;48 :D882–9.10.1093/nar/gkz1062.31713622
40. Bujold  D, de  Lima Morais  DA, Gauthier  C. et al.  The international human epigenome consortium data portal. Cell Syst  2016;3 :496–9. e492.27863956
41. Zheng  R, Wan  C, Mei  S. et al.  Cistrome data browser: expanded datasets and new tools for gene regulatory analysis. Nucleic Acids Res  2019;47 :D729–35.10.1093/nar/gky1094.30462313
42. Noguchi  S, Arakawa  T, Fukuda  S. et al.  FANTOM5 CAGE profiles of human and mouse samples. Sci Data  2017;4 :170112.28850106
43. Reiff  SB, Schroeder  AJ, Kırlı  K. et al.  The 4D Nucleome data portal as a resource for searching and visualizing curated nucleomics data. Nat Commun  2022;13 :2365.35501320
44. Barrett  T, Wilhite  SE, Ledoux  P. et al.  NCBI GEO: archive for functional genomics data sets—update. Nucleic Acids Res  2012;41 :D991–5.23193258
45. Tarhan  L, Bistline  J, Chang  J. et al.  Single Cell Portal: an interactive home for single-cell genomics data, bioRxiv, 2023.
46. Franzén  O, Gan  L-M, Björkegren  JL. PanglaoDB: a web server for exploration of mouse and human single-cell RNA sequencing data. Database  2019;2019 :baz046.30951143
47. Papatheodorou  I, Moreno  P, Manning  J. et al.  Expression atlas update: from tissues to single cells. Nucleic Acids Res  2020;48 :D77–83.10.1093/nar/gkz947.31665515
48. Lindeboom  RG, Regev  A, Teichmann  SA. Towards a human cell atlas: taking notes from the past. Trends Genet  2021;37 :625–30.10.1016/j.tig.2021.03.007.33879355
49. Khurana  E, Fu  Y, Chakravarty  D. et al.  Role of non-coding sequence variants in cancer. Nat Rev Genet  2016;17 :93–108.10.1038/nrg.2015.17.26781813
50. Zhou  J, Theesfeld  CL, Yao  K. et al.  Deep learning sequence-based ab initio prediction of variant effects on expression and disease risk. Nat Genet  2018;50 :1171–9.10.1038/s41588-018-0160-6.30013180
51. Chen  KM, Wong  AK, Troyanskaya  OG. et al.  A sequence-based global map of regulatory activity for deciphering human genetics. Nat Genet  2022;54 :940–9.10.1038/s41588-022-01102-2.35817977
52. Avsec  Ž, Agarwal  V, Visentin  D. et al.  Effective gene expression prediction from sequence by integrating long-range interactions. Nat Methods  2021;18 :1196–203.10.1038/s41592-021-01252-x.34608324
53. Zhou  J, Troyanskaya  OG. Predicting effects of noncoding variants with deep learning–based sequence model. Nat Methods  2015;12 :931–4.10.1038/nmeth.3547.26301843
54. Chen  L, Li  S, Bai  Q. et al.  Review of image classification algorithms based on convolutional neural networks. Remote Sens (Basel)  2021;13 :4712.
55. Van Houdt  G, Mosquera  C, Nápoles  G. A review on the long short-term memory model. Artif Intell Rev  2020;53 :5929–55.
56. Zhang  Y, Liu  Y, Xu  J. et al.  Leveraging the attention mechanism to improve the identification of DNA N6-methyladenine sites. Brief Bioinform  2021;22 :bbab351.34459479
57. Vaswani  A, Shazeer  N, Parmar  N. et al.  Attention is all you need. Adv Neural Inf Process Syst  2017;30 :1–15.
58. Kelley  DR, Snoek  J, Rinn  JL. Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks. Genome Res  2016;26 :990–9.10.1101/gr.200535.115.27197224
59. Quang  D, Xie  X. DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences. Nucleic Acids Res  2016;44 :e107.27084946
60. Park  S, Koh  Y, Jeon  H. et al.  Enhancing the interpretability of transcription factor binding site prediction using attention mechanism. Sci Rep  2020;10 :1–10.31913322
61. Wang  M, Tai  C, Weinan  E. et al.  DeFine: deep convolutional neural networks accurately quantify intensities of transcription factor-DNA binding and facilitate evaluation of functional non-coding variants. Nucleic Acids Res  2018;46 :e69.29617928
62. Hoffman  GE, Bendl  J, Girdhar  K. et al.  Functional interpretation of genetic variants using deep learning predicts impact on chromatin accessibility and histone modification. Nucleic Acids Res  2019;47 :10597–611.10.1093/nar/gkz808.31544924
63. Zhang  H, Hung  C-L, Liu  M. et al.  NCNet: deep learning network models for predicting function of non-coding DNA. Front Genet  2019;10 :432.31191597
64. Pei  G, Hu  R, Jia  P. et al.  DeepFun: a deep learning sequence-based model to decipher non-coding variant effect in a tissue- and cell type-specific manner. Nucleic Acids Res  2021;49 :W131–9.10.1093/nar/gkab429.34048560
65. Kelley  DR, Reshef  YA, Bileschi  M. et al.  Sequential regulatory activity prediction across chromosomes with convolutional neural networks. Genome Res  2018;28 :739–50.10.1101/gr.227819.117.29588361
66. Kelley  DR . Cross-species regulatory sequence activity prediction. PLoS Comput Biol  2020;16 :e1008050.32687525
67. Zrimec  J, Börlin  CS, Buric  F. et al.  Deep learning suggests that gene expression is encoded in all parts of a co-evolving interacting gene regulatory structure. Nat Commun  2020;11 :6141.33262328
68. Schwessinger  R, Gosden  M, Downes  D. et al.  DeepC: predicting 3D genome folding using megabase-scale transfer learning. Nat Methods  2020;17 :1118–24.33046896
69. Fudenberg  G, Kelley  DR, Pollard  KS. Predicting 3D genome folding from DNA sequence with Akita. Nat Methods  2020;17 :1111–7.10.1038/s41592-020-0958-x.33046897
70. Zhou  J . Sequence-based modeling of three-dimensional genome architecture from kilobase to chromosome scale. Nat Genet  2022;54 :725–34.10.1038/s41588-022-01065-4.35551308
71. Umarov  RK, Solovyev  VV. Recognition of prokaryotic and eukaryotic promoters using convolutional deep learning neural networks. PloS One  2017;12 :e0171410.28158264
72. Umarov  R, Kuwahara  H, Li  Y. et al.  Promoter analysis and prediction in the human genome using sequence-based deep learning models. Bioinformatics  2019;35 :2730–7.10.1093/bioinformatics/bty1068.30601980
73. Oubounyt  M, Louadi  Z, Tayara  H. et al.  DeePromoter: robust promoter predictor using deep learning. Front Genet  2019;10 :286.31024615
74. Shujaat  M, Chong  KT. HM-Prom: CNN based Prediction of TATA Promoters from Human and Mouse Sequences. In: Lee DY (ed). 2021 21st International Conference on Control, Automation and Systems (ICCAS). IEEE, Jeju, Korea, 2021, 1848–52.
75. Wang  Y, Peng  Q, Mou  X. et al.  A successful hybrid deep learning model aiming at promoter identification. BMC Bioinformatics  2022;23 :1–20.34983366
76. Min  X, Chen  N, Chen  T. et al.  DeepEnhancer: Predicting enhancers by convolutional neural networks. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, Piscataway, New Jersey, USA, 2016, 637–44.
77. Yang  B, Liu  F, Ren  C. et al.  BiRen: predicting enhancers with a deep-learning-based model using the DNA sequence alone. Bioinformatics  2017;33 :1930–6.10.1093/bioinformatics/btx105.28334114
78. Yang  R, Wu  F, Zhang  C. et al.  iEnhancer-GAN: a deep learning framework in combination with word embedding and sequence generative adversarial net to identify enhancers and their strength. Int J Mol Sci  2021;22 :3589.33808317
79. Li  Y, Kong  F, Cui  H. et al.  SENIES: DNA shape enhanced two-layer deep learning predictor for the identification of enhancers and their strength. IEEE/ACM Trans Comput Biol Bioinform  2022;20 :637–45.
80. Luo  H, Chen  C, Shan  W. et al.  iEnhancer-BERT: A novel transfer learning architecture based on DNA-Language model for identifying enhancers and their strength. In: International Conference on Intelligent Computing. Springer, Building New York City, NY, USA, 2022, 153–65.
81. Mehmood  F, Arshad  S, Shoaib  M. ADH-enhancer: an attention-based deep hybrid framework for enhancer identification and strength prediction. Brief Bioinform  2024;25 :bbae030.38385876
82. Huang  D, Petrykowska  HM, Miller  BF. et al.  Identification of human silencers by correlating cross-tissue epigenetic profiles and gene expression. Genome Res  2019;29 :657–67.10.1101/gr.247007.118.30886051
83. Zeng  W, Chen  S, Cui  X. et al.  SilencerDB: a comprehensive database of silencers. Nucleic Acids Res  2021;49 :D221–8.10.1093/nar/gkaa839.33045745
84. Zhang  T, Li  L, Sun  H. et al.  DeepICSH: a complex deep learning framework for identifying cell-specific silencers and their strength from the human genome. Brief Bioinform  2023;24 :bbad316.37643374
85. Pan  J-H, Du  P-F. SilenceREIN: seeking silencers on anchors of chromatin loops by deep graph neural networks. Brief Bioinform  2024;25 :bbad494.
86. Wang  Y, Chen  L. DeepPerVar: a multi-modal deep learning framework for functional interpretation of genetic variants in personal genome. Bioinformatics  2022;38 :5340–51.10.1093/bioinformatics/btac696.36271868
87. van  Hilten  A, Kushner  SA, Kayser  M. et al.  GenNet framework: interpretable deep learning for predicting phenotypes from genetic data. Commun Biol  2021;4 :1094.34535759
88. Dunham  AS, Beltrao  P, AlQuraishi  M. High-throughput deep learning variant effect prediction with sequence UNET. Genome Biol  2023;24 :110.37161576
89. Popic  V, Rohlicek  C, Cunial  F. et al.  Cue: a deep-learning framework for structural variant discovery and genotyping. Nat Methods  2023;20 :559–68.10.1038/s41592-023-01799-x.36959322
90. Xiao  Y, Wang  J, Li  J. et al.  An analytical framework for decoding cell type-specific genetic variation of gene regulation. Nat Commun  2023;14 :3884.37391400
91. Schwessinger  R, Deasy  J, Woodruff  RT. et al.  Single-cell gene expression prediction from DNA sequence at large contexts. bioRxiv  2023, 1–22.
92. Castro-Mondragon  JA, Riudavets-Puig  R, Rauluseviciute  I. et al.  JASPAR 2022: the 9th release of the open-access database of transcription factor binding profiles. Nucleic Acids Res  2021;50 :D165–73.
93. Uffelmann  E, Huang  QQ, Munung  NS. et al.  Genome-wide association studies. Nat Rev Methods Primers  2021;1 :59.
94. Nica  AC, Dermitzakis  ET. Expression quantitative trait loci: present and future. Philos Trans R Soc B Biol Sci  2013;368 :20120362.
95. Sasse  A, Ng  B, Spiro  AE. et al.  Benchmarking of deep neural networks for predicting personal gene expression from DNA sequence highlights shortcomings. Nat Genet  2023;55 :2060–4.38036778
