
==== Front
Research (Wash D C)
Research (Wash D C)
RESEARCH
Research
2639-5274
AAAS

10.34133/research.0011
0011
Review Article
Biological Sequence Classification: A Review on Data and General Methods
Ao Chunyan 1 2 3
Jiao Shihu 2
Wang Yansu 3
Yu Liang 1 *
Zou Quan 2 3 *
1 School of Computer Science and Technology, Xidian University, Xi’an, China.
2 Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou, China.
3 Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, China.
* Author correspondence to: lyu@xidian.edu.cn (L.Y.); zouquan@nclab.net (Q.Z.)
19 12 2022
2022
2022 001125 7 2022
25 10 2022
Copyright © 2022 Chunyan Ao et al.
2022
Chunyan Ao et al.
https://creativecommons.org/licenses/by/4.0/ Exclusive licensee Science and Technology Review Publishing House. No claim to original U.S. Government Works. Distributed under a Creative Commons Attribution License 4.0 (CC BY 4.0).

With the rapid development of biotechnology, the number of biological sequences has grown exponentially. The continuous expansion of biological sequence data promotes the application of machine learning in biological sequences to construct predictive models for mining biological sequence information. There are many branches of biological sequence classification research. In this review, we mainly focus on the function and modification classification of biological sequences based on machine learning. Sequence-based prediction and analysis are the basic tasks to understand the biological functions of DNA, RNA, proteins, and peptides. However, there are hundreds of classification models developed for biological sequences, and the quite varied specific methods seem dizzying at first glance. Here, we aim to establish a long-term support website (http://lab.malab.cn/~acy/BioseqData/home.html), which provides readers with detailed information on the classification method and download links to relevant datasets. We briefly introduce the steps to build an effective model framework for biological sequence data. In addition, a brief introduction to single-cell sequencing data analysis methods and applications in biology is also included. Finally, we discuss the current challenges and future perspectives of biological sequence classification research.
==== Body
pmcIntroduction

Biological sequences generally refer to sequences of nucleotides or amino acids. Different biological sequences use different experimental methods to obtain sequences. For example, the protein sequence was obtained through experiments such as protein N/C-terminal sequencing, Edman degradation sequencing, complete protein sequence determination, and monoclonal antibody sequencing based on polymerase chain reaction amplification [1]. The nucleotide (RNA and DNA) sequences are obtained using third-generation sequencing technologies such as nanopore sequencing [2–4]. In addition, nanopore single-molecule sequencing technology combined with the dynamic time series data analysis method was used for sequence analysis. Han et al. [5] developed a new algorithm named cwDTW based on dynamic time warping (DTW) to evaluate DNA sequence similarity lifting by cwDTW for the DNA sequence comparison task. In addition, dynamic time series data analysis methods were more widely used in biological sequence analysis: DTW combined with hidden Markov model (HMM) profile was used for protein homolog search [6], and DTW was also used in combination with HMM–HMM for protein fold recognition studies [7] and DTW for subcellular localization classification studies [8] and DNA sequence classification [9]. In the DNA alignment part of the gene signal, DTW was used for position and length alignment of the gene signal [10]. New algorithms have been developed on the basis of DTW, such as the UCR–DTW algorithm applied to DNA sequence data for multiple sequence comparison [11]. Dynamic time series data analysis methods are also applied in several other fields [12–17]. The continuous and rapid development of gene sequencing technology in the post-genome era has produced an exponentially increasing abundance of biological sequence data. With the continuous growth of biological sequence data, researchers hope to comprehensively understand the biological function of the relevant data by mining and analyzing these data with effective computational methods. Therefore, the development of biological sequence data promotes the application of computer technology in its field. Biological sequence classification is a branch of biological sequence analysis research [18,19], which includes many research directions, and the classification of biological sequence functions and modifications is one of them, involving DNA, RNA, and amino acid sequences.

With the fast accumulation of biological sequence data, effective analysis, mining, and visualization of these data have become increasingly difficult tasks. Traditional wet experimental methods such as chromatin immunoprecipitation sequencing, liquid chromatography (LC), mass spectrometry (MS), LC-MS, RNA sequencing (RNA-seq), deoxyribonuclease sequencing, DNA sequencing, chromatin immunoprecipitation, and high-performance LC [20–23] can identify modification sites accurately and reliably. However, they are always prohibitively costly, both in time and financially, thereby limiting the validation and analysis process of biological sequences. The development of machine learning (ML), data mining, and associated technologies in the field of computer science has promoted research in biological sequence data analysis and mining. ML-based methods for biological sequence function prediction/analysis have become popular over the past few years because of their efficiency. Most prediction tasks in the field of biological sequence analysis can be transformed into classification tasks with ML, i.e., binary or multiclass classification tasks. The most widely used traditional ML algorithms in this field include random forest (RF), support vector machine (SVM), naive Bayes (NB), logistic regression (LR), decision tree (DT), light gradient boosting machine (LGBM), and extreme gradient boosting (XGBoost) [19,24,25]. Deep learning has also seen increasing use in biological sequence classification research [26,27].

Various ML-based biological sequence predictive tools have been developed in recent years. Some researchers reviewed and evaluated these biological sequence classification tools. Chen et al. [28,29] reviewed the identification method of RNA modification sites and also discussed the biological function of RNA methylation and its relationship with human diseases. Challenges and developments for future research on RNA modification sites are discussed. However, these reviews only reviewed and discussed the prediction research of RNA modification sites and did not collect relevant data and provide data downloads. In addition, there are some reviews summarizing the computational tools of RNA modification sites for one species or specifically for one or two types of RNA modification sites [30–32]. For the study of protein posttranslational modification (PTM) site prediction, Ramazi et al. [33] and He et al. [34] reviewed online databases relevant to PTM research and a large number of PTM prediction tools. However, the types of PTMs involved in these reviews are not comprehensive, nor do they provide relevant data downloads. Aiming at the classification of imbalanced data, Dou et al. [35] systematically reviewed the prediction research of imbalanced PTM classification and collated the related data on a webpage. It also analyzes the challenges and solutions in unbalanced classification research. Furthermore, some reviews have summarized and analyzed one or more types of modification site computational tools [36,37]. At present, there are few systematic reviews of the classification of biological sequences (DNA, RNA, and amino acids). Focusing on biological sequence modification, we surveyed computational tools for modification of biological sequences and provided downloads of relevant data (http://lab.malab.cn/~acy/PTM_data/), but did not provide a detailed explanation of the performance of each method shown [38].

Building on previous research, we update the biological sequence modification data and demonstrate the performance of each method. In addition, a systematic review of the classification studies of DNA, RNA, and amino acid sequences was carried out. First, the procedure for constructing reliable predictive models for biological sequences is introduced. Second, by comprehensively collecting biological sequence classification data and various computational tools, a webpage with downloadable classification data was constructed, and the dataset, prediction results, and web address were displayed. Finally, we provide suggestions for the challenges of biological sequence classification research and topics for future research. This review is devoted to describing the classification of functions and modifications of biological sequences (DNA, RNA, and amino acids) and analyzes current or popular research by collecting relevant methods and data. It also provides a platform to aggregate and display relevant data and general methods for functional and modification classification research of biological sequences.

General Scheme and Principles of ML in Biological Sequence Classification

Below, we provide the general procedure for constructing reliable ML models of different biological sequences. As shown in Fig. 1, the task flow mainly includes 4 steps: (a) dataset construction, (b) sequence representation and feature selection, (c) model training and evaluation, and (d) web server or standalone software implementation. Each step is briefly elaborated as follows.

Fig. 1. Flowchart for developing machine learning-based predictive models for biological sequences.

Dataset construction

The first task of accurately identifying biological sequences is the collation of a high-quality benchmark dataset. The rough data types contained in the biological sequences are shown in Fig. 2. To construct a biological sequence dataset, the researcher should first collect biological sequence data based on DNA, RNA, and amino acids from related databases and literature. Most of the amino acid sequence data can be obtained from UniProt (https://www.uniprot.org/). The modified sequence data of DNA, RNA, and amino acids has a specially collected database, as shown in Table S1. The initial dataset will have repetitive sequences or sequences with high homology, which will affect the performance of the model. To eliminate these factors, the CD-HIT software [39] could be used to remove the same sequence or a homologous sequence. The threshold range is usually 30% to 90%. The lower the threshold, the lower the homology similarity sequence. The selection of the threshold was determined by the size of the dataset. The obtained dataset could then be divided into a training dataset and an independent test set for the next experiment.

Fig. 2. Biological sequence type.

Sequence representation and feature selection

Sequence feature extraction is a crucial step in biological sequence classification research. Converting biological sequences into effective mathematical expressions to truly reflect the inherent correlation with the target has an important impact on the performance of the prediction model. The researchers have proposed various feature descriptors for 3 types of biological sequences. Fortunately, several software platforms have been developed to generate these features quickly and easily, including iLearnPlus [40], PyFeat [41], iFeature [42], BioSeq-Analysis 2.0 [43], VisFeature [44], and POSSUM [45]. Some of these open-source platforms have graphical or web-based interfaces that can be easily used even by biologists with no programming background. The biological interpretation of these feature encoding methods has been comprehensively summarized in these studies. There are 7 major encoding schemes for DNA and RNA sequences, including nucleic acid composition, residue composition, position-specific tendencies of trinucleotides, electron–ion interaction pseudopotentials, autocorrelation and cross-covariance, physicochemical property, and pseudo-nucleic acid composition. In terms of proteins and peptides, the feature descriptors could be divided into 8 categories, i.e., amino acid composition, grouped amino acid composition, autocorrelation, quasi-sequence–order, pseudo-amino acid composition, residue composition, physicochemical property, and evolutionary information in the form of a position-specific scoring matrix (PSSM). Overall, these feature extraction methods mainly generate numeric vectors by encoding the composition, biological physical, and biological chemical properties of biological sequences for subsequent ML tasks. On the other hand, Wei et al. [46] proposed a novel feature representation learning scheme that integrates the class and probabilistic information into features, which has been demonstrated to be more informative and effective than the traditional sequence-derived features. Because of its automatic feature extraction and powerful feature representation ability, deep learning has also been widely used for sequence analysis of biological sequences. On the basis of the principle of transfer learning, pretrained deep learning models have emerged to encode sequences in a deep representation learning way. Successful examples of these methods include unified representation (UniRep), Tasks Assessing Protein Embeddings (TAPE), MULocDeep, and BiLSTM embedding model [47–50]. The typical feature extraction methods are shown in Fig. 3.

Fig. 3. The commonly used feature extraction methods.

Most researches on biological sequence classification use multiple feature representation methods or feature fusion methods to represent sequence information. Multiple feature extraction methods or mixed features produce redundant features and result in high-dimensional features. Therefore, it is necessary to apply feature selection methods to reduce dimensionality and improve the performance of the prediction model. The most commonly used feature selection methods include analysis of variance [51], F-score [52], maximum relevance and minimum redundancy [53], maximal relevance–maximal distance [54], Pearson correlation coefficient [55], RF, and LGBM algorithms [56]. These techniques are also applied in conjunction with sequential forward feature selection [57] as an effective 2-step feature selection method.

Model training and evaluation

ML mainly includes 3 types of technologies: unsupervised learning, supervised learning, and semi-supervised learning. Supervised or semi-supervised learning is commonly used in sequence classification tasks. The main goal of ML algorithms is to rigorously train the model to accurately classify any unseen data. Depending on the type of input data, different classification tasks are performed. During model training, the inputs (X) are the feature vectors generated by feature extraction or feature selection methods from the training dataset and the response variable (Y). The X and Y of the input model vary according to the data type, for example, for a DNA enhancer classification task, the relevant data are the DNA enhancer sequence, i.e., X is an encoded numerical representation in a form suitable for use in ML model (usually is a scheme for categorical) data. In the binary classification study, Y is the label 1, and 0 (−1) corresponds to the positive sample and the negative sample, respectively. Similarly, for multicategory, Y is the label corresponding to different types of samples. The ML classifier learns the relationship between X and Y. It makes subsequent predictions for any newly provided datasets. The main objective of a well-performing ML model is to generalize from training data to independent data. In the research on the function and modification classification of biological sequences, we expect to build a predictive model through an optimized ML algorithm to achieve high-accuracy identification of whether an unknown sequence is a certain function or modified sequence.

The commonly used ML algorithms for biological sequence classification are shown in Fig. 1C. Deep learning has been widely used for biological sequence classification in recent years. Greener et al. [58] detailed the application of deep learning in biological modeling and the different models of deep learning, including basic neural networks, convolutional neural networks (CNN), multilayer perceptrons, and recurrent neural networks (RNN). CNN and RNN have been used for the prediction research of biological sequence functions and modification sites. CNN is composed of convolutional, nonlinear, and ensemble layers, and RNN is designed to use the sequential information of the input data to make circular connections between building blocks such as gated recursive units or long short-term memory units. For example, Zou et al. [59] developed a CNN-based classifier Gene2vec for prediction of N6-methyladenosine (m6A) modification sites in mammalian messenger RNA (mRNA) sequences. The classifier CNN contains multiple unit structures such as 2 single-layer convolutional layers, pooling layers, and filtering layers. In this method, the m6A-modified sequence of mRNA is input; the feature vector is obtained by different feature encoding methods, and the feature vector is input into CNN; and the final result is the predicted probability of passing the vote.

In addition, Huang et al. [60] developed an RNN deep learning classifier named BERMP for m6A modification sites in different species, which is based on bidirectional gated recurrent unit. The obtained m6A modification site sequences of different species are encoded by the feature extraction method, the obtained feature vector is input into the RNN classifier, and the corresponding probability is output. Experiments show that the BERMP deep learning framework is more suitable for prediction tasks on large datasets. Compared with traditional ML algorithms, deep learning algorithms have more powerful sequence information extraction capabilities and allow for more accurate biological sequence classification. Commonly used deep learning algorithms are shown in Fig. 1. Traditional ML algorithms can be retrieved from scikit-learn (sklearn) [61]. A variety of deep learning frameworks for training and implementing deep learning models are available, including PyTorch, TensorFlow, and Keras [62]. To summarize, the appropriate classification algorithm is chosen according to the classification task, and the performance of the model can be optimized through parameter selection. Moreover, it is necessary to avoid overfitting and underfitting the model.

The following 3 methods are commonly used to evaluate the performance of constructed models: k-fold cross-validation (k-fold CV), leave-one-out CV (LOOCV), and independent tests. Both k-fold CV and LOOCV are CV techniques. The k value of the k-fold CV is often 10 or 5. For example, a 10-fold CV means that the dataset is randomly divided into 10 parts, 9 of which are used as the training set, and the remaining 1 is used as the validation test set. All 10 subsets are tested and averaged to obtain the final results of the 10-fold CV. LOOCV is time-consuming to train and is generally only used with smaller datasets. Independent tests can be used to determine model overfitting and evaluate the generalization ability of the model. The following metrics are commonly used in biological sequence prediction to demonstrate the performance of the model: Acc, Sp, Sn, and MCC. Balanced accuracy (BACC) is usually employed to measure the accuracy the overall performance of a model trained on imbalanced datasets. A receiver operating characteristic (ROC) curve was drawn with 1 specificity on the abscissa and sensitivity on the ordinate. The area under the ROC curve (AUC) is also commonly used to evaluate the overall predictive performance. These performance evaluation indicators were calculated on the basis of the confusion matrix presented in Table 1.

Table 1. Classification confusion matrix.

Prediction	Actual		
	Negative samples	Positive samples		
Negative samples	TN	FN	Sp=TNTN+FP∗100%Sn=TPTP+FN∗100%Acc=TP+TNTP+TN+FP+FN∗100%BACC=12Sp+Sn∗100%MCC=TP×TN−FP×FNTP+FP×TP+FN×TN+FP×TN+FN

	
Positive samples	FP	TP	
TP, true positive; TN, true negative; FP, false positive; FN, false negative; Sp, specificity; Sn, sensitivity; Acc, accuracy; BACC, balanced accuracy (for imbalance data); MCC, Matthew’s correlation coefficient.

Web server development

The developed prediction models and biological sequence datasets should be disclosed to researchers, and a user-friendly web server is highly recommended to facilitate access to prediction tasks and datasets. With the web server, researchers can perform preliminary screening before experimental validation. The web server can be implemented using conventional technologies such as Django, Dash, Flask, Hyper Text Markup Language (HTML), Cascading Style Sheets (CSS), and hypertext preprocessing (PHP). On the other hand, standalone software or source code can benefit the research community if a web server is not provided.

Progress in Biological Sequence Classification

Thousands of computational methods have been developed for the prediction of biological sequences. The specific methods are completely different and may seem confusing at first glance. Here, we do not intend to provide a comprehensive literature review of articles of specific prediction tasks or to describe the detailed mathematics behind each ML algorithm. Instead, we focused on the summary of benchmark datasets and model performance for particular biological sequence data. On the basis of this goal, in this review, we provide a comprehensive survey regarding the state-of-art computational methods according to 3 types of biological sequence data (DNA, RNA, and amino acid). We present a wide scope of aspects, including the dataset information, the core algorithms chosen for each method, performance evaluation strategy, and software availability (Tables S2–S4). With these data, we also constructed a long-term support online webpage (http://lab.malab.cn/~acy/BioseqData/home.html). This webpage will keep relevant researchers clearly informed of the latest progress on each task. The intricacy of biological sequence data presents pitfalls and opportunities for biological sequence analysis using ML techniques. This webpage provides researchers with the ability to compare the classification methods and download related data quickly and conveniently. As shown in Fig. 4, the webpage summarizes the classification methods and related data of DNA, RNA, protein, and peptide biological sequences, respectively. We have collected hundreds of the most advanced classification tools. According to the number of positive and negative samples in a dataset, we have categorized the classification methods into a balance data table and an imbalance data table. In the tables, we provide the group (unequal-length sequence and equal-length sequence), type, literature name, year, data, prediction result, and web address. Biological sequence datasets can also be directly accessed on the webpage. Our systematic summary of the general methods and data of biological sequence classification will aid the development of biological sequence classification research and alleviate a researcher’s overall time investment. We aim to continuously update the webpage based on the latest research.

Fig. 4. Webpage layout of methods and data related to biological sequence classification (http://lab.malab.cn/~acy/BioseqData/home.html).

DNA sequences

DNA is a biological macromolecule in living organisms. It carries the genetic information of life and guides the development of biological functions and the exertion of life functions [63]. The research on the function and modification site classification of DNA sequences is conducive to deeply understanding the biological function of DNA sequences. Classification methods developed on the basis of ML have grown with the proliferation of DNA sequence data [64–77]. DNA sequence-based classification methods and related data are shown in Table S2 and are divided into 2 categories according to sequence length: general prediction methods (unequal-length sequences) and modification site prediction methods (equal-length sequences). Unequal-length sequences include DNA enhancers, promoters, nucleosomes, and functional of DNA sequences. Equal-length sequences (DNA modification sites) contain N6-methyladenine (6mA), N4-methylcytosine (4mC), and 5-methylcytosine (5mC).

Here, we mainly focus on several types of DNA sequence classification studies: enhancers, promoters, and functional effects of DNA sequences. For example, a recently developed 2-layer predictor, iEnhancer-XG [64] integrates a variety of sequence features and uses integrated learning to classify DNA enhancers. iEnhancer-XG uses 5 feature extraction methods, including PSSM, k-spectrum profile, mismatch k-tuple, PseDNC, and subsequence profile, and combines 5 separate output feature vector input integrated learning (Fig. 3B), using XGBoost as a classifier. The training dataset of this method contains 742 strong enhancers, 742 weak enhancers, and 1,484 non-enhancers. An independent test set (100 strong enhancers, 100 weak enhancers, and 200 non-enhancers) was constructed to verify the performance of the model. Under the 10-fold cross-verification and independent test set, the Acc values obtained by the 1-layer predictor were 81.10% and 75.75%, and those obtained by the second layer classifier were 66.74% and 63.50%. However, this performance evaluation should be conducted with a larger dataset, which would likely improve accuracy. Lyu et al. [65] developed a novel method (iPro2L-PSTKNC) for the prediction of promoters in the Escherichia coli genome. The predictor adopts the new feature extraction method PSTKNC to the encoding sequence and the obtained feature vector input integration classifier SVM. The number of positive samples and negative samples in the dataset used by this method is 2,860 and 2,860, respectively. The positive sample contains 6 types of promoters: σ24-484, σ28-134, σ32-291, σ38-163, σ54-94, and σ70-1694. The Acc and MCC values obtained by this classification method were 90.05% and 80.13%, respectively. This prediction method uses synthetic minority oversampling technique (SMOTE) to handle data imbalance. For multitype classification with a smaller sample size, the data quantity of the training model can be increased for stronger migration of the model. A novel classification method, DeepATT [66], has been developed on the basis of hybrid deep neural networks and is used for the functional classification of DNA sequences. The data used in this study include a training dataset of 440,000, a validation dataset of 8,000, and an independent test dataset of 450,024. A hybrid neural network, on the basis of deep learning, was used to construct the predictor. The average area under the ROC obtained by DeepATT is 0.94519, and the performance of this method is much higher than other classification methods. This method effectively reduces the number of parameters to ensure classification accuracy and improves the performance of the model.

DNA modification site prediction studies mainly focus on 4mC and 6mA, with relatively few studies on 5mC. Many 4mC modification site prediction tools have been developed on the basis of ML. 4mC-w2vec [67] and Deep4mcPred [69] use deep learning to classify 4mC in multiple species with high accuracy. 4mC-w2vec is a predictor developed for 2 species, F. vesca and R. chinensis, using training datasets (F. vesca: P-3457/N-3457 and R. chinensis: P-1938/N-1938) to construct model and an independent test set (F. vesca: P-864/N-864, 4320, and 12960 and R. chinensis: P-483/N-2415 and 7245) to test the model. In addition, 4mC-w2vec is also used to conduct comparative analyses on data imbalance using deep learning. The accuracies obtained for F. vesca and R. chinensis were 86.97% and 85.41%, respectively. The accuracy of the independent test set under different proportions was 84.00% to 86.32%. Similarly, Deep4mcPred developed a predictor for 3 species. The training dataset contains Arabidopsis thaliana (20,000/20,000), Caenorhabditis elegans (20,000/20,000), and Drosophila melanogaster (20,000/20,000). The accuracies obtained for A. thaliana, C. elegans, and D. melanogaster were 84.4%, 89.3%, and 87.1%, respectively. Predictors such as EC4mC-SVM [68], 4mcPred-IFL [70], 4mcPred-SVM [78], and 4mCPred [71] use different traditional ML methods to classify 4mC-modified sites. EC4mC-SVM is a predictor developed for E. coli. The training dataset was P-388/N-388, and the independent test set was P-134/N-134. The iEC4mC-SVM predictor had a 10-fold CV accuracy of 85.4% and an independent testing accuracy of 83.2%. 4mcPred-IFL [70], 4mcPred-SVM [78], and 4mCPred [71] use the same dataset. These 3 predictors can identify 4mC modification sites in 6 species, namely, C. elegans: P-1554/N-1554, D. melanogaster: P-1769/N-1769, A. thaliana: P-1978/N-1978, E. coli: P-388/N-388, Geoalkalibacter subterraneus: P-905/N-905, and Geobacter pickeringii: P-569/N-569. Through continuous improvement of feature extraction methods and classification algorithms, model accuracy is gradually improved. The Acc value of each species under the CV of 4mcPred-IFL and 4mcPred-SVM is as follows: 4mcPred-IFL (C. elegans: 88.00%; D. melanogaster: 87.40%; A. thaliana: 82.50%; E. coli: 89.40%; G. subterraneus: 88.60%; and G. pickeringii: 90.70%.) and 4mcPred-SVM (C. elegans: 81.50%; D. melanogaster: 83.00%; A. thaliana: 78.70%; E. coli: 83.30%; G. subterraneus: 83.70%; and G. pickeringii: 86.00%).The 4mCPred predictor jackknife CV accuracy and independent test set accuracy are as follows: jackknife test (C. elegans: 87.71%; D. melanogaster: 87.79%; A. thaliana: 83.37%; E. coli: 94.97%; G. subterraneus: 91.04%; and G. pickeringii: 90.89%) and independent testing (C. elegans: 82.21%; D. melanogaster: 82.63%; A. thaliana: 76.52%; E. coli: 82.69%; G. subterraneus: 83.33%; and G. pickeringii: 77.63%). Classification methods for the 6mA modification site sequence include iRicem6A-CNN [73] and iIM-CNN [79], which are both based on CNN in deep learning. 6mA-RicePred [74] and csDMA [75] are based on prediction models constructed using SVM and Extra Trees, respectively. iIM-CNN can identify 6mA modification sites in multiple species. The training datasets are cross-species: P-2768/N-2716, rice: P-880/N-880, and Mus musculus: P-1934/N-1934. The accuracy of this method under 5-fold CV is cross-species: 82.40%, rice: 87.50%, M. musculus: 96.90%. Similarly, the csDMA predictor uses the same M. musculus (1,934/1,934) and rice (880/880) data. In addition, the cross-species data in the training dataset were 2,768/2,716, and the cross-species independent test set was 554/554. The accuracy under 5-fold CV and independent test set is as follows: 5-fold CV (cross-species: 79.90%; rice: 86.10%; and M. musculus: 96.60%) and independent testing (cross-species: 81.30%). In addition, iRicem6A-CNN (training dataset: P-154000/N-154000 and independent test set: P-880/N-880) and 6mA-RicePred (training dataset: P-880/N-880 and independent test set: P-154000/N-154000) use the same dataset to classify 6mA modification sites in a rice genome sequence. The accuracy obtained by iRicem6A-CNN (Acc value of 93.82% for 5-fold CV and 96.19% for independent testing) is better than that of 6mA-RicePred (Acc value of 87.27% for 10-fold CV and 85.65% for independent testing). Deep learning shows better performance with large datasets. For the classification study of 5mC modification sites, iPromoter-5mC was used to collect the first large-scale 5mC dataset for classification research. The training dataset was p-55800/N-13950, and the independent test set was P-658861/N-164751. Using the down-sampling method for data imbalance processing, the Acc values obtained from the training dataset and independent test set were 90.16% and 90.22%, respectively.

RNA sequences

RNA is a biological macromolecule similar to DNA. Unlike DNA, RNA is single-stranded. RNA participates in life activities and performs specific biological functions in cells. Therefore, the classification study of RNA sequences is helpful for the further study of the biological function and modification mechanism of RNA. With improved data mining efficiency for RNA sequence types and unknown modification site sequence data, the development of general classification methods based on ML is also increasing [24,31,80–94]. General methods and related data for RNA sequence classification are summarized in Table S3. RNA sequence classification is divided into 2 categories: (a) unequal length sequences (used for classification of the eukaryotic mRNA, noncoding RNA, pre-microRNA, circular RNA (circRNA), transfer RNA (tRNA), Piwi-interacting RNA, long noncoding RNA (lncRNA), and single guide RNA) and (b) equal length sequences (used for the classification of RNA posttranscriptional modification sites m6A, N1-methyladenosine (m1A), pseudouridine (Ψ), adenosine-to-inosine (A-to-I), 5-methylcytosine (m5C), N2-methylguanosine (m2G), 2′-O-methylation (Nm), 5-hydroxymethylcytosine (5hmC), dihydrouridine (D), N6,2′-O-dimethyladenosine (m6Am), N7-methylguanosine (m7G), 5-methyluridine (m5U), and inosine (I)).

For the classification research of RNA sequences of unequal length, we introduce the following: SubLocEP [80] is a predictor developed with ML for the subcellular localization of eukaryotic mRNA. This method adopts integrated ML to classify the cytoplasm, endoplasmic reticulum, extracellular region, mitochondria, and nucleus. The training dataset includes cytoplasm: 5,310; endoplasmic reticulum: 1,185; extracellular region: 710; mitochondria: 350; and nucleus: 4,855. The independent test set contains 3 datasets, namely, D1 (cytoplasm: 1,066; endoplasmic reticulum: 241; extracellular region: 145; mitochondria: 71; and nucleus: 976), D2 (cytosol: 91 and nucleus: 148), and D3 (endoplasmic reticulum: 131 and nucleus: 131). The accuracy using this method was as low as 0.659 ± 0.006. The accuracies of the 3 independent testing sets were also low: 0.601, 0.506, and 0.37. The accuracy of this method requires further improvement. A classification tool, CirRNAP [81], was developed for circRNA sequences. This method performs 3 categories of classification work: circRNA vs. protein coding gene (PCG) (training dataset: circRNA-10000/PCG-8000 and testing dataset: circRNA-4084/PCG-1533), circRNA vs. lncRNA (training dataset: circRNA-10000/lncRNA-10000 and testing dataset: circRNA-4084/lncRNA-9722), and circRNA vs. non-circRNA (training dataset: circRNA-1800/non-circRNA-1800 and testing dataset: circRNA-282/non-circRNA-282). Under 10-fold CV, the accuracy of the 3 categories of classification work was 81.50% (circRNA vs. PCG), 80.20% (circRNA vs. lncRNA), and 78.20% (stem cell vs. not), respectively. The accuracy under the independent test set was 82.70% (circRNA vs. PCG), 85.40% (circRNA vs. lncRNA), and 81.20% (stem cell vs. not), respectively. The training dataset was relatively large, and the classification model had strong generalization ability. tRNA-Predict [82] is based on an ensemble classifier. The training dataset contains positive samples of 623 and negative samples of 1,183. The tRNA-Predict predictor had a 10-fold CV accuracy of 95.10%. Research on tRNA classification occurred relatively early, although data are limited. On the basis of this method, a more stable model can be constructed with larger datasets. In response to the previously predicted insufficient activity of sgRNA on the target gene, Niu et al. [85] developed SgRNA-RF, which uses 5 datasets, some of which are severely imbalanced between positive and negative samples. For example, the correlation data of the G5 sequence, hek293t: training dataset: P-1615/N-428 and testing dataset: P-404/N-108; hct116: training dataset: P-3090/N-428 and testing dataset: P-783/N-108; hela: training dataset: P-5923/N-428 and testing dataset: P-782/N-108; and h160: training dataset: P-1973/N-428 and testing dataset: P-494/N-108. The positive and negative sample ratios were between 3.77 and 13.84. To deal with the imbalance problem, the CS-SMOTE method was used to improve the BACC. The Acc value under 10-fold CV was G17-84.7%, Gr-69.4%, and G5-(hct116: 96.9%, hek293t: 94.0%, hela: 97.74%, and h160: 94.10%). The Acc value of the independent test set was G17-86.3%, Gr-91.6%, Gnr-89.4%, Gm-93.8%, and G5-(hct116: 96.50%, hek293t: 78.70%, hela: 97.90%, and h160: 97.30%).

There are only about dozen types involved in the prediction research of RNA posttranscriptional modification sites. On the basis of ML and RNA modification data, many classification methods have been developed (Table S3). Among them, many predictors have been developed specifically for m6A modification sites. Data and species of m6A modification sites are the most common. For example, the m6A prediction tools DNN-m6A [90] and iRNA-m6A [91], developed on the basis of deep learning, used the same dataset and analyzed the brain, kidney, and liver of human, mouse, and rat, respectively. In mouse, m6A modification sequence data were also collected from the heart tissue. The training dataset was human (brain: 4,605/4,605; kidney: 4,574/4,574; and liver: 2,634/2,634), mouse (brain: 8,025/8,025; heart: 2,201/2,201; kidney: 3,953/3,953; liver: 4,133/4,133; and testis: 4,704/4,704), and rat (brain: 2,352/2,352; kidney: 3,433/3,433; and liver: 1,762/1,762). The independent test set was human (brain: 4,604/4,604; kidney: 4,573/4,573; and liver: 2,634/2,634), mouse (brain: 8,025/8,025; heart: 2,200/2,200; kidney: 3,952/3,952; liver: 4,133/4,133; and testis: 4,706/4,706), and rat (brain: 2,351/2,351; kidney: 3,432/3,432; and liver: 1,762/1,762). The accuracy values obtained by DNN-m6A and iRNA-m6A were 72.95% to 83.38% and 68.79% to 81.78%, respectively. HSM6AP [92] was developed for Homo sapiens using data originally from WHISTLE [95], which includes 6 base–resolution datasets (from 5 cell types). Using these data, the authors reconstructed training and independent test datasets. Experimental results show that HSM6AP outperforms previous prediction tools, with average AUC values of approximately 0.975 and 0.900 on full transcript and mature mRNA data, respectively. iMethyl-Deep [93], DeepM6APred [94], and RAM-ESVM [96] also used the same training dataset (D1: 1,307/1,307) to train the model. After continuous improvement of the method, the Acc value was increased to 89.19%, 80.50%, and 78.35%, respectively. For other types of modification sequence data, most studies improve classification methods using existing datasets or develop new classification methods by adding data. For Ψ modification sites, RF-PseU [87] and iPseU-CNN [97] were trained and tested on the same dataset. The training dataset was H. sapiens (495/495), Saccharomyces cerevisiae (314/314), and M. musculus (472/472). The independent test set was H. sapiens (100/100) and S. cerevisiae (100/100). The accuracy obtained by RF-PseU and iPseU-CNN was 64.30% to 77.00% and 66.68% to 73.50%, respectively. In comparison, Dou’s method [88] increased the number of training dataset (H. sapiens: 495/495; S. cerevisiae: 319/319; and M. musculus: 495/495) sequences of S. cerevisiae and M. musculus. For the 3 species, the 10-fold CV accuracy obtained was 62.73%, 70.54%, and 71.72%, respectively. At the same time, the sample size of S. cerevisiae was increased in the independent test set. The accuracy was 60.20% and 77.00%, respectively. In addition, for m2G modification sites, a predictor RFhy-m2G was developed on the basis of RF. The training dataset contained H. sapiens (41/541), M. musculus (27/427), and S. cerevisiae (60/283). The independent test set was H. sapiens (5/60), M. musculus (3/47), and S. cerevisiae (7/31). Because of data imbalance, the RFhy-m2G [24] classification method can correct for the imbalance of positive and negative samples using SMOTE. The accuracy of the 10-fold CV for the 3 species was 99.82%, 100%, and 99.65%, respectively. The accuracy of the independent test set was 94.17%, 90.43%, and 100%, respectively. Dou’s method [98] (training dataset: 140/298 and independent test set: 36/76) and iRNA-m5C_NB [99] (training dataset: 127/808 and independent test set: 157/1,000) both use the hybrid sampling method SMOTE and undersampling method edited nearest neighbors (ENN) (SMOTEENN) to deal with data imbalance. For Dou’s method, the accuracy of the training dataset and the independent test set was 97.24% and 93.75%, respectively. For iRNA-m5C_NB, the accuracy of the training dataset and the independent test set was 82.20% and 74.85%, respectively. For multitype RNA modification site prediction, iMRM [31] is a predictor developed on the basis of ML for multitype RNA modification classification. The species involved include H. sapiens (m1A: 6,366/6,366; m5C: 120/120; m6A: 1,130/1,130; Ψ: 495/195; and A-to-I: 3,000/3,000), S. cerevisiae (m1A: 483/483; m5C: 211/211; m6A: 1,307/1,307; and Ψ: 313/314), and M. musculus (m1A: 1,064/1,064; m5C: 97/97; m6A: 725/725; and Ψ: 472/472). The lowest accuracy obtained by the iMRM method was 66.47%, and the highest accuracy was 100%. This method has room for improvement in the accuracy of Ψ modification. Another predictor, MultiRM [86], was developed on the basis of deep learning and covers 12 types of RNA modification (m6A: 65,178; Ψ: 3,137; m1A: 16,380; m6Am: 2,447; Am: 1,591; Cm: 1,878; Gm: 1,471; Um: 2,253; m5C: 12,936; m7G: 1,036; m5U: 1,696; and I: 52,618). The MultiRM predictor performs a multiclassification task, and the accuracy obtained was 65.00% to 92.00%. In order to solve the problem of data imbalance, the MultiRM method introduces online hard examples mining and uncertain weights to improve accuracy. For other RNA posttranscriptional modifications, information on the related classification methods of Nm, m2G, m7G, 5hmC, and D is shown in Table S2.

Protein sequences

Proteins are composed of 20 different types of amino acids, which are present in nature and encoded by DNA sequences. Proteins play important roles in the cells of organisms, including catalysis of reactions, recognition, regulation, cell signaling, membrane transport, and the provision of structure [100]. As the gap between the number of proteins being discovered and their functional characterization grows (especially due to experimental limitations), reliable prediction of protein function by computational means has become crucial [101–111]. Common protein classification tasks include conventional unequal-length and equal-length sequences (Table S4). By collecting proteins with many studies and significant significance, the conventional unequal length sequence types including DNA-binding proteins, RNA-binding proteins (RBPs), secretory proteins, cancerlectin, phage virion protein, cell wall lytic enzymes, thermophilic proteins, major histocompatibility complex, antioxidant proteins, biological luminescent proteins, electron transport proteins, plant pentatricopeptide repeat, sub-Golgi protein, and type III fluctuation systems. The classification of protein equal-length sequences is related to the prediction of protein PTM sites. Protein PTM involves the modification or addition of chemical groups that occur on amino acid residues. A review of protein PTMs showed that there were about a dozen types of modifications, which have been shown on online webpages.

DNA-binding proteins and RBPs are special binding proteins that are formed by the combination of nucleotides and proteins. The following 3 methods were applied to a classification study of RBPs: rBPDL [106], RBPro-RF [107], and TriPepSVM [108]. The dataset used by rBPDL contained RBP sequences (72,226) and negative sequences (137,003). Deep learning was used for prediction research. The AUC was higher than 0.932. The positive and negative samples in the dataset used by the RBPro-RF and TriPepSVM methods were imbalanced. The ratio of positive and negative samples in the RBPro-RF training dataset was 2.55. The author adopted SMOTE for imbalanced processing. The training dataset was P-2780/N-7093. The constructed independent testing set contained 3 species: H. sapiens (RBPs: 967 and non-RBPs: 597), S. cerevisiae (RBPs: 354 and non-RBPs: 135), and A. thaliana (RBPs: 456 and non-RBPs: 37). Under 10-fold CV, the Acc value of the model was 97.43%. The Acc values obtained from the performance of the 3 species’ test models were 95.63%, 88.82%, and 92.35%, respectively. The training dataset constructed by the TriPepSVM predictor was human (1,625/10,834), Salmonella (275/1,273), and E. coli (460/3,404). The independent test set included human (181/1,204), Salmonella (31/142), and E. coli (52/379). TargetDBP plus is a newly developed DNA-binding protein classification tool [101], which uses weighted convolution features to encode sequence information. This method used 4,500 positive and negative samples in the training dataset, and the positive and negative samples of the independent test set were 381. The accuracies of the jackknife test and independent test were 86.84% and 85.83%, respectively. TargetDBP plus adds the training dataset and an independent test set based on the PSSMEI [102] method and improves the accuracy of the model by improving the method. Moreover, some classification tasks have fewer sequences of data. The quality of the dataset determines the prediction model. For cell wall lytic enzyme classification research based on a single dataset (lyases: 68 and non-lyases: 307), 3 classification tools have been developed: CWLy-RF [103], CWLy-SVM [104], and Jing’s method [105]. The accuracy obtained was 96.09%, 95.50%, and 99.19%, respectively. Although the accuracy obtained was as high as 99.19%, the dataset was too small, and the migration performance of the model was poor.

The classification of protein equal-length sequences is related to the prediction of protein PTM sites (Table S4). Protein PTM involves the modification or addition of chemical groups that occur on amino acid residues. A review of protein PTMs showed that there were about a dozen types of modifications, which have been shown on online webpages (http://lab.malab.cn/~acy/BioseqData/Protein.html). According to the related database of protein PTMs, the top 3 types of modification data are phosphorylation, acetylation, and ubiquitination. Phosphorylation predictors developed on the basis of deep learning are DeepPPSite [112] and DeepPSP [113]. DeepPPSite constructed a training dataset (S: 4,316/4,316; T: 1,551/1,551; and Y: 553/553) and an independent test set (S: 2,773/17,118; T: 941/6,258; and Y: 210/1,296). The accuracy of the 10-fold CV was 80.38% (S), 80.01% (T), and 77.73% (Y). The accuracy of the independent test set was 78.91% (S), 84.81% (T), and 82.73% (Y). The accuracy of the independent test set was 78.91% (S), 84.81% (T), and 82.73% (Y). The datasets used by the DeepPSP method were a training dataset (S/T: P-165787/N-879507 and Y: P-28965/N-134997) and a testing dataset (S/T: P-18588/N-102113 and Y: P-3248/N-14504). For more than 4 positive and negative samples, the developers of DeepPSP used a deep learning algorithm to process the data and solve the imbalance problem. iPhoPred [114] and PhosPred-RF [115] were based on predictors developed by SVM and RF, respectively. The dataset used by iPhoPred was small, with a SerD of 600 and TyrD and ThrD of 200. The AUCs obtained were 0.904, 0.992, and 0.990, respectively, and the model may have been overfitted. PhosPred-RF uses the training dataset (S-type: 4,316/4,316; T-type: 1,551/1,551; and Y-type: 553/553) to train the model, using an independent test set (S-type: 2,273/17,618; T-type: 941/6,258; and Y-type: 296/1,210) to test model performance. The AUCs of the 10-fold CV were 0.851, 0.818, and 0.761, respectively. The AUCs of the independent test set were 0.715, 0.683, and 0.654, respectively. Acetylation is the second most modified type of protein PTM. Since its discovery, many classification methods have been developed on the basis of ML. For lysine acetylation modification sites, DNNAce [111] and ProAcePred [116] used the same positive sample sequence data related to 9 species (training dataset: Archaea: 193/193,1590; Bacillus subtilis:1040/1040,5772; Corynebacterium glutamicum: 1021/1021,4333; Erwinia amylovora: 95/95,718; E. coli:1919/1919,1919; Geobacillus kaustophilus:189/189,1025; Mycobacterium tuberculosis: 866/866,3926; Salmonella typhimuricum: 174/174,1467; and Vibrio parahemolvticus: 1065/1065,5938). DNNAce and ProAcePred use independent test set 1 (Archaea: 21/21; B. subtilis: 115/115; C. glutamicum: 113/113; E .amylovora: 10/10; E. coli:213/213; G. kaustophilus:21/21; M. tuberculosis:96/96; S. typhimuricum:19/19; and V. parahemolvticus:118/118) and independent test set 2 (Archaea: 21/176; B. subtilis:115/641; C. glutamicum: 113/481; E. amylovora: 10/80; E. coli: 213/213; G. kaustophilus: 21/114; M. tuberculosis: 96/436; and S. typhimuricum: 19/163), respectively. The accuracy obtained by DNNAce and ProAcePred was 63.08% to 98.26% and 69.00% to 98.30%, respectively. Another predictor, DeepAcet [52] was developed on the basis of deep learning. This method uses a relatively large dataset (training dataset: P-12886/N-12886 and independent testing dataset: P-3221/N-3221). Under the 10-fold CV and independent test sets, the accuracy obtained was 84.95% and 84.87%, respectively. Less data are available for ubiquitination compared to the other 2 modification types. In the latest report, the predictor DeepTL-Ubi [117] covered 8 species, including the H. sapiens (31,162/31,162), M. musculus (7,746/7,748), S. cerevisiae (3,506/3,506), Rattus norvegicus (1,226/1,226), Aspergillus nidulans (2,299/2,299), A. thaliana (586/587), Toxoplasma gondii (424/424), and Oryza sativa (308/308). The training set using this method showed a low accuracy (0.500–0.604). The accuracy for different species needs to be improved. For A. thaliana, the predictor CNNAthUbi [118] was developed on the basis of a deep learning CNN. The training dataset consists of positive samples (2,043) and negative samples (6,130), and the independent test set consists of positive samples (511) and negative samples (1,533). The Acc value for the 10-fold CV and independent test set was 85.38% and 85.36%, respectively. In addition, for the general methods and data of protein PTM imbalance classification, Dou et al. [35] developed ImbClassi_PTMs, which comprehensively summarized the performance and related data of the general methods, provided online services for dataset downloads, and introduced general methods.

Peptide sequences

Peptides are an important class of naturally occurring biologics, which are amino acid sequences with a length of less than 50. Their unique structures provide them with various functions as hormones, biological messengers, growth factors, anti-infectives, and neurotransmitters [119]. Accurate prediction of biological peptides plays an important role in the discovery and development of efficient peptide-based drugs. However, experimental methods available for the discovery and synthesis of biological peptides are costly, time-consuming, and labor-intensive. Furthermore, this characteristic makes it especially difficult to develop well-performing predictive models by ML due to the short sequences, the more difficult it becomes to extract statistical information. The most commonly studied peptides in classification research include anticancer peptides, antihypertensive peptides, antitubercular peptides, therapeutic peptides, toxic peptides, cell-penetrating peptides, hemolytic peptides, and bitter peptides (Table S4).

There are dozens of classification methods developed for these peptide sequences [120–125], among which, there are many studies of anticancer peptides, therapeutic peptides, and cell-penetrating peptides that are specifically aimed at disease treatment. iACP-DRLF [120] and AntiCP 2.0 [121] are anticancer peptide prediction tools developed by ML. Both methods used the same dataset. The 2 datasets, consisting of both the training dataset and the independent test set, were the main dataset (861/861) and the alternate dataset (970/970). Compared with AntiCP 2.0, iACP-DRLF has improved accuracy in both the training (main dataset: 79.10% and alternate dataset: 94.5%) and independent test sets (main dataset: 77.50% and alternate dataset: 93.0%). However, the accuracy for the main dataset is less than 80%. The ACPred-Fuse [122] (training dataset: 125/125 and independent test set: 82/2628) and ACPred-FL [126] (training dataset: 250/250 and independent test set: 970/970) classification methods used different datasets. The accuracy of 5-fold CV was 82.4% and 91.4%, respectively, and the accuracies of the independent test set were 89.0% and 88.4%, respectively. The training dataset of this method was relatively small (ACPred-Fuse: ACPs-125, non-ACPs-125; ACPred-FL (ACP500): P-250/N-250). For the classification study of therapeutic peptides, the classification methods and related data from the past 3 years have been summarized. The ITP-Pred [123] method is developed on the basis of hybrid low-dimensional features and deep learning and is mainly used for the prediction of quorum-sensing peptide (training dataset: 400/400 and independent test set: 40/40) and cell-penetrating peptide (CPP) (training dataset: 740/740 and independent test set: 92/92) peptides. The dataset increased the number of sequences based on previous research. For QSP, the accuracy of 5-fold CV and independent test sets was 87.00% and 87.30%, respectively. Similarly, for CPP, the accuracy of 5-fold CV and independent test sets was 95.10% and 97.50%, respectively. Several classification methods were developed on the basis of the cell-penetrating peptide sequence, and the training dataset used was CPP924 (CPPs-462 and non-CPPs-462). The accuracies obtained by the classification method were all above 90%. However, the accuracy of the CPPred-RF [127] method on the CPPsites dataset was only 71.1%. Similarly, the accuracy obtained by StackCPPred [125] was 78.3%. The TargetCPP [124] method adopts an independent test set to verify the performance of the model. For the CPPind (CPPs: 111 and non-CPPs: 34) dataset, the accuracy obtained was 88.28%. Research data and methods for other types of peptides are constantly being developed and improved.

Single-Cell Sequencing Data Analysis for Biological Applications

For the analysis of single-cell sequencing data, different analysis processes are used for different sequencing types and research purposes. Single-cell sequencing mainly includes single-cell DNA sequencing, single-cell RNA-seq (scRNA-seq), and single-cell epigenome sequencing, and for single-cell DNA sequencing and single-cell epigenome sequencing, the data analysis process is similar to the traditional high-throughput sequencing data analysis methods [128,129]. Many analysis methods are used for the analysis of scRNA-seq data, and the most commonly used methods are as follows:1. Dimensionality reduction. Dimensionality reduction is actually achieved by combining several original features into some new feature information to obtain a set of compressed and refined feature information [130]. There are linear dimensionality reduction methods and nonlinear dimensionality reduction methods, and the dimensionality reduction methods based on linear decomposition models are principal component analysis, independent component analysis, weighted nonnegative matrix factorization, etc. [131]. Taking principal component analysis as an example, the method has been successfully applied in scRNA-seq data analysis [132–134] to capture the overall structure of cell heterogeneity, with the limitation that it cannot visualize the local structure necessary for cell clustering and cell type identification. Nonlinear dimensionality reduction methods are t-distributed stochastic neighbor embedding (t-SNE), uniform manifold approximation and projection [135], and k-nearest neighbor algorithm [136]. t-SNE is the most widely used data dimensionality reduction method in scRNA-seq data analysis [137].

2. Differential expression analysis. Differentially expressed gene analysis can detect mRNA abundance between different cell types and different cell subpopulations. The effect of different samples or different treatments on gene expression levels, either up- or down-regulated, can be obtained by intergroup comparison [138], which, in turn, allows functional analysis of differentially expressed genes. Differential expression algorithms for scRNA-seq have been developed successively, such as single-cell differential expression [139], MAST [140], Census [141], and BCseq [142].

3. Clustering analysis. Currently, the most popular clustering strategy in scRNA-seq data analysis is the community detection algorithm, and the representative algorithm is Louvain. There are also commonly used clustering strategies, including hierarchical clustering and partitioning clustering. For hierarchical clustering, representative algorithms include Agglomerative Nestling (AGNES) and Divisive Analysis (DIANA). The typical representative of segmentation clustering is the k-means algorithm [143]. In addition, density-based clustering methods (e.g., Density-Based Spatial Clustering of Applications with Noise), neural network-based clustering methods (e.g., self-organizing map), and model-based methods (e.g., expectation maximization algorithm) have also been applied to the field of scRNA-seq data analysis [144].

4. Trajectory inference analysis. In many biological systems, cells are not necessarily in a discrete state; instead, they exist in a continuous state. Some algorithms developed for scRNA-seq pseudotime analysis are Monocle [145], Waterfall [146], Tools for Single Cell Analysis (TSCAN) [147], Sincell [148], Selective Locally Linear Inference of Cellular Expression Relationships (SLICER) [149], Wishbone [150], Wanderlust [151], and single-cell clustering using bifurcation analysis (SCUBA) [152].

5. Prediction of cell communication. Network analysis is one of the most popular research strategies in the field of scRNA-seq, where cell–cell or gene–gene interactions and regulation exist all the time, and the construction of gene and cell-related interaction networks is essential to understand the molecular regulation mechanisms of complex biological traits at microscopic levels. Currently, there are gene co-expression networks, in which the weighted gene co-expression network is used as a classical network analysis method in scRNA-seq research [153]. Furthermore, cell–cell interaction networks are a type of network analysis, and many softwares are available to infer cell–cell interaction networks, including SingleCellSignalR, iTALK, NicheNet, celltalker, CellChat, CellPhoneDB, etc.

In recent years, scRNA-seq technology has emerged as a research method and has gradually become a hot research topic. In addition, scRNA-seq data analysis methods are widely used in the fields of stem cell, developmental biology, tumor, immunology, neurobiology, and skeletal system biology [154–157]. For example, Brunskill et al. [158] used single-cell sequencing to study the gene expression profile during kidney organ development and found that, at different developmental stages, cell populations with the same structure have different origins. Li et al. [159] sequenced 11 colorectal cancer cells and the corresponding normal mucosal cells using single-cell sequencing technology and identified 2 different fibroblastoma subtypes by clustering analysis using the reference component analysis algorithm, in which some up-regulated gene expressions associated with epithelial mesenchymal stem cell differentiation were also identified, providing a good method for characterizing tumor cell heterogeneity. Peng et al. [160] identified 10 types of cells in pancreatic cancer tissues by t-SNE analysis through single-cell transcriptome analysis of 24 preoperative human pancreatic cancer tissues without radiotherapy. The gene expression pattern of pancreatic ductal adenocarcinoma (PDAC) from precancerous to malignant state was investigated using trajectory analysis, and several classical oncogenic pathways, including ErbB and Notch signaling pathways, were found to be activated during the progression of PDAC. Song et al. [161] proposed PseudotimeDE, the first differential gene expression detection tool that takes into account the stochasticity of the pseudotime. This tool uses subsampling to help estimate the stochasticity of the pseudotime, a generalized additive model to fit the relationship between individual gene expression values and the pseudotime, and a permutation test to generate statistically rigorous P values. Compared to existing methods, PseudotimeDE has advantages in accounting for uncertainty in pseudotime inference.

Discussion

Biological sequence classification is an important branch of biological informatics and is mainly related to the classification of DNA, RNA, and amino acid sequences. The goal of the classification task was to build a well-fitted and high-performance prediction model. In Tables S2–S4, we collected more than 100 classification tools for DNA, RNA, proteins, and peptides, as well as their predictive performance. We discuss the existing challenges and future opportunities in model construction for classification tasks with a focus on datasets, classification algorithms, and future trends in biological sequence classification.

Dataset

The construction of a dataset is a key step in biological sequence classification research. The quality of the dataset is related to the performance of the constructed model. The prediction model built using high-quality datasets shows high predictive performance and generalization ability. Some datasets for biological sequence classification have the limitations of small data volumes and an imbalance between positive and negative samples (Tables S2–S4). In most biological sequence classification research, positive and negative samples are manually balanced. In biological informatics research, the process of obtaining datasets is expensive, time-consuming, and laborious. Therefore, positive and negative samples are usually not balanced in the acquired datasets. Data imbalance is a dataset with a positive–negative sample ratio of greater than 3 [35]. The number of positive samples may decrease in sequences analyzed with modern biological technology. For most datasets, a threshold of around 80% was used to remove homologous sequences, but this can lead to data leakage, which can lead to model overfitting and results that are better than they actually look. Therefore, it is necessary to reduce the homology threshold on the data to ensure that it is 25% to 30%.

In the procedure for dataset construction, less data and data imbalance are the major factors affecting data quality. In Table 2, we summarize methods using a small number of datasets and unbalanced datasets in RNA posttranscriptional modification research. For iRNA-m5C [162], the dataset used included four species (H. sapiens, M. musculus, S. cerevisiae, and A. thaliana), among which H. sapiens and M. musculus had smaller datasets (H. sapiens: P-120/N-120 and M. musculus: P-97/N-97). Using the jackknife test, the accuracy of H. sapiens and M. musculus was 100%. Because of the limitation of the number of sequences in the dataset, the accuracy of the model built for these 2 species was 100% and the model was overfitted. If a positive or negative sample is incorrectly predicted, then it will have a greater impact on the accuracy. Similarly, iRNA-PseKNC(2methyl) [163] and iRNA-2OM [53] were developed for Nm using the same dataset (H. sapiens: P-147/N-147). Under 5-fold CV, the Acc values obtained were 98.27% and 97.95%, respectively. In this type of biological sequence classification research, the dataset sequence is small, and there is no independent test set to verify the performance of the model; the accuracy obtained is high, but the model exhibits overfitting and poor migration. When the dataset increases the number of sequences, the prediction performance of the model decreases. The effect of ML is also poor because the sequence dataset and the sequence information obtained are limited. In the case of an imbalanced dataset, the iRNA-m5C_NB [99] predictor method was used, using only a H. sapiens dataset (Met935: training dataset: P-127/N-808; testing dataset: Test1157: P-157/N-1000). The effects of data imbalance were reduced by using SMOTEENN. The Acc values obtained by this method under CV and independent test sets were 82.20% and 74.85%, respectively. Similarly, Dou’s method [98] also adopts SMOTEENN for the imbalance of positive and negative samples in the D-modified dataset. The Acc values obtained using the jackknife test and independent test set were 97.24% and 93.75%, respectively. RFhy-m2G [24] classified m2G modifications and adopted SMOTE to deal with the problem of data imbalance. In addition to the data imbalance in the above method, the dataset was also small, and some positive samples have a limited number of sequences.

Table 2. Summary of 5 RNA posttranscriptional modification datasets.

Type	Method	Year	Dataset	Performance	Imbalance algorithms	
Evaluation	Acc (%)	AUC	
m5C	iRNA-m5C	2020	H. sapiens: TAD (P-120/N-120)	Jackknife test	90.80	0.963	/	
M. musculus: TAD (P-97/N-97)	100	1.000	
S. cerevisiae: TAD (P-211/N-211)	100	1.000	
A. thaliana: TAD (P-5289/N-5289); TSD (P-1000/N-1000)	Jackknife test	70.70	0.765	
IDT	74.00	/	
iRNA-m5C_NB	2020	H. sapiens: Met935: TAD (P-127/N-808); TSD (Test1157: P-157/N-1000)	Jackknife test	82.20	0.91	SMOTEENN	
IDT	74.85	0.83	
Nm	iRNA-PseKNC(2methyl)	2019	H. sapiens: TAD (P-147/N-147)	5-CV	98.27	/	/	
iRNA-2OM	2018	97.95	/	
m2G	RFhy-m2G	2021	H. sapiens: TAD (P-41/N-541); TSD (P-5/N-60)	5-CV	99.82	/	SMOTE	
IDT	94.17	/	
M. musculus: TAD (P-27/N-427); TSD (P-3/N-47)	5-CV	100	/	
IDT	90.43	/	
S. cerevisiae: TAD (P-60/N-283); TSD (P-7/N-31)	5-CV	99.65	/	
IDT	100	/	
D	Dou’s method	2021	Multispecies: TAD (P-140/N-298); TSD (P-36/N-76)	Jackknife test	97.24	0.99	SMOTEEEN	
IDT	93.75	0.87	
P, positive samples; N, negative samples; TAD, training dataset; TSD, testing dataset; 5-CV, 5-fold cross-validation; IDT, independent testing; SMOTE, synthetic minority oversampling technique; SMOTEENN, SMOTE and undersampling method edited nearest neighbors (ENN).

The limitations of datasets and data imbalances are prerequisites for constructing a high-quality dataset. (a) Because of the limitation of the dataset, the amount of data is small, and only by collecting more sequences from related databases or literature, it is best to be verified by experiments. (b) Data imbalance. The solution to the imbalance can be addressed in 3 steps [35,164]. First, the selection of the processing method should be based on the data itself, and positive and negative samples should be balanced at the sampling stage. At present, datasets are collated with various sampling methods, including undersampling, oversampling, and mixed sampling (combined undersampling and oversampling). Second, algorithm-based processing methods, which adjust ML algorithms or deep learning algorithms to correct data imbalances and improve model performance. The most commonly used method is pretraining. In addition, unsupervised pretraining and transfer learning can effectively deal with data imbalances. Third, a cost-sensitive learning method is commonly used to deal with the different costs of misclassified data in a single class. In this method, the objective loss function in deep learning is commonly used. Some researchers have directly used deep learning to study data imbalances. For unbalanced data, BACC was used to calculate the prediction accuracy.

Classification algorithms

Supervised learning classification algorithms are the most commonly used and make up the core of biological sequence classification tasks. This type of algorithm is used to train a model on a sequence of known labels and find a well-performing model so that the new unknown sequence can be input for prediction. Commonly used classification algorithms for biological sequences are divided into traditional ML algorithms and deep learning algorithms:1. The first is a commonly used classification algorithm (RF, SVM, NB, LR, DT, LGBM, and XGBoost; Fig. 1). Construction of the reliable model requires multiple iterations of experiments when using this model. There are more than a dozen traditional algorithms that are commonly used. To find a suitable classification algorithm, the researcher makes a preliminary selection based on experience or literature. This method will take a lot of time, and improper selection of the classification algorithm will affect the prediction results. For some biological sequences, when the prediction results of these traditional classification algorithms are not high, it is impossible to improve the prediction results by improving the algorithm itself. Integrating the weak classification algorithm with the strong classification algorithm only leads to a limited improvement in prediction results. Using traditional classification algorithms, most of the features used are handcrafted features, and the selection of suitable features among many feature coding methods will also be done manually, which influences the prediction performance of the model.

2. The second is the deep learning algorithm (basic neural networks, multilayer perceptrons, CNN, and RNN), which has been widely used in biological sequence classification tasks in recent years. Deep learning algorithms can directly input sequences and classify them by learning or combining manual features with automatic learning features. However, the process of deep learning algorithms is a black box, and it is impossible to explain in detail how it proceeds internally. For small datasets, the prediction effect of using deep learning algorithms is not as good as that of traditional classification algorithms. This may be due to insufficient sequence data for sequence information learning. Deep learning algorithms are suitable for larger datasets and deeper mining of sequence information when conducting biological sequence classification.

Challenges and recommendations

With the continuous development of technologies in molecular biology, the availability of biological sequences has also increased. Computational biologists are required to develop more effective and efficient software for biomedical research. On the basis of this the summary work, we offer our thoughts on the challenges and recommendations for biological sequence classification field.

Dataset

First, we advise caution over the use of datasets in literature or collected by previous studies. Authors should check the datasets according to the assembly steps described in the original articles to ensure that they are correct. On the other hand, the construction procedure should be described adequately in the article if the datasets are being used for the first time. Overall, it is the responsibility of the researchers and authors to ensure that the data used for modeling and evaluation are appropriate. Furthermore, authors should analyze the quality and size of the training and testing datasets, and is there any source of bias, an often neglected but important point. On the basis of our survey, we noticed that some studies did not have sufficient test data or even enough training data. Without proper testing, it may not reflect the true performance of the model on unseen data, which undermines user confidence in the model.

Feature

ML algorithms are used to build a classification model based on sequence statistics to learn the input sequence features and output the prediction results. Various feature encoding methods have been developed [40] on the basis of methods such as sequence, structure, physical–chemical properties, and score matrix, but this is far from enough. Future priorities include not only developing feature descriptors that can contain more deeply hidden information but also exploring how to exploit known structural parts of biological systems, while using neural networks to learn unknown parts. In this way, many heavy models can be replaced by simpler models that are easier to interpret and more robust to unseen data. On the other hand, as mentioned in the “General scheme and principles of ML in biological sequence classification” section, development and selection of appropriate feature selection methods to obtain the optimization feature set are very important for the construction of a robust predictive model.

Standardization

With no standardization, ML algorithms for biological sequence classification have been developed with a very broad range of applications, dataset collection procedures, and performance evaluation metrics. Although the diversity of biological sequence data makes it difficult to develop common standards and guidelines, it will benefit the research community to establish even a few numbers of criteria for specific tasks, not only for authors but also for reviewers.

Collaboration

ML-based models can be thought as the interface between the data analysis and the wet experiment. Biomedical researchers are the prospective users of our developed computational tools. It is very important for computer scientists to collaborate with experimental biologists. Advances in predictive models require joint efforts from developers and users, and prompt feedback from even a small number of users can have a positive impact on model performance. Moreover, the collaboration will also help interpret the built models. Biologists are interested not only in just accurate modeling but also in discovering mechanisms and factors that determine model output. On the other hand, we recommend using reviewers with backgrounds in biology and ML, as they can judge technical and application aspects differently. In this way, models that produce robust predictions are ready to be deployed in real application scenarios.

Software

As described in the “General scheme and principles of ML in biological sequence classification” section, make the trained model long-term available in the form of a user-friendly web server in a suitable repository such as GitHub (https://github.com). This benefits both computational and biological experimental researchers as it allows them to build on the method without having to start from scratch. We noticed that many development servers were no longer accessible even after the article was just published. Additionally, some web servers do not explicitly describe usage guidelines and how their models are implemented. Therefore, we highly recommend the authors take the tool seriously after it is released and try to accommodate support requests.

Conclusions

With the continuous development of biological technology, biological sequences have increased exponentially. Traditional biological experimentation techniques are time-consuming and laborious. Therefore, it is necessary to accurately classify these biological sequences on the basis of the computational tools developed by ML. In this review, we have summarized the data and general classification methods related to 3 types of biological sequences (DNA, RNA, and amino acids). An online webpage was also built to provide downloads of datasets, accuracy and AUC values of related methods, and a web server. In addition, we introduced the model construction process of biological sequence classification, including dataset collection, feature extraction and feature selection, ML algorithms, model performance evaluation, and web server implementation. The classification method based on DNA, RNA, and amino acid sequences was briefly introduced.

Although there are many general methods and related datasets for biological sequence classification, most of the methods are improved on the same dataset, and there is a lot of room for development. There are still challenges in biological sequence classification research tasks, such as small datasets and unbalanced data, and the interpretability and repeatability of ML algorithms. In order to fully understand more information about biological sequences, it is necessary to develop more classification methods and collect more biological sequence data. In the future, we will continue to update data and related methods to provide a public platform and benchmark data for biological sequence function and modification classification research to avoid data and method confusion. It is more helpful for researchers to have a simple understanding of the study of biological sequence classification. The discussion and analysis of the existing shortcomings of biological sequence research provides insights for future biological sequence classification research. For single-cell sequencing data analysis, we briefly introduce the commonly used methods and their applications in systems biology. In the future, we will give a more in-depth review of the hot research on single-cell sequencing data analysis.

Acknowledgments

Funding: This work was supported by the Fundamental Research Funds for the Central Universities (no. YJS2205 and no. JB180307), the Innovation Fund of Xidian University (no. YJS2205), the Natural Science Foundation of China (no. 62072353 and no. 61922020), the China Postdoctoral Science Foundation (no. 2022T150095), the Sichuan Provincial Science Fund for Distinguished Young Scholars (2021JDJQ0025), and the Special Science Foundation of Quzhou (2021D004). Author contributions: Data curation, formal analysis, methodology, software, and writing (original draft): C.A. Validation and investigation: S.J. Data curation and revision: Y.W. Project administration, funding acquisition, and resources: L.Y. Conceptualization, project administration, and writing (review and editing): Q.Z. All authors contributed to the final manuscript. Competing interests: The authors declare that they have no competing interests.

Supplementary Materials

Supplementary Materials Table S1. Biological sequence modification site database.

Table S2. Summary of DNA sequence classification methods and dataset.

Table S3. Summary of RNA sequence classification methods and dataset.

Table S4. Summary of amino acid sequence classification methods and dataset.
==== Refs
References

1. Standing KG. Peptide and protein de novo sequencing by mass spectrometry. Curr Opin Struct Biol. 2003;13 (5 ):595–601.14568614
2. Guo B-Y, Zeng T, Wu H-C. Recent advances of DNA sequencing via nanopore-based technologies. Sci Bull. 2015;60 (3 ):287–295.
3. Zhao N, Cao J, Xu J, Liu B, Liu B, Chen D, Xia B, Chen L, Zhang W, Zhang Y, et al. Targeting RNA with next- and third-generation sequencing improves pathogen identification in clinical samples. Adv Sci. 2021;8 (23 ):2102593.
4. Fan X, Tang D, Liao Y, Li P, Zhang Y, Wang M, Liang F, Wang X, Gao Y, Wen L, et al. Single-cell RNA-seq analysis of mouse preimplantation embryos by third-generation sequencing. PLOS Biol. 2020;18 (12 ):e3001017.33378329
5. Han R, Li Y, Gao X, Wang S. An accurate and rapid continuous wavelet dynamic time warping algorithm for end-to-end mapping in ultra-long nanopore sequencing. Bioinformatics. 2018;34 (17 ):i722–i731.30423085
6. Raimondi D, Orlando G, Moreau Y, Vranken WF. Ultra-fast global homology detection with discrete cosine transform and dynamic time warping. Bioinformatics. 2018;34 (18 ):3118–3125.29684140
7. Lyons J, Paliwal KK, Dehzangi A, Heffernan R, Tsunoda T, Sharma A. Protein fold recognition using HMM-HMM alignment and dynamic programming. J Theor Biol. 2016;393 :67–74.26801876
8. Gao C, Shen W, Zhang Y. A dynamic-time distance based on wavelet decomposition for subcellular localization classification. IEEE Access. 2020;8 :220293–220301.
9. Dakhli A, Amar CB. Power spectrum and dynamic time warping for DNA sequences classification. Evol Syst. 2020;11 (4 ):637–646.
10. Skutkova H, Vitek M, Babula P, Kizek R, Provaznik I. Classification of genomic signals using dynamic time warping. BMC Bioinformatics. 2013;14 (10 ):S1.
11. Rakthanmanon T, Campana B, Mueen A, Batista G, Westover B, Zhu Q, Zakaria J, Keogh E. Searching and mining trillions of time series subsequences under dynamic time warping. Paper presented at: KDD 2012. Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2012 Aug 12–16; Beijing, China.
12. Gogolou A, Tsandilas T, Palpanas T, Bezerianos A. Comparing similarity perception in time series visualizations. IEEE Trans Vis Comput Graph. 2019;25 (1 ):523–533.
13. Guo S, Xu K, Zhao R, Gotz D, Zha H, Cao N. EventThread: Visual summarization and stage analysis of event sequence data. IEEE Trans Vis Comput Graph. 2018;24 (1 ):56–65.28866586
14. Stitz H, Gratzl S, Aigner W, Streit M. ThermalPlot: Visualizing multi-attribute time-series data using a thermal metaphor. IEEE Trans Vis Comput Graph. 2016;22 (12 ):2594–2607.26731767
15. Strobelt H, Gehrmann S, Pfister H, Rush AM. LSTMVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans Vis Comput Graph. 2018;24 (1 ):667–676.28866526
16. Bernard J, Mittelstädt S, Lücke-Tieke H, Keim D, May T, Kohlhammer J. Visual analysis of time-series similarities for anomaly detection in sensor networks. Comput Graph Forum. 2014;33 (3 ):401–410.
17. Rind A, Lammarsch T, Aigner W, Alsallakh B, Miksch S. TimeBench: A data model and software library for visual analytics of time-oriented data. IEEE Trans Vis Comput Graph. 2013;19 (12 ):2247–2256.24051791
18. Lambert SA, Jolma A, Campitelli LF, Das PK, Yin Y, Albu M, Chen X, Taipale J, Hughes TR, Weirauch MT. The human transcription factors. Cell. 2018;172 (4 ):650–665.29425488
19. Yu L, Shi Y, Zou Q, Wang S, Zheng L, Gao L. Exploring drug treatment patterns based on the action of drug and multilayer network model. Int J Mol Sci. 2020;21 (14 ):5014.32708644
20. Lo Giudice C, Pesole G, Picardi E. High-throughput sequencing to detect DNA-RNA changes. Methods Mol Biol. 2021;2181 :193–212.32729082
21. Meyer CA, Liu XS. Identifying and mitigating bias in next-generation sequencing methods for chromatin biology. Nat Rev Genet. 2014;15 (11 ):709–721.25223782
22. Aebersold R, Mann M. Mass spectrometry-based proteomics. Nature. 2003;422 (6928 ):198–207.12634793
23. Zhang Z, Cui F, Wang C, Zhao L, Zou Q. Goals and approaches for each processing step for single-cell RNA sequencing data. Brief Bioinform. 2021;22 (4 ):bbaa314.33316046
24. Ao C, Zou Q, Yu L. RFhy-m2G: Identification of RNA N2-methylguanosine modification sites based on random forest and hybrid features. Methods. 2021;203 :32–39.34033879
25. Niu M, Ju Y, Lin C, Zou Q. Characterizing viral circRNAs and their application in identifying circRNAs in viruses. Brief Bioinform 2022;23 (1 ):bbab404.34585234
26. Cui F, Zhang Z, Zou Q. Sequence representation approaches for sequence-based protein prediction tasks that use deep learning. Brief Funct Genomics. 2021;20 (1 ):61–73.33527980
27. Lv Z, Ao C, Zou Q. Protein function prediction: From traditional classifier to deep learning. Proteomics. 2019;19 (14 ):e1900119.31187588
28. Chen W, Lin H. Recent advances in identification of RNA modifications. Noncoding RNA. 2016;3 (1 ):1.29657273
29. Chen X, Sun Y-Z, Liu H, Zhang L, Li J-Q, Meng J. RNA methylation and diseases: Experimental results, databases, Web servers and computational models. Brief Bioinform. 2019;20 (3 ):896–917.29165544
30. Zhu X, He J, Zhao S, Tao W, Xiong Y, Bi S. A comprehensive comparison and analysis of computational predictors for RNA N6-methyladenosine sites of Saccharomyces cerevisiae. Brief Funct Genomics. 2019;18 (6 ):367–376.31609411
31. Liu K, Chen W. iMRM: A platform for simultaneously identifying multiple kinds of RNA modifications. Bioinformatics. 2020;36 (11 ):3336–3342.32134472
32. Xu X, Liu S, Yang Z, Zhao X, Deng Y, Zhang G, Pang J, Zhao C, Zhang W. A systematic review of computational methods for predicting long noncoding RNAs. Brief Funct Genomics. 2021;20 (3 ):162–173.33754153
33. Ramazi S, Zahiri J. Post-translational modifications in proteins: Resources, tools and prediction methods. Database. 2021;2021 :baab012.33826699
34. He W, Wei L, Zou Q. Research progress in protein posttranslational modification site prediction. Brief Funct Genomics. 2019;18 (4 ):220–229.
35. Dou L, Yang F, Xu L, Zou Q. A comprehensive review of the imbalance classification of protein post-translational modifications. Brief Bioinform. 2021;22 (5 ):bbab089.33834199
36. Ren J, Gao X, Liu Z, Cao J, Ma Q, Xue Y. Computational analysis of phosphoproteomics: Progresses and perspectives. Curr Protein Pept Sci. 2011;12 (7 ):591–601.21827424
37. Hasan MM, Khatun MS, Kurata H. A comprehensive review of in silico analysis for protein S-sulfenylation sites. Protein Pept Lett. 2018;25 (9 ):815–821.30182830
38. Ao C, Yu L, Zou Q. Prediction of bio-sequence modifications and the associations with diseases. Brief Funct Genomics. 2021;20 (1 ):1–18.33313647
39. Huang Y, Niu B, Gao Y, Fu L, Li W. CD-HIT Suite: A web server for clustering and comparing biological sequences. Bioinformatics. 2010;26 (5 ):680–682.20053844
40. Chen Z, Zhao P, Li C, Li F, Xiang D, Chen YZ, Akutsu T, Daly RJ, Webb GI, Zhao Q, et al. iLearnPlus: A comprehensive and automated machine-learning platform for nucleic acid and protein sequence analysis, prediction and visualization. Nucleic Acids Res. 2021;49 (10 ):e60.33660783
41. Muhammod R, Ahmed S, Md Farid D, Shatabda S, Sharma A, Dehzangi A. PyFeat: A python-based effective feature generation tool for DNA, RNA and protein sequences. Bioinformatics. 2019;35 (19 ):3831–3833.30850831
42. Chen Z, Zhao P, Li F, Leier A, Marquez-Lago TT, Wang Y, Webb GI, Smith AI, Daly RJ, Chou K-C, et al. iFeature: A Python package and web server for features extraction and selection from protein and peptide sequences. Bioinformatics. 2018;34 (14 ):2499–2502.29528364
43. Liu B, Gao X, Zhang H. BioSeq-Analysis2.0: An updated platform for analyzing DNA, RNA and protein sequences at sequence level and residue level based on machine learning approaches. Nucleic Acids Res. 2019;47 (20 ):e127.31504851
44. Wang J, Du PF, Xue XY, Li GP, Zhou YK, Zhao W, Lin H, Chen W. VisFeature: A stand-alone program for visualizing and analyzing statistical features of biological sequences. Bioinformatics. 2020;36 (4 ):1277–1278.31504195
45. Wang J, Yang B, Revote J, Leier A, Marquez-Lago TT, Webb G, Song J, Chou K-C, Lithgow T. POSSUM: A bioinformatics toolkit for generating numerical sequence feature descriptors based on PSSM profiles. Bioinformatics. 2017;33 (17 ):2756–2758.28903538
46. Wei L, Zhou C, Su R, Zou Q. PEPred-Suite: Improved and robust prediction of therapeutic peptides using adaptive feature representation learning. Bioinformatics. 2019;35 (21 ):4272–4280.30994882
47. Jiang Y, Wang D, Yao Y, Eubel H, Künzler P, Møller IM, Xu D. MULocDeep: A deep-learning framework for protein subcellular and suborganellar localization prediction with residue-level interpretation. Comput Struct Biotechnol J. 2021;19 :4825–4839.34522290
48. Nambiar A, Liu S, Hopkins M, Heflin M, Maslov S, Ritz A. Transforming the language of life: Transformer neural networks for protein prediction tasks. Paper presented at: ACM-BCB 2020. Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics; 2020 Sep 21–24; USA.
49. Rao R, Bhattacharya N, Thomas N, Duan Y, Chen X, Canny J, Abbeel P, Song YS. Evaluating protein transfer learning with TAPE. Adv Neural Inf Process Syst. 2019;32 :9689–9701.33390682
50. Lv Z, Wang P, Zou Q, Jiang Q. Identification of sub-Golgi protein localization by use of deep representation learning features. Bioinformatics. 2020;36 (24 ):5600–5609.
51. Wang H, Liu Y, Guan H, Fan G-L. The regulation of target genes by co-occupancy of transcription factors, c-Myc and Mxi1 with max in the mouse cell line. Curr Bioinformatics. 2020;15 (6 ):581–588.
52. Wu M, Yang Y, Wang H, Xu Y. A deep learning method to more accurately recall known lysine acetylation sites. BMC Bioinformatics. 2019;20 (1 ):49.30674277
53. Yang H, Lv H, Ding H, Chen W, Lin H. iRNA-2OM: A sequence-based predictor for identifying 2 '-O-methylation sites in homo sapiens. J Comput Biol. 2018;25 (11 ):1266–1277.30113871
54. Chen W, Lv H, Nie F, Lin H. i6mA-Pred: Identifying DNA N-6 - methyladenine sites in the rice genome. Bioinformatics. 2019;35 (16 ):2796–2800.30624619
55. Zheng W, Wuyun Q, Cheng M, Hu G, Zhang Y. Two-level protein methylation prediction using structure model-based features. Sci Rep. 2020;10 (1 ):6008.32265459
56. Wen P-P, Shi S-P, Xu H-D, Wang L-N, Qiu J-D. Accurate in silico prediction of species-specific methylation sites based on information gain feature optimization. Bioinformatics. 2016;32 (20 ):3107–3115.27354692
57. Wu X, Wei Y, Jiang T, Wang Y, Jiang S. A micro-aggregation algorithm based on density partition method for anonymizing biomedical data. Curr Bioinformatics. 2019;14 (7 ):667–675.
58. Greener JG, Kandathil SM, Moffat L, Jones DT. A guide to machine learning for biologists. Nat Rev Mol Cell Biol. 2022;23 (1 ):40–55.34518686
59. Zou Q, Xing P, Wei L, Liu B. Gene2vec: Gene subsequence embedding for prediction of mammalian N-6-methyladenosine sites from mRNA. RNA. 2019;25 (2 ):205–218.30425123
60. Huang Y, He N, Chen Y, Chen Z, Li L. BERMP: A cross-species classifier for predicting m6A sites by integrating a deep learning algorithm and a random forest approach. Int J Biol Sci. 2018;14 (12 ):1669–1677.30416381
61. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, et al. Scikit-Learn: Machine learning in python. J Mach Learn Res. 2011;12 :2825–2830.
62. Chen KM, Cofer EM, Zhou J, Troyanskaya OG. Selene: A PyTorch-based deep learning library for sequence data. Nat Methods. 2019;16 (4 ):315–318.30923381
63. Yang A, Zhang W, Wang J, Yang K, Han Y, Zhang L. Review on the application of machine learning algorithms in the sequence data mining of DNA. Front Bioeng Biotechnol. 2020;8 :1032.33015010
64. Cai L, Ren X, Fu X, Peng L, Gao M, Zeng X. iEnhancer-XG: Interpretable sequence-based enhancers and their strength predictor. Bioinformatics. 2021;37 (8 ):1060–1067.33119044
65. Lyu Y, He W, Li S, Zou Q, Guo F. iPro2L-PSTKNC: A two-layer predictor for discovering various types of promoters by position specific of nucleotide composition. IEEE J Biomed Health Inform. 2021;25 (6 ):2329–2337.32976109
66. Li J, Pu Y, Tang J, Zou Q, Guo F. DeepATT: A hybrid category attention neural network for identifying functional effects of DNA sequences. Brief Bioinform. 2021;22 (3 ):bbaa159.32778871
67. Khanal J, Tayara H, Zou Q, Chong KT. Identifying DNA N4-methylcytosine sites in the rosaceae genome with a deep learning model relying on distributed feature representation. Comput Struct Biotechnol J. 2021;19 :1612–1619.33868598
68. Lv Z, Wang D, Ding H, Zhong B, Xu L. Escherichia coli DNA N-4-methycytosine site prediction accuracy improved by light gradient boosting machine feature selection technology. IEEE Access. 2020;8 :14851–14859.
69. Zeng R, Liao M. Developing a multi-layer deep learning based predictive model to identify DNA N4-methylcytosine modifications. Front Bioeng Biotechnol. 2020;8 :274.32373597
70. Wei L, Su R, Luan S, Liao Z, Manavalan B, Zou Q, Shi X. Iterative feature representations improve N4-methylcytosine site prediction. Bioinformatics. 2019;35 (23 ):4930–4937.31099381
71. He W, Jia C, Zou Q. 4mCPred: Machine learning methods for DNA N-4-methylcytosine sites prediction. Bioinformatics. 2019;35 (4 ):593–601.30052767
72. Manavalan B, Basith S, Shin TH, Wei L, Lee G. Meta-4mCpred: A sequence-based meta-predictor for accurate DNA 4mC site prediction using effective feature representation. Mol Ther Nucleic Acids. 2019;16 :733–744.31146255
73. Lv Z, Ding H, Wang L, Zou Q. A convolutional neural network using dinucleotide one-hot encoder for identifying DNA N6-methyladenine sites in the rice genome. Neurocomputing. 2021;422 :214–221.
74. Huang Q, Zhang J, Wei L, Guo F, Zou Q. 6mA-RicePred: A method for identifying DNA N6-methyladenine sites in the rice genome based on feature fusion. Front Plant Sci. 2020;11 :4.32076430
75. Liu Z, Dong W, Jiang W, He Z. csDMA: An improved bioinformatics tool for identifying DNA 6 mA modifications via Chou's 5-step rule. Sci Rep. 2019;9 (1 ):13109.31511570
76. Liu M, Sun Z-L, Zeng Z, Lam K-M. MGF6mARice: Prediction of DNA N6-methyladenine sites in rice by exploiting molecular graph feature and residual block. Brief Bioinform. 2022;23 (3 ):bbac082.35325050
77. Li Z, Jiang H, Kong L, Chen Y, Lang K, Fan X, Zhang L, Pian C. Deep6mA: A deep learning framework for exploring similar patterns in DNA N6-methyladenine sites across different species. PLOS Comput Biol. 2021;17 (2 ):e1008767.33600435
78. Wei L, Luan S, Nagai LAE, Su R, Zou Q. Exploring sequence-based features for the improved prediction of DNA N4-methylcytosine sites in multiple species. Bioinformatics. 2019;35 (8 ):1326–1333.30239627
79. Wahab A, Ali S D, Tayara H, Chong KT. iIM-CNN: Intelligent identifier of 6mA sites on different species by using convolution neural network. IEEE Access. 2019;7 :178577–178583.
80. Li J, Zhang L, He S, Guo F, Zou Q. SubLocEP: A novel ensemble predictor of subcellular localization of eukaryotic mRNA based on machine learning. Brief Bioinform. 2021;22 (5 ):bbaa401.33388743
81. Niu M, Zhang J, Li Y, Wang C, Liu Z, Ding H, Zou Q, Ma Q. CirRNAPL: A web server for the identification of circRNA based on extreme learning machine. Comput Struct Biotechnol J. 2020;18 :834–842.32308930
82. Zou Q, Guo J, Ju Y, Wu M, Zeng X, Hong Z. Improving tRNAscan-SE annotation results via ensemble classifiers. Mol Inform. 2015;34 (11–12 ):761–770.27491037
83. Ali SD, Alam W, Tayara H, Chong KT. Identification of functional piRNAs using a convolutional neural network. IEEE/ACM Trans Comput Biol Bioinform. 2020;19 (3 ):1661–1669.
84. Liu B, Yang F, Chou K-C. 2L-piRNA: A two-layer ensemble classifier for identifying piwi-interacting RNAs and their function. Mol Ther Nucleic Acids. 2017;7 :267–277.28624202
85. Niu M, Zou Q. SgRNA-RF: Identification of SgRNA on-target activity with imbalanced datasets. IEEE/ACM Trans Comput Biol Bioinform. 2021;19 (4 ):2442–2453.
86. Song Z, Huang D, Song B, Chen K, Song Y, Liu G, Su J, Magalhães JP, Rigden DJ, Meng J. Attention-based multi-label neural networks for integrated prediction and interpretation of twelve widely occurring RNA modifications. Nat Commun. 2021;12 (1 ):4011.34188054
87. Lv Z, Zhang J, Ding H, Zou Q. RF-PseU: A random forest predictor for RNA pseudouridine sites. Front Bioeng Biotechnol. 2020;8 :134.32175316
88. Dou L, Li X, Ding H, Xu L, Xiang H. Is there any sequence feature in the RNA pseudouridine modification prediction problem? Mol Ther Nucleic Acids. 2020;19 :293–303.31865116
89. Dou L, Li X, Ding H, Xu L, Xiang H. Prediction of m5C modifications in RNA sequences by combining multiple sequence features. Mol Ther Nucleic Acids. 2020;21 :332–342.32645685
90. Zhang L, Qin X, Liu M, Xu Z, Liu G. DNN-m6A: A cross-species method for identifying RNA N6-methyladenosine sites based on deep neural network with multi-information fusion. Genes. 2021;12 (3 ):354.33670877
91. Dao F-Y, Lv H, Yang Y-H, Zulfiqar H, Gao H, Lin H. Computational identification of N6-methyladenosine sites in multiple tissues of mammals. Comput Struct Biotechnol J. 2020;18 :1084–1091.32435427
92. Li J, He S, Guo F, Zou Q. HSM6AP: A high-precision predictor for the Homo sapiens N6-methyladenosine (m6 A) based on multiple weights and feature stitching. RNA Biol. 2021;18 (11 ):1882–1892.33446014
93. Mahmoudi O, Wahab A, Chong KT. iMethyl-Deep: N6 methyladenosine identification of yeast genome with automatic feature extraction technique by using deep learning algorithm. Genes. 2020;11 (5 ):529.32397453
94. Wei L, Su R, Wang B, Li X, Zou Q, Xing G. Integration of deep feature representations and handcrafted features to improve the prediction of N-6-methyladenosine sites. Neurocomputing. 2019;324 :3–9.
95. Chen K, Wei Z, Zhang Q, Wu X, Rong R, Lu Z, Su J, de Magalhães JP, Rigden DJ, Meng J. WHISTLE: A high-accuracy map of the human N6-methyladenosine (m6A) epitranscriptome predicted using a machine learning approach. Nucleic Acids Res. 2019;47 (7 ):e41.30993345
96. Chen W, Xing P, Zou Q. Detecting N6-methyladenosine sites from RNA transcriptomes using ensemble support vector machines. Sci Rep. 2017;7 :40242.28079126
97. Tahir M, Tayara H, Chong KT. iPseU-CNN: Identifying RNA pseudouridine sites using convolutional neural networks. Mol Ther Nucleic Acids. 2019;16 :463–470.31048185
98. Dou L, Zhou W, Zhang L, Xu L, Han K. Accurate identification of RNA D modification using multiple features. RNA Biol. 2021;18 (12 ):2236–2246.33729104
99. Dou L, Li X, Ding H, Xu L, Xiang H. iRNA-m5C_NB: A novel predictor to identify RNA 5-methylcytosine sites based on the naive bayes classifier. IEEE Access. 2020;8 :84906–84917.
100. Bonetta R, Valentino G. Machine learning techniques for protein function prediction. Proteins. 2020;88 (3 ):397–413.31603244
101. Hu J, Rao L, Zhu YH, Zhang GJ, Yu DJ. TargetDBP+: Enhancing the performance of identifying DNA-binding proteins via weighted convolutional features. J Chem Inf Model. 2021;61 (1 ):505–515.33410688
102. Luo J, Meng Y, Zhai J, Zhu Y, Li Y. Screening of SLE-susceptible SNPs in one chinese family with systemic lupus erythematosus. Curr Bioinformatics. 2020;15 (7 ):778–787.
103. Jiao S, Xu L, Ju Y. CWLy-RF: A novel approach for identifying cell wall lyases based on random forest classifier. Genomics. 2021;113 (5 ):2919–2924.34186189
104. Meng C, Guo F, Zou Q. CWLy-SVM: A support vector machine-based tool for identifying cell wall lytic enzymes. Comput Biol Chem. 2020;87 :107304.32580129
105. Jing X-Y, Li F-M. Predicting cell wall lytic enzymes using combined features. Front Bioeng Biotechnol. 2021;8 :627335.33585423
106. Niu M, Wu J, Zou Q, Liu Z, Xu L. rBPDL:Predicting RNA-binding proteins using deep learning. IEEE J Biomed Health Inform. 2021;25 (9 ):3668–3676.33780344
107. Sun X, Jin T, Chen C, Cui X, Ma Q, Yu B. RBPro-RF: Use Chou's 5-steps rule to predict RNA-binding proteins via random forest with elastic net. Chemom Intell Lab Syst. 2020;197 :103919.
108. Bressin A, Schulte-Sasse R, Figini D, Urdaneta EC, Beckmann BM, Marsico A. TriPepSVM: De novo prediction of RNA-binding proteins based on short amino acid motifs. Nucleic Acids Res. 2019;47 (9 ):4406–4417.30923827
109. Li S-H, Li S-D, Wu K-L, Li J-Y, Li H-J, Wang W-Q, Yang L-J, Xu J-J, Chang G-J, Zhang Y-L, et al. Transcriptome analysis reveals possible virulence factors of Paragonimus proliferus. Curr Bioinformatics. 2021;16 (2 ):197–206.
110. Wang D, Liu D, Yuchi J, He F, Jiang Y, Cai S, Li J, Xu D. MusiteDeep: A deep-learning based webserver for protein post-translational modification site prediction and visualization. Nucleic Acids Res. 2020;48 (W1 ):W140–W146.32324217
111. Yu B, Yu Z, Chen C, Ma A, Liu B, Tian B, Ma Q. DNNAce: Prediction of prokaryote lysine acetylation sites through deep neural networks with multi-information fusion. Chemom Intell Lab Syst. 2020;200 :103999.
112. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71 (3 ):209–249.33538338
113. Wu C, Lin B, Shi K, Zhang Q, Gao R, Yu Z, De Marinis Y, Zhang Y, Liu Z-P. PEPRF: Identification of essential proteins by integrating topological features of PPI network and sequence-based features via random forest. Curr Bioinformatics. 2021;16 (9 ):1161–1168.
114. Li S-H, Zhang J, Zhao Y-W, Dao F-Y, Ding H, Chen W, Tang H. iPhoPred: A predictor for identifying phosphorylation sites in human protein. IEEE Access. 2019;7 :177517–177528.
115. Wei L, Xing P, Tang J, Zou Q. PhosPred-RF: A novel sequence-based predictor for phosphorylation sites using sequential information only. IEEE Trans Nanobioscience. 2017;16 (4 ):240–247.28166503
116. Chen G, Cao M, Luo K, Wang L, Wen P, Shi S. ProAcePred: Prokaryote lysine acetylation sites prediction based on elastic net feature optimization. Bioinformatics. 2018;34 (23 ):3999–4006.29868863
117. Liu Y, Li A, Zhao X-M, Wang M. DeepTL-Ubi: A novel deep transfer learning method for effectively predicting ubiquitination sites of multiple species. Methods. 2021;192 :103–111.32791338
118. Wang X, Yan R, Chen YZ, Wang Y. Computational identification of ubiquitination sites in Arabidopsis thaliana using convolutional neural networks. Plant Mol Biol. 2021;105 (6 ):601–610.33527202
119. Basith S, Manavalan B, Hwan Shin T, Lee G. Machine intelligence in peptide therapeutics: A next-generation tool for rapid disease screening. Med Res Rev. 2020;40 (4 ):1276–1314.31922268
120. Lv Z, Cui F, Zou Q, Zhang L, Xu L. Anticancer peptides prediction with deep representation learning features. Brief Bioinform. 2021;22 (5 ):bbab008.33529337
121. Agrawal P, Bhagat D, Mahalwal M, Sharma N, Raghava GPS. AntiCP 2.0: An updated model for predicting anticancer peptides. Brief Bioinform. 2021;22 (3 ):bbaa153.32770192
122. Rao B, Zhou C, Zhang G, Su R, Wei L. ACPred-Fuse: Fusing multi-view information improves the prediction of anticancer peptides. Brief Bioinform. 2020;21 (5 ):1846–1855.31729528
123. Cai L, Wang L, Fu X, Xia C, Zeng X, Zou Q. ITP-Pred: An interpretable method for predicting, therapeutic peptides with fused features low-dimension representation. Brief Bioinform. 2021;22 (4 ):bbaa367.33313672
124. Arif M, Ahmad S, Ali F, Fang G, Li M, Yu D-J. TargetCPP: Accurate prediction of cell-penetrating peptides from optimized multi-scale features using gradient boost decision tree. J Comput Aided Mol Des. 2020;34 (8 ):841–856.32180124
125. Fu X, Cai L, Zeng X, Zou Q. StackCPPred: A stacking and pairwise energy content-based prediction of cell-penetrating peptides and their uptake efficiency. Bioinformatics. 2020;36 (10 ):3028–3034.32105326
126. Wei L, Zhou C, Chen H, Song J, Su R. ACPred-FL: A sequence-based predictor using effective feature representation to improve the prediction of anti-cancer peptides. Bioinformatics. 2018;34 (23 ):4007–4016.29868903
127. Wei L, Xing P, Su R, Shi G, Ma ZS, Zou Q. CPPred-RF: A sequence-based predictor for identifying cell penetrating peptides and their uptake efficiency. J Proteome Res. 2017;16 (5 ):2044–2053.28436664
128. Guo H, Zhu P, Wu X, Li X, Wen L, Tang F. Single-cell methylome landscapes of mouse embryonic stem cells and early embryos analyzed using reduced representation bisulfite sequencing. Genome Res. 2013;23 (12 ):2126–2135.24179143
129. Chen C, Xing D, Tan L, Li H, Zhou G, Huang L, Xie XS. Single-cell whole-genome analyses by linear amplification via transposon insertion (LIANTI). Science. 2017;356 (6334 ):189–194.28408603
130. Townes FW, Hicks SC, Aryee MJ, Irizarry RA. Feature selection and dimension reduction for single-cell RNA-Seq based on a multinomial model. Genome Biol. 2019;20 (1 ):295.31870412
131. Hie B, Peters J, Nyquist SK, Shalek AK, Berger B, Bryson BD. Computational methods for single-cell RNA sequencing. Annu Rev Biomed Data Sci. 2020;3 :339–364.
132. AlJanahi AA, Danielsen M, Dunbar CE. An introduction to the analysis of single-cell RNA-sequencing data. Mol Ther Methods Clin Dev. 2018;10 :189–196.30094294
133. Fletcher RB, Das D, Ngai J. Creating lineage trajectory maps via integration of single-cell RNA-sequencing and lineage tracing: Integrating transgenic lineage tracing and single-cell RNA-sequencing is a robust approach for mapping developmental lineage trajectories and cell fate changes. Bioessays. 2018;40 (8 ):e1800056.29944188
134. Hwang B, Lee JH, Bang D. Single-cell RNA sequencing technologies and bioinformaticspipelines. Exp Mol Med. 2018;50 (8 ):1–14.
135. Becht E, McInnes L, Healy J, Dutertre CA, Kwok IWH, Ng LG, Ginhoux F, Newell EW. Dimensionality reduction for visualizing single-cell data using UMAP. Nat Biotechnol. 2019;37 (1 ):38–44.
136. Luecken MD, Theis FJ. Current best practices in single-cell RNA-seq analysis: A tutorial. Mol Syst Biol. 2019;15 (6 ):e8746.31217225
137. Linderman GC, Rachh M, Hoskins JG, Steinerberger S, Kluger Y. Fast interpolation-based t-SNE for improved visualization of single-cell RNA-seq data. Nat Methods. 2019;16 (3 ):243–245.30742040
138. Sharma A, Cao EY, Kumar V, Zhang X, Leong HS, Wong AML, Ramakrishnan N, Hakimullah M, Teo HMV, Chong FT, et al. Longitudinal single-cell RNA sequencing of patient-derived primary cells reveals drug-induced infidelity in stem cell hierarchy. Nat Commun. 2018;9 (1 ):4931.30467425
139. Kharchenko PV, Silberstein L, Scadden DT. Bayesian approach to single-cell differential expression analysis. Nat Methods. 2014;11 (7 ):740–742.24836921
140. Finak G, McDavid A, Yajima M, Deng J, Gersuk V, Shalek AK, Slichter CK, Miller HW, McElrath MJ, Prlic M, et al. MAST: A flexible statistical framework for assessing transcriptional changes and characterizing heterogeneity in single-cell RNA sequencing data. Genome Biol. 2015;16 (1 ):278.26653891
141. Qiu X, Hill A, Packer J, Lin D, Ma Y-A, Trapnell C. Single-cell mRNA quantification and differential analysis with Census. Nat Methods. 2017;14 (3 ):309–315.28114287
142. Chen L, Zheng S. BCseq: Accurate single cell RNA-seq quantification with bias correction. Nucleic Acids Res. 2018;46 (14 ):e82.29718338
143. Grün D, Lyubimova A, Kester L, Wiebrands K, Basak O, Sasaki N, Clevers H, van Oudenaarden A. Single-cell messenger RNA sequencing reveals rare intestinal cell types. Nature. 2015;525 (7568 ):251–255.26287467
144. Kiselev VY, Andrews TS, Hemberg M. Challenges in unsupervised clustering of single-cell RNA-seq data. Nat Rev Genet. 2019;20 (5 ):273–282.30617341
145. Perešíni P, Kuźniar M, Kostić D. Dynamic, fine-grained data plane monitoring with monocle. IEEE/ACM Trans Netw. 2018;26 (1 ):534–547.
146. Shin J, Berg DA, Zhu Y, Shin JY, Song J, Bonaguidi MA, Enikolopov G, Nauen DW, Christian KM, Ming GL, et al. Single-Cell RNA-seq with waterfall reveals molecular cascades underlying adult neurogenesis. Cell Stem Cell. 2015;17 (3 ):360–372.26299571
147. Ji Z, Ji H. TSCAN: Pseudo-time reconstruction and evaluation in single-cell RNA-seq analysis. Nucleic Acids Res. 2016;44 (13 ):e117.27179027
148. Juliá M, Telenti A, Rausell A. Sincell: An R/Bioconductor package for statistical assessment of cell-state hierarchies from single-cell RNA-seq. Bioinformatics. 2015;31 (20 ):3380–3382.26099264
149. Welch JD, Hartemink AJ, Prins JF. SLICER: Inferring branched, nonlinear cellular trajectories from single cell RNA-seq data. Genome Biol. 2016;17 (1 ):106.27215581
150. Setty M, Tadmor MD, Reich-Zeliger S, Angel O, Salame TM, Kathail P, Choi K, Bendall S, Friedman N, Pe'er D. Wishbone identifies bifurcating developmental trajectories from single-cell data. Nat Biotechnol. 2016;34 (6 ):637–645.27136076
151. Bendall SC, Davis KL, Amir E-AD, Tadmor MD, Simonds EF, Chen TJ, Shenfeld DK, Nolan GP, Pe'er D. Single-cell trajectory detection uncovers progression and regulatory coordination in human B cell development. Cell. 2014;157 (3 ):714–725.24766814
152. Marco E, Karp RL, Guo G, Robson P, Hart AH, Trippa L, Yuan GC. Bifurcation analysis of single-cell gene expression data reveals epigenetic landscape. Proc Natl Acad Sci USA. 2014;111 (52 ):E5643–E5650.25512504
153. Xue Z, Huang K, Cai C, Cai L, Jiang CY, Feng Y, Liu Z, Zeng Q, Cheng L, Sun YE, et al. Genetic programs in human and mouse early embryos revealed by single-cell RNA sequencing. Nature. 2013;500 (7464 ):593–597.23892778
154. Wang J, Song Y. Single cell sequencing: A distinct new field. Clin Transl Med. 2017;6 (1 ):10.28220395
155. Björklund ÅK, Forkel M, Picelli S, Konya V, Theorell J, Friberg D, Sandberg R, Mjösberg J. The heterogeneity of human CD127+ innate lymphoid cells revealed by single-cell RNA sequencing. Nat Immunol. 2016;17 (4 ):451–460.26878113
156. Meredith M, Zemmour D, Mathis D, Benoist C. Aire controls gene expression in the thymic epithelium with ordered stochasticity. Nat Immunol. 2015;16 (9 ):942–949.26237550
157. Poulin J-F, Gaertner Z, Moreno-Ramos OA, Awatramani R. Classification of midbrain dopamine neurons using single-cell gene expression profiling approaches. Trends Neurosci. 2020;43 (3 ):155–169.32101709
158. Brunskill EW, Park J-S, Chung E, Chen F, Magella B, Potter SS. Single cell dissection of early kidney development: Multilineage priming. Development. 2014;141 (15 ):3093–3101.25053437
159. Li H, Courtois ET, Sengupta D, Tan Y, Chen KH, Goh JJL, Kong SL, Chua C, Hon LK, Tan WS, et al. Reference component analysis of single-cell transcriptomes elucidates cellular heterogeneity in human colorectal tumors. Nat Genet. 2017;49 (5 ):708–718.28319088
160. Peng J, Sun B-F, Chen C-Y, Zhou J-Y, Chen Y-S, Chen H, Liu L, Huang D, Jiang J, Cui G-S, et al. Single-cell RNA-seq highlights intra-tumoral heterogeneity and malignant progression in pancreatic ductal adenocarcinoma. Cell Res. 2019;29 (9 ):725–738.31273297
161. Song D, Li JJ. PseudotimeDE: Inference of differential gene expression along cell pseudotime with well-calibrated p-values from single-cell RNA sequencing data. Genome Biol. 2021;22 (1 ):124.33926517
162. Lv H, Zhang Z-M, Li S-H, Tan J-X, Chen W, Lin H. Evaluation of different computational methods on 5-methylcytosine sites identification. Brief Bioinform. 2020;21 (3 ):982–995.31157855
163. Tahir M, Tayara H, Chong KT. iRNA-PseKNC(2methyl): Identify RNA 2 '-O-methylation sites by convolution neural network and Chou's pseudo components. J Theor Biol. 2019;465 :1-6.30590059
164. Min S, Lee B, Yoon S. Deep learning in bioinformatics. Brief Bioinform. 2017;18 (5 ):851–869.27473064
