
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39227685
67626
10.1038/s41598-024-67626-1
Article
Short-term air quality prediction based on EMD-transformer-BiLSTM
Dong Jie 12
Zhang Yaoli 2
Hu Jiang 20150007@huat.edu.cn

3
1 https://ror.org/013q1eq08 grid.8547.e 0000 0001 0125 2443 Fudan University, Shanghai, 200433 China
2 https://ror.org/04yqxxq63 grid.443621.6 0000 0000 9429 2040 Zhongnan University of Economics and Law, Wuhan, 430073 China
3 https://ror.org/039m95m06 grid.443568.8 0000 0004 1799 0602 School of Economics and Management, Hubei University of Automotive Technology, Shiyan, 442002 China
3 9 2024
3 9 2024
2024
14 2051322 2 2024
15 7 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Actual acquired air quality time series data are highly volatile and nonstationary, and accurately predicting nonlinear time series data containing complex noise is an ongoing challenge. This paper proposes an air quality prediction method based on empirical mode decomposition (EMD), a transformer and a bidirectional long short-term memory neural network (BiLSTM), which is good at addressing the ultrashort-term prediction of nonlinear time-series data and shows good performance for application to the air quality dataset of Patna, India (6:00 am on October 3, 2015–0:00 pm on July 1, 2020). The AQI sequence is first decomposed into intrinsic mode functions (IMFs) via EMD and subsequently predicted separately via the improved transformer algorithm based on BiLSTM, where linear prediction is performed for IMFs with simple trends. Finally, the predicted values of each IMF are integrated using BiLSTM to obtain the predicted AQI values. This paper predicts the AQI in Patna with a time window of 5 h, and the RMSE, MAE and MAPE are as low as 5.6853, 2.8230 and 2.23%, respectively. Moreover, the scalability of the proposed model is validated on air quality datasets from several other cities, and the results prove that the proposed hybrid model has high performance and broad application prospects in real-time air quality prediction.

Keywords

Hourly forecast
Air quality index
Transformer
BiLSTM
EMD
Subject terms

Environmental sciences
Engineering
Mathematics and computing
Doctoral Research FundBK202010 Hu Jiang issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

In recent years, global air quality problems have become increasingly serious. According to the World Meteorological Organization, if greenhouse gas emissions remain high, ground-level ozone levels are expected to rise by the second half of the twenty-first century in heavily polluted regions, especially in Asia, including Pakistan, northern India and Bangladesh1, where they have already increased by 20%. On Nov. 3, the air quality in New Delhi, India, reached its worst since the beginning of winter, seriously affecting economic activity in India. However, real-time monitoring of existing monitoring systems in most countries has a certain lag, and air pollution conditions cannot be predicted in advance, allowing reliable early warning information to be provided to the government. Therefore, how to predict air quality well to provide suggestions for social development has become an urgent problem.

The air quality index (AQI) is a composite index that converts the levels of various pollutants into a quantitative description of air quality levels. However, real-life measurements of AQIs usually involve complex noise with typical nonlinear characteristics. To retain the original physical characteristics of the data, this paper uses the EMD method to decompose the input AQI series, and the resulting IMF components can represent different local characteristics of the original data2. If the decomposed IMFs are separately predicted in parallel, the time needed to identify the features of the signal is reduced, which greatly improves the efficiency and accuracy of algorithm training. For the predicted component data, most of the previous studies used the direct summation method for integration3. In this paper, to improve the accuracy of the model as much as possible and considering that there is still some correlation between the decomposed IMFs, the BiLSTM model is integrated to obtain the final AQI prediction results. Overall, the advantage of this approach lies in utilizing empirical mode decomposition (EMD) to decompose time series, enabling each component to be separately predicted by predictive models, thereby better capturing variations in different frequencies within the sequence. Subsequently, a neural network algorithm is trained based on the initial decomposition components to integrate the prediction results of each component into an overall prediction result, effectively enhancing the accuracy and robustness of the predictions.

For the prediction model, one of the disadvantages of the transformer architecture is the limited size of the context and prediction window due to the traditional squared operations and memory requirements, and some experiments have shown this architecture to be ineffective for long-term prediction4. Since its calculation method may cause spurious correlations in the data at different times, obviously more input data increase the probability of this phenomenon. Therefore, this paper aims to investigate whether the transformer architecture can be useful for short-term prediction. On the other hand, the pointwise dot product self-attention in the transformer is not sensitive to local information, which makes the model prone to abnormalities in the prediction, while BiLSTM is good at capturing bidirectional time series information dependencies; thus, the transformer-BiLSTM model that combines the two methods can balance the global and local sequence information well. In addition, when the transformer is used for time series data prediction, the self-attention mechanism of the decoder accumulates errors layer by layer. Chaos can easily be generated, and the prediction effect in many cases is even worse than that of replacing the decoder part with a fully connected layer5. Therefore, it is appropriate to apply the BiLSTM network to improve the performance of the decoder. Furthermore, considering that the decomposed components after EMD exhibit different frequencies and trends, not all component sequences are suitable for prediction via complex algorithms. Therefore, the prediction result for each component is determined as the one with the minimum root mean square error (RMSE) among Transformer-BiLSTM and simple linear regression.

The contributions of this paper are as follows. (1) Expand the application methods when signal decomposition methods such as EMD are used for time series prediction: For the prediction of IMFs, two prediction methods are used in a selective manner considering the different characteristics of the data; in the signal reconstruction process, BiLSTM is used to predict the final results considering the possible correlation among the original data and all IMFs. (2) Investigate whether the improved transformer architecture using the BiLSTM model can perform well for short-term prediction: the transformer architecture does not perform as well as expected in long-term time series prediction tasks, and this paper further investigates whether this architecture is good for short-term prediction tasks by improving the decoder part to enhance the local information perception capability of the model. (3) Enrich the library of methods for air quality prediction in India: This paper presents the first example of an EMD-transformer-BiLSTM model and demonstrates through experiments that the accuracy of this model is improved compared with that of other models with similar structures used for air quality prediction.

The paper is structured as follows. Section “Literature review” reviews the literature on research methods for air quality prediction. Section “Models” presents the framework of the model proposed in this paper and some of its computational processes. Section “Experimental results and analysis” presents the preparation before model training, the results of model training and a detailed explanation and analysis of the results. In Section “Extension analysis”, the prediction results of the proposed model for several different cities are given to verify the scalability of the model. Section “Conclusions” summarizes the results of this paper and the limitations of the model and proposes future research directions.

Literature review

Statistical analysis

The commonly used methods for predicting air quality can be divided into two main categories: statistical measures and machine learning algorithms. Statistical methods make predictions by applying regression models based on mathematical statistics, such as autoregressive integrated moving average (ARIMA) models and generalized autoregressive conditional heteroskedasticity (GARCH) models. Polydorasa et al.6 compared the effectiveness of air quality prediction using partial differential equations and univariate Box‒Jenkins models. Alsoltany et al.7 suggested that residual errors are sometimes caused by the uncertainty or inaccuracy of the model structure; therefore, the fuzzy linear regression parameter estimation method was used to predict the concentrations of urban pollutants. However, these methods can only describe the trend state of time series in the field of linear regression and rely heavily on the assumption of a normal distribution of data, making it difficult to more accurately fit real data in real life. In the field of measurement, in recent years, several scholars have proposed using the generalized Pareto distribution (DCP) model to fit the time dependence of air pollutant concentrations and provide a better fit for the tail correlation of the data8.

Machine learning

With the development of big data technology, since machine learning and deep learning methods based on nonparametric statistics can fit complex multiple interactive relationships and nonlinear relationships, an increasing number of scholars have applied these algorithms to prediction tasks.

In the realm of nonparametric machine learning algorithms, Donnelly et al.9 employed a nonparametric kernel regression approach, integrating temporal variations in pollutant concentrations, historical correlations with meteorological factors, and seasonal and diurnal periodicity factors to achieve real-time forecasting of air quality for the next 48 h with low resource requirements and high accuracy. Castelli et al.10 utilized support vector regression (SVR) to predict hourly pollutant concentrations and the air quality index (AQI), experimentally indicating that the radial basis function (RBF) is the kernel type allowing SVR to achieve the most accurate predictions. Mengash et al.11 also developed an automated atmospheric particulate matter concentration prediction tool based on various machine learning algorithms, employing the chi-square feature selection method for feature screening after computing multimodal features of particulate matter concentration in dynamic environments. Zhan et al.12, taking representative machine learning algorithms as an example, introduced a novel framework based on feature selection, error correction models, and a novel kernel acceleration method for residual estimation that is capable of handling large-scale data and significantly reducing the prediction interval.

In the domain of deep learning algorithms, Neagu et al.13 combined fuzzy inference and neural networks for the prediction of air quality. Corani14 used feedforward neural network (FFNN), pruning neural network (PNN), and lazy learning (LL) methods to predict ozone concentrations in Milan and found that LL, as a local linear prediction algorithm, can eliminate the overfitting problem, update faster and is more interpretable. Kim et al.15 devised a data-driven method for predicting the indoor air quality in subway stations by leveraging recurrent neural networks (RNNs). They utilized the partial least squares (PLS) method during preprocessing to establish a linear relationship model between the input and output variables, thereby facilitating the selection of key input variables to optimize the prediction model. The experimental results showed that the prediction results of RNN models have good performance and high interpretability. Mellit16 used least squares support vector machines (LS-SVMs) for short-term forecasting of meteorological time series and verified that LS-SVMs produced significantly better results than artificial neural networks (ANNs). Singh et al.17 used partial least squares regression (PLSR), multiple polynomial regression (MPR) and ANN models to predict the levels of atmospheric air pollutants such as SO and ultimately found that the performance of the nonlinear model was relatively better than that of the linear model. Li et al.18 worked on building deep networks and considered spatiotemporal correlation to propose an air quality prediction method based on spatiotemporal deep learning (STDL) and stacked autoencoder (SAE) models, and the results demonstrated that the model could predict air quality at multiple stations simultaneously. Yi et al.19 proposed Deepair, a deep neural network (DNN)-based method consisting of a spatial transformation component and a deep distributed fusion network, and experimental results based on data from Chinese cities showed that Deepair outperformed 10 classical prediction methods. In summary, the development of machine learning methods in the field of air quality prediction has evolved from traditional statistical approaches to deep learning. These methods offer more powerful and accurate tools for addressing air quality prediction issues. On the one hand, they have improved the selection and extraction of features related to air quality. On the other hand, they employ model fusion and ensemble methods to combine results from multiple models.

Improvement of the LSTM algorithm

Although nonlinear machine learning methods achieve strong generalizability in predicting air quality, these methods have difficulty capturing the effects of long lags between series, thus limiting the prediction accuracy. As a classic time series forecasting model, LSTM models offer advantages in short-term air quality prediction by effectively capturing both long-term and short-term dependencies within time series data and possessing stronger memory and sequence modeling capabilities. Compared to LSTM models, BiLSTM models can further enhance predictive performance by better leveraging contextual information. To make full use of more historical data in time series, many scholars have started to analyze and improve the structure of LSTMs in recent years. Li et al.20 argued that the existing methods for predicting air pollutant concentrations at that time could not effectively model long-term dependence, and most of them ignored spatial correlation; thus, a method that merges meteorological data and time-stamped data was proposed. Therefore, they proposed a new long- and short-term memory neural network extension (LSTME) model that simultaneously considers spatiotemporal correlations, and multiscale predictions were performed for different periods. The results prove that the model can achieve satisfactory performance. Wen et al.21 proposed a spatiotemporal convolutional long short-term memory neural network extension (C-LSTME) model for predicting air quality concentrations, incorporating historical air pollutant concentrations at each site as well as adaptive k-nearest neighboring sites to improve the model prediction performance. Ma22 worked on solving the problem of data shortages in air quality prediction tasks and proposed a stacked bidirectional long short-term memory (TLS-BLSTM) network based on migration learning for predicting air quality at new stations lacking data. Li et al.23 applied a model based on a one-dimensional convolutional neural network (CNN), LSTM, and attention mechanism for urban PM2.5 concentration prediction; additionally, they added meteorological data and data from neighboring air quality monitoring stations to their input data while using an attention mechanism to capture the importance level of the influence of different temporal feature states in the past on future PM2.5 concentrations to improve the prediction accuracy. Zhang et al.24 proposed a semisupervised model based on empirical modal decomposition (EMD) and a bidirectional long short-term memory (BiLSTM) network to improve short-term trend prediction, especially for the identification of unexpected situations.

Decomposition ensemble methods

Air quality data typically exhibit complex nonlinearities, comprising multiple components with different frequency characteristics, making accurate trend prediction, especially long-term trend prediction, challenging. To address this issue, numerous scholars have sought to improve prediction accuracy and reliability from the perspective of information decomposition ensemble methods. After decomposing PM2.5 data into multiple components via empirical mode decomposition (EMD), Jin et al.25 employed convolutional neural networks (CNNs) to group all the components based on their frequency characteristics. Finally, they applied gated recurrent units (GRUs) to predict each group and fused the results to obtain the outcome. Song and Fu26 integrated three single prediction models, the radial basis function neural network (RBFNN), the RBFNN algorithm based on ensemble empirical mode decomposition (EEMD), and the ARIMA algorithm based on EEMD-RBFNN, into a composite forecasting model (CFM). Through weight allocation, they achieved decomposition integration, providing a novel ensemble method for AQI prediction. Wang et al.27 proposed a novel multiscale hybrid learning framework based on robust local mean decomposition and a moving window integration strategy for particle concentration prediction. This framework can capture linear and nonlinear patterns and improve prediction accuracy and generalizability through ensemble methods. Subsequently, Wang et al.28 introduced a new forecasting method capable of effectively capturing trends and fluctuations in AQI. Specifically, by constructing ternary interval value sequences of AQI data and performing multiscale decomposition using multivariate variational mode decomposition (MVMD), they conducted separate predictions followed by simple addition for integration. Cai et al.29 proposed a novel decomposition-ensemble-reconstruction prediction framework that utilizes entropy to compute and study decomposed subcomponents. They employed different prediction tools (ARIMA, CNN, and TCN) to capture different time-scale patterns of reconstructed subtime series data, demonstrating significant superiority over classical deep learning algorithms. It is evident that utilizing decomposition ensemble methods based on EMD for air quality forecasting can better handle complex air quality data and capture multiple components with different frequency characteristics, thereby improving the prediction accuracy and reliability. Moreover, this approach can adapt to different scale prediction requirements, better meeting the practical application scenarios of air quality forecasting.

Among the above four categories of methods, whether based on traditional neural network models or statistical econometric models, many have more elaborate parameter settings, longer training cycles, and complex model hierarchies, which make it difficult to verify the generalization performance of the models. In the application of decomposition ensemble methods, while previous research has considered using nonsingle prediction methods for forecasting components, there has been a slight oversight in the fusion of prediction results by employing only simple ensemble methods. Moreover, although deep learning methods can obtain good results in air quality prediction, the implementation of these methods requires sufficient historical datasets, and the amount of data significantly limits the model performance. Some research methods fail to effectively extract the spatiotemporal characteristics of air pollutant concentration data or gauge the impact of different temporal characteristics on future air quality, and most of them fail to effectively model the spatiotemporal dependence of air quality indices at the same time, showing low accuracy in long-term predictions and unexpected situations. Therefore, considering the shortcomings of existing research, this paper integrates BiLSTM and Transformer architectures to better address the potential issues of information forgetting and gradient vanishing in BiLSTM when dealing with long sequences, as well as the inadequate modeling of long-term dependency relationships in Transformers. Additionally, by utilizing EMD for sequence decomposition followed by ensemble neural network algorithms, the predictive capability of the model is further enhanced.

Models

Model framework

In the model of this paper, the input data are first decomposed using EMD to obtain intrinsic mode functions30. To accurately predict these component sequences, the data are first processed by positional encoding, and then a vector containing information about the data features is output to the decoder after a multihead self-attention mechanism and a feedforward neural network based on a residual network structure. Then, BiLSTM outputs the predicted sequence after learning the relationship31. Finally, a fully connected layer is used to interpret each time step in the output sequence and output the prediction probability.

However, considering that the final decomposed IMF usually has a very simple linear trend, using a neural network with complex parameters for prediction would not only have no beneficial effect on the enhancement of prediction results but also increase the overall complexity of the algorithm. It was found experimentally that the final results after predicting all IMFs using only neural networks were worse than the final results after using the model proposed in this paper. Therefore, the model in this study compares the transformer-BiLSTM model and simple linear regression, and the model with the smallest RMSE is selected as the prediction result of the IMF sequence from the two prediction results. In this module, after the components obtained through EMD are initially predicted using the Transformer-BiLSTM algorithm, simple linear regression is applied for prediction. Subsequently, the prediction value with the smaller RMSE is selected as the predicted result for that component. The introduction of this selective structure helps ensure the optimal prediction performance of the model by choosing the most suitable prediction method in different scenarios.

Moreover, based on the training set data decomposed into IMFs, a training set is constructed with N IMFs as N input features and the corresponding real AQI values as labels, training a BiLSTM model. At this point, the total sample size of this training set is 23,174. Finally, using this model to reconstruct the prediction results of the IMF sequence, i.e., using the predicted results sequence of N components as input, the final AQI prediction sequence is obtained using the trained BiLSTM model, where both the input and output sequence sample sizes are 772432. The model flow used in this paper is shown in Fig. 1.Figure 1 Flow of the EMD-transformer-BiLSTM hybrid model.

EMD

In the preprocessing method of time series data, the EMD algorithm can reflect the original physical characteristics of the system more accurately than the wavelet algorithm, which seems more effective in dealing with nonlinear and nonstationary signals33. The steps of the method are as follows: for signal x(t), all the extreme value points and all the minimal value points are fitted with two threefold spline curves, and the average of the upper and lower 2 extreme value envelopes is denoted by m(t). Let ht=xt-mt. If h(t) satisfies the condition (Eq. 1), then h(t) is the first IMF; otherwise, h(t) is considered x(t). h1,k(t) is the difference between the obtained signal and the mean value of the envelope after k repetitions.1 ∑t=0Th1,k-1(t)-h1,k(t)2h1,k-1(t)21/2<0.1

Suppose h1,k(t) is the first IMF; let x(t)=x(t)-h1,k(t), and repeat the above steps. When the residual is a monotonic function or the amplitude is less than a predetermined value, several components Ci(t) can be obtained, and the residual is r(t).

Moreover, compared with other advanced signal decomposition algorithms, EMD excels in balancing performance stability with relatively low computational complexity. Methods such as EEMD and CEEMD are essential improvements to the EMD method. The computational complexity of these improved methods is usually greater due to the introduction of additional steps and iterative processes aimed at enhancing decomposition performance. Moreover, given its widespread recognition and ample research support, we ultimately opt to use EMD as the decomposition method to ensure reliable and stable results.

Position encoding

The transformer model discards the traditional RNN and CNN models. To avoid losing the sequential information of the time series data, this paper uses positional encoding (hereafter referred to as PE) to sum the position information of the data one by one into the input vector of the model so that the self-attention mechanism can determine the absolute and relative position information of each data point in the overall sequence. The data at the even-numbered position are transformed by the sine function, and the data at the odd-numbered position are transformed by the cosine function so that the variable values can be scaled between [0,1], thus avoiding the interference of data orders. 100002i/dmodel-fold reduction of the sequential values first can effectively avoid situations in which the data positions are different but the PE values are the same34. The calculation method of the position encoding is shown in Eqs. (2)–(3):2 PE(pos,2i)=sin(pos/100002i/dmodel)

3 PE(pos,2i+1)=cos(pos/100002i/dmodel)

Multihead self-attention

Compared with the recurrent neural network structure, the self-attention mechanism can compute the input data in parallel in less time and space complexity, and it also enhances the interpretability of the whole model because the correlation between the data can be visualized. The self-attention mechanism plays a role in allowing the model to observe the correlations between different data throughout the input, thus discovering and solving the problem of lagged intercorrelations of time series data.

Assuming that Q0 represents the information of certain data and K0 represents the information of the rest of the input data, Q0 and K0 are pointwise dot products used to obtain a weight matrix containing the correlation information between the two vectors, which can be multiplied by the original data to obtain a weighted summed output. To enhance the fitting ability of the model, three trainable parameter matrices, Q, K, and V, are used; these matrices are obtained by linear transformation of the input matrix X with different parameter matrices. The self-attention mechanism is defined in Eq. (4):4 AttentionQ,K,V=softmaxQKTdkV

Compared to single self-attention, multihead self-attention promotes the advantages of integrated learning. The input matrices Q, K, and V are linearly transformed so that each attention mechanism function is responsible for only one subspace of the final output sequence and that the results are independent of each other, which fully utilizes the original information of the data and effectively reduces the risk of overfitting.

Residual network

The main problems encountered by deep learning for network depth are gradient disappearance and gradient explosion. During the training process, each layer extracts features from the previous layer; thus, the network degenerates as the number of layers increases. On the other hand, residual networks take a jump connection approach to avoid these problems. Usually, a residual block consists of a direct mapping part and a residual part, which enables a connection between the input and output so that the newly added layer needs to learn only new features based on the original input layer, i.e., learning the residuals, thus avoiding the phenomenon that the error in the training set increases as the network deepens. The general manifestation of the residual block is shown in Eq. (5). F is a function of the residuals, xl is the input to the l th layer, and Wl is the parameter corresponding to the l th layer.5 xl+1=xl+F(xl,Wl)

BiLSTM

To solve the problem of gradient disappearance caused by ordinary recurrent neural networks when the input sequence is too long, the BiLSTM algorithm is jointly determined by 2 LSTM models in opposite directions; these models use bidirectional results to enhance access to information, forget nonessential information, and retain critical information. The LSTM network is composed of input data at time t Xt, cell state Ct, temporary cell state Ct~, hidden layer state ht, forgetting gate ft, memory gate it, and output gate ot. By dynamically memorizing and forgetting information, a network transmits effective information and discards invalid information to solve the problem that RNNs cannot establish long-term associations35. Among them, forgetting, memory and output are controlled by the forget gate, memory gate and output gate, respectively, which are calculated from the hidden layer state at the last moment ht-1 and the current input information Xt. The specific computational flowchart of BiLSTM is shown in Fig. 2.Figure 2 Flow of the BiLSTM algorithm.

During the training process, the model also adds a masking mechanism to the decoder. When using the transformer construct for training, the model compiles and converts the entire output results into feature vectors and inputs them, but the output of the decoder is expected to be obtained using the previous results to avoid future information from being used in advance. Thus, this paper adds the mask tensor to the BiLSTM input.

Experimental results and analysis

Datasets and configurations

To verify the effectiveness of the proposed method, the AQI dataset of Patna city in India (6:00 am on October 3, 2015–0:00 pm on July 1, 2020) is used for prediction with a time interval of 1 h. The data were obtained from the Central Pollution Control Board, the official website of the Government of India (https://airquality.cpcb.gov.in/AQI_India/). In this paper, the air quality index at T + 1 was predicted based on the air quality index of the previous T hour. This method involves single-step and single-variable prediction. The structure of the data is shown in Fig. 3.Figure 3 The structure of the AQI data.

Since the missing values of the original data are not large compared to the total amount of data and the missing values are continuous for a long time, they are removed from this paper directly. The final dataset used in the experiments consisted of 30,898 samples. After that, this paper uses the sliding window method to construct time series samples, which serve to divide the input series into a training set and its labels by delaying Δt time units. Since it is a single-step prediction, if the input is 1–10, its labels are 2–11 to fit the transformer's seq2seq output form. Seventy-five percent of the data are included in the training sets; i.e., the training set contains 23,174 samples, and the test set contains 7724 samples. Let the sample size of the training set be n. Then, the total number of time series samples to be constructed is n-Δt+1. Usually, the value of Δt affects the number of time series samples and the number of features in each sample, thus affecting the performance of the model; therefore, the choice of the value of Δt is important22. In this paper, to verify the application of the model to ultrashort-term air quality index prediction, experiments are conducted for Δt in the range of values from 1 to 5.

The experiments in this paper were conducted in Python (version 3.8) using CUDA 11.3 and the deep learning development framework PyTorch (version 1.11.0) to construct the network model. All the experiments were conducted on a remote PC equipped with a 15-core Intel processor and 80 GB of RAM; the specific environment configuration is shown in Table 1.Table 1 Experimental environment configuration.

Experimental environment	Specific settings	
GPU	RTX A5000 * 1/Memory:24GB	
CPU	15-core Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60 GHz/RAM:80GB	
Default hard disk	System disk:20 GB/Data disk:50GB	
Additional hard disk	None	
Port mapping	None	
Network	Upstream broadband:10MB/s/Downstream broadband:10MB/s	

Parameter setting

When preprocessing the data using EMD, the AQI sequence was decomposed into 12 IMFs. This indicates that the air quality data used in this study are complex and exhibit nonlinear and oscillatory behaviors, requiring 12 IMFs to effectively capture the structure of the data. The components obtained after EMD of the AQI data in this article are shown in Fig. 4.Figure 4 The IMFs obtained after EMD.

To train the transformer-BiLSTM model for IMF component prediction, this paper sets adaptive moment estimation with decoupled weight decay (AdamW) as the optimizer. The AdamW optimizer decouples the weight decay and the learning rate based on the Adam optimizer, which has better generalization performance and a wider range of optimal hyperparameters than the latter. The specific settings are shown in Table 2. For the BiLSTM model used for component sequence reconstruction, 20 hidden units are used in the experiments.Table 2 Main parameters of the model.

Parameter item	Parameter value	
Global	Window size	{1, 2, 3, 4, 5}	
Allocation ratio of the training set	1%	
Epoch	100	
Batch size	64	
Learning rate	0.001	
Decay rate	0.95	
Encoder	Embedding Size	250	
Number of encoder layers	1	
Dropout rate	0.1	
Head of self-attention	10	
Decoder	Number of decoder layers	2	
Hidden size of BiLSTM	2	

Evaluation metrics

After constructing the time series samples and initializing the model parameters, this paper applies the EMD-transformer-BiLSTM network to model the training set data for predicting the AQI in the next hour. Since the performances of most algorithms on training data are biased and often overfit, the results of this study are based on the test set. The root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) are used to evaluate the predictive performance of the hybrid model. The specific definitions are shown in Eqs. (6)–(8):6 RMSE=1n∑i=1n(yi-yi∗)2

7 MAE=1n∑i=1nyi-yi∗

8 MAPE=1n∑i=1nyi-yi∗yi

Results

In this paper, the training process for the prediction of the components is recorded, and a chart of the training loss and test loss is obtained. Some training processes for the data are shown in Fig. 5, where the images are logarithmically transformed on the y-axis scale so that the difference between the two types of errors can be more clearly observed. For the third IMF, the training loss is almost consistently smaller than the test loss, which indicates that the model has converged and that no overfitting has occurred. For the later IMFs, the test loss is lower than the training loss in the later stages of training. The reason is likely that the original data were leaked when EMD was used, and the model was trained with a known test set; thus, the prediction effect on the test set is naturally better than that on the training set. By analyzing the ninth function, it is found that the model converges gradually more slowly, which is mainly because the earlier the IMF component is obtained, the simpler the trend is, and it is difficult to obtain better accuracy for prediction using a model with complex parameters instead.Figure 5 Model training process—loss for IMFs.

In addition, in the hybrid model proposed in this paper, as it involves decision-making steps for result assessment and model selection, the overall parameter complexity lies between n(IMFs)∗O(linear)+O(bilstm) and n(IMFs)∗O(transformer)+O(bilstm). This paper estimates the computational complexity based on the number of parameters and finds that the overall parameter scale of the model ranges from 0.02 to 127.01 MB. The time complexity of neural networks is typically challenging to precisely compute using traditional analytical methods. For convolutional neural networks, one can consider the computational load of convolution and pooling operations and estimate their time complexity based on the network's hierarchical structure. However, for more complex neural network architectures such as recurrent neural networks or deep residual networks, their time complexity often requires assessment through experimentation or simulation. In our study, the training runtime on a computer configured with the specified parameters is approximately between 15 and 45 min, which is shorter than the prediction interval (1 h). This demonstrates that the model's training runtime does not exceed the data sampling interval, maintaining the practicality of the model.

To analyze the characteristics of the hybrid model proposed in this paper from three perspectives—the data preprocessing, decoder and encoder models—the comparison models chosen in this paper are ARIMA, RNN, LSTM, BiLSTM, and BiLSTM based on the wavelet transform36; the transformer with the encoder as a linear layer37; and the transformer with the encoder as an LSTM38 and BiLSTM39. The latter two are essentially improved models based on RNNs with self-attention, which is the current research direction of many scholars. In the comparison model, the most classical time series model ARIMA (p,d,q) is first added, which includes an autoregressive part of order p and a moving average part of order q, and a d-order difference is performed on the original sequence. The baseline transformer model is less effective when used for time series prediction tasks because its decoder accumulates errors; therefore, the comparison model in this paper removes the original decoder and leaves only one linear layer to output the final prediction results of the model, which is also more convenient for us to compare the performance of the target model from the perspective of control variables. The transformer-LSTM model turns the decoder part of the transformer model into an LSTM network with two hidden layers40.

The five models are used to predict the AQI series with time windows of 1, 2, 3, 4, and 5 h. The use of a time window ranging from 1 to 5 h is intended to meet the demand for ultrashort-term forecasting while considering data availability and forecasting objectives. For air quality data, we have sufficient historical data for modeling, and the monitoring frequency is relatively high, making shorter time windows more suitable. The final prediction results and best case for each model are obtained, as shown in Tables 3 and 4.Table 3 Performance of each model in the experiment.

Window size	1 h	2 h	3 h	
Model	RMSE	MAE	MAPE (%)	RMSE	MAE	MAPE (%)	RMSE	MAE	MAPE (%)	
ARIMA	7.6254	3.1340	2.26	7.5747	3.0862	2.25	7.5508	3.0855	2.26	
RNN	11.0375	8.2253	5.43	10.3749	7.4516	5.02	11.2471	8.4678	5.56	
LSTM	12.0059	8.6868	7.37	8.5041	4.4886	4.05	8.5306	4.6756	4.07	
BiLSTM	8.1379	4.0548	3.48	8.5041	4.4886	4.05	11.7364	8.0747	7.63	
Wavelet-BiLSTM	6.4211	3.1996	2.64	8.5428	5.6258	4.76	10.1685	7.0923	6.45	
Transformer (-Linear)	15.2598	13.142	10.12	15.4502	13.408	10.45	15.3298	13.333	10.11	
Transformer-LSTM	53.3007	45.0149	34.94	8.7666	5.7037	3.76	7.6878	3.7221	2.68	
Transformer-BiLSTM	8.0003	3.9352	3.09	7.5647	3.3338	2.70	8.0002	3.9308	3.31	
EMD-Transformer-BiLSTM	7.2421	3.6044	2.60	5.9586	3.0370	2.35	5.7704	2.8755	2.26	
Window size	4 h	5 h	
Model	RMSE	MAE	MAPE (%)	RMSE	MAE	MAPE (%)	
ARIMA	7.5353	3.0873	2.27	7.5317	3.0904	2.28	
RNN	10.7178	7.8524	5.22	10.7366	7.8739	5.24	
LSTM	8.553	4.6981	4.11	8.5812	4.7445	4.14	
BiLSTM	12.932	9.4522	8.73	13.791	10.4999	9.41	
Wavelet-BiLSTM	11.0512	8.0952	7.24	11.7920	9.0281	7.82	
Transformer (-Linear)	14.58	12.3491	9.56	14.3897	12.1258	9.30	
Transformer-LSTM	8.1011	4.1805	2.95	7.8993	4.1013	2.95	
Transformer-BiLSTM	7.9834	3.7744	3.14	8.6614	5.0771	4.22	
EMD-Transformer-BiLSTM	5.6877	2.8316	2.24	5.6853	2.8230	2.23	

Table 4 Comparison of the optimal performances of the models.

Model	RMSE	MAE	MAPE (%)	Window size (h)	
ARIMA	7.5317	3.0904	2.28	5	
RNN	10.7178	7.8524	5.22	4	
LSTM	8.5041	4.4886	4.05	2	
BiLSTM	8.1379	4.0548	3.48	1	
Wavelet-BiLSTM	6.4211	3.1996	2.64	1	
Transformer (-Linear)	14.3897	12.1258	9.30	5	
Transformer-LSTM	7.6878	3.7221	2.68	3	
Transformer-BiLSTM	7.5647	3.3338	2.70	2	
EMD-Transformer-BiLSTM	5.6853	2.8230	2.23	5	

A comparison of the results reveals that the EMD-transformer-BiLSTM model yields the best results in all the ultrashort-term series prediction experiments. In the experimental phase of this study, we conducted a minimum of three repeated experiments for each experimental set, and experimental validation was carried out across five different short-term time windows. Overall, our proposed model outperformed the other comparative models. The consistency and stability of the experimental results suggest that the experiments were not influenced by randomness. According to Table 4, the EMD-transformer-BiLSTM has the lowest values for all three evaluation metrics, which indicates that this hybrid model has significant optimal properties. Among the best results of each model, the RMSE, MAE and MAPE of the EMD-transformer-BiLSTM model were 11.46%, 11.77% and 15.53% lower, respectively, than those of the second-ranked Wavelet-BiLSTM model. The traditional ARIMA model is well suited for time series data with seasonal trends. When applied to static forecasting of AQI data, it performed remarkably well, second only to the hybrid model proposed in this paper. In contrast, although RNNs can capture long-term dependencies in sequences, they did not realize their full potential in our experiments, resulting in performance inferior to that of ARIMA. The Naive (Persistence) model was also employed for comparative experiments. This model is particularly effective when the time series exhibits strong autocorrelation, making it highly suitable for air quality data prediction. The experimental results indicate that the Naive (Persistence) model achieved a prediction RMSE of approximately 7.5943, comparable to the performance of the ARIMA model. However, it still falls short of the hybrid model proposed in this paper. Figure 6 shows the best prediction results for all models across different time windows, except for the Naive (Persistence) model, which does not use a sliding time window and serves solely as a benchmark comparison model.Figure 6 Comparison of the final prediction results of each model.

Furthermore, by comparing the Transformer-BiLSTM and EMD-Transformer-BiLSTM models, it can be seen that the proposed improved EMD decomposition ensemble method based on a selective branching structure and neural network integration does indeed enhance the prediction accuracy. To validate the reliability of the results, these two models were tested in ten independent repeated experiments that were conducted randomly, and the Mann‒Whitney U test was used to compare the differences in the experimental outcomes. Nonparametric testing methods were utilized here due to the difficulty in determining the overall distribution of experimental result data, coupled with a relatively small sample size. The computed Mann‒Whitney U statistic is approximately 98, with a two-tailed test p value of approximately 0.00033, which is significantly less than 0.05. This indicates the rejection of the null hypothesis that the two sets of data are drawn from the same distribution. Hence, from a statistical testing perspective, the novel decomposition ensemble method proposed in this paper contributes to improving the prediction accuracy.

To demonstrate the reliability of the experimental results, this paper conducted a robustness test by varying the training set and test set data partition ratios. The results obtained are shown in Fig. 7. The numerical fluctuations of the three metrics are relatively minor, indicating that the experimental results are quite robust.Figure 7 The results of robustness testing.

Discussion

Data preprocessing

The wavelet-BiLSTM model has a poorer prediction ability for time series with large and frequent fluctuations, possibly because using the noise reduction processed data for training does not make the model too easy to overfit or fail to learn the law; rather, the model's prediction ability for mutated series decreases. The wavelet transform does not decompose the signal according to its characteristics but requires the prior selection of a suitable wavelet basis with the appropriate number of decomposition layers, and these subjective selection factors can affect the characteristics of the reconstructed sequence to some extent; however, EMD avoids this aspect, so this method has some advantages compared with the wavelet method41. Furthermore, for wavelet transformation, the choice of wavelet basis and other parameters can significantly impact the results for the input data42. However, given the substantial variations in air quality data across different regions, opting for EMD as a more objective preprocessing method can enhance the generalizability of the model.

Transformer architecture

The LSTM and BiLSTM models have better fitting effects than the Transformer (-Linear) model. BiLSTM has fewer fitting errors because it can take into account the information in both directions of the sequences. However, both models fit the tail sequences poorly, with some lag effects on the predicted values, and the overall values are larger than the true values, which is obviously the opposite of the results of the Transformer (-Linear) model. It is worth noting that the Transformer-LSTM model fits the tail sequences and mutated partial sequences better than the LSTM and BiLSTM models. The transformer is good at capturing long-term dependencies and has an outstanding ability to interact with information due to its ability to use global information; however, the self-attention mechanism using dot product multiplication has a disadvantage in extracting local information43, and it does not perform as well as LSTM in this regard. This leads to the fact that although the Transformer (-Linear) can predict the global sequence trend more accurately, the predicted values are not accurate, and there is a short-term lag effect.

Decoder section

The transformer-LSTM model does not fit the clustered effect sequences well enough and addresses the problems of the transformer (-Linear) model; however, there is a great improvement in this model, which confirms, to some extent, the effectiveness of introducing the LSTM model into the decoder part. For RNN-based LSTM and BiLSTM networks, the outputs of their neurons can act directly on themselves in the next period. This sequential traversal makes the LSTM network very good at extracting local information. However, this sequential model makes it difficult to control the mutated sequences well and leads to poor fitting of the sequences at the tail of the test set. The best-case transformer-BiLSTM model shows encouraging performance, combining the advantages of transformers and long short-term memory (LSTM) networks to better identify clustered effect sequences and mutated sequences with no significant lag in overall prediction.

The final hybrid model

In contrast to the prediction results of the previous models, the EMD-transformer-BiLSTM model does not generally have low or high prediction results compared with the true values; rather, it has a slightly stronger prediction for periods of frequent fluctuations, i.e., the prediction results may be higher when the true values increase suddenly, and vice versa. In this paper, the proposed EMD-transformer-BiLSTM hybrid model uses EMD to decompose the input sequences into multiple sequences, which are predicted separately; finally, the prediction results are integrated using the model with BiLSTM. The transformer-BiLSTM model used for prediction embeds a BiLSTM that can capture the bidirectional time series information dependence in the decoder to overcome the local agnosticism of the transformer, and the model also has a better learning effect on the global contours of the sequences; thus, the model can balance the global and local contextual information well.

Extension analysis

The generalization ability of a model is important. If a model has good generalization performance, it tends to perform well on data that have not been previously reported. Moreover, a neural network with good generalization ability helps to improve network interpretation and leads to a more regular and reliable model architecture design44. In contrast, the proposed model performs better on the dataset for all cities with different air quality levels. Therefore, this paper obtained the air quality of each city in India, and the exponential distribution diagram is shown in Fig. 8. Finally, four cities with very poor or very good air quality and four cities with moderate air quality are selected for prediction to determine whether the prediction model works well for cities with different air quality levels.Figure 8 Air quality distribution in India (Data from IQAir, Switzerland).

This paper chooses datasets with different data volumes for experiments to verify whether the data volume has a large impact on the performance of the model and to examine the robustness of the model. The hyperparameter settings of the model are consistent with those in Table 2. The final selected city datasets and experimental results are summarized in Table 5, where the cutoff time for all of these datasets is 2020-7-1 at 0:00:00.Table 5 Model generalizability validation.

Dataset	Start time	Sample size	Average AQI in the interval	RMSE	MAE	MAPE (%)	
Thiruvananthapuram	2017-6-23 7:00	24,734	76.11	3.6292	2.3494	3.48	
Bengaluru	2015-3-21 3:00	45,497	94.41	2.2717	1.4981	1.89	
Mumbai	2018-5-7 3:00	18,171	105.95	2.6632	2.2222	2.78	
Visakhapatnam	2016-7-7 2:00	26,766	117.73	3.9454	1.9448	2.42	
Amritsar	2017-2-28 4:00	25,840	120.53	6.0158	2.1767	1.99	
Brajrajnagar	2017-12-8 12:00	16,047	151.42	5.924	2.5102	1.94	
Gurugram	2016-1-23 11:00	33,420	223.88	3.5712	1.9198	1.67	
Delhi	2015-1-1 16:00	47,694	260.15	1.8013	0.9523	0.58	

The EMD-transformer-BiLSTM hybrid model proposed in this paper shows encouraging performance for the AQI prediction tasks in eight cities, including Thiruvananthapuram and Bengaluru, with the best application on the Delhi dataset and an RMSE, MAE and MAPE as low as 1.8013, 0.9523 and 0.58%, respectively. The model has the best prediction results for the cities of Delhi and Bengaluru, which also have the largest datasets, indicating that the amount of data still improves the performance of the model. The prediction results of the model on different datasets are shown in Fig. 9.Figure 9 Results of the generalization performance verification experiment.

Conclusions

Hybrid model

Accurate predictions of air quality can have an important impact on economic development, but the actual obtained time series data are often highly volatile, nonstationary and nonlinear. To predict the AQI more accurately, this paper proposes a hybrid model based on EMD, transformers and BiLSTM. The original data are first decomposed using EMD, after which the component sequences are positionally encoded separately. The vectors containing the data feature information are subsequently fed into the BiLSTM-based decoder via multihead self-attention and a residual network. A comparison is made with the prediction results obtained from a simple linear regression model, and the result with the smallest RMSE is chosen as the predicted value of each component. Finally, the BiLSTM model is used to reconstruct the predicted values of each component to obtain the final predicted AQI values.

Results analysis

According to the experimental results, the EMD-transformer-BiLSTM model achieves the best performance compared with other structurally similar models for air quality prediction. The RMSE, MAE and MAPE of the EMD-transformer-BiLSTM model were 11.46, 11.77 and 15.53% lower than those of the second-ranked models, respectively. The EMD method for data preprocessing can better reflect the original physical characteristics of the data, which in turn can handle nonlinear and nonstationary time series well. The pointwise dot product self-attention in the transformer is insensitive to local information, which makes the model prone to anomalies in the time series, while BiLSTM is good at capturing two-way time series information. LSTM has a disadvantage in parallel processing, while the multihead self-attention in transformers makes it superior in parallel computing; thus, the transformer-BiLSTM model combining the two methods can balance the global and local sequence information well.

Innovations

We use either complex or simple prediction methods to predict the decomposed components, taking into account the different characteristics of IMFs. In the signal reconstruction process, BiLSTM is used to predict the final result considering the possible correlation between the original data and all IMFs. Based on previous experience that the Transformer architecture does not perform as well as expected in long-term time series prediction tasks, this paper further investigates whether the architecture can be applied to ultrashort-term prediction tasks by improving the decoder.

Limitations

However, the study in this paper still does not address the limitation of data leakage when using the EMD method for prediction; one approach that can be used is to construct the sample using the sliding window method and perform an EMD process for each step forward in the prediction45. Moreover, although the study in this paper focused on univariate time series data, the proposed method is expected to be further extended to model spatiotemporal data that can utilize auxiliary data such as meteorological data and time-stamped data while considering spatiotemporal correlation, which is a direction for future exploration.

Author contributions

Conceptualization, J.D.; methodology, J.D.; formal analysis, J.H.; data curation, Y.Z.; supervision, Y.Z.; writing—original draft preparation, J.D.; writing—review and editing, J.D. and J.H. All the authors have read and agreed to the published version of the manuscript.

Funding

Hubei Provincial Social Science Foundation Early Funding Project “Research on the Value Co-creation Mechanism and Realization Path of Hubei New Energy Vehicle Green Supply Chain from the Perspective of Configuration” (23ZD148); Hubei University of Automotive technology Doctoral Research Start-up Fund “Research on the Long-term Mechanism and Path of Green and High-quality Development of the Yangtze River Economic Belt from the Perspective of Digital Empowerment” (BK202010).

Data availability

The data used in this study were obtained from the Central Pollution Control Board of India (https://airquality.cpcb.gov.in/AQI_India/), and the hourly data of the air quality indices were obtained from Patna, Thiruvananthapuram, Bengaluru, Visakhapatnam, Amritsar, Gurugram and Delhi from 16:00 pm on January 1, 2015, to 0:00 pm on July 1, 2020.

Competing interests

The authors declare no competing interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Bikkina S Andersson A Kirillova EN Air quality in megacity Delhi affected by countryside biomass burning Nat. Sustain. 2019 2 3 200 205 10.1038/s41893-019-0219-0
Bikkina, S. et al. Air quality in megacity Delhi affected by countryside biomass burning. Nat. Sustain. 2(3), 200–205 (2019).10.1038/s41893-019-0219-0
2. Meng E Huang S Huang Q A robust method for non-stationary streamflow prediction based on improved EMD-SVM model J. Hydrol. 2019 568 462 478 10.1016/j.jhydrol.2018.11.015
Meng, E. et al. A robust method for non-stationary streamflow prediction based on improved EMD-SVM model. J. Hydrol. 568, 462–478 (2019).10.1016/j.jhydrol.2018.11.015
3. Abedinia O Lotfi M Bagheri M Improved EMD-based complex prediction model for wind power forecasting IEEE Trans. Sustain. Energy 2020 11 4 2790 2802 10.1109/TSTE.2020.2976038
Abedinia, O. et al. Improved EMD-based complex prediction model for wind power forecasting. IEEE Trans. Sustain. Energy 11(4), 2790–2802 (2020).10.1109/TSTE.2020.2976038
4. Zeng, A., Chen, M., Zhang, L. et al. Are transformers effective for time series forecasting?. arXiv preprint arXiv:2205.13504 (2022).
5. Zhou, H., Zhang, S., Peng, J. et al. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence Vol. 35 11106–11115 (2021).
6. Polydoras GN Anagnostopoulos JS Bergeles GC Air quality predictions: dispersion model vs Box-Jenkins stochastic models. An implementation and comparison for Athens, Greece Appl. Therm. Eng. 1998 18 11 1037 1048 10.1016/S1359-4311(98)00016-7
Polydoras, G. N., Anagnostopoulos, J. S. & Bergeles, G. C. Air quality predictions: dispersion model vs Box-Jenkins stochastic models. An implementation and comparison for Athens, Greece. Appl. Therm. Eng. 18(11), 1037–1048 (1998).10.1016/S1359-4311(98)00016-7
7. Alsoltany SN Alnaqash IA Estimating fuzzy linear regression model for air pollution predictions in Baghdad City Al Nahrain J. Sci. 2015 18 2 157 166
Alsoltany, S. N. & Alnaqash, I. A. Estimating fuzzy linear regression model for air pollution predictions in Baghdad City. Al Nahrain J. Sci. 18(2), 157–166 (2015).
8. Huang C Zhao X Cheng W Statistical inference of dynamic conditional generalized Pareto distribution with weather and air quality factors Mathematics 2022 10 9 1433 10.3390/math10091433
Huang, C. et al. Statistical inference of dynamic conditional generalized Pareto distribution with weather and air quality factors. Mathematics 10(9), 1433 (2022).10.3390/math10091433
9. Donnelly A Misstear B Broderick B Real time air quality forecasting using integrated parametric and non-parametric regression techniques Atmos. Environ. 2015 103 53 65 10.1016/j.atmosenv.2014.12.011
Donnelly, A., Misstear, B. & Broderick, B. Real time air quality forecasting using integrated parametric and non-parametric regression techniques. Atmos. Environ. 103, 53–65 (2015).10.1016/j.atmosenv.2014.12.011
10. Castelli M Clemente FM Popovič A A machine learning approach to predict air quality in California Complexity 2020 10.1155/2020/8049504
Castelli, M. et al. A machine learning approach to predict air quality in California. Complexity10.1155/2020/8049504 (2020).10.1155/2020/8049504
11. Mengash HA Hussain L Mahgoub H Smart cities-based improving atmospheric particulate matters prediction using chi-square feature selection methods by employing machine learning techniques Appl. Artif. Intell. 2022 36 1 2067647 10.1080/08839514.2022.2067647
Mengash, H. A. et al. Smart cities-based improving atmospheric particulate matters prediction using chi-square feature selection methods by employing machine learning techniques. Appl. Artif. Intell. 36(1), 2067647 (2022).10.1080/08839514.2022.2067647
12. Zhan H Zhu X Hu J A probabilistic forecasting approach for air quality spatio-temporal data based on kernel learning method Appl. Soft Comput. 2023 132 109858 10.1016/j.asoc.2022.109858
Zhan, H., Zhu, X. & Hu, J. A probabilistic forecasting approach for air quality spatio-temporal data based on kernel learning method. Appl. Soft Comput. 132, 109858 (2023).10.1016/j.asoc.2022.109858
13. Neagu CD Kalapanidas E Avouris N Air quality prediction using neuro-fuzzy tools IFAC Proc. Vol. 2001 34 8 229 235 10.1016/S1474-6670(17)40822-6
Neagu, C. D. et al. Air quality prediction using neuro-fuzzy tools. IFAC Proc. Vol. 34(8), 229–235 (2001).10.1016/S1474-6670(17)40822-6
14. Corani G Air quality prediction in Milan: Feed-forward neural networks, pruned neural networks and lazy learning Ecol. Model. 2005 185 2–4 513 529 10.1016/j.ecolmodel.2005.01.008
Corani, G. Air quality prediction in Milan: Feed-forward neural networks, pruned neural networks and lazy learning. Ecol. Model. 185(2–4), 513–529 (2005).10.1016/j.ecolmodel.2005.01.008
15. Kim, M. H., Kim, Y. S., Sung. S. W. et al. Data-driven prediction model of indoor air quality by the preprocessed recurrent neural networks. In 2009 ICCAS-SICE 1688–1692 (IEEE, 2009).
16. Mellit A Pavan AM Benghanem M Least squares support vector machine for short-term prediction of meteorological time series Theor. Appl. Climatol. 2013 111 1 297 307 10.1007/s00704-012-0661-7
Mellit, A., Pavan, A. M. & Benghanem, M. Least squares support vector machine for short-term prediction of meteorological time series. Theor. Appl. Climatol. 111(1), 297–307 (2013).10.1007/s00704-012-0661-7
17. Singh KP Gupta S Kumar A Linear and nonlinear modeling approaches for urban air quality prediction Sci. Total Environ. 2012 426 244 255 10.1016/j.scitotenv.2012.03.076 22542239
Singh, K. P. et al. Linear and nonlinear modeling approaches for urban air quality prediction. Sci. Total Environ. 426, 244–255 (2012).22542239 10.1016/j.scitotenv.2012.03.076
18. Li X Peng L Hu Y Deep learning architecture for air quality predictions Environ. Sci. Pollut. Res. 2016 23 22 22408 22417 10.1007/s11356-016-7812-9
Li, X. et al. Deep learning architecture for air quality predictions. Environ. Sci. Pollut. Res. 23(22), 22408–22417 (2016).10.1007/s11356-016-7812-9
19. Yi X, Zhang J, Wang Z, et al. Deep distributed fusion network for air quality prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 965–973 (2018).
20. Li X Peng L Yao X Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation Environ. Pollut. 2017 231 997 1004 10.1016/j.envpol.2017.08.114 28898956
Li, X. et al. Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation. Environ. Pollut. 231, 997–1004 (2017).28898956 10.1016/j.envpol.2017.08.114
21. Wen C Liu S Yao X A novel spatiotemporal convolutional long short-term neural network for air pollution prediction Sci. Total Environ. 2019 654 1091 1099 10.1016/j.scitotenv.2018.11.086 30841384
Wen, C. et al. A novel spatiotemporal convolutional long short-term neural network for air pollution prediction. Sci. Total Environ. 654, 1091–1099 (2019).30841384 10.1016/j.scitotenv.2018.11.086
22. Ma J Li Z Cheng JCP Air quality prediction at new stations using spatially transferred bi-directional long short-term memory network Sci. Total Environ. 2020 705 135771 10.1016/j.scitotenv.2019.135771 31972931
Ma, J. et al. Air quality prediction at new stations using spatially transferred bi-directional long short-term memory network. Sci. Total Environ. 705, 135771 (2020).31972931 10.1016/j.scitotenv.2019.135771
23. Li S Xie G Ren J Urban PM2.5 concentration prediction via attention-based CNN–LSTM Appl. Sci. 2020 10 6 1953 10.3390/app10061953
Li, S. et al. Urban PM2.5 concentration prediction via attention-based CNN–LSTM. Appl. Sci. 10(6), 1953 (2020).10.3390/app10061953
24. Zhang L Liu P Zhao L Air quality predictions with a semi-supervised bidirectional LSTM neural network Atmos. Pollut. Res. 2021 12 1 328 339 10.1016/j.apr.2020.09.003
Zhang, L. et al. Air quality predictions with a semi-supervised bidirectional LSTM neural network. Atmos. Pollut. Res. 12(1), 328–339 (2021).10.1016/j.apr.2020.09.003
25. Jin XB Yang NX Wang XY Deep hybrid model based on EMD with classification by frequency characteristics for long-term air quality prediction Mathematics 2020 8 2 214 10.3390/math8020214
Jin, X. B. et al. Deep hybrid model based on EMD with classification by frequency characteristics for long-term air quality prediction. Mathematics 8(2), 214 (2020).10.3390/math8020214
26. Song C Fu X Research on different weight combination in air quality forecasting models J. Clean. Prod. 2020 261 121169 10.1016/j.jclepro.2020.121169
Song, C. & Fu, X. Research on different weight combination in air quality forecasting models. J. Clean. Prod. 261, 121169 (2020).10.1016/j.jclepro.2020.121169
27. Wang Z Chen H Zhu J Daily PM2.5 and PM10 forecasting using linear and nonlinear modeling framework based on robust local mean decomposition and moving window ensemble strategy Appl. Soft Comput. 2022 114 108110 10.1016/j.asoc.2021.108110
Wang, Z. et al. Daily PM2.5 and PM10 forecasting using linear and nonlinear modeling framework based on robust local mean decomposition and moving window ensemble strategy. Appl. Soft Comput. 114, 108110 (2022).10.1016/j.asoc.2021.108110
28. Wang Z Gao R Wang P A new perspective on air quality index time series forecasting: A ternary interval decomposition ensemble learning paradigm Technol. Forecast. Soc. Change 2023 191 122504 10.1016/j.techfore.2023.122504
Wang, Z. et al. A new perspective on air quality index time series forecasting: A ternary interval decomposition ensemble learning paradigm. Technol. Forecast. Soc. Change 191, 122504 (2023).10.1016/j.techfore.2023.122504
29. Cai P Zhang C Chai J Forecasting hourly PM2.5 concentrations based on decomposition-ensemble-reconstruction framework incorporating deep learning algorithms Data Sci. Manag. 2023 6 1 46 54 10.1016/j.dsm.2023.02.002
Cai, P., Zhang, C. & Chai, J. Forecasting hourly PM2.5 concentrations based on decomposition-ensemble-reconstruction framework incorporating deep learning algorithms. Data Sci. Manag. 6(1), 46–54 (2023).10.1016/j.dsm.2023.02.002
30. Meng Z Xie Y Sun J Short-term load forecasting using neural attention model based on EMD Electr. Eng. 2022 104 3 1857 1866 10.1007/s00202-021-01420-4
Meng, Z., Xie, Y. & Sun, J. Short-term load forecasting using neural attention model based on EMD. Electr. Eng. 104(3), 1857–1866 (2022).10.1007/s00202-021-01420-4
31. Zhang Y Li C Jiang Y Accurate prediction of water quality in urban drainage network with integrated EMD-LSTM model J. Clean. Prod. 2022 354 131724 10.1016/j.jclepro.2022.131724
Zhang, Y. et al. Accurate prediction of water quality in urban drainage network with integrated EMD-LSTM model. J. Clean. Prod. 354, 131724 (2022).10.1016/j.jclepro.2022.131724
32. Jiang, B., Liu, Y., Xie, H. Super short-term wind speed prediction based on CEEMD decomposition and BILSTM-Transformer model. In 2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications (ICPECA) 876–882 (IEEE, 2023).
33. Qiu X Ren Y Suganthan PN Empirical mode decomposition based ensemble deep learning for load demand time series forecasting Appl. Soft Comput. 2017 54 246 255 10.1016/j.asoc.2017.01.015
Qiu, X. et al. Empirical mode decomposition based ensemble deep learning for load demand time series forecasting. Appl. Soft Comput. 54, 246–255 (2017).10.1016/j.asoc.2017.01.015
34. Vaswani, A., Shazeer, N., Parmar, N. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017).
35. Hochreiter S Schmidhuber J Long short-term memory Neural Comput. 1997 9 8 1735 1780 10.1162/neco.1997.9.8.1735 9377276
Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997).9377276 10.1162/neco.1997.9.8.1735
36. Li, S., Jin, X., Xuan, Y. et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Adv. Neural Inf. Process. Syst. 32 (2019).
37. Liang X Ge Z Sun L LSTM with wavelet transform based data preprocessing for stock price prediction Math. Probl. Eng. 2019 10.1155/2019/1340174
Liang, X. et al. LSTM with wavelet transform based data preprocessing for stock price prediction. Math. Probl. Eng.10.1155/2019/1340174 (2019).10.1155/2019/1340174
38. Mohammadi Farsani R Pazouki E A transformer self-attention model for time series forecasting J. Electr. Comput. Eng. Innov. JECEI 2020 9 1 1 10
Mohammadi Farsani, R. & Pazouki, E. A transformer self-attention model for time series forecasting. J. Electr. Comput. Eng. Innov. JECEI 9(1), 1–10 (2020).
39. Zeyer, A., Bahar, P., Irie, K. et al. A comparison of transformer and lstm encoder decoder models for asr. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) 8–15 (IEEE, 2019).
40. Yan Q Lu Z Liu H An improved feature-time Transformer encoder-Bi-LSTM for short-term forecasting of user-level integrated energy loads Energy Build. 2023 297 113396 10.1016/j.enbuild.2023.113396
Yan, Q. et al. An improved feature-time Transformer encoder-Bi-LSTM for short-term forecasting of user-level integrated energy loads. Energy Build. 297, 113396 (2023).10.1016/j.enbuild.2023.113396
41. Rhif M Ben Abbes A Farah IR Wavelet transform application for/in non-stationary time-series analysis: A review Appl. Sci. 2019 9 7 1345 10.3390/app9071345
Rhif, M. et al. Wavelet transform application for/in non-stationary time-series analysis: A review. Appl. Sci. 9(7), 1345 (2019).10.3390/app9071345
42. Yu C Li Y Chen Q Matrix-based wavelet transformation embedded in recurrent neural networks for wind speed prediction Appl. Energy 2022 324 119692 10.1016/j.apenergy.2022.119692
Yu, C. et al. Matrix-based wavelet transformation embedded in recurrent neural networks for wind speed prediction. Appl. Energy 324, 119692 (2022).10.1016/j.apenergy.2022.119692
43. Huang, Z., Xu, P., Liang, D. et al. TRANS-BLSTM: Transformer with bidirectional LSTM for language understanding. arXiv preprint arXiv:2003.07000 (2020).
44. Zhang C Bengio S Hardt M Understanding deep learning (still) requires rethinking generalization Commun. ACM 2021 64 3 107 115 10.1145/3446776
Zhang, C. et al. Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64(3), 107–115 (2021).10.1145/3446776
45. Qian Z Pei Y Zareipour H A review and discussion of decomposition-based hybrid models for wind energy forecasting applications Appl. Energy 2019 235 939 953 10.1016/j.apenergy.2018.10.080
Qian, Z. et al. A review and discussion of decomposition-based hybrid models for wind energy forecasting applications. Appl. Energy 235, 939–953 (2019).10.1016/j.apenergy.2018.10.080
