
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39223221
70384
10.1038/s41598-024-70384-9
Article
Short-term natural gas load forecasting based on EL-VMD-Transformer-ResLSTM
Zhao Mingzhi 1
Guo Guangrong 3
Fan Lijun 3
Han Long 3
Yu Qiancheng 1999019@nmu.edu.cn

12
Wang Ziyi 1
1 https://ror.org/05xjevr11 grid.464238.f 0000 0000 9488 1187 The College of Computer Science and Engineering, North Minzu University, Yinchuan, 750021 China
2 https://ror.org/05xjevr11 grid.464238.f 0000 0000 9488 1187 The Key Laboratory of Images and Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan, 750021 China
3 Ningxia Hanas Gas Group Co., Ltd, Yinchuan, 750021 China
2 9 2024
2 9 2024
2024
14 203432 7 2024
16 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Due to changes in urban residents’ consumption habits and lifestyles, accurately predicting natural gas consumption has become increasingly important. To address this issue, this paper proposes a forecasting model that combines Ensemble Learning (EL), Variational Mode Decomposition (VMD), Transformer, and LSTM. First, XGBoost, CatBoost, and LightGBM are used as base learners in the ensemble learning framework, with the predictions generated by the ensemble model integrated into the original dataset. Next, the VMD method is employed to decompose the natural gas load sequence into several intrinsic mode functions (IMFs), effectively extracting the inherent features of the natural gas load sequence. Finally, the data is input into the Transformer-ResLSTM network for prediction. This network replaces the original Transformer decoder structure with an LSTM network and fully connected layers, creating a new decoder structure. Additionally, a residual connection mechanism is introduced in both the encoder of the Transformer network and the new decoder structure. Experimental results show that, compared to traditional models such as ARIMA, Transformer, GRU, and LSTM, the proposed hybrid model significantly improves prediction accuracy, reducing MSE by 92–98% and MAE by 74–83%. In summary, this method demonstrates significant potential and practical value in enhancing the accuracy of natural gas load forecasting.

Subject terms

Natural gas
Computer science
Yinchuan City University-Enterprise Joint Innovation Project2022XQZD009 Ningxia Key Research and Development Program (Talent Intro-duction Special Project) Project2022YCZX0013 North Minzu University 2022 School level Scientific Research Platform Digital Agriculture Enabling Ningxia Rural Revitalization Innovation Team2022PT_S10 Ningxia Key Research and Development Program (Key Project) Project2023BDE02001 issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

China has established a diversified natural gas supply system, characterized by “west-to-east gas transmission, north-to-south gas adjustment, offshore gas landing, and nearby external supply”1. Yinchuan, as a key node in the west-to-east gas transmission, plays a crucial role in ensuring natural gas supply for urban residents. In this context, natural gas load forecasting becomes particularly important. It not only helps urban gas companies to plan gas procurement efficiently but also optimizes gas storage and transportation arrangements, thereby improving the efficiency and stability of the supply chain. During the load forecasting process, gas compressors2,3 provide stable delivery data and key operational parameters. This data enhances the accuracy of forecasting models, allowing companies to better adjust gas procurement and resource allocation, thereby optimizing the management of natural gas supply.

Currently, methods for natural gas load forecasting can be categorized into three main types4: traditional forecasting methods, which include statistical and time series-based models; the second type is based on machine learning and deep learning methods, encompassing various algorithms such as XGBoost5, support vector machines(SVM)6, random forests7, and neural networks8,9; the last type is ensemble forecasting methods, which combine multiple forecasting models to improve accuracy and stability.

In traditional natural gas load forecasting methods, Haydar et al.10 developed auto-regressive time series forecasting models for both heating and non-heating periods to estimate residential heating gas consumption. Pinar11 used ARIMA to forecast natural gas consumption for the next ten days, effectively capturing the trends in gas consumption. However, early traditional forecasting methods have certain limitations in handling multivariate time series data, especially when dealing with high-dimensional data, as they struggle to fully capture intrinsic features, leading to lower prediction accuracy.

In contrast, natural gas load forecasting methods based on machine learning and deep learning analyze large volumes of data and extract more features, resulting in superior forecasting performance. Brown et al. developed prediction models based on artificial neural networks and traditional statistical methods. However, as research progresses and the demand for higher prediction accuracy increases, single forecasting methods face several issues in practical applications. For example, a single model may struggle with the non-linearity and non-stationarity of the data, and its predictive accuracy and robustness can be limited by the model’s architecture and parameter choices. Consequently, an increasing number of researchers are turning to combined forecasting methods. In urban natural gas load forecasting, combined methods analyze detailed characteristics of gas usage and consider the actual usage environment and features, integrating multiple approaches to achieve more accurate prediction results.

Next, we will focus on the combination of machine learning and deep learning methods and their applications in forecasting from a technical perspective. In practical predictions, however, load data is often affected by noise, making feature extraction from load sequences challenging. To address this issue, preprocessing techniques such as Empirical Mode Decomposition (EMD) and Variational Mode Decomposition (VMD) are widely applied to enhance the feature extraction capability of load data.

Gong et al.12 introduced a combined forecasting model utilizing EMD phase space reconstruction and Least Squares Support Vector Machine (LS-SVM). Initially, the original nonlinear time series was decomposed into mutually independent modal components using EMD. Subsequently, Particle Swarm Optimization was used to optimize the parameters of LS-SVM, and regression prediction was carried out on each IMF component. Yang et al.13 adopted a hybrid energy storage allocation strategy based on the EMD technique and the two-stage robust method. The EMD technique transforms large fluctuations in wind power into smaller ones and smooths the wind power fluctuation curves at the minute and hour levels. However, the IMFs derived from EMD decomposition are prone to mode mixing, and endpoint effects may also affect the decomposition results. Additionally, EMD’s sensitivity to noise or outliers can lead to instability and inconsistency in the decomposition results.

In contrast, VMD excels at avoiding mode mixing. For example, Fang et al.14 proposed a model combining VMD and LSTM networks, applied to predict pressure pulsation signals in pumped storage power stations to prevent potential equipment failures. Wang et al.15 used a VMD-SSO-LSTM model for landslide dynamic displacement prediction. This method decomposes cumulative displacement and rainfall data to extract trend, periodic, and fluctuation components of displacement, as well as low-frequency and high-frequency components of rainfall. Zhang et al.16 proposed a monthly runoff volume prediction method based on the VMD-SSA-BiLSTM coupled model, which uses VMD for signal decomposition and preprocessing.

Although VMD is effective in avoiding mode mixing, it requires manual determination of the number of decompositions. If the number of decompositions is set too high, it can lead to over-decomposition of the signal, increasing computational complexity; if set too low, it may not fully extract the features of the signal17. Therefore, Liu et al.18 proposed a method to optimize the mode number and penalty parameters in VMD using the RIME algorithm, which avoids the randomness in parameter selection and the resulting uncertainty. Wang et al.19 also proposed a rotor unbalance fault diagnosis method based on RIME-VMD, aiming to improve the detection accuracy of small unbalance faults in low signal-to-noise ratio vibration signals. By optimizing the parameters in VMD and selecting the most relevant decomposition components, this method can more effectively extract fault information.

Many researchers have combined VMD with ensemble learning to further enhance predictive performance. In this context, ensemble learning algorithms such as XGBoost and CatBoost are used in conjunction with VMD to leverage their respective advantages. By applying VMD for signal preprocessing and feature extraction, and then feeding these processed features into ensemble learning models, higher-quality predictions can be achieved. For example, Zhang et al.20 introduced a sophisticated prediction model that combines LSTM neural networks with LightGBM, enhanced by VMD. Firstly, VMD is used to decompose features into modal components of various scales, effectively reducing the nonstationarity of the original sequence. LSTM is then used to individually predict each modal component, which are subsequently combined as multi-feature inputs into LightGBM for comprehensive load forecasting. Lu et al.21 addressed the issues of imperfect feature selection and poor stability in traditional GNSS height time series prediction models by proposing a predictive model that integrates VMD and XGBoost algorithms. This combination of VMD with ensemble learning takes full advantage of VMD’s strengths in feature extraction and ensemble learning’s powerful multi-model prediction capabilities, achieving higher prediction accuracy and better model performance. This approach not only enhances the handling of complex data but also improves the robustness and accuracy of the model. However, Dai et al.22 proposed an improved electricity load forecasting method based on BiLSTM and XGBoost. They utilized the Weighted Grey Relational Projection algorithm for data preprocessing, which also provided more accurate input data for subsequent predictions.

Nowadays, the Transformer model has been widely applied in various fields such as stock prediction23, wind power24, and electricity load forecasting25. However, its application in natural gas load forecasting remains relatively limited. Considering that natural gas load data is often influenced by factors such as seasonal variations, weather conditions, and industrial activities, the powerful capabilities of the Transformer model in feature extraction and sequence modeling, especially its unique multi-head attention mechanism, show great potential in the field of natural gas load forecasting.

Taking into account the advantages and disadvantages of the aforementioned methods, this paper proposes a novel combined forecasting approach. Its main contributions can be summarized as follows:We have proposed a method named EL-VMD-Transformer-ResLSTM for short-term natural gas load forecasting.

We employ XGBoost, CatBoost, and LightGBM as the base learners for ensemble learning, integrating the prediction results generated by the ensemble model into the original dataset.

We have improved the decoder of the traditional Transformer network by discarding the original decoder structure and replacing it with LSTM networks and fully connected layers to construct a new decoder structure. At the same time, we introduced residual connections and integrated them into both the encoder of the Transformer network and the new decoder structure.

The following organization of this paper is as follows: Section “Materials and methods”, provides a detailed explanation of how the load prediction model, the proposed method, is constructed. Section “Data preprocessing and normalization” comprehensively explains data preprocessing, whereas section “Experimental results and analysis” provides experimental results. Section “Conclusion and future work” concludes the discussion and provides an overview of future possibilities.

Materials and methods

Ensemble learning

The core idea of ensemble learning is to combine multiple relatively simple and moderately accurate models to form a single, more precise, and powerful learning model. Within this framework, the independent, relatively weak models are referred to as “base learners,” while the high-performance model obtained through ensemble integration is called the “strong learner.” To ensure the integrity of the training process, the training and validation sets are strictly partitioned, effectively preventing information leakage during the model learning phase. Using the ensemble learning strategy, the model selected three representative base learners: XGBoost, CatBoost, and LightGBM. As shown in Fig. 1.Figure 1 Ensemble learning layer.

XGBoost is a powerful and scalable implementation of the gradient boosting framework. With each iteration, it fits a new model to the residuals of the previous models’ predictions, progressively refining the final prediction by addressing past errors. The objective function in XGBoost comprises a loss function and a regularization term.1 Obj(θ)=L(y,y^)+Ω(f)Ω(f)=γT+12λ∑j=1Twj2

Among them, θ represents the set of parameters for the entire model; L represents the loss function, which quantifies the error between model predictions and true values; Ω denotes the regularization term used to prevent over-fitting, f is an individual decision tree; T is the number of leaf nodes, w is the weight of the leaf nodes, and γ and λ are regularization parameters. In XGBoost, the regularization term imposes constraints on the weights of the leaf nodes.

CatBoost26 is a gradient-boosting decision tree algorithm renowned for its efficient handling of categorical features. It introduces Ordered Boosting, a novel strategy designed for time-related data. In this approach, data is randomly shuffled, and the model is trained iteratively based on this new order. During each iteration, CatBoost uses a greedy algorithm to maximize the gain of the objective function, identifying the optimal split point for continuous features.2 Gain=(∑i∈ILgi)2∑i∈ILhi+λ+(∑i∈IRgi)2∑i∈IRhi+λ+(∑i∈Igi)2∑i∈Ihi+λ

In Eq. (2), IL and IR represent the instance sets of the left and right subtrees, respectively. gi is the first derivative of the loss function, hi is the second derivative, and λ is the regularization term for weight decay.

LightGBM27 is an efficient gradient-boosting framework based on decision tree algorithms. Unlike traditional GBDT, which requires traversing all data points to find the best-split point, LightGBM uses histogram-based reduction. This technique discretizes continuous feature values into bins and constructs histograms, making it more efficient to find optimal split points. Additionally, LightGBM employs a leaf-wise growth strategy, selecting the leaf with the maximum split gain from all current leaves for splitting at each step until reaching the maximum depth, thereby reducing the loss more effectively. To accelerate training, LightGBM introduces GOSS (Gradient-based One-Side Sampling), which retains data points with large gradients (crucial for boosting the model) and randomly samples from data points with small gradients.

By combining three base learners, the final prediction result is not merely the output of a single model but the weighted average of the predictions from these models, as follows:3 y(x)=1M∑i=1Mwiyi(x)

In Eq. (3) y(x) denotes the prediction output of the ensemble model, where M is the total number of models used in the ensemble, x refers to the sample being predicted, yi(x) indicates the prediction from the i-th model, and wi represents the weight assigned to the i-th model. These weights are required to be non-negative and sum up to 1, as shown below:4 wi>0,∑i=1Mwi=1

In this layer, the boosting strategy is employed for ensemble learning. It starts with a base learner and enhances the model’s performance through iterative weighting and optimization. In each iteration, base learners are trained sequentially, with the training set adjusted according to a predefined strategy to focus on the previous round’s errors. After all iterations, the predictions of each base learner are linearly combined using assigned weights to produce the final prediction result, as follows:5 f(x)=∑m=1Mαmhm(x)

In Eq. (5), M represents the number of classifiers, αm denotes the weight of the m-th base learner, and hm(x) represents the m-th base learner.

Variational mode decomposition

VMD28 is a fully intrinsic adaptive variational method that integrates Wiener filtering, Hilbert transformation, and alternating direction method of multipliers. This method allows users to manually set an appropriate number of intrinsic mode functions K to reduce model complexity. Its core idea is to utilize an optimal solution search mechanism to adaptively decompose the input sequence x(t), generating K IMFs with different frequency characteristics. By continuously optimizing the central frequency and bandwidth of each IMF, with the objective of minimizing bandwidth estimation, an optimized decomposition of x(t) is achieved. The specific steps of the algorithm are as follows:

Perform the Hilbert transformation on the decomposed intrinsic mode functions uk(t)of the natural gas load data to obtain the single-sided spectrum:6 [δ(t)+jπt]∗uk(t)

In Eq. (6),δ(t) represents the Diarc distribution function,* represents the convolution.

Use the single-sided spectrum of the natural gas load data and perform aliasing operations with the exponential term corresponding to its center frequency. Modulate the spectrum of each intrinsic mode function uk(t) to the corresponding “baseband.”7 [δ(t)+jπt]∗uk(t)e-jωkt

Utilize the Gaussian smoothing method to estimate the bandwidth of each demodulated intrinsic mode function. This bandwidth, represented by the squared gradient norm, is then used to solve the constrained variational problem. The expression for the constrained variational problem is as follows:8 minuk,ωk∑k=1K∂tδ(t)+jπt∗uk(t)e-jωkt22s.t.∑k=1Kuk(t)=x(t)

In Eq. (8), K represents the number of modes that need to be decomposed, uk ωk corresponding to the k-th modal component and center frequency after decomposition, respectively, and x(t) represents the natural gas load original data.

By introducing two parameters, the quadratic penalty factor α and the Lagrange operator λ(t), the constraint variational problem is transformed into an unconstrained variational problem. The expression for the extended Lagrange function is as follows:9 Luk,ωk,λ=α∑k∂tδ(t)+jπtuk(t)e-jωkt22+x(t)-∑kuk(t)22+λ(t),x(t)-∑kuk(t)

In Eq. (9),α represents a quadratic penalty factor, controlling the degree of bandwidth penalty to ensure that the bandwidth of each IMF is as small as possible. In this paper, α is set to 2000. The first term represents the bandwidth constraint of the signal modes, ensuring the smoothness of each mode function by limiting its bandwidth in the frequency domain. The second term is the sum of the squared reconstruction errors, representing the difference between the original signal and the sum of all mode functions. It aims to minimize this error to achieve precise signal reconstruction. The third term is the Lagrange multiplier term, which imposes a constraint to ensure the consistency between the original signal and the reconstructed signal, thereby maintaining the constraint during the optimization process.

Using the alternating direction method of multipliers combined with Parseval/Plancherel, Fourier equispaced transform, the optimization is carried out to obtain the modal components and central frequencies. Additionally, the saddle points of the augmented Lagrange function are searched. The expressions for the iteratively optimized variables uk,ωk and λ after alternating optimization iterations are shown below:10 u^kn+1(ω)=x^(ω)-∑i<ku^in+1(ω)+λ^(ω)21+2α(ω-ωk)2

11 ωkn+1=∫0∞ωu^k(ω)2dω∫0∞u^k(ω)2dω

12 λn+1(ω)=λn(ω)+τx-∑k=1Kukn+1(ω)

Among them, τ represents the bandwidth constraint factor. A larger τ value results in narrower modal bandwidths, which may lead to over-smoothing and the loss of some important information. Conversely, a smaller τ value results in wider modal bandwidths, which can capture signal variations more easily but may also cause overlap between modes. In this paper, τ is set to 1e–7. u^k(ω), f^(ω) and λ^(ω) are obtained by performing the Fourier transform on uk(t), x(t) and λ(t) respectively.

If the following expression is met, the algorithm stops iterating.13 1ukn22∑k=1Kukn+1-ukn22<ϵ

In Eq. (13), ϵ represents the discriminative accuracy, ||.||22 represents the L2 norm, ukn represents the modal function at the nth iteration.

When decomposing natural gas load sequences, selecting an appropriate value of K is crucial for obtaining accurate forecasting results. An overly large value of K may lead to excessive decomposition and model overfitting, whereas a too-small value of K may result in inadequate decomposition. We use the minimum envelope entropy to optimize the signal decomposition process. Lower envelope entropy indicates a more concentrated energy distribution of the signal, implying purer decomposed IMFs. The formula for calculating envelope entropy is as follows:14 Ek=-∑t=1TAk(t)logAk(t)Etotal=∑k=1KEk

In Eq. (14), Ek represents the envelope entropy of the k-th IMF, Ak(t) represents the instantaneous envelope of the k-th IMF at time t, and Etotal represents the total envelope entropy of all IMFs.

Transformer-ResLSTM network model construction

Position encoding layer and transformer encoding layer

The Transformer29 model has been widely applied in sequence data processing, particularly excelling in natural language processing. To handle positional information within sequences, a positional encoding layer has been introduced, which adds corresponding positional vectors to each position. However, without additional encoding, the inherent sequence order may be lost. Therefore, an explicit encoding method is needed to manage positional information. In this study, we use sine and cosine functions for positional encoding. The periodicity and continuity of sine and cosine functions allow unique encoding to be assigned to each position in the sequence while preserving the relative relationships between positions. The computational formula is as follows:15 PE(pos,2i)=sin(pos100002idmodel)PE(pos,2i+1)=cos(pos100002idmodel)

In Eq. (15), PE(pos,2i) represents the sine value at the position pos in the 2i dimension, PE(pos,2i+1)represents the cosine value at the position pos in the (2i+1)th dimension, dmodel represents the size of the embedded spatial dimension.

The main structure of the Transformer encoding layer consists of multi-head attention mechanisms and feed-forward layers. Layer normalization operations are added after each multi-head attention mechanism and feed-forward layer to reduce issues such as gradient explosion and improve the stability of the network. The core part of the encoding layer is the multi-head attention mechanism, which is a key component of the Transformer-ResLSTM model for accurate prediction. The multi-head attention mechanism is composed of multiple heads with self-attention. Each self-attention mechanism (head) aggregates the weighted parts of the input sequence, computing attention scores between queries (Q), keys (K), and values (V) to determine the importance of each position to others. After scaling, the attention scores are fused with the value matrix through weighted summation to generate the final output. As illustrated in Fig. 2.Figure 2 Self attention(left) and muti-head attention(right).

In practical computation, the model simultaneously computes the attention function for a set of query vectors, forming them into a matrix (Q), while also forming the key and value vectors into matrices (K) and (V), respectively. The output of the query vectors is determined by the hidden vectors encoded by the previous layer, and the matrices (K) and (V) are assigned the same values as (Q) in self-attention. The output computation results from the compatibility operation between query vectors and key vectors, obtaining weighted values. Here, f=fii=1t represents the input to the multi-head self-attention module. The computation formulas for key vectors, weight vectors, and query vectors are shown in Eq. (16).16 Kj=fWjkVj=fWjvQj=fWjq

In Eq. (16), Wjk,Wjv,Wjq∈Rd×dk represent trainable projection matrices. Utilizing the derived results, a scaled dot-product attention computation is performed. The attention calculation formula is as follows:17 Attention(Q,K,V)j=softmax(QjKjTdk)Vj

In order to comprehensively capture multidimensional representation subspaces information from different positions within the input sequence, the multi-head attention mechanism is crucial. Specifically, we will employ H parallel attention calculation processes, each focusing on a different representation subspace of the input data. In this configuration, the weight matrix WA of each attention mechanism is parameterized as an Rd×Hdk matrix, where H represents the number of parallel attention, dk is the key dimension of each attention, and d is the output dimension. Consequently, the detailed computation process of the multi-head attention mechanism is illustrated by Eq. (18). This approach enables the model to simultaneously consider multiple positions within the input sequence and extract relevant information from different representation subspaces, thereby enhancing its understanding and processing capabilities of complex input data.18 MultiHead(Q,K,V)=Concat(headj=1H)×WA

The functionality of the encoding layer is further achieved through the feed-forward network, which consists of two linear transformations connected by a ReLU activation function. This network is applied independently and identically to each time step. The linear transformation formula is expressed as shown in Eq. (19).19 y=WTx+b

In Eq. (19),WT represents the transpose of the weight matrix W, x represents the input vector, b represents the bias term.

ResLSTM decoder

This paper improves the decoder of the traditional Transformer network by replacing it with an LSTM network and fully connected layers, thereby constructing a new decoder structure. Additionally, a residual connection mechanism is introduced and incorporated into both the encoder of the Transformer network and the new decoder structure. The decoder architecture is illustrated in Fig. 3.Figure 3 Transformer decoder (left) and ResLSTM decoder (right).

Upon observation, we can see that the improved structure has significant differences from the traditional Transformer decoder structure. The traditional Transformer decoder utilizes multi-head self-attention mechanisms, encoder-decoder attention mechanisms, and feed-forward networks to process each layer’s input sequence, including attention computation and information transmission. In the improved decoder, these traditional components are replaced by an LSTM network and fully connected layers.

Specifically, In the improved decoder structure, the input sequence is first processed by the Transformer encoder and then passed to the LSTM network. The LSTM network is responsible for modeling these input sequences to capture long-term dependencies within the data. Subsequently, the output of the LSTM network is fed into fully connected layers, which convert the LSTM output into the final prediction results. Additionally, residual connections have been introduced in both the encoder and the improved decoder structure. These connections facilitate effective gradient flow, enhance model performance, and preserve the original input information, thereby improving the learning of complex dependencies.

The output X of the Transformer encoder is used as the input to the ResLSTM decoder layers. The ResLSTM decoder layers consist of N stacked LSTM layers, each with residual connections. The specific formulas are as follows:

For the n-th residual LSTM layer:20 it(n)=σ(Wi(n)ht-1(n)+Ui(n)Ht(n-1)+bi(n))ft(n)=σ(Wf(n)ht-1(n)+Uf(n)Ht(n-1)+bf(n))ot(n)=σ(Wo(n)ht-1(n)+Uo(n)Ht(n-1)+bo(n))gt(n)=tanh(Wg(n)ht-1(n)+Ug(n)Ht(n-1)+bg(n))ct(n)=ft(n)⊙ct-1(n)+it(n)⊙gt(n))ht(n)=ot(n)⊙tanh(ct(n))H(n)=Dropout(h(n))+H(n-1)

In Eq. (20), where H(n) represents the output of the n-th LSTM layer, and H(0)=y, y represents the output of the Transformer encoder, as shown in Eq. (19). ; it(n), ft(n), ot(n), gt(n), ct(n), and ht(n) represent the input gate, forget gate, output gate, candidate memory cell, memory cell state, and hidden state of the n-th LSTM unit at time t, respectively; Wi(n), Wf(n), Wo(n), and Wg(n) are the weight matrices for the input gate, forget gate, output gate, and candidate memory cell of the n-th LSTM unit, respectively; Ui(n),Uf(n), Uo(n), and Ug(n) are the recurrent weight matrices for the input gate, forget gate, output gate, and candidate memory cell of the n-th LSTM unit, respectively; bi(n), bf(n), bo(n), and bg(n) are the bias vectors for the input gate, forget gate, output gate, and candidate memory cell of the n-th LSTM unit, respectively.

Finally, the output of the LSTM layers is mapped to the final prediction values through a linear layer:21 Y=H(N)Wout+bout

In Eq. (21), Wout∈Rdmodel×1, bout∈R, and Y∈RN×1 is the final prediction output.

EL-VMD-Transformer-ResLSTM network model construction

The comprehensive framework of the EL-VMD-Transformer-ResLSTM model is illustrated in Fig. 4.Figure 4 EL-VMD-Transformer-ResLSTM.

The detailed procedure for using the model is as follows:Preprocess the collected data to ensure data continuity.

Use XGBoost, CatBoost, and LightGBM as the base learners for ensemble learning, and enhance the original dataset with the prediction results generated by the ensemble model.

Apply the VMD method to the preprocessed natural gas load data to decompose it into several IMFs. Use the minimum envelope entropy as the criterion to select the appropriate value of K, where envelope entropy measures the complexity and uncertainty of the signal. The smaller the entropy value, the lower the complexity and uncertainty of the signal.

Combine each IMF individually with the preprocessed features data to form multiple datasets to be tested. Each dataset is paired with a Transformer-ResLSTM network. Input these datasets into their respective Transformer-ResLSTM networks for prediction to obtain corresponding prediction results.

Finally, aggregate and reconstruct all the prediction results to output the final prediction result.

Data preprocessing and normalization

The quality of data significantly determines the effectiveness of predictive outcomes. To ensure high-quality predictions, data preprocessing is essential. Initially, employing the technique of same-period filling to handle missing natural gas load data ensures continuous numerical values across time intervals.

Data normalization is a common data processing method. Normalization can help data to converge to a certain extent, thereby reducing the problem of gradient explosion during prediction. The normalization method used in this paper is MinMaxScaler, as shown in Eq. (22). This method linearly transforms the original data to the range [0,1], thereby improving convergence speed and preventing gradient explosion.22 Xnorm=Xi-XminXmax-Xmin

In Eq. (22), Xmin and Xmax represent the minimum and maximum values of the input variable, Xnorm represents the normalized data, while Xi represents the current input value of the variable.

Experimental results and analysis

Dataset

The data selected in this article is the natural gas pipeline load data of Yinchuan City, selected from January 1, 2021, to September 30, 2023, with a total of 24072 pipeline load data. The sampling point interval is 1 h. When partitioning the dataset, 80% of it is allocated for training, 10% for validation, and the remaining 10% is reserved for testing. During training, validation is performed every 10 epochs. It’s important to note that all reported results are based on evaluations conducted on the test set.

Simultaneously, we collected meteorological data corresponding to time, including temperature, humidity, pressure, and precipitation, as well as five natural gas consumption growth characteristics: year-over-year growth (YOY), quarter-on-quarter growth for the previous hour (QOQ_H), previous day (QOQ_D), previous week (QOQ_W), and previous month (QOQ_M). As shown in Table 1. Yinchuan City, located in northwest China, has a typical temperate continental climate. The average annual precipitation is approximately 200 millimeters, leading to low humidity levels, with an average annual relative humidity of around 50%. This dryness is especially pronounced in winter and spring. Temperature variations are significant, with marked differences between seasons. In analyzing Yinchuan’s climate data, we selected mean, Pearson correlation coefficient, variance, and median as statistical indicators to gain a more comprehensive understanding of Yinchuan’s climate characteristics.

The Pearson correlation coefficient was utilized to assess the correlation between natural gas load data and meteorological as well as growth characteristic data. The value range of ρ is [−1, 1], where the magnitude reflects the strength of the correlation. If ρ>0, there is a positive correlation between the two variables; if ρ<0, there is a negative correlation between the two variables; if ρ=1 or ρ=-1, it indicates a perfect correlation between the two variables; and if ρ=0, the two variables are uncorrelated. Particularly noteworthy is the strong negative correlation observed between temperature and consumption, with a correlation coefficient of 0.74. Therefore, in the process of natural gas load prediction, temperature is an important influencing factor. The Pearson calculation formula is as follows:23 ρ=∑(Xi-X¯)(Yi-Y¯)∑i=1n(Xi-X¯)2∑i=1n(Yi-Y¯)2

In Eq. (23), n is the total number of samples; Xi and Yi are the values corresponding to the i-th vector X and Y; X¯, Y¯ are the sample mean.Table 1 Dataset introduction.

Column	Explains	Unit	ρ	Mean	Variance	Median	
Temperature	Real time temperature in Yinchuan city	∘C	-0.74	11.63	143.91	13.055	
Humidity	Real time humidity in Yinchuan city	%	-0.19	44.18	394.03	41.772	
Pressure	Real time atmospheric pressure in Yinchuan City	hPa	0.57	886.74	47.467	886.319	
Precipitation	Real time rainfall in Yinchuan City	mm/h	-0.07	0.023	0.026	0.000064	
YOY	Compared to the growth rate of natural gas consumption in the previous year	–	-0.01	0.078	37.854	-0.036	
QOQ_H	Compared to the growth rate of natural gas consumption in the previous hour	–	-0.03	0.040	0.88	-0.005	
QOQ_D	Compared to the growth rate of natural gas consumption the previous day	–	-0.03	0.067	1.52	-0.006	
QOQ_W	Compared to the growth rate of natural gas consumption in the previous week	–	-0.01	0.058	2.13	-0.015	
QOQ_M	Compared to the growth rate of natural gas consumption in the previous month	–	0.08	0.171	2.94	-0.068	
Consumption	Natural gas consumption	104m3	1	5.023	17.47	3.208	
The meteorological data in this article were purchased from the Xihe Energy Meteorological Big Data Platform, the website of which is as follows: https://xihe-energy.com/.

Experimental setup and training process

This experiment was conducted on the Pycharm platform under the Windows 10 operating system, with an Intel (R) Xeon (R) Gold 6154 CPU, 128GB of memory, NVIDIA TITAN V GPU, and 12GB of graphics memory.

The hyperparameters of the proposed model are shown in Table 2. This study adopts the time-sliding window method to transform time-series data into a supervised learning problem. The fundamental idea of this method is to extract consecutive windows from the time series, with each window’s data serving as features and the data point following the window as the target. This approach converts time series data into a dataset suitable for supervised learning, enabling the utilization of supervised learning models for training and prediction. Therefore, this prediction is a single-step forecast.Table 2 Hyperparameters settings of model.

Parameter	Value	
Batch size	32	
Epochs	200	
Optimizer	Adam	
Loss function	MSE	
Dropout	0.1	
Number LSTM layers	2	
Learning rate	0.01	
Hidden size	32	
Window size	12	
Encoder layers	1	
Model dimension	32	
Head	2	
Dimension FeedForward	128	
The hyperparameter settings for the comparison models are the same as those for the proposed model.

Model Dimension represents the embedding dimension of the Transformer encoder, while Hidden Size indicates the number of neurons in the LSTM hidden layer; these two should be equal. Number of LSTM Layers refers to the number of stacked LSTM layers. The parameters required for ELVMDTransformer are consistent with this setting. In ELVMDTResGRU, the number of GRU layers is set to 2, with the remaining parameters matching those in the table.

Evaluation metrics

In this paper, multiple metrics are used to evaluate the performance of the models in the experiments. The selected metrics include the coefficient of determination (R2), mean absolute error (MAE), mean absolute percentage error (MAPE), and mean square error (MSE). Among these indicators, the lower the values of MSE, MAE, and MAPE, the higher the value of R2, indicating higher prediction accuracy of the model.

Natural gas load sequence decomposition

The original natural gas load sequence exhibits characteristics such as nonlinearity, pseudo-periodicity, volatility, and randomness, as shown in Fig. 5. These features are clearly visible in the figure, with the load fluctuating over time, influenced by various factors. Nonlinearity is manifested in irregular patterns and sudden changes. Pseudo-periodicity indicates the presence of inconsistent intervals or irregular repetitive patterns, which may be influenced by seasonal variations, similar to the “cold” and “warm” annotations in the figure. Volatility is reflected in the sudden and significant changes in natural gas usage, which may be induced by weather conditions, economic factors, or other external events.Figure 5 Natural gas consumption.

Directly predicting the natural gas load sequence as an input vector is challenging as it struggles to effectively extract temporal features and is susceptible to various types of noise, significantly impacting prediction accuracy. To fully exploit the latent temporal feature information in the original load sequence, reduce its volatility and randomness, and enhance the accuracy of the prediction model, decomposition of the original load sequence(consumption) is necessary. The original load sequence is decomposed using VMD, EMD, EEMD, and CEEMDAN methods, as shown in Fig. 6. During this process, Gaussian white noise with an amplitude of 0.01 is added, and the average number of trials for the signal is set to 100.

Observing Figs. 5 and 6a, it is evident that IMF1 effectively captures the historical trend of the original load data. IMF1 exhibits a fluctuation pattern highly consistent with the overall trend of the original data, with its range of variation and main fluctuation characteristics consistent with the fluctuation pattern of the original load data. This indicates that IMF1 successfully captures the major periodicity and long-term trend in the original load data, demonstrating strong descriptive capability. Thus, IMF1 plays a crucial role in the decomposed signal. Observing Fig. 6b–d, it can be seen that the components processed by decomposition methods such as EMD, EEMD, and CEEMDAN exhibit evident mode mixing phenomena. This phenomenon may impact the accuracy and stability of models in natural gas load forecasting tasks.Figure 6 Results of natural gas original load decomposition obtained by four modal decomposition methods. (a) VMD (b) EMD (c) EEMD (d) CEEMDAN.

Comparison and analysis of load forecasting results

To validate the superiority of the proposed EL-VMD-Transformer-ResLSTM (ELVMDTResLSTM) model, two sets of comparative experiments were conducted. The first set of experiments employed individual models for natural gas prediction, specifically using the LSTM, ARIMA, GRU, and Transformer models, as depicted in Fig. 7. Figure 7a shows the prediction results for the test set of 110 h, while Fig. 7b displays the prediction errors of the individual models.

Observations reveal that, compared to other models, the LSTM model exhibits significant errors in some data predictions, resulting in relatively lower prediction accuracy. In contrast, the ARIMA model’s predictions accurately reflect the changing trends of the actual load sequence, demonstrating higher prediction accuracy. Compared to Transformer, GRU, and LSTM, the ARIMA model reduces the MAPE by 13.534%, 11.31%, and 16.272%, respectively. It is noteworthy that the coefficient of determination R2 for individual models exceeds 0.8, indicating that most individual models can fit the trend of the data well.

In the prediction results of single models, we also included a comparison between the Transformer-ResLSTM and the individual models. Through comparative analysis, we found that the Transformer-ResLSTM model significantly outperforms the single models in various evaluation metrics. Specifically, compared to the ARIMA model, the Transformer-ResLSTM model reduces MSE by 0.004, MAE by 0.007, MAPE by 0.059, and increases R2 by 0.002. These results indicate that the Transformer-ResLSTM has superior performance in terms of prediction accuracy and capturing data trends.Figure 7 Prediction results of the single model.

The second set of experiments compared the prediction results of different combination models, including the proposed ELVMDTResLSTM model, as well as EL-LSTM(ELLSTM), EL-Transformer-ResLSTM(ELTResLSTM), EL-EMD-Transformer-ResLSTM(ELEMDTResLSTM), EL-EEMD-Transformer-ResLSTM(ELEEMDTResLSTM), EL-CEEMDAN-Transformer-ResLSTM(ELCEEMDANTResLSTM), EL-VMD-Transformer(ELVMDTransformer) and EL-VMD-Transformer-ResGRU(ELVMDTResGRU). In these experiments, the natural gas original load sequence(consumption) underwent VMD, EMD, EEMD, and CEEMDAN, and the resulting IMF components were fed into the Transformer-ResLSTM models for prediction. The error magnitudes obtained on the test set are shown in Table 3.

Observing Table 3 and Fig. 7b, it is evident that the proposed hybrid model ELVMDTResLSTM significantly outperforms the individual models ARIMA, Transformer, GRU, and LSTM in terms of prediction performance. Specifically, MSE is reduced by 92–98%, MAE is reduced by 74–83%, MAPE is reduced by 76–88%, and R2 is improved by 5–22%. In summary, the hybrid model demonstrates better performance in capturing the underlying patterns and trends in the data, effectively reducing prediction errors, and significantly enhancing prediction accuracy.

From Table 3, it can be observed that ELEMDTResLSTM and ELEEMDTResLSTM perform excellently in terms of prediction errors when tested on the Yinchuan dataset. However, they are slightly inferior to ELVMDTResLSTM, particularly in terms of Mean Squared Error (MSE), where they are higher by 0.073 and 0.057, respectively. The forecasting results of the combined models on the Yinchuan dataset are shown in Fig. 8. On the U.S. Natural Gas dataset, ELVMDTResLSTM also shows significantly higher prediction accuracy compared to other models and aligns well with the trend of natural gas consumption.

To validate the effectiveness of the Transformer-ResLSTM structure proposed in this paper, we compared EL-VMD-Transformer and EL-VMD-Transformer-ResGRU. As shown in Table 3 for the Yinchuan dataset, the Transformer-ResLSTM reduces MSE by 0.007, MAE by 0.028, MAPE by 1.948, and increases R2 by 0.008 compared to the traditional Transformer. To further validate the effectiveness of the LSTM in the ResLSTM decoder, we also compared it to GRU. The results indicated that while both ELVMDTResLSTM and ELVMDTResGRU had relatively low prediction errors, ELVMDTResGRU was slightly less accurate, further demonstrating the effectiveness of the ResLSTM decoder.Table 3 Comparison of evaluation indicators.

Dataset	Model	MSE	MAE	MAPE	R2	Time/h	
Yinchuan dataset	ELLSTM	0.158	0.313	23.880	0.806	0.25	
ELTResLSTM	0.221	0.362	31.249	0.729	0.27	
ELEMDTResLSTM	0.078	0.212	15.755	0.904	2.98	
ELEEMDTResLSTM	0.062	0.207	17.704	0.924	3.00	
ELCEEMDANTResLSTM	0.296	0.472	41.157	0.638	2.44	
ELVMDTransformer	0.012	0.080	6.073	0.986	2.52	
ELVMDTResGRU	0.018	0.119	10.999	0.978	1.96	
ELVMDTResLSTM(our)	0.007	0.058	4.135	0.992	2.40	
U.S. Natural gas	ELLSTM	2468.399	38.639	10.285	0.968	–	
ELTResLSTM	3458.982	49.073	17.607	0.955	–	
ELEMDTResLSTM	7399.741	63.905	18.633	0.904	–	
ELEEMDTResLSTM	7677.303	76.351	29.208	0.900	–	
ELCEEMDANTResLSTM	6275.533	69.757	29.580	0.918	–	
ELVMDTransformer	8462.979	69.465	21.531	0.896	–	
ELVMDTResGRU	3082.091	45.902	18.578	0.962	–	
ELVMDTResLSTM(our)	2004.031	36.125	12.714	0.975	–	
Due to the small size of the dataset, the runtime can be disregarded and is represented by ’–’.

U.S. Natural Gas:https://www.eia.gov/naturalgas/monthly/.

Figure 8 Combination model prediction performance.

We also conducted validation on the U.S. Natural Gas Deliveries to Residential Consumers dataset. The experimental results indicate that the proposed model consistently achieved remarkable performance. From Table 3, we can conclude that the proposed model demonstrates strong generalization capability across different datasets and application scenarios, further affirming its effectiveness and robustness in time series forecasting tasks.

From Table 3, we can see that the training time of the model proposed in this paper is not the shortest. This is because our model is constructed based on the number of decomposed IMFs, meaning the number of IMFs is equal to the number of models. As the number of IMFs increases, the computational complexity grows linearly, leading to longer processing times and significantly higher memory usage. These factors collectively determine the overall computational load. Nevertheless, our model still demonstrates a clear advantage in terms of accuracy.Table 4 Comparison of different models.

Model	MSE	MAE	MAPE	R2	
PSO-LSTM 30	0.054	0.183	15.801	0.932	
CEEMD-LSTM 31	0.320	0.498	40.745	0.609	
ISSA-LSTM 32	0.063	0.217	18.502	0.918	
ELVMDTResLSTM(our)	0.007	0.058	4.135	0.992	
The size of the hyperparameter settings follows the parameter configurations provided in the relevant literature

The comparative experimental results are summarized in Table 4:

Compared to the PSO-LSTM model, the proposed ELVMDTResLSTM model demonstrates significantly superior performance in natural gas load forecasting, with MSE and MAE reduced by 0.047 and 0.125, respectively. This notable difference is likely due to the PSO-LSTM model using a wavelet threshold denoising algorithm for IMF denoising during the data preprocessing phase, which led to the loss of some important information and consequently affected the prediction accuracy.

Compared to the CEEMD-LSTM model, the ELVMDTResLSTM model proposed in this chapter demonstrates superior performance, with MSE and MAPE reduced by 0.313 and 36.61, respectively. This difference is likely due to the use of more effective feature extraction methods in this study. The Transformer encoder used in this research excels at capturing complex patterns and long-term dependencies in time series data, significantly enhancing prediction performance. Additionally, VMD effectively avoids mode mixing, further improving the model’s accuracy compared to CEEMD.

Compared to the ISSA-LSTM model, the ELVMDTResLSTM model reduces MSE and MAPE by 0.056 and 11.666, respectively. This difference is largely due to the data augmentation method. The ELVMDTResLSTM model uses ensemble learning methods to generate predictions, which are then added as new features to the original dataset. By capturing more data features and patterns, this data augmentation strategy significantly enhances the model’s generalization and accuracy. In contrast, the ISSA-LSTM model relies solely on ISSA for data preprocessing and feature extraction. Although ISSA is an improved Singular Spectrum Analysis method that effectively removes noise by analyzing subsequence distributions, its feature extraction capabilities are relatively limited and may not fully capture important features such as trends and seasonal components in the data.

Conclusion and future work

This paper introduces a natural gas load forecasting model called EL-VMD-Transformer-ResLSTM. Firstly, it uses XGBoost, CATBoost, and LightGBM as base learners for ensemble learning, enhancing the original dataset with the predictions generated by the ensemble model as new features. Secondly, the VMD method is employed to decompose the original natural gas load sequence into several IMFs, effectively avoiding the mode mixing phenomenon. The traditional Transformer network’s decoder is improved by replacing its original structure with an LSTM network and fully connected layers, constructing a new decoder structure. Additionally, a residual connection mechanism is introduced and integrated into both the encoder and the new decoder structure of the Transformer network. Experimental results show that compared to existing single models, our method reduces MAE and MSE in natural gas load forecasting by 74–83% and 92–98%, respectively. This indicates that our model can accurately capture and predict the complex patterns in natural gas load data, providing a reliable foundation for energy management and pipeline planning. The robustness of this model is also demonstrated in the U.S. natural gas dataset.

However, the proposed model has some limitations. Firstly, since predictions need to be made for each IMF, the number of Transformer-ResLSTM networks required equals the number of IMFs, resulting in high overall computational complexity. Secondly, natural gas consumption is influenced by various factors besides weather, such as urban development plans, economic activity levels, population density, and energy prices. Urban development plans may lead to regional growth or changes in natural gas demand, economic activity fluctuations directly affect industrial and commercial gas usage, and changes in population density alter residential gas demand distribution. Lastly, energy price fluctuations can impact user behavior and consumption patterns. Therefore, future research could consider integrating these diverse data sources to further enhance the accuracy and applicability of the forecasting model. This will help to more comprehensively capture the complexity and diversity of natural gas load variations, providing support for more precise energy management and pipeline planning.

Author contributions

Constructing the framework: M.Z., Z.W. and L.H.; correction of errors and writing instructions: Q.Y. and L.F.; data processing: M.Z. and G.G.; paper writing: M.Z., L.F. and Q.Y. All authors have read and agreed to the published version of the manuscript.

Data availability

The data that support the findings of this study are available from Ningxia Hanas Gas Group Co., Ltd., but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. However, data are available from the corresponding author (Lijun Fan: 4196553@qq.com) upon reasonable request and with permission of Ningxia Hanas Gas Group Co., Ltd.

Competing interests

The authors declare no competing interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Wu Yan SC Li H Exploration of key technologies for natural gas energy measurement China Test. 2023 49 18 23
Wu Yan, S. C. & Li, H. Exploration of key technologies for natural gas energy measurement. China Test. 49, 18–23 (2023).
2. Rigatos, G. et al. Nonlinear control of a gas compressor driven by an electric motor. In AIP Conference Proceedings, vol. 3094, (AIP Publishing, 2024). 10.1063/5.0210169.
3. Rigatos G Nonlinear optimal control for a gas compressor driven by an induction motor Results Control Optim. 2023 11 100226 10.1016/j.rico.2023.100226
Rigatos, G. et al. Nonlinear optimal control for a gas compressor driven by an induction motor. Results Control Optim. 11, 100226. 10.1016/j.rico.2023.100226 (2023).10.1016/j.rico.2023.100226
4. Xu P Wei LIU Du J Research progress on urban natural gas load prediction methods Oil Gas Storage Transp. 2023 42 481 491 10.6047/j.issn.1000-8241.2023.05.001
Xu, P., Wei, L. I. U. & Du, J. Research progress on urban natural gas load prediction methods. Oil Gas Storage Transp. 42, 481–491. 10.6047/j.issn.1000-8241.2023.05.001 (2023).10.6047/j.issn.1000-8241.2023.05.001
5. Chen T Guestrin C Xgboost: A scalable tree boosting system In Knowl. Discov. Data Min. 2016 10.1145/2939672.2939785
Chen, T. & Guestrin, C. Xgboost: A scalable tree boosting system. In Knowl. Discov. Data Min.10.1145/2939672.2939785 (2016).10.1145/2939672.2939785
6. Cortes C Vapnik V Support-vector networks Mach. Learn. 1995 20 273 297 10.1007/BF00994018
Cortes, C. & Vapnik, V. Support-vector networks. Mach. Learn. 20, 273–297. 10.1007/BF00994018 (1995).10.1007/BF00994018
7. Breiman L Random forest Mach. Learn. 2001 45 5 32 10.1023/A:1010933404324
Breiman, L. Random forest. Mach. Learn. 45, 5–32. 10.1023/A:1010933404324 (2001).10.1023/A:1010933404324
8. Brown, R. H., Kharouf, P., Feng, X., Piessens, L. P. & Nestor, D. Development of feed-forward network models to predict gas consumption. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), vol. 2, 802–805 (IEEE, 1994). 10.1109/ICNN.1994.374281.
9. Brown, R. H. & Matin, I. Development of artificial neural network models to predict daily gas consumption. In Proceedings of IECON’95-21st Annual Conference on IEEE Industrial Electronics, vol. 2, 1389–1394 (IEEE, 1995), 10.1109/IECON.1995.484153.
10. Aras H Aras N Forecasting residential natural gas demand Energy Sources 2004 26 463 472 10.1080/00908310490429740
Aras, H. & Aras, N. Forecasting residential natural gas demand. Energy Sources 26, 463–472. 10.1080/00908310490429740 (2004).10.1080/00908310490429740
11. Cihan P Impact of the COVID-19 lockdowns on electricity and natural gas consumption in the different industrial zones and forecasting consumption amounts: Turkey case study Int. J. Electr. Power Energy Syst. 2022 134 107369 10.1016/j.ijepes.2021.107369
Cihan, P. Impact of the COVID-19 lockdowns on electricity and natural gas consumption in the different industrial zones and forecasting consumption amounts: Turkey case study. Int. J. Electr. Power Energy Syst. 134, 107369. 10.1016/j.ijepes.2021.107369 (2022).10.1016/j.ijepes.2021.107369
12. Gong, C. Z, Yang, J., Li, L. L. & Zhu, K. J. Short term load prediction of urban gas pipeline network based on EMD-PSR-LSSVM. Syst. Eng. Theory Pract. 34, 8. 10.12011/1000-6788(2014)11-3001 (2014).
13. Yang X Hybrid energy storage configuration method for wind power microgrid based on EMD decomposition and two-stage robust approach Sci. Rep. 2024 14 2733 10.1038/s41598-024-53101-4 38302532
Yang, X. et al. Hybrid energy storage configuration method for wind power microgrid based on EMD decomposition and two-stage robust approach. Sci. Rep. 14, 2733. 10.1038/s41598-024-53101-4 (2024).38302532 10.1038/s41598-024-53101-4
14. Fang M The influence of optimization algorithm on the signal prediction accuracy of VMD-LSTM for the pumped storage hydropower unit J. Energy Storage 2024 78 1.1 1.12 10.1016/j.est.2023.110187
Fang, M. et al. The influence of optimization algorithm on the signal prediction accuracy of VMD-LSTM for the pumped storage hydropower unit. J. Energy Storage 78, 1.1-1.12. 10.1016/j.est.2023.110187 (2024).10.1016/j.est.2023.110187
15. Wang H Ao Y Wang C Zhang Y Zhang X A dynamic prediction model of landslide displacement based on VMD-SSO-LSTM approach Sci. Rep. 2024 14 9203 10.1038/s41598-024-59517-2 38649403
Wang, H., Ao, Y., Wang, C., Zhang, Y. & Zhang, X. A dynamic prediction model of landslide displacement based on VMD-SSO-LSTM approach. Sci. Rep. 14, 9203. 10.1038/s41598-024-59517-2 (2024).38649403 10.1038/s41598-024-59517-2
16. Zhang X Wang X Li H Sun S Liu F Monthly runoff prediction based on a coupled VMD-SSA-BiLSTM model Sci. Rep. 2023 13 13149 10.1038/s41598-023-39606-4 37573389
Zhang, X., Wang, X., Li, H., Sun, S. & Liu, F. Monthly runoff prediction based on a coupled VMD-SSA-BiLSTM model. Sci. Rep. 13, 13149. 10.1038/s41598-023-39606-4 (2023).37573389 10.1038/s41598-023-39606-4
17. Wu, L. L, Bian, Y., Tai, Q. R. & Li, Y. H. Research on regional carbon emission trading price prediction based on GA-VMD and CNN BiLSTM attention model. Oper. Res. Manag. 1–8 (2024).
18. Liu W Bai Y Yue X Wang R Song Q A wind speed forcasting model based on rime optimization based VMD and multi-headed self-attention-LSTM Energy 2024 294 130726 10.1016/j.energy.2024.130726
Liu, W., Bai, Y., Yue, X., Wang, R. & Song, Q. A wind speed forcasting model based on rime optimization based VMD and multi-headed self-attention-LSTM. Energy 294, 130726. 10.1016/j.energy.2024.130726 (2024).10.1016/j.energy.2024.130726
19. Wang Q Hu S Wang X Detection of incipient rotor unbalance fault based on the RIME-VMD and modified-WKN Sci. Rep. 2024 14 4683 10.1038/s41598-024-54984-z 38409246
Wang, Q., Hu, S. & Wang, X. Detection of incipient rotor unbalance fault based on the RIME-VMD and modified-WKN. Sci. Rep. 14, 4683. 10.1038/s41598-024-54984-z (2024).38409246 10.1038/s41598-024-54984-z
20. Zhang, W. et al. Short-term power load forecasting based on VMD-LSTM-LIGHTGBM with multi-feature integration. South Power Grid Technol. 17, 74–81. 10.13648/j.cnki.issn1674-0629.2023.02.009 (2023).
21. Tieding, L. U., Zhen, L. I., Xiaoxing, H. E. & Shijian, Z. H. O. Gnss elevation time series prediction method integrating VMD and XGBoost algorithms. Acta Geodaetica et Cartographica Sinica 52, 1235–1244. 10.11947/j.AGCS.2023.20220052 (2023).
22. Dai Y Zhou Q Leng M Yang X Wang Y Improving the BI-LSTM model with XGBoost and attention mechanism: A combined approach for short-term power load prediction Appl. Soft Comput. 2022 130 109632 10.13255/j.cnki.jusst.20220312002
Dai, Y., Zhou, Q., Leng, M., Yang, X. & Wang, Y. Improving the BI-LSTM model with XGBoost and attention mechanism: A combined approach for short-term power load prediction. Appl. Soft Comput. 130, 109632. 10.13255/j.cnki.jusst.20220312002 (2022).10.13255/j.cnki.jusst.20220312002
23. Tao Z Wu W Wang J Series decomposition transformer with period-correlation for stock market index prediction Expert Syst. Appl. 2024 237 121424 10.1016/j.eswa.2023.121424
Tao, Z., Wu, W. & Wang, J. Series decomposition transformer with period-correlation for stock market index prediction. Expert Syst. Appl. 237, 121424. 10.1016/j.eswa.2023.121424 (2024).10.1016/j.eswa.2023.121424
24. Lin S Wang S Xu X Li R Shi P Gaoformer: An adaptive spatiotemporal feature fusion transformer utilizing gat and optimizable graph matrixes for offshore wind speed prediction Energy 2024 292 130404 10.1016/j.energy.2024.130404
Lin, S., Wang, S., Xu, X., Li, R. & Shi, P. Gaoformer: An adaptive spatiotemporal feature fusion transformer utilizing gat and optimizable graph matrixes for offshore wind speed prediction. Energy 292, 130404. 10.1016/j.energy.2024.130404 (2024).10.1016/j.energy.2024.130404
25. Chan JW Yeo CK A transformer based approach to electricity load forecasting Electr. J. 2024 37 107370 10.1016/j.tej.2024.107370
Chan, J. W. & Yeo, C. K. A transformer based approach to electricity load forecasting. Electr. J. 37, 107370. 10.1016/j.tej.2024.107370 (2024).10.1016/j.tej.2024.107370
26. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V. & Gulin, A. Catboost: Unbiased boosting with categorical features. Adv. Neural Inf. Process. Syst. 10.48550/arXiv.1706.09516 (2018).
27. Ke, G. et al. Lightgbm: A highly efficient gradient boosting decision tree. Adv. Neural Inf. Process. Syst. 30 (2017).
28. Dragomiretskiy K Zosso D Variational mode decomposition IEEE Trans. Signal Process. 2013 62 531 544 10.1109/TSP.2013.2288675
Dragomiretskiy, K. & Zosso, D. Variational mode decomposition. IEEE Trans. Signal Process. 62, 531–544. 10.1109/TSP.2013.2288675 (2013).10.1109/TSP.2013.2288675
29. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30. 10.48550/arXiv.1706.03762 (2017).
30. Jianfeng Man EA Lei H Study on the multiple gas nodes load forecast of natural gas pipelines based on PSO-LSTM mixed model Oil, Gas, New Energy 2022 34 91 100 10.3969/j.issn.2097-0021.2022.06.013
Jianfeng Man, E. A. & Lei, H. Study on the multiple gas nodes load forecast of natural gas pipelines based on PSO-LSTM mixed model. Oil, Gas, New Energy 34, 91–100. 10.3969/j.issn.2097-0021.2022.06.013 (2022).10.3969/j.issn.2097-0021.2022.06.013
31. Ke Yan EA Xingyu P Short term natural gas load forecasting model based on CEEMD-LSTM Oil Gas Storage Transp. 2024 43 351 359 10.6047/j.issn.1000-8241.2024.03.012
Ke Yan, E. A. & Xingyu, P. Short term natural gas load forecasting model based on CEEMD-LSTM. Oil Gas Storage Transp. 43, 351–359. 10.6047/j.issn.1000-8241.2024.03.012 (2024).10.6047/j.issn.1000-8241.2024.03.012
32. Wei N Li C Peng X Li Y Zeng F Daily natural gas consumption forecasting via the application of a novel hybrid model Appl. Energy 2019 250 358 368 10.1016/j.apenergy.2019.05.023
Wei, N., Li, C., Peng, X., Li, Y. & Zeng, F. Daily natural gas consumption forecasting via the application of a novel hybrid model. Appl. Energy 250, 358–368. 10.1016/j.apenergy.2019.05.023 (2019).10.1016/j.apenergy.2019.05.023
