
==== Front
Heliyon
Heliyon
Heliyon
2405-8440
Elsevier

S2405-8440(24)12267-0
10.1016/j.heliyon.2024.e36236
e36236
Research Article
Remaining useful life prediction of Lithium-ion batteries using spatio-temporal multimodal attention networks
Suh Sungho Sungho.Suh@dfki.de
ab⁎
Mittal Dhruv Aditya b
Bello Hymalai ab
Zhou Bo ab
Jha Mayank Shekhar c
Lukowicz Paul ab
a German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany
b Department of Computer Science, RPTU Kaiserslautern-Landau, Kaiserslautern, Germany
c Centre de Recherche en Automatique de Nancy (CRAN), University of Lorraine, Nancy, France
⁎ Corresponding author at: German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany. Sungho.Suh@dfki.de
20 8 2024
30 8 2024
20 8 2024
10 16 e362367 8 2024
12 8 2024
© 2024 The Author(s)
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Lithium-ion batteries are widely used in various applications, including electric vehicles and renewable energy storage. The prediction of the remaining useful life (RUL) of batteries is crucial for ensuring reliable and efficient operation, as well as reducing maintenance costs. However, determining the life cycle of batteries in real-world scenarios is challenging, and existing methods have limitations in predicting the number of cycles iteratively. In addition, existing works often oversimplify the datasets, neglecting important features of the batteries such as temperature, internal resistance, and material type. To address these limitations, this paper proposes a two-stage RUL prediction scheme for Lithium-ion batteries using a spatio-temporal multimodal attention network (ST-MAN). The proposed ST-MAN is to capture the complex spatio-temporal dependencies in the battery data, including the features that are often neglected in existing works. Despite operating without prior knowledge of end-of-life (EOL) events, our method consistently achieves lower error rates, boasting mean absolute error (MAE) and mean square error (MSE) of 0.0275 and 0.0014, respectively, compared to existing convolutional neural networks (CNN) and long short-term memory (LSTM)-based methods. The proposed method has the potential to improve the reliability and efficiency of battery operations and is applicable in various industries.

Keywords

Remaining useful life prediction
Multimodal learning
Transformer
Battery health management
==== Body
pmcNomenclature

ADLSTM Adaptive Dropout LSTM

Ah Ampere Hour

CNN Convolution Neural Network

EOL End of Life

ELM Extreme Learning Machine

FC Fully Connected

FLOPs Floating Point Operations

FPC First Prediction Cycle

GPR Gaussian Process Regression

GRU Gated Recurrent Units

HS Health State

HUST Huazhong University of Science and Technology

ISSA-MKELM Sparrow Search Algorithm

LSTM Long Short Term Memory

MCT Minimum Cycle Threshold

MAPE Mean Absolute Percentage Error

MSE Mean Squared Error

MKELM Multiple Kernel ELM

PSO Particle Swarm Optimization

RMSE Root Mean Squared Error

RUL Remaining Useful Life

SOC State of Charge

ST-MAN Spatio-Temporal Multimodal Attention Network

TCN Temporal Convolutional Network

1 Introduction

Lithium-ion batteries have become indispensable power sources across diverse applications, spanning from electric vehicles and renewable energy storage to consumer electronics and industrial systems [5]. As their significance continues to grow, accurate prediction of the Remaining Useful Life (RUL) of these batteries assumes paramount importance. RUL prediction not only optimizes resource utilization but also enhances operational reliability and minimizes maintenance expenses, safeguarding the efficiency and longevity of battery-driven systems. In today's industrial landscape, reliable and efficient Lithium-ion battery operation underpins a spectrum of outcomes, from extended electric vehicle ranges to optimized renewable energy systems. Nonetheless, predicting RUL reliably is a formidable challenge due to the intricate interplay of factors governing battery degradation.

Past attempts to predict RUL have yielded diverse methodologies encompassing model-based and data-driven approaches. Model-based methods use mathematical equations to build mechanism models to capture internal electrochemical reactions and prognosticate RUL. These models mainly consider internal attenuation mechanism factors, including loss of lithium and loss of active material in the electrodes [2], [41]. Empirical models, such as exponential models and polynomial models, have been utilized to predict the degradation trend of Lithium-ion battery capacities [53]. In addition, adaptive filter techniques, including the Kalman filter, have been adopted to update the model parameters [51]. However, these approaches often exhibit significant RUL prediction errors when confronted with real-world complexities and uncertainties, despite the advantages of interpretability and the lack of a training procedure associated with data-driven methods.

Conversely, data-driven methods have garnered substantial attention for their adaptability and ability to learn from data without necessitating exhaustive knowledge of underlying physics. Data-driven methods predict the RUL by training the degradation data of the Lithium-ion batteries with statistical and machine learning algorithms. Various machine learning algorithms, including support vector regression (SVR) [56], naive Bayes (NB) [26], Gaussian process regression (GPR) [50], convolutional neural networks (CNN) [57], [20], and long short-term memory (LSTM) networks [16], [22], [42], have been utilized to predict the RUL of Lithium-ion batteries. However, data-driven methods require a substantial volume of historical degradation data for model training and exhibit limitations, notably in capturing intricate correlations between various degradation influencers. The majority of prevailing techniques predominantly focus on discharge capacity as a health indicator, failing to account for pivotal factors substantially shaping battery performance and degradation. In particular, existing data-driven methods require discharge capacity data from more than 25% of total charge-discharge cycles, potentially overlooking sudden degradations in battery performance.

Furthermore, conventional methods for RUL prediction grapple with limitations that curtail their efficacy in real-world scenarios [31], [24]. Conventional deep learning models, for instance, attempt to predict discharge values for later timestamps/cycles by inputting a percentage of the initial discharge capacity. Additionally, these data-driven approaches assume uniformity in data sources, presupposing that both training and test data originate from identical sensors under comparable operating conditions or share identical distributions.

These conventional approaches face challenges in iteratively forecasting the number of cycles until battery end-of-life (EOL), relying on static models ill-suited for adapting to dynamic operational variations. Moreover, they tend to oversimplify datasets, disregarding crucial variables such as temperature, internal resistance, and material type [3], [20]. Neglecting these essential parameters compromises the models' ability to accurately capture the intricacies of battery degradation mechanisms. In summary, the limitations of prevailing RUL prediction methods are twofold. Firstly, many struggle to dynamically forecast RUL across multiple cycles, limiting their applicability in dynamic real-world settings. Secondly, these approaches often overlook vital factors like temperature, internal resistance, and material type, resulting in imprecise and overly simplistic predictions.

Furthermore, precise identification of a battery's health state (HS) is paramount for accurate RUL prediction. Unfortunately, many conventional hybrid methods fall short of recognizing the first prediction cycle (FPC), which signifies the commencement of the unhealthy stage [39], [40]. Neglecting to pinpoint the FPC can lead to suboptimal RUL predictions, particularly when the HS of the battery exhibits minimal differences in the run-to-failure training dataset.

To overcome these limitations, in this paper, we propose a novel two-stage RUL prediction scheme for Lithium-ion batteries employing a spatio-temporal multimodal attention network (ST-MAN) architecture, aimed at addressing the critical challenge of RUL estimation in real-world scenarios where precise EOL information is often unavailable. In the initial stage, our approach delineates the transition to the unhealthy state, effectively segmenting degradation data into discernible health stages. Subsequently, in the second stage, we predict the RUL as a percentage in the unhealthy state after the FPC. By considering a broader range of features, including discharge capacity, charge capacity, temperature, internal resistance, and charge time, we improve prediction accuracy and robustness. To improve the performance of the RUL prediction and generality of the model, we propose the ST-MAN which can adeptly capture intricate spatio-temporal connections inherent in multimodal battery degradation data. Our novel ST-MAN architecture for the RUL prediction combines CNN, LSTM, and spatio-temporal attention units to effectively capture intricate degradation patterns. Through a series of experiments on two public Lithium-ion battery datasets including the MIT dataset [34] and HUST dataset [21], we demonstrate the effectiveness of the proposed two-stage scheme and the network architecture.

The primary contributions of this paper can be summarized as follows.• We introduce a two-stage RUL prediction scheme, involving the prediction of FPC in the first stage and the degradation pattern prediction in the second stage.

• We automatically annotate the degradation pattern as a percentage by determining the FPC to train the proposed ST-MAN model.

• We introduce an innovative ST-MAN architecture, combining CNN, spatio-temporal attention, and LSTM units, to capture intricate degradation patterns effectively.

• We validate our proposed method through rigorous experimentation, showcasing its superiority over existing CNN and LSTM-based methods on widely recognized battery degradation datasets.

The remainder of this paper is structured as follows. Section 2 introduces the related work. Section 3 provides the details of the proposed two-stage RUL prediction scheme and ST-MAN architecture. Section 4 presents the descriptions of the datasets used in our experiments and provides quantitative and qualitative experimental results on the two datasets. Finally, Section 5 concludes the paper.

2 Related work

The landscape of data-driven models for predicting RUL in Lithium-ion batteries has witnessed significant development. Deep learning techniques have proven effective in enhancing RUL prediction, thereby augmenting predictive energy management capabilities [44]. Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) algorithms have emerged as prominent strategies [54], [9], [52]. Dynamic LSTM variants, such as the work by Song et al. [37], have explored online RUL prediction employing indirect voltage measures for health index creation, deviating from traditional capacity-based approaches. Ensemble methods coupled with LSTM, as proposed by Liu et al. [19], have accounted for uncertainties using Bayesian model averaging. Ren et al. [29] integrated Particle Swarm Optimization (PSO) with LSTM, enabling efficient parameter estimation.

LSTM derivatives like Gated Recurrent Units (GRUs) have also garnered attention. Existing works such as Hannan et al. [7] addressed State of Charge (SOC) estimation under varying temperatures, while Huang et al. [10] introduced convolutional GRUs, and Cui et al. [4] employed attention mechanisms within GRUs for battery prognostics. Complementary approaches include the application of Convolutional Neural Networks (CNNs), often combined with other techniques for enhanced performance. Zhou et al. [57] incorporated causal and dilated convolutions to capture local contextual information. Hybrid architectures like CNNs combined with Bi-LSTM, as showcased by Yang et al. [46], have effectively integrated feature extraction and temporal modeling. Fusion strategies such as RNN-CNN combinations, as presented by Zhao et al. [55], have demonstrated merit in feature extraction and time dependency management.

Extreme Learning Machine (ELM) has garnered attention due to its rapid learning speed and generalization performance, proving stable in RUL estimation [44], [23], [49]. Variations like Kernel ELM (KELM) [8], Multiple Kernel ELM (MKELM) [17], and the incorporation of MKELM with the Sparrow Search Algorithm (ISSA-MKELM) [49] have emerged to adapt to complex data relationships. Hybrid models have also surfaced; for instance, Wang et al. [43] adopted a Bi-LSTM scheme with an attention mechanism for State of Health (SOH) estimation. Meanwhile, Tong et al. [42] innovatively combined Adaptive Dropout LSTM (ADLSTM) and Monte Carlo simulation for RUL prediction, achieving accuracy comparable to methods utilizing more data.

Incorporating domain knowledge, Ma et al. [20] integrated CNN and Gaussian Process Regression (GPR) for multi-stage RUL prediction. Similarly, Yao et al. [47] harnessed time series data and regression models for multi-cycle RUL prediction. Several works focus on using two stages for RUL prediction: Xu et al. [45] for smart manufacturing systems, Song et al. [36] for prognostic health management, Qu et al. [28] for railroad systems, and Zhang et al. [48] for mechanical equipment degradation. The data used to implement such methods differs from battery data, which is more complex due to varying physical elements and hidden chemical reactions inside the battery [35], [30], [13]. Liu et al. [18] proposed an end-to-end intelligent fault diagnosis and RUL prediction framework on ball-bearing data, which contains features such as load ratings, bearing life, fatigue, failure modes, and vibration analysis. In contrast, the battery data contains features like charge/discharge cycles, energy density, power density, internal resistance, lifespan, failure modes, etc. Our work presents the application of fault diagnosis and RUL prediction on battery data where the physical and chemical properties vary from bearings data.

Building upon our prior work [25], which introduced a two-stage LSTM-based RUL prediction framework, this paper presents a novel RUL prediction method with the proposed ST-MAN architecture. By capturing intricate spatio-temporal dependencies, this method aims to address limitations in existing approaches and advance the accuracy and robustness of RUL prediction for Lithium-ion batteries.

3 Proposed method

In this section, we present our innovative RUL prediction framework, depicted in Fig. 1, strategically crafted to address the intricacies of diverse degradation patterns observed in distinct battery cells. Departing from conventional methodologies, our framework adopts a two-stage paradigm. It initiates by categorizing the data from each cycle into either a healthy or unhealthy state, a pivotal classification step that markedly enhances the accuracy of RUL predictions. Our framework has two stages: an HS division stage and a RUL prediction stage. Within the HS division stage, our goal is to distinguish between healthy and unhealthy states in the degradation patterns of the battery cells. This discrimination plays a crucial role in identifying the FPC, signaling the commencement of the unhealthy phase—a pivotal factor for precise RUL predictions.Figure 1 Overview of the proposed framework. Stage 1 is the health state (HS) division step using the LSTM classifier and Stage 2 is a remaining useful life (RUL) prediction step using the proposed spatio-temporal multimodal attention network (ST-MAN) architecture.

Figure 1

The FPC prediction is needed because typically, any system remains in its nominal state of functioning in the initial stages of operation, gradually leading to failure as time advances. This remains the case across the domains (fuel cells, rotary machine components like turbines, bearings, etc.) mainly because most of the degradation mechanisms have slow dynamics (crack propagation, battery discharge degradation, etc.). Moreover, the need for prognostics (i.e. prediction of end of life) is felt when the system is beyond the initial stages of operation i.e. after an initial 40-50% of useful life has been consumed. Usually, RUL forecasts are mainly needed for predictive maintenance and as such, their real utility becomes apparent when the real End of life (EOL) approaches. Given these two aspects, prognostics of most of the systems are generally not done from the very first second of operation and are usually triggered/” switched-on” after the system has functioned for some considerable part of the time (say 40-50%). For example, as proposed in the paper [32] thus, leading to such a practice since then within the community) the “True RUL” is considered constant (straight line) from the start till a certain point in time owing to the general understanding that assessing prognostics efficiency right from the beginning is not useful.

Once the FPC is established, we define the subsequent data as degradation data. Utilizing an extensive array of features, including discharge capacity, charge capacity, charging time, temperature, and internal resistance, we forecast the RUL as a percentage from the FPC to the end-of-life (EOL). To effectively capture intricate degradation patterns, we propose the ST-MAN, which integrates CNN, LSTM, and spatio-temporal attention units.

3.1 First prediction cycle decision

In this subsection, we address the intricate challenge of predicting time-series data with limited availability and inherently non-linear degradation patterns. Time-series data in the context of Lithium-ion battery health is notably influenced by a multitude of factors, including usage patterns, load variations, and dynamic operating conditions. These complexities hinder the precise measurement of the internal reactions occurring inside the battery. To mitigate the data limitations, we employ a sliding window approach, characterized by a window size of 50 and a step size of 1, including the different number of channels depending on the dataset. A window size of 50 is selected by referring to the works presented in [6], [1], [27], [34], [15], [33].

The first stage of our proposed approach is centered on determining the FPC. This pivotal point signifies the transition from a healthy state to an unhealthy state within the battery cell degradation data. However, annotating the data to distinguish between healthy and unhealthy segments for training the HS division network poses a significant challenge. To overcome this limitation, we adopt a unique approach wherein we designate the initial 10% of the cycles as the healthy state and the final 10% as the unhealthy state in the training dataset, as illustrated in Fig. 1.

This strategy not only decreases the need for manual labeling but also enhances the accuracy of HS division, particularly in cases where ground truth HS labels are unavailable in most open-access Lithium-ion battery datasets.

The percentage value equal to 10% is selected because the early life of the Lithium-ion battery is defined between the third cycle and 40% of the total number of cycles of the battery and the end of life is defined after 80% of the battery's total number of cycles [38]. Hence, we have used as training data for our model the first 10% of the cycles labeled as healthy, and the last 10% of the battery cycles labeled as unhealthy to guarantee the data is between the above-mentioned boundaries.

To provide further justification, Table 1 demonstrates how the discharge capacity varies the initial and last 10% of the total number of battery cycles, it is evident that the discharge capacity remains relatively stable during the initial 10% of the data, whereas significant fluctuations are observed during the last 10%. Moreover, the distribution of cycles in the initial and last 10% of the total number of cycles along with the distribution of the total number of cycles in different batteries is presented in Table 2. To provide visualization, Fig. 3 shows how the discharge capacity curves look during the initial and last 10% of representative battery life in both datasets.Table 1 Mean standard deviation of discharge capacity during initial and last 10% of total cycles.

Table 1Dataset	Initial 10%	Last 10%	
MIT [34]	0.002	0.024	
HUST [21]	0.004	0.014	

Table 2 Information related to number of cycles.

Table 2Dataset	All Data	Initial 10%	Last 10%	
MIT [34]	810 ± 369	75 ± 36	80 ± 36	
HUST [21]	1888 ± 387	183 ± 38	188 ± 38	

Consider Xi=[x1i,...xni] as input data, representing all features of the i-th battery cell. A sliding window of size nw is applied to accommodate the nf features, with xji∈Rnf×nw containing the battery cycle data within that window. The corresponding HS labels are defined as:(1) yHSji={0,ifj<EOLi×p1,ifj>EOLi×(1−p)

where yHSji denotes the HS label of xji, EOLi represents the total number of cycles in the i-th battery cell data, and p is the percentage of the total degradation process set to 10%. Consequently, 20% of the training data is labeled for training the HS division network model in the FPC decision stage. The shape of the variable xji is equal to [Window Size, Number of Channels]. For instance, the input dimension for training the model on MIT [34] dataset is [50, 7], where 50 is window size and 7 is the number of features.

The HS division model employs a 1D LSTM architecture, as illustrated in Fig. 2. This network model incorporates one LSTM module, comprising three layers. Following the LSTM layers, one fully connected (FC) layer with sigmoid activation functions is utilized to classify the HS. For training the HS division model, we use the binary cross entropy (BCE) loss function, quantifying the discrepancy between predictions and corresponding HS labels:(2) LBCE=EyHSi,yˆHSi[yHSilog⁡yˆHSi+(1−yHSi)log⁡(1−yˆHSi)]

where yˆHSi and yHSi denote the HS division prediction and the corresponding HS label of the i-th battery cell data, respectively.Figure 2 HS division classifier network architecture based on the LSTM units.

Figure 2

Figure 3 Discharge capacity at initial and last 10% of the total cycles examples from MIT (Upper) and HUST dataset (Lower).

Figure 3

After the HS division model is trained, it can categorize both the remaining training and test data into healthy and unhealthy states without the necessity for a predefined threshold value. By assimilating the feature distinctions between labeled healthy and unhealthy data during training, the model can discern degradation patterns across the entire dataset. Determining the FPC is crucial, marking the initial stages of Lithium-ion battery deterioration. However, identifying the FPC poses a challenge due to typically weaker degradation features during this phase compared to those in the labeled unhealthy state.

To identify the relevant FPC, we employ a straightforward continuous trigger mechanism. In this setup, the battery cell is classified as unhealthy when consecutively predicted as such by the HS division model for a defined number of instances. This trigger mechanism helps mitigate unnecessary oscillations and uncertainties in FPC determination, being a widely adopted strategy in HS division approaches [14], [39]. In our study, the FPC is determined when the HS division model designates the input data as unhealthy for five consecutive instances, contributing to a more reliable indication of the FPC. To further improve the robustness of FPC determination, we incorporate additional measures within the continuous trigger mechanism. The pseudo-code for determining the FPC point is mentioned in Algorithm 1.Algorithm 1 FPC point prediction algorithm.

Algorithm 1

In particular, we introduce a temporal element to the trigger mechanism to ensure that the FPC is not identified too early in the battery's life cycle. This added temporal constraint guards against prematurely labeling the battery as unhealthy, which could result in inaccurate RUL predictions.

Under this extended mechanism, the battery is considered to have reached its FPC only if the HS division model classifies it as unhealthy for five consecutive times and this classification occurs beyond a defined threshold cycle, which we refer to as the minimum cycle threshold (MCT). We selected the threshold value for consecutive prediction of the unhealthy stage as five to eliminate oscillations between healthy and unhealthy stage predictions and to pinpoint the FPC accurately. When calculating the FPC point for each battery, we observed that in most cases, no oscillation was present. However, to safeguard our model from potential random oscillations. The MCT is a dynamic value calculated as a percentage of the total cycle count. By setting an MCT, we ensure that the FPC identification occurs after a certain proportion of the battery's expected lifetime has elapsed, adding a temporal context to the determination process.

The introduction of this temporal constraint is crucial for several reasons. First, it aligns with the practical reality that the FPC often occurs after a certain number of cycles, rather than immediately. Second, it prevents premature FPC identification, which can be especially beneficial in cases where initial battery cycles exhibit varying degrees of degradation before stabilizing. Finally, this temporal constraint enhances the adaptability of our framework to different battery types and operating conditions, as it accounts for variations in degradation behavior.

3.2 Remaining useful life prediction

In the stage of predicting RUL, our objective is to estimate the RUL percentage after the FPC. For this purpose, we introduce the ST-MAN architecture, a novel design adept at capturing intricate degradation patterns while maintaining computational efficiency. This architectural choice seeks a fine balance between effectiveness and computational resources, presenting a lightweight yet powerful model for deep learning in RUL prediction.

To effectively train our RUL prediction model, our approach utilizes time-series data by applying sliding windows from the FPC to the EOL as inputs, while the RUL percentage serves as the ground truth for training. Since the precise RUL information is generally unavailable in real-world scenarios, we calculate the RUL percentage label after the FPC as follows.(3) yLji=EOLi−jEOLi−FPCi

where yLji represents the RUL label of xji post FPC, while EOLi and FPCi denote the EOL and FPC of the i-th battery cell, respectively. This formulation allows us to express the RUL as a percentage, providing a more practical and interpretable measure of battery health.

Our approach fundamentally differs from conventional approaches that rely exclusively on discharge capacity as a health indicator and predict discharge capacity values as RUL estimates. Recognizing the tendency of discharge capacity values to exhibit sudden drops, making accurate RUL predictions challenging, we define RUL as a linear percentage. This approach considers the period from FPC to EOL, enabling the prediction of remaining life cycles from the testing cycle, given its linear relationship with the period from FPC to the cycle point and the remaining life cycles from the cycle point to EOL.

The proposed ST-MAN architecture is depicted in Fig. 4, along with its detailed layer-wise description in Table 3. This architecture is specifically tailored to the unique characteristics of our battery degradation data, which is inherently multimodal and temporally dependent. We start by applying convolutional operations to the input data xji∈Rnf×nw, which represents battery degradation data with different features nf using a sliding window of length nw. Importantly, we treat each channel individually during this process to emphasize local context and account for the distinct contributions of each channel. To understand the relationships between different channels and determine the relative importance of each channel, we employ a transformer encoder block. This block enables the model to learn interactions between channels and their significance in the overall degradation process. The relative importance calculated by the transformer is then added back into the previous input data.Figure 4 The architecture overview of the proposed spatial-temporal multimodal attention networks (ST-MAN). The network consists of convolutional operations, a cross-channel Transformer encoder block, a fully connected layer to fuse cross-channel information, an LSTM module to extract temporal information, and a temporal attention module to improve the temporal information extraction.

Figure 4

Table 3 The details of the proposed ST-MAN network architecture for training on example MIT dataset [34].

Table 3Network	Layers	Description	Activation	Output	
Individual Convolution Sublet	Input (7x50)		-	-	
Reshape	-	-	1x7x50	
Permute	-	-	1x50x7	
Conv2d	CNN (input = 1, output = 20, kernel size = (5,1), stride = (1, 1) )	ReLU	20x46x7	
Conv2d	CNN (input = 20, output = 20, kernel size = (5,1), stride = (2, 1) )	ReLU	20x 21x 7	
Conv2d	CNN (input = 20, output = 20, kernel size = (5,1), stride = (1, 1) )	ReLU	20x17x7	
Conv2d	CNN (input =20, output = 20, kernel size = (5,1), stride = (2, 1) )	ReLU	20x7x7	


				
Channel Interaction	SelfAttention	Query(20,20), Key(20,20), Value(20,20)	Sigmoid	7x20x7	


	
Channel Fusion	FC	Linear(140,40)	-	7x40	
Dropout	Dropout(0.1)	-	7x40	


	
Temporal Fusion	LSTM	LSTM( input = 40, hidden size = 40)	Tanh	7x40	


	
Temporal Fusion	FC	Linear(40,40)	Tanh	40	
FC	Linear(40,1)	Softmax	1	


	
Prediction	FC	Linear(40,1)	-	1	

After learning channel interactions, we proceed to fuse the channel information using an FC layer. Unlike the self-attention mechanism, the FC layer allows different features within the same sensor channel to have varying weights, enhancing the feature fusion and capturing the unique contributions of each. To model temporal dependencies across the sequence of data, we utilize LSTM units. These units enable the network to understand the global temporal relationships between different time steps within the input window. Considering that not all time steps contribute equally to RUL prediction, it is vital to assess the importance of features at each time step in the sequence. To achieve this, we generate a global contextual representation through a weighted average sum of hidden states at each time step. These weights are determined by a temporal self-attention layer. Since the feature at the last time step represents the entire sequence, the resulting global representation is reintegrated into the previous feature. We introduce a trainable multiplier parameter for the global representation, affording the model flexibility in deciding whether to utilize or disregard the generated global representation. Finally, an FC layer is applied to the represented feature to obtain the final RUL output.

To train the RUL prediction model effectively, we define the RUL prediction loss, which measures the difference between the predicting RUL and the corresponding RUL labels as a percentage.(4) LRUL=LMAE+LRMSE+LMAPE

(5) LMAE=1EOLi−FPCi∑j=FPCiEOLi|yLji−yLˆji|

(6) LRMSE=1EOLi−FPCi∑j=FPCiEOLi(yLji−yLˆji)2

(7) LMAPE=1EOLi−FPCi∑j=FPCiEOLi|yLji−yLˆji|yLji

where yLji and yLˆji represent the corresponding RUL label and RUL prediction of the i-th battery cell data, respectively. LMAE denotes the mean absolute error (MAE) between the predicting RUL and the corresponding RUL labels, LRMSE is root mean square error (RMSE), LMAPE is mean absolute percentage error (MAPE), and the final RUL prediction loss is the sum of the MAE, RMSE, and MAPE. This comprehensive loss function ensures that the model learns to make precise predictions while considering both the magnitude and relative importance of prediction errors. Combining MAE, RMSE, and MAPE into a composite loss function offers a comprehensive evaluation of prediction accuracy, considering error magnitude, relative errors, and sensitivity to outliers. This approach ensures a balanced assessment, robustness to individual metric idiosyncrasies, and simplicity in interpretation, providing a holistic evaluation of model performance across various error dimensions. By incorporating these architectural elements and the corresponding loss functions, our proposed ST-MAN architecture is equipped to effectively capture and learn from the complex degradation patterns exhibited by Lithium-ion batteries, leading to accurate and robust RUL predictions.

4 Experimental results

4.1 Datasets

To evaluate the proposed framework on the real battery cell degradation data, we used two different datasets: the MIT dataset [34] and the HUST dataset [21]. While the MIT dataset is widely recognized in the field, the HUST dataset represents one of the latest additions, offering valuable battery discharge data. The MIT dataset is the product of a collaborative effort between the Toyota Research Institute, Stanford, and MIT [34]. This dataset comprises 124 commercial lithium-ion battery cells, specifically the A123 Systems model APR18650M1A, boasting a nominal capacity of 1.1 Ah. These batteries were subjected to rigorous cycling under fast-charging conditions. The experimentation involved cycling the batteries within horizontal cylindrical fixtures using a 48-channel Arbin LBT potentiostat, all within a controlled forced convection temperature chamber maintained at 30C∘. The key specifications of the cells in this dataset include a nominal capacity of 1.1 Ah and a nominal voltage of 3.3 V. Charging was executed following either a one-step or two-step fast-charging policy, denoted as ‘C1(Q1)-C2’. Here, C1 and C2 represent the initial and subsequent constant-current steps, while Q1 signifies the state-of-charge (SOC, %) at which the current transitions between the two steps. Following this, charging at 1C CC-CV ensued, concluding at 80% SOC, which is EOL. The upper and lower cutoff potentials adhere to the manufacturer's specifications at 3.6 V and 2.0 V, respectively. The dataset spans a wide range of cycle lives, varying from 150 to 2,300 cycles, with an average of approximately 810 cycles and a standard deviation of 340. Each cycle corresponds to a complete battery discharge and recharge. This rich dataset comprises crucial parameters, including cycle number, internal resistance, temperature statistics (minimum, average, maximum), and charge and discharge capacities. In Fig. 5 (a), we present a visualization of the discharge capacities of the 124 cells, demonstrating the significant variability in cycle numbers among the lithium-ion batteries.Figure 5 Discharge capacity degradation patterns of (a) 124 Lithium-ion battery cells from the MIT dataset [34], (b) 77 Lithium-ion battery cells from HUST dataset [21].

Figure 5

The HUST dataset [21] is provided by the Huazhong University of Science and Technology and involves 77 commercial batteries subjected to over 140,000 charge-discharge cycles. These batteries, all LFP/graphite A123 APR18650M1A with a nominal capacity of 1.1 Ah and a nominal voltage of 3.3 V, undergo testing with various multi-stage discharge protocols. However, they follow an identical fast-charging protocol in two thermostatic chambers at 30C∘. The number of cycles ranges from 1,100 to 2,700 cycles, with an average of 1,898 and a standard deviation of 387, each representing information about discharge capacity, charge capacity, and charge voltage. Ma et al. [21] claim that this is the largest dataset for diverse protocols. Moreover, we used the feature generation mechanism used in [21] to generate two additional features, where the difference of the charge voltage and charge capacity curve between each cycle and the 10th cycle is taken into account, making a total of 5 features for each cycle. The high performance of differences in charge capacity features and voltage capacity features are inspired by Severson et al. [34] and Jiang et al. [11], respectively. As illustrated in Fig. 5 (b), the discharge capacity for all 77 cells in the HUST dataset exhibits diverse patterns. Notably, the distinct properties of both datasets make them valuable resources for conducting experiments and analyzing results, thereby contributing to the development of robust solutions for RUL prediction tasks.

4.2 Implementation details and evaluation metrics

The experiments were carried out using Python scripts within the PyTorch framework. The models were optimized using the Adam optimizer with a learning rate of 0.0001, β1 at 0.9, and β2 at 0.99. The training process involved a batch size of 8, running for 100 epochs with early stopping, implemented with a patience of 20 epochs to mitigate overfitting.

To demonstrate the advantage of our proposed framework, we conducted comparisons with established models, specifically the Temporal Convolutional Network (TCN) [57] and ADLSTM [42]. These conventional methods traditionally rely on discharge capacity as the sole input and presume prior knowledge of the EOL. The RUL prediction scheme for conventional methods, as illustrated in Fig. 6, employs 40% of the battery cell data as input, predicting the remaining 60% as output. To address variations in cycle numbers across datasets, padding was implemented to align input and output sizes. For example, on the MIT dataset, the input and output sizes were set at 894 and 1342, respectively, while for the HUST dataset, sizes of 1072 and 1608 were utilized.Figure 6 The conventional RUL prediction scheme with input and output on the same battery cell.

Figure 6

Additionally, we compare the proposed ST-MAN with the TCN and ADLSTM, all using our two-stage RUL prediction scheme and the same number of input features. These experiments highlight the effectiveness of the ST-MAN architecture for RUL prediction.

In the initial stage of our framework, the goal is to predict the HS of the battery cell at various cycle points and identify the FPC. In our experiments, we labeled the first 10% of the total cycle data for each cell as the healthy state and the last 10% as the unhealthy state, leaving the remaining 80% for FPC prediction. To mitigate overfitting, we performed 5-fold cross-validation. For instance, on the MIT dataset, 99-100 battery cell data were used for the training dataset, and 24-25 battery cell data for the test dataset, iteratively changing the training and test dataset split to include all cells in the test dataset. To pass input of uniform size we use the fixed sliding window of size 50 to train the HS classification model and the RUL prediction model. For training the HS classification model in Stage 1 we show the number of cycles labeled as healthy (initial 10%) and unhealthy (last 10%) in Table 2.

Moreover, we use 7 and 5 input features from the MIT and HUST datasets, respectively. The input features of the MIT dataset include discharge capacity, internal resistance, charge capacity, average temperature, minimum and maximum temperature, and charge time. The HUST dataset input features comprise charge voltage, discharge capacity, charge capacity, the difference of the charge voltage between each cycle and the 10th cycle (ΔVi−10=Vi−V10), and the difference of the charge capacity between each cycle and the 10th cycle (ΔQi−10=Qi−Q10).

For the evaluation and comparison of the proposed framework with conventional methods, three evaluation metrics—MAE, MSE, and MAPE—were employed, defined as follows:(8) MAE=1N∑i=1N|yi−yˆi|

(9) MSE=1N∑i=1N(yi−yˆi)2

(10) MAPE=100%N∑i=0N−1|yi−yˆi|yi.

where yˆi represents the predicted RUL percentage and yi represents the ground truth. These evaluation metrics allow us to quantitatively assess the accuracy and performance of our models in predicting remaining useful life.

4.3 Results and discussion

In Fig. 7, instances of the health state (HS) division outcomes from the initial stage of our proposed scheme are demonstrated using the MIT dataset. The results illustrate a distinct classification between the healthy and unhealthy states. Significantly, the identification of FPCs occurred at precise junctures corresponding to the onset of degradation in both discharge and charge capacity values. To provide further insight into the FPC determination process, we include the percentage of discharge capacity at the FPC point for both datasets in Table 4. This percentage represents the proportion of remaining discharge capacity when the FPC occurs. For the MIT dataset, on average, the FPCs were identified at approximately 94% of the total discharge capacity, while for the HUST dataset, they occurred at roughly 90% of the total discharge capacity. This consistent trend across both datasets demonstrates the robustness and stability of our proposed first-stage approach in pinpointing the FPCs accurately. These findings reinforce the reliability of our method in identifying the critical transition points within battery degradation cycles, which is paramount for precise remaining useful life prediction. Furthermore, the discharge capacity curves before and after the FPC point in the MIT [34] dataset can be visualized in Fig. 8, where the discharge capacity is relatively stable before the FPC point as compared to the discharge capacity after the FPC point.Figure 7 Illustration of first predicted cycle (FPC) decisions based on seven features for three battery cells from the MIT dataset: Discharge capacity, internal resistance, charge capacity, average temperature, minimum and maximum temperature, and charge time. The cells depicted are numbered First 101, Second 102, Third 103.

Figure 7

Table 4 Percentage of Discharge Capacity at FPC.

Table 4Dataset	Train	Test	
MIT [34]	94.70	94.34	
HUST [21]	90.771	86.69	

Figure 8 Discharge Capacities for MIT [34] dataset before and after the FPC.

Figure 8

In Fig. 9, we present a visual comparison of the RUL prediction results achieved by our proposed two-stage framework and conventional schemes [57], [42]. It is worth noting that there is a fundamental difference in how RUL percentage prediction commences between the proposed method and conventional approaches, as depicted in Fig. 9. Specifically, our proposed method initiates RUL percentage prediction after the FPC point, which signifies the starting point of battery degradation. Conversely, conventional methods base their predictions on data covering the first 40% of the overall cycle, making the initial prediction points distinctly dissimilar.Figure 9 RUL percentage prediction, First two rows show RUL prediction on batteries 51 and 56 of MIT dataset, last two rows show RUL prediction on batteries 51 and 56 on HUST dataset.

Figure 9

This disparity in prediction starting points has profound implications for the accuracy of RUL predictions. In the case of conventional methods, which necessitate prior knowledge of the EOL, the predicted RUL exhibits noticeable gaps when compared to the ground truth. Moreover, conventional models struggle to accurately anticipate sudden drops in RUL, often leading to discontinuous predictions. In contrast, our proposed method excels in producing smooth and stable RUL predictions. By commencing RUL estimation after the FPC point, our approach circumvents the need for prior knowledge of the EOL, resulting in more reliable and continuous predictions. This is particularly beneficial in real-world scenarios where exact EOL information may not be readily available.

For a comprehensive evaluation of the RUL prediction models, we provide a detailed comparison in Table 5, utilizing three prominent evaluation metrics. Notably, our proposed method consistently outperforms the two conventional methods across all these metrics. This superior performance underscores the effectiveness of our approach in accurately predicting RUL, all while obviating the need for prior knowledge of the EOL. These results highlight the potential of our two-stage framework in enhancing the precision of RUL prediction in various real-world applications.Table 5 Evaluation of RUL prediction on the MIT and HUST datasets.

Table 5Methods	MIT [34]	HUST [21]	
MAE	MSE	MAPE	MAE	MSE	MAPE	
TCN [57]	0.0451 ± 0.0091	0.0065 ± 0.0046	0.1308 ± 0.0360	0.0226 ± 0.0081	0.0014 ± 0.0015	0.0776 ± 0.0237	
ADLSTM [42]	0.0461 ± 0.0146	0.0072 ± 0.0067	0.1149 ± 0.0477	0.0237 ± 0.0030	0.0006 ± 0.0002	0.0834 ± 0.0148	
ST-MAN (Ours)	0.0275 ± 0.0046	0.0014 ± 0.0005	0.0494 ± 0.0100	0.0168 ± 0.0091	0.0005 ± 0.0006	0.0384 ± 0.0193	


	
Two-stage TCN	0.1034 ± 0.0110	0.0160 ± 0.0027	0.1462 ± 0.0172	0.0939 ± 0.0389	0.0139 ± 0.0093	0.1670 ± 0.0821	
Two-stage LSTM	0.0293 ± 0.0043	0.0015 ± 0.0004	0.0544 ± 0.0077	0.0181 ± 0.0023	0.0006 ± 0.0001	0.0438 ± 0.0048	

Furthermore, we compare the proposed ST-MAN with the TCN and LSTM, all employing our two-stage RUL prediction scheme and the same number of input features. Fig. 10 demonstrates examples of the RUL prediction results on the MIT and HUST datasets by the proposed ST-MAN, two-stage TCN, and two-stage LSTM. Compared to the results obtained by conventional schemes in Fig. 9, the proposed scheme improved the performance of the RUL prediction by the TCN and LSTM models. In Table 5, the proposed ST-MAN also outperformed the two-stage TCN and two-stage LSTM. These experiments demonstrate the effectiveness of the ST-MAN architecture for RUL prediction and its potential to outperform conventional and alternative deep learning models in this critical task.Figure 10 RUL percentage prediction, first two rows show RUL prediction on batteries 8 and 18 of HUST dataset, last two rows show RUL prediction on batteries 8 and 18 on MIT dataset.

Figure 10

In Table 6, we present a comprehensive overview of the computational complexities and performance metrics associated with the proposed method compared to the TCN and LSTM models when employing the two-stage RUL prediction scheme on the MIT dataset. Our assessment of computational costs encompasses both the number of network parameters and floating-point operations (FLOPs), a metric widely used to represent the execution time and overall computational complexity of neural network models [12].Table 6 Comparison of computation complexity and the number of parameters.

Table 6Model	FLOPs	Parameters	
TCN [57]	527,424	46,049	
ADLSTM [42]	1,003,648	190,721	
ST-MAN (Ours)	1,200,869	27,861	

Remarkably, as illustrated in Table 6, our proposed network architecture shows the smallest number of parameters. However, it is noteworthy that the FLOPs associated with the proposed ST-MAN model are relatively larger compared to the TCN and LSTM models. This signifies that our ST-MAN model exhibits a higher computational demand during execution. Nevertheless, the smaller number of weight parameters, despite the increased FLOPs, underscores the model's efficiency and demonstrates that it can achieve robust predictive performance with a leaner set of learnable parameters. This balance between FLOPs and parameters demonstrates the capability of the model to deliver favorable results while maintaining computational efficiency.

4.4 Ablation study

In this section we discuss the ablation study performed to select the best set of parameters for our proposed model. Table 7 investigates the number of parameters required to train the HS Division Classifier for FPC prediction using various hyperparameters. Based on this analysis, we have chosen our HS classifier to consist of 1 LSTM block with 3 hidden layers inside it. This configuration strikes a balance between early FPC detection and the number of parameters, optimizing the performance of our model.Table 7 Ablation Study for HS Division Model by changing the number layers and the number of LSTM blocks in the model.

Table 7Hidden Layers in each block	No of LSTM Blocks	
1	2	
MIT	HUST	MIT	HUST	
Params	Train	Test	Params	Train	Test	Params	Train	Test	Params	Train	Test	
1	65,457	94.6	93.5	52,657	90.1	83.9	85,857	94.5	93.4	73108	92.2	85.6	
2	85,857	94.8	94.1	73,057	92.3	85.7	126,657	96.4	96.6	113908	92.4	85.8	
3	106,257	96.8	98.4	93,457	91.8	85.7	167,457	96.2	98.6	154708	91.4	85.7	
4	126,657	96.8	98.42	113,857	92.4	85.8	208,257	95.4	95.8	195508	91.3	86.8	

Table 8 shows how the RUL prediction performance is impacted after removing certain blocks from the proposed architecture. It can be easily observed that removing any of the subsections in the proposed model provided inferior performance in the performance metrics compared to the complete.Table 8 Ablation study of ST-MAN architecture.

Table 8Methods	MIT [34]	HUST [21]	
MAE	MSE	MAPE	MAE	MSE	MAPE	
Without Initial Convnet	0.1646 ± 0.0115	0.0396 ± 0.0060	0.3180 ± 0.02691	0.2255 ± 0.0235	0.0727 ± 0.0191	0.4329 ± 0.02857	
Without Channel Interaction	0.0500 ± 0.0147	0.0046 ± 0.0046	0.1043 ± 0.0314	0.0471 ± 0.0511	0.0074 ± 0.0132	0.0954 ± 0.1123	
Without Temporal LSTM	0.0358 ± 0.00542	0.0024 ± 0.0008	0.0728 ± 0.0116	0.0176 ± 0.0042	0.0005 ± 0.0002	0.0331 ± 0.0072	
ST-MAN (Ours)	0.0275 ± 0.0046	0.0014 ± 0.0005	0.0494 ± 0.0100	0.0168 ± 0.0091	0.0005 ± 0.0006	0.0384 ± 0.0193	

5 Conclusion

In this study, we proposed a novel two-stage framework for RUL prediction of Lithium-ion batteries, addressing the critical challenge of RUL estimation in real-world scenarios where precise EOL information is often unavailable. Our proposed framework, which combines health state classification and RUL prediction, offers several key contributions and advantages. First, we introduced the concept of the FPC, which serves as a pivotal point to initiate RUL prediction. By identifying the FPC through health state classification, we effectively circumvent the need for prior knowledge of the EOL. This feature is particularly valuable in practical applications, where obtaining exact EOL information can be challenging or impossible. To achieve accurate health state classification and RUL prediction, we designed the ST-MAN, a novel architecture tailored to the unique characteristics of battery degradation data. ST-MAN contains CNNs, Transformer encoders, and LSTM units to capture intricate degradation patterns while maintaining computational efficiency. The use of multi-attention mechanisms and temporal self-attention enables ST-MAN to learn both local and global dependencies in the data, resulting in robust RUL predictions. Our experimental results demonstrate the effectiveness of the proposed framework and ST-MAN architecture. We compared our approach with conventional methods and alternative deep learning models, consistently achieving superior RUL prediction accuracy across multiple evaluation metrics. Notably, our method produces smooth and stable RUL predictions, addressing the challenges of discontinuity and inaccuracy associated with conventional methods.

Furthermore, for the integration of an embedded solution, it is necessary to refine our strategy with a focus on reducing FLOPs in future iterations. In this work, we propose a hierarchical method comprising two stages. In the first stage, the healthy and unhealthy stages of the battery are determined. Subsequently, the second stage is activated when the battery status is confirmed as unhealthy. Both stages utilize high-performance LSTM-based models, which are computationally expensive. To mitigate the computational burden, it is recommended to revise the model architecture for the FPC detection (First stage) by transitioning to a CNN-based model, for example, employing knowledge distillation techniques. In the case of battery monitoring in embedded devices, only the first stage of our algorithm will be deployed. This decision is based on two main considerations: 1. When the battery of the embedded device is identified as unhealthy, the degradation of the device performance makes it an unreliable source for subsequent calculations, hence the second stage must be performed on an external device, such as the cloud. 2. By deploying only the first stage of our algorithm, the computational burden for the embedded device will be considerably alleviated, thereby improving its operational efficiency and resource utilization.

Future work may involve further refinement of the framework, exploration of additional datasets, and deployment in real-world battery management systems. By advancing the field of battery health management and extending the lifespan of energy storage technologies, our research has the potential to revolutionize various industries, including the automotive and renewable energy sectors.

CRediT authorship contribution statement

Sungho Suh: Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing. Dhruv Aditya Mittal: Data curation, Formal analysis, Software, Visualization, Writing – original draft. Hymalai Bello: Formal analysis, Investigation, Validation, Visualization, Writing – original draft, Writing – review & editing. Bo Zhou: Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing. Mayank Shekhar Jha: Conceptualization, Validation, Writing – review & editing. Paul Lukowicz: Funding acquisition, Project administration, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work was supported by Carl-Zeiss Stiftung under the Sustainable Embedded AI project (P2021-02-009 ) and by the 10.13039/501100000780 European Union under the HumanE AI Network (H2020-ICT-2019-3 #952026 ).
==== Refs
References

1 Attia P.M. Bills A. Planella F.B. Dechent P. Dos Reis G. Dubarry M. Gasper P. Gilchrist R. Greenbank S. Howey D. “Knees” in lithium-ion battery aging trajectories J. Electrochem. Soc. 169 2022 060517
2 Birkl C.R. Roberts M.R. McTurk E. Bruce P.G. Howey D.A. Degradation diagnostics for lithium ion cells J. Power Sources 341 2017 373 386
3 Chen M. Ma G. Liu W. Zeng N. Luo X. An overview of data-driven battery health estimation technology for battery management system Neurocomputing 2023
4 Cui S. Shin J. Woo H. Hong S. Joe I. State-of-health estimation of lithium-ion batteries with attention-based deep learning Software Engineering Perspectives in Intelligent Systems: Proceedings of 4th Computational Methods in Systems and Software 2020 vol. 24 2020 Springer 322 331
5 Dos Reis G. Strange C. Yadav M. Li S. Lithium-ion battery data and where to find it Energy AI 5 2021 100081
6 Fermín-Cueto P. McTurk E. Allerhand M. Medina-Lopez E. Anjos M.F. Sylvester J. Dos Reis G. Identification and machine learning prediction of knee-point and knee-onset in capacity degradation curves of lithium-ion cells Energy AI 1 2020 100006
7 Hannan M. How D.N. Mansor M. Lipu M.H. Ker P.J. Muttaqi K.M. State-of-charge estimation of li-ion battery at variable ambient temperature with gated recurrent unit network 2020 IEEE Industry Applications Society Annual Meeting 2020 IEEE 1 8
8 Heidari A.A. Abbaspour R.A. Chen H. Efficient boosted grey wolf optimizers for global search and kernel extreme learning machine training Appl. Soft Comput. 81 2019 105521
9 Hong J. Wang Z. Chen W. Yao Y. Synchronous multi-parameter prediction of battery systems on electric vehicles using long short-term memory networks Appl. Energy 254 2019 113648
10 Huang Z. Yang F. Xu F. Song X. Tsui K.L. Convolutional gated recurrent unit–recurrent neural network for state-of-charge estimation of lithium-ion batteries IEEE Access 7 2019 93139 93149
11 Jiang B. Gent W.E. Mohr F. Das S. Berliner M.D. Forsuelo M. Zhao H. Attia P.M. Grover A. Herring P.K. Bayesian learning for rapid prediction of lithium-ion battery-cycling protocols Joule 5 2021 3187 3203
12 Jouppi N.P. Young C. Patil N. Patterson D. Agrawal G. Bajwa R. Bates S. Bhatia S. Boden N. Borchers A. In-datacenter performance analysis of a tensor processing unit Proceedings of the 44th Annual International Symposium on Computer Architecture 2017 1 12
13 Kouhestani H.S. Liu L. Wang R. Chandra A. Data-driven prognosis of failure detection and prediction of lithium-ion batteries J. Energy Storage 70 2023 108045
14 Lei Y. Li N. Guo L. Li N. Yan T. Lin J. Machinery health prognostics: a systematic review from data acquisition to rul prediction Mech. Syst. Signal Process. 104 2018 799 834
15 Li W. Sengupta N. Dechent P. Howey D. Annaswamy A. Sauer D.U. One-shot battery degradation trajectory prediction with deep learning J. Power Sources 506 2021 230024
16 Li X. Zhang L. Wang Z. Dong P. Remaining useful life prediction for lithium-ion batteries based on a hybrid model combining the long short-term memory and Elman neural networks J. Energy Storage 21 2019 510 518
17 Liu R. Remaining useful life prediction of lithium-ion batteries using multiple kernel extreme learning machine Recent Adv. Comput. Sci. Commun. (Formerly: Recent Patents on Computer Science) 15 2022 715 721
18 Liu R. Yang B. Hauptmann A.G. Simultaneous bearing fault recognition and remaining useful life prediction using joint-loss convolutional neural network IEEE Trans. Ind. Inform. 16 2019 87 96
19 Liu Y. Zhao G. Peng X. Deep learning prognostics for lithium-ion battery based on ensembled long short-term memory networks IEEE Access 7 2019 155130 155142
20 Ma G. Wang Z. Liu W. Fang J. Zhang Y. Ding H. Yuan Y. A two-stage integrated method for early prediction of remaining useful life of lithium-ion batteries Knowl.-Based Syst. 259 2023 110012
21 Ma G. Xu S. Jiang B. Cheng C. Yang X. Shen Y. Yang T. Huang Y. Ding H. Yuan Y. Real-time personalized health status prediction of lithium-ion batteries using deep transfer learning Energy Environ. Sci. 15 2022 4083 4094
22 Ma G. Zhang Y. Cheng C. Zhou B. Hu P. Yuan Y. Remaining useful life prediction of lithium-ion batteries based on false nearest neighbors and a hybrid neural network Appl. Energy 253 2019 113626
23 Ma Y. Wu L. Guan Y. Peng Z. The capacity estimation and cycle life prediction of lithium-ion batteries using a new broad extreme learning machine approach J. Power Sources 476 2020 228581
24 Miao Q. Xie L. Cui H. Liang W. Pecht M. Remaining useful life prediction of lithium-ion battery with unscented particle filter technique Microelectron. Reliab. 53 2013 805 810
25 Mittal D.A. Bello H. Zhou B. Jha M.S. Suh S. Lukowicz P. Two-stage early prediction framework of remaining useful life for lithium-ion batteries IECON 2023-49th Annual Conference of the IEEE Industrial Electronics Society 2023 IEEE 1 7
26 Ng S.S. Xing Y. Tsui K.L. A naive Bayes model for robust remaining useful life prediction of lithium-ion battery Appl. Energy 118 2014 114 123
27 Paulson N.H. Kubal J. Ward L. Saxena S. Lu W. Babinec S.J. Feature engineering for machine learning enabled early prediction of battery lifetime J. Power Sources 527 2022 231127
28 Qu Z. Zhu L. Ma S. Zhang B. Remaining useful life prediction of high-speed railroad contact network based on stacking integrated attention-lstm-cnn deep learning Arab. J. Sci. Eng. 2024 1 18
29 Ren X. Liu S. Yu X. Dong X. A method for state-of-charge estimation of lithium-ion batteries based on pso-lstm Energy 234 2021 121236
30 Richardson R.R. Osborne M.A. Howey D.A. Battery health prediction under generalized conditions using a Gaussian process transition model J. Energy Storage 23 2019 320 328
31 Salkind A.J. Fennie C. Singh P. Atwater T. Reisner D.E. Determination of state-of-charge and state-of-health of batteries by fuzzy logic methodology J. Power Sources 80 1999 293 300
32 Saxena A. Celaya J. Balaban E. Goebel K. Saha B. Saha S. Schwabacher M. Metrics for evaluating performance of prognostic techniques 2008 International Conference on Prognostics and Health Management 2008 IEEE 1 17
33 Saxena S. Ward L. Kubal J. Lu W. Babinec S. Paulson N. A convolutional neural network model for battery capacity fade curve prediction using early life data J. Power Sources 542 2022 231736
34 Severson K.A. Attia P.M. Jin N. Perkins N. Jiang B. Yang Z. Chen M.H. Aykol M. Herring P.K. Fraggedakis D. Data-driven prediction of battery cycle life before capacity degradation Nat. Energy 4 2019 383 391
35 Singh A. Feltner C. Peck J. Kuhn K.I. Data driven prediction of battery cycle life before capacity degradation arXiv preprint arXiv:2110.09687 2021
36 Song J.W. Park Y.I. Hong J.J. Kim S.G. Kang S.J. Attention-based bidirectional lstm-cnn model for remaining useful life estimation 2021 IEEE International Symposium on Circuits and Systems (ISCAS) 2021 IEEE 1 5
37 Song Y. Li L. Peng Y. Liu D. Lithium-ion battery remaining useful life prediction based on gru-rnn 2018 12th International Conference on Reliability, Maintainability, and Safety (ICRMS) 2018 IEEE 317 322
38 Strange C. Ibraheem R. dos Reis G. Online lifetime prediction for lithium-ion batteries with cycle-by-cycle updates, variance reduction, and model ensembling Energies 16 2023 3273
39 Suh S. Jang J. Won S. Jha M.S. Lee Y.O. Supervised health stage prediction using convolutional neural networks for bearing wear Sensors 20 2020 5846 33081097
40 Suh S. Lukowicz P. Lee Y.O. Generalized multiscale feature extraction for remaining useful life prediction of bearings with generative adversarial networks Knowl.-Based Syst. 237 2022 107866
41 Tian J. Xu R. Wang Y. Chen Z. Capacity attenuation mechanism modeling and health assessment of lithium-ion batteries Energy 221 2021 119682
42 Tong Z. Miao J. Tong S. Lu Y. Early prediction of remaining useful life for lithium-ion batteries based on a hybrid machine learning method J. Clean. Prod. 317 2021 128265
43 Wang F.K. Amogne Z.E. Chou J.H. Tseng C. Online remaining useful life prediction of lithium-ion batteries using bidirectional long short-term memory with attention mechanism Energy 254 2022 124344
44 Wang S. Jin S. Bai D. Fan Y. Shi H. Fernandez C. A critical review of improved deep learning methods for the remaining useful life prediction of lithium-ion batteries Energy Rep. 7 2021 5562 5574
45 Xu X. Li X. Ming W. Chen M. A novel multi-scale cnn and attention mechanism method with multi-sensor signal for remaining useful life prediction Comput. Ind. Eng. 169 2022 108204
46 Yang H. Wang P. An Y. Shi C. Sun X. Wang K. Zhang X. Wei T. Ma Y. Remaining useful life prediction based on denoising technique and deep neural network for lithium-ion capacitors ETransportation 5 2020 100078
47 Yao J. Powell K. Gao T. A two-stage deep learning framework for early-stage lifetime prediction for lithium-ion batteries with consideration of features from multiple cycles Front. Energy Res. 10 2022 1059126
48 Zhang L. Wang B. Yuan X. Liang P. Remaining useful life prediction via improved cnn, gru and residual attention mechanism with soft thresholding IEEE Sens. J. 22 2022 15178 15190
49 Zhang Y. Ma H. Wang S. Li S. Guo R. Indirect prediction of remaining useful life for lithium-ion batteries based on improved multiple kernel extreme learning machine J. Energy Storage 64 2023 107181
50 Zhang Y. Tang Q. Zhang Y. Wang J. Stimming U. Lee A.A. Identifying degradation patterns of lithium ion batteries from impedance spectroscopy using machine learning Nat. Commun. 11 2020 1706 32249782
51 Zhang Y. Tu L. Xue Z. Li S. Tian L. Zheng X. Weight optimized unscented Kalman filter for degradation trend prediction of lithium-ion battery with error compensation strategy Energy 251 2022 123890
52 Zhang Y. Xiong R. He H. Liu Z. A lstm-rnn method for the lithuim-ion battery remaining useful life prediction 2017 Prognostics and System Health Management Conference (PHM-Harbin) 2017 IEEE 1 4
53 Zhang Y. Xiong R. He H. Pecht M. Validation and verification of a hybrid method for remaining useful life prediction of lithium-ion batteries J. Clean. Prod. 212 2019 240 249
54 Zhang Y. Xiong R. He H. Pecht M.G. Long short-term memory recurrent neural network for remaining useful life prediction of lithium-ion batteries IEEE Trans. Veh. Technol. 67 2018 5695 5705
55 Zhao F. Li Y. Wang X. Bai L. Liu T. Lithium-ion batteries state of charge prediction of electric vehicles using rnns-cnns neural networks IEEE Access 8 2020 98168 98180
56 Zhao Q. Qin X. Zhao H. Feng W. A novel prediction method based on the support vector regression for the remaining useful life of lithium-ion batteries Microelectron. Reliab. 85 2018 99 108
57 Zhou D. Li Z. Zhu J. Zhang H. Hou L. State of health monitoring and remaining useful life prediction of lithium-ion batteries based on temporal convolutional network IEEE Access 8 2020 53307 53320
