|
|
A MLP-Mixer and mixture of expert model for remaining useful life prediction of lithium-ion batteries |
Lingling ZHAO1, Shitao SONG2, Pengyan WANG3, Chunyu WANG1, Junjie WANG4(), Maozu GUO5() |
1. Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China 2. School of Electrical Engineering, Liaoning University of Technology, Jinzhou 121004, China 3. School of Computer Science, Northeast Electric Power University, Jilin 132000, China 4. Department of Medical Informatics, School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing 211166, China 5. School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, Beijing 100044, China |
|
|
Abstract Accurately predicting the Remaining Useful Life (RUL) of lithium-ion batteries is crucial for battery management systems. Deep learning-based methods have been shown to be effective in predicting RUL by leveraging battery capacity time series data. However, the representation learning of features such as long-distance sequence dependencies and mutations in capacity time series still needs to be improved. To address this challenge, this paper proposes a novel deep learning model, the MLP-Mixer and Mixture of Expert (MMMe) model, for RUL prediction. The MMMe model leverages the Gated Recurrent Unit and Multi-Head Attention mechanism to encode the sequential data of battery capacity to capture the temporal features and a re-zero MLP-Mixer model to capture the high-level features. Additionally, we devise an ensemble predictor based on a Mixture-of-Experts (MoE) architecture to generate reliable RUL predictions. The experimental results on public datasets demonstrate that our proposed model significantly outperforms other existing methods, providing more reliable and precise RUL predictions while also accurately tracking the capacity degradation process. Our code and dataset are available at the website of github.
|
Keywords
lithium-ion battery
remaining useful life
deep learning
MLP-Mixer
mixture-of-experts
|
Corresponding Author(s):
Junjie WANG,Maozu GUO
|
Just Accepted Date: 20 July 2023
Issue Date: 21 September 2023
|
|
1 |
X, Tang K, Liu K, Li W D, Widanage E, Kendrick F Gao . Recovering large-scale battery aging dataset with machine learning. Patterns, 2021, 2( 8): 100302
|
2 |
Z, Wang N, Liu Y Guo . Adaptive sliding window LSTM NN based RUL prediction for lithium-ion batteries integrating LTSA feature reconstruction. Neurocomputing, 2021, 466: 178–189
|
3 |
M F, Ge Y, Liu X, Jiang J Liu . A review on state of health estimations and remaining useful life prognostics of lithium-ion batteries. Measurement, 2021, 174: 109057
|
4 |
H, Rauf M, Khalid N Arshad . Machine learning in state of health and remaining useful life estimation: theoretical and technological development in battery degradation modelling. Renewable and Sustainable Energy Reviews, 2022, 156: 111903
|
5 |
Q, Zhai Z S Ye . RUL prediction of deteriorating products using an adaptive wiener process model. IEEE Transactions on Industrial Informatics, 2017, 13( 6): 2911–2921
|
6 |
Y, Wang J, Tian Z, Sun L, Wang R, Xu M, Li Z Chen . A comprehensive review of battery modeling and state estimation approaches for advanced battery management systems. Renewable and Sustainable Energy Reviews, 2020, 131: 110015
|
7 |
Z, Deng L, Yang H, Deng Y, Cai D Li . Polynomial approximation pseudo-two-dimensional battery model for online application in embedded battery management system. Energy, 2018, 142: 838–850
|
8 |
L, Yang Y, Cai Y, Yang Z Deng . Supervisory long-term prediction of state of available power for lithium-ion batteries in electric vehicles. Applied Energy, 2020, 257: 114006
|
9 |
J, Son S, Zhou C, Sankavaram X, Du Y Zhang . Remaining useful life prediction based on noisy condition monitoring signals using constrained Kalman filter. Reliability Engineering & System Safety, 2016, 152: 38–50
|
10 |
X, Su S, Wang M, Pecht L, Zhao Z Ye . Interacting multiple model particle filter for prognostics of lithium-ion batteries. Microelectronics Reliability, 2017, 70: 59–69
|
11 |
J, Tian R, Xu Y, Wang Z Chen . Capacity attenuation mechanism modeling and health assessment of lithium-ion batteries. Energy, 2021, 221: 119682
|
12 |
Y, Li K, Liu A M, Foley A, Zülke M, Berecibar E, Nanini-Maury Mierlo J, Van H E Hoster . Data-driven health estimation and lifetime prediction of lithium-ion batteries: a review. Renewable and Sustainable Energy Reviews, 2019, 113: 109254
|
13 |
S, Li H, Fang B Shi . Remaining useful life estimation of lithium-ion battery based on interacting multiple model particle filter and support vector regression. Reliability Engineering & System Safety, 2021, 210: 107542
|
14 |
X, Shu G, Li J, Shen Z, Lei Z, Chen Y Liu . A uniform estimation framework for state of health of lithium-ion batteries considering feature extraction and parameters optimization. Energy, 2020, 204: 117957
|
15 |
Z, Liu Y, Cheng P, Wang Y, Yu Y Long . A method for remaining useful life prediction of crystal oscillators using the Bayesian approach and extreme learning machine under uncertainty. Neurocomputing, 2018, 305: 27–38
|
16 |
D, Shen L, Wu G, Kang Y, Guan Z Peng . A novel online method for predicting the remaining useful life of lithium-ion batteries considering random variable discharge current. Energy, 2021, 218: 119490
|
17 |
B, Yang R, Liu E Zio . Remaining useful life prediction based on a double-convolutional neural network architecture. IEEE Transactions on Industrial Electronics, 2019, 66( 12): 9521–9530
|
18 |
P, Ding X, Liu H, Li Z, Huang K, Zhang L, Shao O Abedinia . Useful life prediction based on wavelet packet decomposition and two-dimensional convolutional neural network for lithium-ion batteries. Renewable and Sustainable Energy Reviews, 2021, 148: 111287
|
19 |
Y, Zhang R, Xiong H, He M G Pecht . Long short-term memory recurrent neural network for remaining useful life prediction of lithium-ion batteries. IEEE Transactions on Vehicular Technology, 2018, 67( 7): 5695–5705
|
20 |
S, Zhao C, Zhang Y Wang . Lithium-ion battery capacity and remaining useful life prediction using board learning system and long short-term memory neural network. Journal of Energy Storage, 2022, 52: 104901
|
21 |
D, Chen W, Hong X Zhou . Transformer network for remaining useful life prediction of lithium-ion batteries. IEEE Access, 2022, 10: 19621–19628
|
22 |
L, Zheng Y, He X, Chen X Pu . Optimization of dilated convolution networks with application in remaining useful life prediction of induction motors. Measurement, 2022, 200: 111588
|
23 |
M, Ragab Z, Chen M, Wu C K, Kwoh R, Yan X Li . Attention-based sequence to sequence model for machine remaining useful life prediction. Neurocomputing, 2021, 466: 58–68
|
24 |
J Y, Wu M, Wu Z, Chen X L, Li R Yan . Degradation-aware remaining useful life prediction with LSTM autoencoder. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1–10
|
25 |
R, Jin Z, Chen K, Wu M, Wu X, Li R Yan . Bi-LSTM-based two-stream network for machine remaining useful life prediction. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1–10
|
26 |
L, Xiao L, Zhang F, Niu X, Su W Song . RETRACTED: remaining useful life prediction of wind turbine generator based on 1D-CNN and Bi-LSTM. International Journal of Fatigue, 2022, 163: 107051
|
27 |
Ardeshiri R, Rouhi C Ma . Multivariate gated recurrent unit for battery remaining useful life prediction: a deep learning approach. International Journal of Energy Research, 2021, 45( 11): 16633–16648
|
28 |
Z, Chen M, Wu R, Zhao F, Guretno R, Yan X Li . Machine remaining useful life prediction via an attention-based deep learning approach. IEEE Transactions on Industrial Electronics, 2021, 68( 3): 2521–2531
|
29 |
I O, Tolstikhin N, Houlsby A, Kolesnikov L, Beyer X, Zhai T, Unterthiner J, Yung A, Steiner D, Keysers J, Uszkoreit M, Lucic A Dosovitskiy . MLP-Mixer: an all-MLP architecture for vision. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021
|
30 |
H, Touvron P, Bojanowski M, Caron M, Cord A, El-Nouby E, Grave G, Izacard A, Joulin G, Synnaeve J, Verbeek H Jegou . ResMLP: feedforward networks for image classification with data-efficient training. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45( 4): 5314–5321
|
31 |
Chen S, Xie E, Ge C, Chen R, Liang D, Luo P. CycleMLP: a MLP-like architecture for dense prediction. In: Proceedings of the 10th International Conference on Learning Representations. 2022
|
32 |
T, Yu X, Li Y, Cai M, Sun P Li . S2-MLP: spatial-shift MLP architecture for vision. In: Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2022, 3615−3624
|
33 |
R A, Jacobs M I, Jordan S J, Nowlan G E Hinton . Adaptive mixtures of local experts. Neural Computation, 1991, 3( 1): 79–87
|
34 |
H, Liu Z, Liu W, Jia X Lin . Remaining useful life prediction using a novel feature-attention-based end-to-end approach. IEEE Transactions on Industrial Informatics, 2021, 17( 2): 1197–1207
|
35 |
A, Vaswani N, Shazeer N, Parmar J, Uszkoreit L, Jones A N, Gomez Ł, Kaiser I Polosukhin . Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010
|
36 |
Y, Wu W, Li Y, Wang K Zhang . Remaining useful life prediction of lithium-ion batteries using neural network and bat-based particle filter. IEEE Access, 2019, 7: 54843–54854
|
37 |
J, Liu A, Saxena K, Goebel B, Saha W Wang . An adaptive recurrent neural network for remaining useful life prediction of lithium-ion batteries. In: Proceedings of Annual Conference of the Prognostics and Health Management Society. 2010
|
38 |
N, Williard W, He M, Osterman M Pecht . Comparative analysis of features for determining state of health in lithium-ion batteries. International Journal of Prognostics and Health Management, 2013, 4(1): 1−7
|
39 |
Z, Shi A Chehade . A dual-LSTM framework combining change point detection and remaining useful life prediction. Reliability Engineering & System Safety, 2021, 205: 107257
|
40 |
V M, Nagulapati H, Lee D, Jung B, Brigljevic Y, Choi H Lim . Capacity estimation of batteries: influence of training dataset size and diversity on data driven prognostic models. Reliability Engineering & System Safety, 2021, 216: 108048
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|