A hybrid Wavelet-CNN-LSTM deep learning model for short-term urban water demand forecasting
Zhengheng Pu1,2, Jieru Yan1,2, Lei Chen1,2, Zhirong Li1,2, Wenchong Tian1,2, Tao Tao1,2, Kunlun Xin1,2()
1. College of Environmental Science and Engineering, Tongji University, Shanghai 200092, China 2. Smart Water Joint Innovation RD Center, Tongji University, Shanghai 200092, China
● A novel deep learning framework for short-term water demand forecasting.
● Model prediction accuracy outperforms other traditional deep learning models.
● Wavelet multi-resolution analysis automatically extracts key water demand features.
● An analysis is performed to explain the improved mechanism of the proposed method.
Short-term water demand forecasting provides guidance on real-time water allocation in the water supply network, which help water utilities reduce energy cost and avoid potential accidents. Although a variety of methods have been proposed to improve forecast accuracy, it is still difficult for statistical models to learn the periodic patterns due to the chaotic nature of the water demand data with high temporal resolution. To overcome this issue from the perspective of improving data predictability, we proposed a hybrid Wavelet-CNN-LSTM model, that combines time-frequency decomposition characteristics of Wavelet Multi-Resolution Analysis (MRA) and implement it into an advanced deep learning model, CNN-LSTM. Four models - ANN, Conv1D, LSTM, GRUN - are used to compare with Wavelet-CNN-LSTM, and the results show that Wavelet-CNN-LSTM outperforms the other models both in single-step and multi-steps prediction. Besides, further mechanistic analysis revealed that MRA produce significant effect on improving model accuracy.
ranging from 0.01 to 0.0005 with a decrease of 0.0005
0.0005
Batch size
[50,100,150,200]
100
ANN
Number of layers
[2,3,4]
3
Number of Nodes(1st)
[48,32,24]
32
Number of Nodes(2nd)
[16,8,4]
8
Conv1D1
Number of layers
[2,3,4]
3
Number of Nodes(1st)
[4,8,16]
16
Kernel size
[3,5,7]
3
Number of Nodes(dense)
[64,32,16]
32
LSTM2
Number of Nodes
[50,49,48,47,46,32,24]
50
Wavelet-CNN-LSTM
Number of layers
[2,3,4]
2
Number of Nodes(1st)
[4,8,16]
16
Kernel size
[3,5,7]
3
Number of Nodes (LSTM layer)
[50,49,48,47,46,32,24]
50
Tab.3
Model
Prediction step
R2
MAPE (%)
MAE (m3/h)
RMSE (m3/h)
ANN
15 min
0.79
2.59
133.71
182.24
1 h
0.63
4.08
210.74
276.01
Conv1D
15 min
0.71
3.05
154.01
197.68
1 h
0.53
4.67
246.85
310.23
LSTM
15 min
0.85
2.32
118.31
164.89
1 h
0.62
4.49
231.79
304.13
GRUN
15 min
0.86
2.39
122.72
170.71
1 h
0.61
4.31
223.54
294.3
Wavelet-CNN-LSTM
15 min
0.93
1.25
62.86
79.23
1 h
0.71
2.97
150.28
201.13
Tab.4
Fig.5
Fig.6
Fig.7
Fig.8
Fig.9
Fig.10
1
J F Adamowski. (2008). Peak daily water demand forecast modeling using artificial neural networks. Journal of Water Resources Planning and Management, 134( 2): 119– 128 https://doi.org/10.1061/(ASCE)0733-9496(2008)134:2(119
2
S I Amari, S Wu. (1999). Improving support vector machine classifiers by modifying kernel functions. Neural networks: the official journal of the International Neural Network Society or Neural Netw, 12( 6): 783– 789 https://doi.org/10.1016/S0893-6080(99)00032-5
pmid: 12662656
R B Billings, C V Jones ( 2011). Forecasting Urban Water Demand. Washington, DC: America Water Works Association
5
A Boggess, F J Narcowich. (2015). A first course in wavelets with Fourier analysis. 2nd ed. John Wiley & Sons, 183– 217
6
J Bougadis, K Adamowski, R Diduch. (2005). Short-term municipal water demand forecasting. Hydrological Processes, 19( 1): 137– 148 https://doi.org/10.1002/hyp.5763
A Candelieri, I Giordani, F Archetti, K Barkalov, I Meyerov, A Polovinkin, A Sysoyev, N Zolotykh. (2019). Tuning hyperparameters of a SVM-based water demand forecasting system through parallel global optimization. Computers & Operations Research, 106 : 202– 209 https://doi.org/10.1016/j.cor.2018.01.013
9
J Cao, J Wang. (2019). Stock price forecasting model based on modified convolution neural network and financial time series analysis. International Journal of Communication Systems, 32( 12): e3987 https://doi.org/10.1002/dac.3987
10
G Chen, T Long, J Xiong, Y Bai. (2017). Multiple random forests modelling for urban water consumption forecasting. Water Resources Management, 31( 15): 4715– 4729 https://doi.org/10.1007/s11269-017-1774-7
J Cheng, Y Liu, Y Ma. (2020). Protein secondary structure prediction based on integration of CNN and LSTM model. Journal of Visual Communication and Image Representation, 71 : 102844 https://doi.org/10.1016/j.jvcir.2020.102844
13
S Dara P Tumma ( 2018) . Feature extraction by using deep learning: a survey. In: Second International Conference on Electronics 2018, Communication and Aerospace Technology (ICECA), IEEE, Coimbatore, RVS Technical Campus, Coimbatore, India, 1795– 1801
14
I Duerr, H R Merrill, C Wang, R Bai, M Boyer, M D Dukes, N Bliznyuk. (2018). Forecasting urban household water demand with statistical and machine learning methods using large space-time data: a comparative study. Environmental Modelling & Software, 102 : 29– 38 https://doi.org/10.1016/j.envsoft.2018.01.002
15
M Firat M E Turan M A (2010) Yurdusev. Comparative analysis of neural network techniques for predicting water consumption time series. Journal of Hydrology (Amsterdam), 384(1-2): 46– 51
16
M Ghiassi, D K Zimbra, H Saidane. (2008). Urban water demand forecasting with a dynamic artificial neural network model. Journal of Water Resources Planning and Management, 134( 2): 138– 146 https://doi.org/10.1061/(ASCE)0733-9496(2008)134:2(138
17
C Goodchild. (2003). Modelling the impact of climate change on domestic water demand. Water and Environment Journal: the Journal/the Chartered Institution of Water and Environmental Management, 17( 1): 8– 12 https://doi.org/10.1111/j.1747-6593.2003.tb00423.x
18
G Guo, S Liu, Y Wu, J Li, R Zhou, X Zhu. (2018). Short-term water demand forecast based on deep learning method. Journal of Water Resources Planning and Management, 144( 12): 04018076 https://doi.org/10.1061/(ASCE)WR.1943-5452.0000992
19
G Hafeez, K S Alimgeer, I Khan. (2020). Electric load forecasting based on deep learning and optimized by heuristic algorithm in smart grid. Applied Energy, 269 : 114915 https://doi.org/10.1016/j.apenergy.2020.114915
20
M Herrera, L Torgo, J Izquierdo, R P’erez-Garc’ıa. (2010). Predictive models for forecasting hourly urban water demand. Journal of Hydrology (Amsterdam), 387( 1–2): 141– 150 https://doi.org/10.1016/j.jhydrol.2010.04.005
21
S Hochreiter ( 1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-based Systems, 6(2): 107– 116
P Hu J Tong J Wang Y Yang L de Oliveira Turci ( 2019). A hybrid model on CNN and bi-LSTM for urban water demand prediction. In: 2019 IEEE Congress on evolutionary computation (CEC), Wellington, New Zealand, 1088– 1094
24
C W Huang C T Chiang Q Li ( 2017). A study of deep learning networks on mobile traffic forecasting. In: IEEE 28th annual international 2017 symposium on personal, indoor, and mobile radio communications (PIMRC), Montreal, Quebec, Canada , 1– 6
G Kurata, B Ramabhadran, G Saon, A Sethy ( 2017). Language modeling with highway lstm. In: IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) 2017, Okinawa Japan, 244– 251
L Mu, F Zheng, R Tao, Q Zhang, Z Kapelan. (2020). Hourly and daily urban water demand predictions using a long short-term memory-based model. Journal of Water Resources Planning and Management, 146( 9): 05020017 https://doi.org/10.1061/(ASCE)WR.1943-5452.0001276
29
H J Nussbaumer 1981 The Fast fourier transform. In: Fast Fourier Transform and Convolution Algorithms, 80– 111
30
G H Oliveira R C Cavalcante G G Cabral L L Minku A L Oliveira ( 2017). series forecasting in the presence of concept drift: A pso-based approach. In: IEEE 29th International Conference on Tools with Acritical Intelligence (ICTAI), Boston, USA, 239– 246
31
S Pincus ( 1995). Approximate entropy (apen) as a complexity measure−Chaos: an interdisciplinary Journal of Nonlinear Science, 5( 1): 110– 117
32
C Peña-Guzmán, J Melgarejo, D Prats. (2016). Forecasting water demand in residential, commercial, and industrial zones in Bogotá, Colombia, using least-squares support vector machines. Mathematical Problems in Engineering, 2016
33
T N Sainath O Vinyals A Senior H Sak( 2015). Convolutional, long short-term memory fully connected deep neural networks. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, Brisbane, Queensland, Australia, 4580– 4584
34
S Sharma, S Sharma, A Athaiya. (2017). Activation functions in neural net-works. Towards Data Science, 6( 12): 310– 316
35
A Sherstinsky. (2020). Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D. Nonlinear Phenomena, 404 : 132306 https://doi.org/10.1016/j.physd.2019.132306
36
K Simonyan A Zisserman ( 2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556
37
S K Sønderby O Winther ( 2014). Protein secondary structure prediction with long short-term memory networks. arXiv preprint arXiv:14127828
38
X Song, Y Liu, L Xue, J Wang, J Zhang, J Wang, L Jiang, Z Cheng. (2020). Time-series well performance prediction based on long short-term memory (LSTM) neural network model. Journal of Petroleum Science Engineering, 186 : 106682 https://doi.org/10.1016/j.petrol.2019.106682
39
Sundermeyer M, Ney H, Schlüter R (2015). From feedforward to recurrent LSTM neural networks for language modeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23(3): 517−529
40
J F Torres, A M Fernández, A Troncoso, F Martínez-Álvarez. (2017). Deep learning-based approach for time series forecasting with application to electricity load. In: International Work-Conference on the Interplay between Natural and Artificial Computation. Berlin: Springer, 203– 212
41
Ullah A, Ahmad J, Muhammad K, Sajjad M, Baik S W (2018). Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access: Practical Innovations, Open Solutions, 6: 1155−1166
42
M Xenochristou Z Kapelan C Hutton J Hofman ( 2017). Identifying relationships between weather variables and domestic water consumption using smart metering. Proceedings of the CCWI
43
J Yang J Li ( 2017). Application of deep convolution neural network. In: 14th International Computer Conference on Wavelet Active Media and Technology Information Processing 2017 (ICCWAMTIP), University of Electronic Science And Technology of China, China, 229– 232
44
B Yu H Yin Z Zhu ( 2017a). Spatio-temporal graph convolutional net- works: a deep learning framework for traffic forecasting. arXiv preprint arXiv:170904875
45
R Yu Y Li C Shahabi U Demiryurek Y Liu ( 2017b). Deep learning: A generic approach for extreme condition traffic forecasting. In: Proceedings of the 2017 SIAM international Conference on Data Mining, Houston, Texas, USA, 777– 785
46
H Zen ( 2015). Acoustic modeling in statistical parametric speech synthesis-from HMM to LSTM-RNN: Conference: RTTH Summer School on Speech Technology, A Deep Learning Perspective, Barcelona, Spain
47
G P Zhang. (2001). An investigation of neural networks for linear time-series forecasting. Computers & Operations Research, 28( 12): 1183– 1202 https://doi.org/10.1016/S0305-0548(00)00033-2
48
S L Zhou, T A McMahon, A Walton, J Lewis. (2000). Forecasting daily urban water demand: a case study of Melbourne. Journal of Hydrology (Amsterdam), 236( 3–4): 153– 164 https://doi.org/10.1016/S0022-1694(00)00287-0
