Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

邮发代号 80-970

2019 Impact Factor: 1.275

Frontiers of Computer Science  2023, Vol. 17 Issue (1): 171601   https://doi.org/10.1007/s11704-021-1080-7
  本期目录
Effective ensemble learning approach for SST field prediction using attention-based PredRNN
Baiyou QIAO1,2(), Zhongqiang WU1, Ling MA1, Yicheng Zhou1, Yunjiao SUN1
1. School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
2. Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang 110169, China
 全文: PDF(4736 KB)   HTML
Abstract

Accurate prediction of sea surface temperature (SST) is extremely important for forecasting oceanic environmental events and for ocean studies. However, the existing SST prediction methods do not consider the seasonal periodicity and abnormal fluctuation characteristics of SST or the importance of historical SST data from different times; thus, these methods suffer from low prediction accuracy. To solve this problem, we comprehensively consider the effects of seasonal periodicity and abnormal fluctuation characteristics of SST data, as well as the influence of historical data in different periods, on prediction accuracy. We propose a novel ensemble learning approach that combines the Predictive Recurrent Neural Network(PredRNN) network and an attention mechanism for effective SST field prediction. In this approach, the XGBoost model is used to learn the long-period fluctuation law of SST and to extract seasonal periodic features from SST data. The exponential smoothing method is used to mitigate the impact of severely abnormal SST fluctuations and extract the a priori features of SST data. The outputs of the two aforementioned models and the original SST data are stacked and used as inputs for the next model, the PredRNN network. PredRNN is the most recently developed spatiotemporal deep learning network, which simulates both spatial and temporal representations and is capable of transferring memory across layers and time steps. Therefore, we used it to extract the spatiotemporal correlations of SST data and predict future SSTs. Finally, an attention mechanism is added to capture the importance of different historical SST data, weigh the output of each step of the PredRNN network, and improve the prediction accuracy. The experimental results on two ocean datasets confirm that the proposed approach achieves higher training efficiency and prediction accuracy than the existing SST field prediction approaches do.

Key wordsSST prediction    ensemble learning    XGBoost    PredRNN    attention mechanism
收稿日期: 2021-02-19      出版日期: 2022-03-01
Corresponding Author(s): Baiyou QIAO   
 引用本文:   
. [J]. Frontiers of Computer Science, 2023, 17(1): 171601.
Baiyou QIAO, Zhongqiang WU, Ling MA, Yicheng Zhou, Yunjiao SUN. Effective ensemble learning approach for SST field prediction using attention-based PredRNN. Front. Comput. Sci., 2023, 17(1): 171601.
 链接本文:  
https://academic.hep.com.cn/fcs/CN/10.1007/s11704-021-1080-7
https://academic.hep.com.cn/fcs/CN/Y2023/V17/I1/171601
Fig.1  
Fig.2  
Fig.3  
Fig.4  
Variable Definition and explanation Dimension
χt The input vector at time t 5-D
Ht?1l The l th hidden layer output of the PredRNN at time t?1 5-D
Htl The lth hidden layer output of the PredRNN at time t 5-D
Ct?1l Thelth layer standard temporal cell at time t?1 5-D
Ctl The lth layer standard temporal cell that is delivered from the previous node at t-1 to the current time step within each ST-LSTM unit 5-D
Mtl?1 The l?1th layer spatiotemporal memory cell at time t 5-D
Mtl the spatiotemporal memory, which is conveyed vertically from l?1 layer to the current node at the same time t step. For the bottom ST-LSTM layer where l=1, Mtl?1=Mt?1L 5-D
ft Output of the forget gate. The value of each of its element is between 0 and 1. It controls the temporal information that is forgotten in the old cell state Ct?1l 5-D
ft Output of the forget gate. The value of each of its element is between 0 and 1. It controls the spatiotemporal information that is forgotten in the old cell state Mtl?1 5-D
it Output of the input gate. The value of each of its element is between 0 and 1. It controls how much of the temporal information gt will be stored in the new state Ctl. 5-D
it Output of the input gate. The value of each of its element is between 0 and 1. It controls how much of the spatiotemporal information gt will be stored in the new state Mtl 5-D
ot Output of the output gate. The value of each of its element is between 0 and 1. It controls the amount of information output to Htl from the current state Ctl and Mtl 5-D
Tab.1  
Fig.5  
Fig.6  
Datasets Total Training set Validation set Testing set
Bohai Sea 13514 12784 365 365
South China Sea 13514 12784 365 365
Tab.2  
Fig.7  
Fig.8  
Fig.9  
Dataset Bohai Sea dataset South China Sea dataset
MSE RMSE MAE R2 MSE RMSE MAE R2
Approaches PredRNN 0.210 0.458 0.323 0.981 0.103 0.325 0.258 0.937
PredRNN-TF 0.208 0.457 0.316 0.981 0.092 0.303 0.229 0.946
PredRNN-ExpS 0.202 0.450 0.313 0.982 0.096 0.311 0.234 0.943
PredRNN-AT 0.198 0.445 0.318 0.982 0.090 0.299 0.226 0.947
ELA-PredRNN-AT 0.183 0.425 0.315 0.982 0.081 0.285 0.215 0.952
Tab.3  
Fig.10  
Fig.11  
Model Step RMSE MAE R2
SVR 1 0.6434 0.4660 0.9786
2 0.7844 0.5747 0.9758
3 0.8866 0.6452 0.9735
FC-LSTM 1 0.6245 0.4585 0.9789
2 0.7661 0.5655 0.9762
3 0.8771 0.6405 0.9737
CNN-LSTM 1 0.5708 0.4280 0.9798
2 0.7227 0.5378 0.9771
3 0.8406 0.6192 0.9746
ConvLSTM 1 0.6288 0.4847 0.8207
2 0.6901 0.4980 0.9777
3 0.8282 0.5876 0.9748
ELA-PredRNN-AT 1 0.5397 0.4084 0.9812
2 0.6262 0.5145 0.9710
3 0.7342 0.6038 0.9758
Tab.4  
Model Step RMSE MAE R2
SVR 1 0.4901 0.3226 0.9029
2 0.4705 0.3359 0.8708
3 0.5594 0.4271 0.8436
FC-LSTM 1 0.3909 0.2990 0.9110
2 0.4369 0.3369 0.8887
3 0.4723 0.3661 0.8695
CNN-LSTM 1 0.3790 0.2912 0.9162
2 0.4365 0.3389 0.8892
3 0.4905 0.3821 0.8593
ConvLSTM 1 0.3493 0.2688 0.9291
2 0.4100 0.3159 0.9018
3 0.4483 0.3478 0.8854
ELA-PredRNN-AT 1 0.3452 0.2637 0.9305
2 0.4046 0.3135 0.9046
3 0.4429 0.3446 0.8854
Tab.5  
Fig.12  
Prediction step size SVR FC-LSTM CNN-LSTM ConvLSTM ELA-PredRNN-AT
1 0.979 0.979 0.9805 0.979 0.981
2 0.977 0.976 0.977 0.976 0.977
3 0.974 0.973 0.975 0.976 0.975
4 0.972 0.971 0.973 0.972 0.974
5 0.970 0.969 0.9713 0.968 0.972
6 0.969 0.967 0.97 0.965 0.975
7 0.968 0.966 0.969 0.966 0.971
Tab.6  
Prediction step size SVR FC-LSTM CNN-LSTM ConvLSTM ELA-PredRNN-AT
1 0.918 0.936 0.943 0.940 0.952
2 0.887 0.895 0.912 0.910 0.916
3 0.859 0.862 0.889 0.880 0.892
4 0.841 0.832 0.873 0.860 0.876
5 0.826 0.802 0.858 0.835 0.865
6 0.805 0.770 0.844 0.815 0.817
7 0.787 0.742 0.830 0.795 0.844
Tab.7  
  
  
  
  
  
1 F J Wentz , C Gentemann , D Smith , D Chelton . Satellite measurements of sea surface temperature through clouds. Science, 2000, 288( 5467): 847– 850
2 H U Solanki , D Bhatpuria , P Chauhan . Integrative analysis of AltiKa-SSHa, MODIS-SST, and OCM-chlorophyll signatures for fisheries applications. Marine Geodesy, 2015, 38( S1): 672– 683
3 C C Funk , A Hoell . The leading mode of observed and CMIP5 ENSO-residual sea surface temperatures and associated changes in Indo-Pacific climate. Journal of Climate, 2015, 28( 11): 4309– 4329
4 S G Aparna , S D’souza , N B Arjun . Prediction of daily sea surface temperature using artificial neural networks. International Journal of Remote Sensing, 2018, 39( 12): 4214– 4231
5 Y Liu , W Fu . Assimilating high-resolution sea surface temperature data improves the ocean forecast potential in the Baltic Sea. Ocean Science, 2018, 14( 3): 525– 541
6 T N Stockdale , M A Balmaseda , A Vidard . Tropical Atlantic SST prediction with coupled ocean–atmosphere GCMs. Journal of Climate, 2006, 19( 23): 6047– 6061
7 Y Xue , A Leetmaa . Forecasts of tropical Pacific SST and sea level using a Markov model. Geophysical Research Letters, 2000, 27( 17): 2701– 2704
8 I D Lins , M Araujo , M das Chagas Moura . Prediction of sea surface temperature in the tropical Atlantic by support vector machines. Computational Statistics & Data Analysis, 2013, 61 : 187– 198
9 K Patil , M C Deo , M Ravichandran . Prediction of sea surface temperature by combining numerical and neural techniques. Journal of Atmospheric and Oceanic Technology, 2016, 33( 8): 1715– 1726
10 Q He , C Zha , M Sun , X Y Jiang , F M Qi , D M Huang , W Song . Surface temperature parallel prediction algorithm under Spark platform. Marine Science Bulletin, 2019, 38( 3): 280– 289
11 Y LeCun , Y Bengio , G Hinton . Deep learning. Nature, 2015, 521( 7553): 436– 444
12 Q Zhang , H Wang , J Dong , G Zhong , X Sun . Prediction of sea surface temperature using long short-term memory. IEEE Geoscience and Remote Sensing Letters, 2017, 14( 10): 1745– 1749
13 Y Yang , J Dong , X Sun , E Lima , Q Mu , X Wang . A CFCC-LSTM model for sea surface temperature prediction. IEEE Geoscience and Remote Sensing Letters, 2018, 15( 2): 207– 211
14 C Xiao , N Chen , C Hu , K Wang , Z Xu , Y P Cai , L Xu , Z Chen , J Gong . A spatiotemporal deep learning model for sea surface temperature field prediction using time-series satellite data. Environmental Modelling & Software, 2019, 120 : 104502–
15 L Wei , L Guan , L Qu . Prediction of sea surface temperature in the South China Sea by artificial neural networks. IEEE Geoscience and Remote Sensing Letters, 2020, 17( 4): 558– 562
16 Y Wang, M Long, J Wang. PredRNN: recurrent neural networks for predictive learning using spatiotemporal LSTMs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 879– 888
17 X Shi, Z Chen, H Wang, D Y Yeung, W K Wong, W C Woo. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015, 802– 810
18 T Chen, C Guestrin. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016, 785– 794
19 J H Friedman . Greedy function approximation: a gradient boosting machine. The Annals of Statistics, 2001, 29( 5): 1189– 1232
20 J Feng, Y Yu, Z-H Zhou. Multi-layered gradient boosting decision trees. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 3555−3565
21 A Vaswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A N Gomez, Ł Kaiser, I Polosukhin. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010
22 J Xie , J Zhang , J Yu , L Xu . An adaptive scale sea surface temperature predicting method based on deep learning with attention mechanism. IEEE Geoscience and Remote Sensing Letters, 2020, 17( 5): 740– 744
23 X Shi, D-Y Yeung. Machine learning for spatiotemporal sequence forecasting: a survey. 2018, arXiv preprint arXiv: 1808.06865
[1] Highlights Download
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed