A hybrid spatial-temporal deep learning prediction model of industrial methanol-to-olefins process
Jibin Zhou1, Xue Li1, Duiping Liu2, Feng Wang3, Tao Zhang1(), Mao Ye1(), Zhongmin Liu1
1. National Engineering Research Center of Lower-Carbon Catalysis Technology, Dalian Institute of Chemical Physics, Chinese Academy of Sciences, Dalian 116023, China 2. Yulin Innovation Institute for Clean Energy, Clean Energy Innovation Institute of Chinese Academy of Sciences, Yulin 719053, China 3. Xuelang Industrial Intelligence Technology Co., Ltd., Wuxi 214000, China
Methanol-to-olefins, as a promising non-oil pathway for the synthesis of light olefins, has been successfully industrialized. The accurate prediction of process variables can yield significant benefits for advanced process control and optimization. The challenge of this task is underscored by the failure of traditional methods in capturing the complex characteristics of industrial processes, such as high nonlinearities, dynamics, and data distribution shift caused by diverse operating conditions. In this paper, we propose a novel hybrid spatial-temporal deep learning prediction model to address these issues. Firstly, a unique data normalization technique called reversible instance normalization is employed to solve the problem of different data distributions. Subsequently, convolutional neural network integrated with the self-attention mechanism are utilized to extract the temporal patterns. Meanwhile, a multi-graph convolutional network is leveraged to model the spatial interactions. Afterward, the extracted temporal and spatial features are fused as input into a fully connected neural network to complete the prediction. Finally, the outputs are denormalized to obtain the ultimate results. The monitoring results of the dynamic trends of process variables in an actual industrial methanol-to-olefins process demonstrate that our model not only achieves superior prediction performance but also can reveal complex spatial-temporal relationships using the learned attention matrices and adjacency matrices, making the model more interpretable. Lastly, this model is deployed onto an end-to-end Industrial Internet Platform, which achieves effective practical results.
. [J]. Frontiers of Chemical Science and Engineering, 2024, 18(4): 42.
Jibin Zhou, Xue Li, Duiping Liu, Feng Wang, Tao Zhang, Mao Ye, Zhongmin Liu. A hybrid spatial-temporal deep learning prediction model of industrial methanol-to-olefins process. Front. Chem. Sci. Eng., 2024, 18(4): 42.
Catalyst density of the dense phase in the reactor
kg·m−3
TI1134A
Regenerator temperature
°C
PIC1110
Regenerator pressure
MPa
WI1105
Catalyst inventory in the regenerator
t
WZ1101
Catalyst inventory in the reactor and regenerator
t
FIC1104B
Upper stripping steam feed
Nm3·h−1
FIC1105B
Lower stripping steam feed
Nm3·h−1
FIC1113B
Steam delivery feed
Nm3·h−1
ZI1102
Value of slide valve of regenerated catalysts
%
DI1106
Regenerated catalyst density
kg·m−3
TI1119
Regenerated catalyst temperature
°C
TI1135B
Lower stripping temperature
°C
FIC1121A
Air feed
Nm3·h−1
FIC1001
C4 feed
kg·h−1
FIC1103
Nitrogen feed
Nm3·h−1
Q_PDI1113
Catalyst circulation rate
t·h
PDI1113
Pressure drop of the slide valve of regenerated catalysts
kPa
PDI1106
Pressure drop of the standby valve of coked catalysts
kPa
AI1603G
Ethylene yield
%
AI1603I
Propylene yield
%
Tab.1
Fig.3
Models
Horizon = 1 (2 h)
Horizon = 3 (6 h)
Horizon = 5 (10 h)
MAE
MAPE
MAE
MAPE
MAE
MAPE
Classical
VAR
15.25
1.21%
88.37
0.29%
96.63
4.09%
LSTM
24.39
1.40%
27.57
0.23%
37.70
2.98%
GRU
12.94
0.94%
23.35
0.15%
22.96
2.08%
Attention
DSANet
6.65
0.54%
9.32
0.62%
10.73
0.75%
LSTNet
7.67
0.62%
10.08
0.70%
11.85
0.85%
Graph
Graph WaveNet
8.31
0.70%
13.36
0.85%
16.53
1.19%
STGCN
7.91
0.81%
11.27
0.86%
12.97
1.10%
MTGNN
7.25
0.71%
10.93
0.79%
11.85
0.87%
This work
CSA-MGCN
6.33
0.49%
9.10
0.58%
10.28
0.66%
Improvements
+4.83%
+8.13%
+2.41%
+7.05%
+4.20%
+11.23%
Tab.2
Fig.4
Fig.5
Fig.6
Fig.7
Fig.8
Fig.9
Fig.10
1
J Zhou, M Gao, J Zhang, W Liu, T Zhang, H Li, Z Xu, M Ye, Z Liu. Directed transforming of coke to active intermediates in methanol-to-olefins catalyst to boost light olefins selectivity. Nature Communications, 2021, 12(1): 17 https://doi.org/10.1038/s41467-020-20193-1
C Q Li, Y Q Chen, Y L Shang. A review of industrial big data for decision making in intelligent manufacturing. Engineering Science and Technology an International Journal, 2022, 29: 101021
4
M Pirdashti, S Curteanu, M H Kamangar, M H Hassim, M A Khatami. Artificial neural networks: applications in chemical engineering. Reviews in Chemical Engineering, 2013, 29(4): 205–239 https://doi.org/10.1515/revce-2013-0013
5
L H Chiang, B Braun, Z Wang, I Castillo. Towards artificial intelligence at scale in the chemical industry. AIChE Journal, 2022, 68(6): e17644 https://doi.org/10.1002/aic.17644
6
L T Zhu, X Z Chen, B Ouyang, W C Yan, H Lei, Z Chen, Z H Luo. Review of machine learning for hydrodynamics, transport, and reactions in multiphase flows and reactors. Industrial & Engineering Chemistry Research, 2022, 61(28): 9901–9949 https://doi.org/10.1021/acs.iecr.2c01036
7
Z Q Wang, L Wang, Z H Yuan, B Z Chen. Data-driven optimal operation of the industrial methanol to olefin process based on relevance vector machine. Chinese Journal of Chemical Engineering, 2021, 34: 106–115 https://doi.org/10.1016/j.cjche.2020.09.040
8
H L Zhang, A Q Zhu, J Xu, W Ge. Gas-solid reactor optimization based on EMMS-DPM simulation and machine learning. Particuology, 2024, 89: 131–143 https://doi.org/10.1016/j.partic.2023.10.007
9
L Yao, Z Q Ge. Big data quality prediction in the process industry: a distributed parallel modeling framework. Journal of Process Control, 2018, 68: 1–13 https://doi.org/10.1016/j.jprocont.2018.04.004
10
Q Q Sun, Z Q Ge. A Survey on deep learning for data-driven soft sensors. IEEE Transactions on Industrial Informatics, 2021, 17(9): 5853–5866 https://doi.org/10.1109/TII.2021.3053128
11
X F Yuan, Z Z Jia, L Li, K Wang, L J Ye, Y L Wang, C H Yang, W H Gui. A SIA-LSTM based virtual metrology for quality variables in irregular sampled time sequence of industrial processes. Chemical Engineering Science, 2022, 249: 117299 https://doi.org/10.1016/j.ces.2021.117299
12
Y S Lee, J H Chen. Developing semi-supervised latent dynamic variational autoencoders to enhance prediction performance of product quality. Chemical Engineering Science, 2023, 265: 118192 https://doi.org/10.1016/j.ces.2022.118192
13
F Yang, Y S Sang, J C Lv, J Cao. Prediction of gasoline yield in fluid catalytic cracking based on multiple level LSTM. Chemical Engineering Research & Design, 2022, 185: 119–129 https://doi.org/10.1016/j.cherd.2022.06.040
14
J C Li, B Yang, H G Li, Y J Wang, C Qi, Y Liu. DTDR–ALSTM: extracting dynamic time-delays to reconstruct multivariate data for improving attention-based LSTM industrial time series prediction models. Knowledge-Based Systems, 2021, 211: 106508 https://doi.org/10.1016/j.knosys.2020.106508
15
X Hao, G Huang, Z Li, L Zheng, Y Zhao. A spatio-temporal data decoupling convolution network model for specific surface area prediction in cement grind process. ISA Transactions, 2023, 135: 380–397 https://doi.org/10.1016/j.isatra.2022.10.006
16
C H Zhao. Perspectives on nonstationary process monitoring in the era of industrial artificial intelligence. Journal of Process Control, 2022, 116: 255–272 https://doi.org/10.1016/j.jprocont.2022.06.011
17
Y C Jiang, S Yin, J W Dong, O Kaynak. A review on soft sensors for monitoring, control, and optimization of industrial processes. IEEE Sensors Journal, 2021, 21(11): 12868–12881 https://doi.org/10.1109/JSEN.2020.3033153
Y H Kuo, A Kusiak. From data to big data in production research: the past and future trends. International Journal of Production Research, 2019, 57(15–16): 4828–4853 https://doi.org/10.1080/00207543.2018.1443230
20
S KumarL HussainS BanarjeeM Reza. Energy load forecasting using deep learning approach-LSTM and GRU in spark cluster. In: 2018 Fifth International Conference on Emerging Applications of Information Technology. New York: IEEE, 2018, 1–4
21
Y J Wang, Y M Ren, H G Li. Symbolic multivariable hierarchical clustering based convolutional neural networks with applications in industrial process operating trend predictions. Industrial & Engineering Chemistry Research, 2020, 59(34): 15133–15145 https://doi.org/10.1021/acs.iecr.0c02084
22
F Yan, C J Yang, X M Zhang. DSTED: a denoising spatial-temporal encoder-decoder framework for multistep prediction of burn-through point in sintering process. IEEE Transactions on Industrial Electronics, 2022, 69(10): 10735–10744 https://doi.org/10.1109/TIE.2022.3151960
23
J T Connor, R D Martin, L E Atlas. Recurrent neural networks and robust time series prediction. IEEE Transactions on Neural Networks, 1994, 5(2): 240–254 https://doi.org/10.1109/72.279188
K ChoB Van MerrienboerC GulcehreD BahdanauF Bougares H SchwenkY Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078, 2014
26
K O’SheaR Nash. An introduction to convolutional neural networks. arXiv:1511.08458, 2015
27
Y J Wang, Y C Zhang, Z Wu, H G Li, P D Christofides. Operational trend prediction and classification for chemical processes: a novel convolutional neural network method based on symbolic hierarchical clustering. Chemical Engineering Science, 2020, 225: 115796 https://doi.org/10.1016/j.ces.2020.115796
28
J Zhou, G Q Cui, S D Hu, Z Y Zhang, C Yang, Z Y Liu, L F Wang, C C Li, M S Sun. Graph neural networks: a review of methods and applications. AI Open, 2020, 1: 57–81 https://doi.org/10.1016/j.aiopen.2021.01.001
29
D BahdanauK ChoY Bengio. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473, 2014
30
X YinY Han H SunZ Xu H YuX Duan. A multivariate time series prediction schema based on multi-attention in recurrent neural network. In: 2020 IEEE Symposium on Computers and Communications (ISCC). New York: IEEE, 2020, 1–7
31
Y Yang, Q Xiong, C Wu, Q Zou, Y Yu, H Yi, M Gao. A study on water quality prediction by a hybrid CNN-LSTM model with attention mechanism. Environmental Science and Pollution Research International, 2021, 28(39): 55129–55139 https://doi.org/10.1007/s11356-021-14687-8
32
A VaswaniN ShazeerN ParmarJ UszkoreitL Jones A N GomezL KaiserI Polosukhin. Attention is all you need. In: Advances in Neural Information Processing Systems. New York: Curran Associates Inc., 2017
33
X B FuF GaoJ WuX Y WeiF W Duan. Spatiotemporal attention networks for wind power forecasting. In: 2019 International Conference on Data Mining Workshops. New York: IEEE, 2019, 149–154
34
S T HuangD L WangX WuA Tang. Dsanet: dual self-attention network for multivariate time series forecasting. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: Association for Computing Machinery, 2019, 2129–2132
35
N WuB Green X BenS O’Banion. Deep transformer models for time series forecasting: the influenza prevalence case. arXiv:2001.08317, 2020
36
F Scarselli, M Gori, A C Tsoi, M Hagenbuchner, G Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20(1): 61–80 https://doi.org/10.1109/TNN.2008.2005605
37
B YuH T Yin Z X Zhu. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. arXiv:1709.04875, 2017
38
Z H WuS R PanG D LongJ JiangC Q Zhang. Graph wavenet for deep spatial-temporal graph modeling. arXiv:1906.00121, 2019
39
B LuX Y Gan H M JinL Y FuH S Zhang. Spatiotemporal adaptive gated graph convolution network for urban traffic flow forecasting. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: Association for Computing Machinery, 2020, 1025–1034
40
C Amornbunchornvej, E Zheleva, T Berger-Wolf. Variable-lag granger causality and transfer entropy for time series analysis. ACM Transactions on Knowledge Discovery from Data, 2021, 15(4): 1–30 https://doi.org/10.1145/3441452
41
H Y XuY D HuangZ H DuanJ FengP Y Song. Multivariate time series forecasting based on causal inference with transfer entropy and graph neural network. arXiv:2005.01185, 2020
42
K W He, X Chen, Q Wu, S Yu, Z Zhou. Graph attention spatial-temporal network with collaborative global-local learning for citywide mobile traffic prediction. IEEE Transactions on Mobile Computing, 2022, 21(4): 1244–1256 https://doi.org/10.1109/TMC.2020.3020582
43
Z H WuS R PanG D LongJ JiangX J Chang C Q Zhang. Connecting the dots: multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: Association for Computing Machinery, 2020, 753–763
44
T KimJ Kim Y TaeC Park J H ChoiJ Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In: International Conference on Learning Representations, 2022
45
G Y JinZ X XiH Y ShaY H FengJ C Huang. Deep multi-view spatiotemporal virtual graph neural network for significant citywide ride-hailing demand prediction. arXiv:2007.15189, 2020
46
D F Li, K X Lin, X T Li, J B Liao, R Du, D Q Chen, A Madden. Improved sales time series predictions using deep neural networks with spatiotemporal dynamic pattern acquisition mechanism. Information Processing & Management, 2022, 59(4): 102987 https://doi.org/10.1016/j.ipm.2022.102987
47
D ChaiL WangQ Yang. Bike flow prediction with multi-graph convolutional networks. In: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York: Association for Computing Machinery, 2018, 397–400
48
O Troyanskaya, M Cantor, G Sherlock, P Brown, T Hastie, R Tibshirani, D Botstein, R B Altman. Missing value estimation methods for DNA microarrays. Bioinformatics, 2001, 17(6): 520–525 https://doi.org/10.1093/bioinformatics/17.6.520
49
G K LaiW C ChangY M YangH X Liu. Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR Conference on Research & Development in Information Retrieval. New York: Association for Computing Machinery, 2018, 95–104
50
J Fan, K Zhang, Y Huang, Y Zhu, B Chen. Parallel spatio-temporal attention-based TCN for multivariate time series prediction. Neural Computing & Applications, 2023, 35(18): 13109–13118 https://doi.org/10.1007/s00521-021-05958-z
