Using machine learning models to explore the solution space of large nonlinear systems underlying flowsheet simulations with constraints

doi:10.1007/s11705-021-2073-7

Front. Chem. Sci. Eng.

2022, Vol. 16

Issue (2) : 183-197 https://doi.org/10.1007/s11705-021-2073-7

RESEARCH ARTICLE

Using machine learning models to explore the solution space of large nonlinear systems underlying flowsheet simulations with constraints

Patrick Otto Ludl¹(

), Raoul Heese¹, Johannes Höller¹, Norbert Asprion², Michael Bortz¹

¹. Fraunhofer ITWM Optimization Department, Kaiserslautern 67663, Germany
². Chemical and Process Engineering BASF SE, Ludwigshafen 67056, Germany

Download: PDF(2983 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Flowsheet simulations of chemical processes on an industrial scale require the solution of large systems of nonlinear equations, so that solvability becomes a practical issue. Additional constraints from technical, economic, environmental, and safety considerations may further limit the feasible solution space beyond the convergence requirement. A priori, the design variable domains for which a simulation converges and fulfills the imposed constraints are usually unknown and it can become very time-consuming to distinguish feasible from infeasible design variable choices by simply running the simulation for each choice. To support the exploration of the design variable space for such scenarios, an adaptive sampling technique based on machine learning models has recently been proposed. However, that approach only considers the exploration of the convergent domain and ignores additional constraints. In this paper, we present an improvement which particularly takes the fulfillment of constraints into account. We successfully apply the proposed algorithm to a toy example in up to 20 dimensions and to an industrially relevant flowsheet simulation.

Keywords machine learning flowsheet simulations constraints exploration

Corresponding Author(s): Patrick Otto Ludl

Online First Date: 26 August 2021 Issue Date: 10 January 2022

Cite this article:

Patrick Otto Ludl,Raoul Heese,Johannes Höller, et al. Using machine learning models to explore the solution space of large nonlinear systems underlying flowsheet simulations with constraints[J]. Front. Chem. Sci. Eng., 2022, 16(2): 183-197.

URL:

https://academic.hep.com.cn/fcse/EN/10.1007/s11705-021-2073-7
https://academic.hep.com.cn/fcse/EN/Y2022/V16/I2/183

Fig.1 Outline of the unconstrained adaptive sampling strategy [14].

Fig.2 Constrained adaptive sampling: starting from the results of flowsheet simulations, additional constraints can be imposed. Training ML models using the information how strong the constraints are violated allows to suggest new sampling points which are expected to fulfil the constraints. Subsequently, these new points x_new are evaluated by solving the system of equations of the flowsheet simulation.

1.	function EXPLORATION $(D init, χ, N max, w)$
2.		$D expl ← D init$
3.		$N ← size (D expl)$
4.		while $N < N max$ do
5.			$x new ←$ SUGGESTIONUNCONSTRAINED $(D expl, χ, w)$
6.			$D expl ← D expl ∪ ? {$ SIMULATION $(x new)}$
7.			$N ← N + 1$
8.		end while
9.		return $D expl$
10.	end function
11.
12.	function SUGGESTIONUNCONSTRAINED $(D expl, χ, w)$
13		$C ←$ TRAINCLASSIFIER $(D expl)$
14.		$R t ←$ TRAINREGRESSOR $(D expl)$
15.		function UTILITY $(x, D expl, C, R t, w)$
16.			$u ← (U s (C, x), U o (R t, x), U r (D expl, x)) T$
17.			return $w T u / \|\| w \|\| 1$
18.		end function
19.		return $a r g m a x x ∈ χ$ UTILITY $(x, D expl, C, R t, w)$
20.	end function

Algorithm 1 Outline of the unconstrained adaptive sampling algorithm from ref. [14].

1:	function SUGGESTIONCONSTRAINED $(D expl, χ, D f, w)$
2:		????? $C ←$ TRAINCLASSIFIER $(D expl)$
3:		????? $R f ←$ TRAINREGRESSOR $(D expl)$
4:		?????function UTILITY $(x, D expl, C, R f, D f, w)$
5:			????????? $u ← (U s (C, x), U r' (D expl, x), U c (R f, D f, x)) T$
6:			?????????return $w T u / \|\| w \|\| 1$
7:		?????end function
8:		?????return $a r g m a x x ∈ χ$ UTILITY $(x, D expl, C, R f, D f, w)$
9:	end function

Algorithm 2 Outline of our proposed constrained adaptive sampling algorithm. The new function SUGGESTIONCONSTRAINED replaces SUGGESTIONUNCONSTRAINED in Line 5 of Algorithm 1.

Fig.3 The divergent (red), convergent feasible (green) and convergent infeasible (blue) regions of the design variable space χ of the toy example for n = 2 dimensions.

Fig.4 An illustration of the behavior of the algorithm for the n = 2 toy example. As initial state D_init, we use 15 randomly generated points in the lower left quadrant. Each column of the above matrix of plots corresponds to one run of the sampling algorithm. The column titles show the chosen weights

w = (w s, w r, w c) .

The different rows show the generated points at different stages, the total number of sampled points (including the 15 initial points) being shown on the very left of the figure. The three different types of points are plotted as red triangles (divergent), blue squares (convergent but infeasible) and green circles (convergent and feasible), respectively. The black dashed lines show the boundaries of the regions.

Fig.5 The number of divergent (red), convergent feasible (green) and convergent infeasible (blue) points as a function of the total number of sampled points for the n = 2 toy example. For a fixed

w = (w s, w r, w c),

we perform 50 runs of algorithm 2 with different random initial configurations D_init consisting of 15 points each. The lines are the averages of the 50 runs and the shaded regions are the 1σ-error bands. The results for

w = (1, 1, 0), (1, 1, 1)

and

(1, 1, 5)

are shown in plots (a), (b) and (c), respectively.

Tab.1 The regions

χ init ⊆ χ

from which the initial configurations D_init for the tests of the n-dimensional toy example are drawn randomly

Fig.6 Mean number of non-divergent (feasible+ infeasible) points with 1σ-error bands for the dimensions n = 2, n = 10 and n = 20 of the toy example. The subsets

χ init ⊆ χ

of the design variable space from which the initial configurations are drawn randomly (see Table 1) are chosen in such a way that a comparable number of non-divergent points in

D init

is achieved for all dimensions n (The curves for n = 3 and n = 5 are not shown for the sake of clarity. They run between those for n = 2 and n = 10, as expected).

Fig.7 Mean number of feasible points with 1σ-error bands for the dimensions n = 2, n = 10 and n = 20 of the toy example. The subsets

χ init ⊆ χ

of the design variable space from which the initial configurations are drawn randomly (see Table 1) are chosen in such a way that a comparable number of non-divergent points in

D init

is achieved for all dimensions n (The curves for n = 3 and n = 5 run between those for n = 2 and n = 10 and are not shown for the sake of clarity).

Fig.8 Mean number of feasible points for the analysis with nonzero constraint weight with 1σ-error bands for the dimensions n = 2, n = 10 and n = 20 of the toy example. The subsets

χ init ⊆ χ

of the design variable space from which the initial configurations are drawn randomly (see Table 1) are chosen in such a way that a comparable number of non-divergent points in

D init

is achieved for all dimensions n. The curves for n = 3 and n = 5 are very similar to the one for n = 2 and are therefore not shown for the sake of clarity.

Fig.9 Simplified flowsheet for the pressure swing distillation of a mixture of chloroform and acetone. A mixture containing 86 mass percent chloroform and 14 mass percent acetone is fed into the column C1 operating at 1 bar. Since the feed contains more chloroform than the azeotropic point at 1 bar, chloroform will enrich in the top (distillate) stream. The bottom liquid (sump) stream of C1 is fed into column C2 operating at 10 bar. The distillate stream of C2 is rich in acetone. The bottom liquid stream of C2 is recycled by combining it with the input mixture stream.

Fig.10 An illustration of the behavior of the algorithm for the chloroform/acetone pressure swing distillation. As initial state D_init we use 10 randomly generated points in the four-dimensional design space χ. The plots themselves show projections into the

(m a c, m c l)

-plane. Each column of the above matrix of plots corresponds to one run of the sampling algorithm. The column title shows the chosen weights

w = (w s, w r, w c)

. The different rows show the generated points at different stages, the total number of sampled points (including the 10 initial points) being shown on the very left of the figure. The three different types of points are plotted as red triangles (divergent), blue squares (convergent but infeasible) and green circles (convergent and feasible), respectively.

Fig.11 The number of divergent (red), convergent feasible (green) and convergent infeasible (blue) points as a function of the total number of sampled points for the pressure swing distillation example. For a fixed

w = (w s, w r, w c), ? we ? perform

50 runs of algorithm 2 with different random initial configurations D_init consisting of 10 points each. The lines are the averages of the 50 runs and the shaded regions represent the 1σ-error bands. The results for

w = (1, 1, 0), ? (1, 1, 1)

and

(1, 1, 5)

are shown in plots (a), (b) and (c), respectively.

Fig.12 Comparison of the results of algorithm 2 (lines with 1σ-error bands) to mere random sampling (lines without error bands; for the sake of clarity, the error bars for the random sampling results are not shown) for the pressure swing distillation example. For a fixed

w = (w s, w r, w c),

we perform 50 runs with different random initial configurations D_init consisting of 10 points each. The results for

w = (1, 1, 0), (1, 1, 1)

and

(1, 1, 5)

are shown in plots (a), (b) and (c), respectively.

1	I E Grossmann, R W H Sargent. Optimum design of chemical plants with uncertain parameters. AIChE Journal. American Institute of Chemical Engineers, 1978, 24(6): 1021–1028 https://doi.org/10.1002/aic.690240612
2	K P Halemane, I E Grossmann. Optimal process design under uncertainty. AIChE Journal. American Institute of Chemical Engineers, 1983, 29(3): 425–433 https://doi.org/10.1002/aic.690290312
3	F Boukouvala, M G Ierapetritou. Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method. Computers & Chemical Engineering, 2012, 36: 358–368 https://doi.org/10.1016/j.compchemeng.2011.06.005
4	F Boukouvala, M G Ierapetritou. Derivative-free optimization for expensive constrained problems using a novel expected improvement objective function. AIChE Journal. American Institute of Chemical Engineers, 2014, 60(7): 2462–2474 https://doi.org/10.1002/aic.14442
5	Z Wang, M G Ierapetritou. A novel feasibility analysis method for black-box processes using a radial basis function adaptive sampling approach. AIChE Journal. American Institute of Chemical Engineers, 2017, 63(2): 532–550 https://doi.org/10.1002/aic.15362
6	A Rogers, M G Ierapetritou. Feasibility and flexibility analysis of black-box processes Part 1: surrogate-based feasibility analysis. Chemical Engineering Science, 2015, 137: 986–1004 https://doi.org/10.1016/j.ces.2015.06.014
7	B Shahriari, K Swersky, Z Wang, R P Adams, N de Freitas. Taking the human out of the loop: a review of Bayesian optimization. Proceedings of the Institute of Electrical and Electronics Engineers, 2016, 104(1): 148–175 https://doi.org/10.1109/JPROC.2015.2494218
8	G Bano, Z Wang, P Facco, F Bezzo, M Barolo, M G Ierapetritou. A novel and systematic approach to identify the design space of pharmaceutical processes. Computers & Chemical Engineering, 2018, 115: 309–322 https://doi.org/10.1016/j.compchemeng.2018.04.021
9	R B Gramacy, H K H Lee. Optimization Under Unknown Constraints, Bayesian Statistics 9: Proceedings of the Ninth Valencia International Meeting, 2011, 9, 229–256
10	A Tran, J Sun, J M Furlan, K V Pagalthivarthi, R J Visintainer, Y Wang. A batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics. Computer Methods in Applied Mechanics and Engineering, 2019, 347: 827–852 https://doi.org/10.1016/j.cma.2018.12.033
11	M A Gelbart, J Snoek, R P Adams. Bayesian optimization with unknown constraints. arXiv:1403.5607, 2014
12	R Griffiths, J M Hernández-Lobato. Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chemical Science (Cambridge), 2020, 11(2): 577–586 https://doi.org/10.1039/C9SC04026A
13	L S Dias, M G Ierapretrou. Data-driven feasibility analysis for the integration of planning and scheduling problems. Optimization and Engineering, 2019, 20(4): 1029–1066 https://doi.org/10.1007/s11081-019-09459-w
14	R Heese, M Walczak, T Seidel, N Asprion, M Bortz. Optimized data exploration applied to the simulation of a chemical process. Computers & Chemical Engineering, 2019, 124: 326–342 https://doi.org/10.1016/j.compchemeng.2019.01.007
15	M Schonlau, W J Welch, D R Jones. Global versus local search in constrained optimization of computer models. Institute of Mathematical Statistics Lecture Notes—Monograph Series, 1998, 34: 11–25
16	M A Gelbart. Constrained Bayesian optimization and applications. Dissertation for the Doctoral Degree. Cambridge (Massachusetts): Harvard University, 2015
17	C E Rasmussen, C K I Williams. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). Cambridge (Massachusetts): The MIT Press, 2005
18	J R Gardner, M J Kusner, Z Xu, K Q Weinberger, J P Cunningham. Bayesian optimization with inequality constraints. ICML'14: Proceedings of the 31st International Conference on International Conference on Machine Learning, 2014, 32: 937–945
19	B Schölkopf. The kernel trick for distances. In: Advances in Neural Information Processing Systems. Cambridge (Massachusetts): The MIT Press, 2001, 301–307
20	R Heese, M Walczak, M Bortz, J Schmid. Calibrated simplex mapping classification., 2021
21	R H Byrd, P Lu, J Nocedal, C Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 1995, 16(5): 1190–1208 https://doi.org/10.1137/0916069
22	C Zhu, R H Byrd, P Lu, J Nocedal. Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software, 1997, 23(4): 550–560 https://doi.org/10.1145/279232.279236
23	P Virtanen, R Gommers, T E Oliphant, M Haberland, T Reddy, D Cournapeau, E Burovski, P Peterson, W Weckesser, J Bright, et al.. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods, 2020, 17(3): 261–272 https://doi.org/10.1038/s41592-019-0686-2
24	F Pedregosa, G Varoquaux, A Gramfort, V Michel, B Thirion, O Grisel, M Blondel, P Prettenhofer, R Weiss, V Dubourg, et al.. Scikit-learn: machine learning in Python. Journal of Machine Learning Research, 2011, 12: 2825–2830
25	GPy. GPy: a gaussian process framework in python. The website of github, 2012
26	L T Biegler, I E Grossmann, A W Westerberg. Systematic Methods for Chemical Process Design. New Jersey: Prentice Hall, 1997
27	H Renon, J M Prausnitz. Local compositions in thermodynamic excess functions for liquid mixtures. AIChE Journal. American Institute of Chemical Engineers, 1968, 14(1): 135–144 https://doi.org/10.1002/aic.690140124
28	M Bortz, J Burger, N Asprion, S Blagov, R Böttcher, U Nowak, A Scheithauer, R Welke, K H Küfer, H Hasse. Multi-criteria optimization in chemical process design and decision support by navigation on pareto sets. Computers & Chemical Engineering, 2014, 60: 354–363 https://doi.org/10.1016/j.compchemeng.2013.09.015

[1]

FCE-20122-OF-LPO_suppl_1

Download

[1]	Quirin Göttl, Dominik G. Grimm, Jakob Burger. Automated synthesis of steady-state continuous processes using reinforcement learning[J]. Front. Chem. Sci. Eng., 2022, 16(2): 288-302.
[2]	Haoqin Fang, Jianzhao Zhou, Zhenyu Wang, Ziqi Qiu, Yihua Sun, Yue Lin, Ke Chen, Xiantai Zhou, Ming Pan. Hybrid method integrating machine learning and particle swarm optimization for smart chemical process operations[J]. Front. Chem. Sci. Eng., 2022, 16(2): 274-287.
[3]	Ewan Chee, Wee Chin Wong, Xiaonan Wang. An integrated approach for machine-learning-based system identification of dynamical systems under control: application towards the model predictive control of a highly nonlinear reactor system[J]. Front. Chem. Sci. Eng., 2022, 16(2): 237-250.

Viewed

Full text

Abstract

Cited

Shared

Discussed