| Industrial Engineering and Intelligent Manufacturing |
|
|
|
Intelligent smelting process, management system: Efficient and intelligent management strategy by incorporating large language model |
Tianjie FU1, Shimin LIU2( ), Peiyu LI1 |
1. The School of Mechanical Engineering, Zhejiang University, Hangzhou 310058, China
2. Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, China; State Key Laboratory of Ultra-precision Machining Technology, Department of Industrial Systems and Engineering, The Hong Kong Polytechnic University, Hong Kong, China |
|
|
|
|
Abstract In the steelmaking industry, enhancing production cost-effectiveness and operational efficiency requires the integration of intelligent systems to support production activities. Thus, effectively integrating various production modules is crucial to enable collaborative operations throughout the entire production chain, reducing management costs and complexities. This paper proposes, for the first time, the integration of Vision-Language Model (VLM) and Large Language Model (LLM) technologies in the steel manufacturing domain, creating a novel steelmaking process management system. The system facilitates data collection, analysis, visualization, and intelligent dialogue for the steelmaking process. The VLM module provides textual descriptions for slab defect detection, while LLM technology supports the analysis of production data and intelligent question-answering. The feasibility, superiority, and effectiveness of the system are demonstrated through production data and comparative experiments. The system has significantly lowered costs and enhanced operational understanding, marking a critical step toward intelligent and cost-effective management in the steelmaking domain.
|
| Keywords
smelting steel
process management
large language models
intelligent Q & A
ChatGPT
|
|
Corresponding Author(s):
Shimin LIU
|
|
Just Accepted Date: 04 June 2024
Online First Date: 11 July 2024
Issue Date: 26 September 2024
|
|
| 1 |
J B, Alayrac J, Donahue P, Luc A, Miech I, Barr Y, Hasson K, Lenc A, Mensch K, Millican M, Reynolds R, Ring (2022). Reynolds M. Flamingo: A visual language model for few-shot learning. Advances in Neural Information Processing Systems, 35: 23716–23736
https://doi.org/10.48550/arXiv.2204.14198
|
| 2 |
P AndersonB FernandoM JohnsonS Gould (2016). Spice: Semantic propositional image caption evaluation. In: Proceedings of European Conference on Computer Vision (ECCV): 382–398
|
| 3 |
P AndersonX HeC BuehlerD TeneyM Johnson S GouldL Zhang (2018). Bottom-up and top-down attention for image captioning and visual question answering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 6077–6086
|
| 4 |
Z, Bao D, He M K, Khan M, Luo Q, Xie (2023). PBidm: Privacy-preserving blockchain-based identity management system for industrial internet of things. IEEE Transactions on Industrial Informatics, 19( 2): 1524–1534
https://doi.org/10.1109/TII.2022.3206798
|
| 5 |
P, Bellavista M, Fogli C, Giannelli C, Stefanelli (2023). Application-aware network traffic management in MEC-integrated industrial environments. Future Internet, 15( 2): 42
https://doi.org/10.3390/fi15020042
|
| 6 |
A M, Bessarabov V E, Trokhin A K, Popov A S, Radetskaya (2023). CALS project: Hardware and technological design of a modular water management system for industrial applications. Chemical and Petroleum Engineering, 58( 9–10): 855–864
https://doi.org/10.1007/s10556-023-01172-x
|
| 7 |
A A, Borkowski (2023). Applications of ChatGPT and large language models in medicine and health care: Benefits and pitfalls. Federal Practitioner, 40( 6): 170–173
https://doi.org/10.12788/fp.0386
|
| 8 |
Y CuiS Niekum A GuptaV KumarA Rajeswaran (2022). Can foundation models perform zero-shot task specification for robot manipulation? In: Proceedings of 4th Annual Learning for Dynamics and Control Conference, Stanford, USA
|
| 9 |
Curtò J, De Zarzà I, De C T, Calafate (2023). Semantic scene understanding with large language models on unmanned aerial vehicles. Drones, 7( 2): 114
https://doi.org/10.3390/drones7020114
|
| 10 |
K, Demertzis S, Demertzis L, Iliadis (2023). A selective survey review of computational intelligence applications in the primary subdomains of civil engineering specializations. Applied Sciences-Basel, 13( 6): 3380
https://doi.org/10.3390/app13063380
|
| 11 |
A DosovitskiyL BeyerA KolesnikovD WeissenbornX ZhaiT T UnterthinerM DehghaniM MindererG Heigold S GellyJ Uszkoreit (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In: Proceedings of International Conference on Learning Representations 2021
|
| 12 |
L, Fang F, Su Z, Kang H, Zhu (2023). Artificial neural network model for temperature prediction and regulation during molten steel transportation process. Processes, 11( 6): 1629
https://doi.org/10.3390/pr11061629
|
| 13 |
D’Souza R, Franco S, Amanullah M, Mathew K M, Surapaneni (2023). Appraising the performance of ChatGPT in psychiatry using 100 clinical case vignettes. Asian Journal of Psychiatry, 89: 103770
https://doi.org/10.1016/j.ajp.2023.103770
|
| 14 |
T, Fu P, Li S, Liu (2024a). An imbalanced small sample slab defect recognition method based on image generation. Journal of Manufacturing Processes, 118: 376–388
https://doi.org/10.1016/j.jmapro.2024.03.028
|
| 15 |
T, Fu S, Liu P, Li (2024b). Digital twin-driven smelting process management method for converter steelmaking. Journal of Intelligent Manufacturing, 2024: 1–17
https://doi.org/10.1007/s10845-024-02366-7
|
| 16 |
X GuT Y O’LearyW KuoY Cui (2022). Open-vocabulary object detection via vision and language knowledge distillation. In: Proceedings of International Conference on Learning Representations 2022
|
| 17 |
F, Hein-Pensel H, Winkler A, Brückner M, Wölke I, Jabs I J, Mayan A, Kirschenbaum J, Friedrich C, Zinke-Wehlmann (2023). Maturity assessment for Industry 5.0: A review of existing maturity models. Journal of Manufacturing Systems, 66: 200–210
https://doi.org/10.1016/j.jmsy.2022.12.009
|
| 18 |
H C, Huang C H, Tsai H C, Lin (2023). Development of 5G cyber-physical production system. International Journal of Networked and Distributed Computing, 11( 1): 9–19
https://doi.org/10.1007/s44227-022-00003-4
|
| 19 |
W HuangP AbbeelD PathakI Mordatch (2022). Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In: Proceedings of 39th International Conference on Machine Learning (ICML), Baltimore, MA, USA
|
| 20 |
R, Iwańkowicz R, Rutkowski (2023). Digital twin of shipbuilding process in Shipyard 4.0. Sustainability, 15( 12): 9733
https://doi.org/10.3390/su15129733
|
| 21 |
M M, Jaber M H, Ali S K, Abd M M, Jassim A, Alkhayyat E H, Kadhim A R, Alkhuwaylidee S, Alyousif (2023). AHI: A hybrid machine learning model for complex industrial information systems. Journal of Combinatorial Optimization, 45( 2): 58
https://doi.org/10.1007/s10878-023-00988-w
|
| 22 |
A, Jadhav S K, Shandilya I, Izonin M, Gregus (2023). Effective software effort estimation leveraging machine learning for digital transformation. IEEE Access: Practical Innovations, Open Solutions, 11: 83523–83536
https://doi.org/10.1109/ACCESS.2023.3293432
|
| 23 |
D KouzapasN StylianidisC G PanayiotouD G (2023) Eliades. Ontology-based reasoning to reconFigure industrial processes for energy efficiency. In: Proceedings of 2023 31st Mediterranean Conference on Control and Automation (MED). 79–84
|
| 24 |
S, Li Z, Guo X, Zang (2023). Advancing the production of clinical medical devices through ChatGPT. Annals of Biomedical Engineering, 52( 3): 441–445
https://doi.org/10.1007/s10439-023-03300-3
|
| 25 |
X J LiX YinC Y LiP C ZhangX W Hu L ZhangL WangH HuL DongF Wei Y Choi (2020). Oscar: Object-semantics aligned pre-training for vision-language tasks. In: Proceedings of 16th European Conference on Computer Vision (ECCV 2020). 121–137
|
| 26 |
C Y LinF J Och (2004). Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In: Proceedings of 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), 605–612
|
| 27 |
R, Liu X, Xie (2024). Improve the industrial digital transformation through Industrial Internet platforms. Frontiers of Engineering Management, 11( 1): 167–174
https://doi.org/10.1007/s42524-023-0286-9
|
| 28 |
C A, Mallio A C, Sertorio C, Bernetti B, Beomonte Zobel (2023). Large language models for structured reporting in radiology: performance of GPT-4, ChatGPT-3.5, Perplexity and Bing. La Radiologia Medica, 128( 7): 808–812
https://doi.org/10.1007/s11547-023-01651-4
|
| 29 |
P A, Massey C, Montgomery A S, Zhang (2023). Comparison of ChatGPT-3.5, ChatGPT-4, and orthopaedic resident performance on orthopaedic assessment examinations. Journal of the American Academy of Orthopaedic Surgeons, 31( 23): 1173–1179
https://doi.org/10.5435/JAAOS-D-23-00396
|
| 30 |
R MokadyA HertzA H Bermano (2021). ClipCap: CLIP prefix for image captioning. Computer Science. arXiv: 2111.09734
|
| 31 |
S NairA RajeswaranV KumarC FinnA Gupta (2022). R3M: A universal visual representation for robot manipulation. arXiv: 2203.12601
|
| 32 |
D E, O’Leary (2023). Enterprise large language models: Knowledge characteristics, risks, and organizational activities. Intelligent Systems in Accounting, Finance & Management, 30( 3): 113–119
https://doi.org/10.1002/isaf.1541
|
| 33 |
J, Pavlopoulos A, Romell J, Curman O, Steinert T, Lindgren M, Borg K, Randl (2023). Automotive fault nowcasting with machine learning and natural language processing. Machine Learning, 113( 2): 843–861
https://doi.org/10.1007/s10994-023-06398-7
|
| 34 |
G, Peng Y, Cheng Y, Zhang J, Shao H, Wang W, Shen (2022). Industrial big data-driven mechanical performance prediction for hot-rolling steel using lower upper bound estimation method. Journal of Manufacturing Systems, 65: 104–114
https://doi.org/10.1016/j.jmsy.2022.08.014
|
| 35 |
A RadfordJ W KimC HallacyA RameshG Goh S AgarwalG SastryA AskellP MishkinJ Clark G KruegerI Sutskever (2021). Learning transferable visual models from natural language supervision. In: Proceedings of 38th International Conference on Machine Learning, Virtual
|
| 36 |
J RedmonA Farhadi (2017). YOLO9000: Better, faster, stronger. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 6517–6525
|
| 37 |
Y S, Semenov Y I, Shumelchyk V V, Horupakha I Y, Semion S V, Vashchenko O Y, Khudyakov I V, Chychov I H, Hulina R H, Zakharov (2022). Development and implementation of decision support systems for blast smelting control in the conditions of PrJSC “Kamet-Steel”. Metals, 12( 6): 985
https://doi.org/10.3390/met12060985
|
| 38 |
P SharmaN DingS GoodmanR Soricut (2018). Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In: Proceedings of 56th Annual Meeting of the Association-for-Computational-Linguistics (ACL), Melbourne, Australia, 2556–2565
|
| 39 |
J J, Shi S, Zeng X, Meng (2017). Intelligent data analytics is here to change engineering management. Frontiers of Engineering Management, 4( 1): 41–48
https://doi.org/10.15302/J-FEM-2017003
|
| 40 |
Y, Shi (2015). Challenges to engineering management in the big data era. Frontiers of Engineering Management, 2( 3): 293–303
https://doi.org/10.15302/J-FEM-2015042
|
| 41 |
J, Sievers T, Blank (2023). A systematic literature review on data-driven residential and industrial energy management systems. Energies, 16( 4): 1688
https://doi.org/10.3390/en16041688
|
| 42 |
C L SnoswellA J SnoswellJ T KellyL J CafferyA C Smith (2023). Artificial intelligence: Augmenting telehealth with large language models. Journal of Telemedicine and Telecare: 1357633X2311690
|
| 43 |
V K, Stepanov M S, Madzhumder D D, Begunova (2023). Exploring the potential of applying the artificial intelligence language model ChatGPT-3.5 in library and bibliographic activities. Scientific and Technical Information Processing, 50( 3): 166–175
https://doi.org/10.3103/S0147688223030036
|
| 44 |
R, Thiébaut B, Hejblum F, Mougin C, Tzourio L, Richert (2023). ChatGPT and beyond with artificial intelligence (AI) in health: Lessons to be learned. Joint, Bone, Spine, 90( 5): 105607
https://doi.org/10.1016/j.jbspin.2023.105607
|
| 45 |
R VedantamC L ZitnickD Parikh (2015). Cider: Consensus-based image description evaluation. In: Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4566–4575
|
| 46 |
J WeiY Tay R BommasaniC RaffelB ZophS BorgeaudD Yogatama M BosmaD ZhouD MetzlerE H Chi (2022). Emergent abilities of large language models. arXiv: 2206.07682
|
| 47 |
Y, Xiao S, Zheng J, Shi X, Du J, Hong (2023). Knowledge graph-based manufacturing process planning: A state-of-the-art review. Journal of Manufacturing Systems, 70: 417–435
https://doi.org/10.1016/j.jmsy.2023.08.006
|
| 48 |
Z, Yu Y, Gong (2024). ChatGPT, AI-generated content, and engineering management. Frontiers of Engineering Management, 11( 1): 159–166
https://doi.org/10.1007/s42524-023-0289-6
|
| 49 |
A ZengM AttarianB IchterK ChoromanskiA Wong S WelkerF TombariA PurohitM RyooV SindhwaniJ Lee (2022b). Socratic models: Composing zero-shot multimodal reasoning with language. arXiv: 2204.00598
|
| 50 |
A ZengP FlorenceJ TompsonS WelkerJ Chien M AttarianT ArmstrongI KrasinD DuongV SindhwaniJ Lee (2022a). Transporter networks: Rearranging the visual world for robotic manipulation. arXiv: 2010.14406
|
| 51 |
H, Zheng S, Liu H, Zhang J, Yu J, Bao (2024). Visual triggered contextual guidance for lithium battery disassembly: A multi-modal event knowledge graph approach. Journal of Engineering Design, 2024: 1–26
https://doi.org/10.1080/09544828.2024.2301876
|
| 52 |
L, Zhou H, Palangi L, Zhang H, Hu J, Corso J, Gao (2020). Unified vision-language pretraining for image captioning and VQA. Proceedings of the AAAI Conference on Artificial Intelligence, 34( 7): 13041–13049
https://doi.org/10.1609/aaai.v34i07.7005
|
| 53 |
T, Zhu X, Wang Y, Yu C, Li Q, Yao Y, Li (2023). Multi-process and multi-pollutant control technology for ultra-low emissions in the iron and steel industry. Journal of Environmental Sciences, 123: 83–95 (in Chinese)
https://doi.org/10.1016/j.jes.2022.01.044
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
