Knowledge enhanced graph inference network based entity-relation extraction and knowledge graph construction for industrial domain

doi:10.1007/s42524-023-0273-1

Front. Eng

2024, Vol. 11

Issue (1) : 143-158 https://doi.org/10.1007/s42524-023-0273-1

Knowledge enhanced graph inference network based entity-relation extraction and knowledge graph construction for industrial domain

Zhulin HAN(

), Jian WANG

College of Electronic and Information Engineering, Tongji University, Shanghai 200092, China

Download: PDF(6630 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

With the escalating complexity in production scenarios, vast amounts of production information are retained within enterprises in the industrial domain. Probing questions of how to meticulously excavate value from complex document information and establish coherent information links arise. In this work, we present a framework for knowledge graph construction in the industrial domain, predicated on knowledge-enhanced document-level entity and relation extraction. This approach alleviates the shortage of annotated data in the industrial domain and models the interplay of industrial documents. To augment the accuracy of named entity recognition, domain-specific knowledge is incorporated into the initialization of the word embedding matrix within the bidirectional long short-term memory conditional random field (BiLSTM-CRF) framework. For relation extraction, this paper introduces the knowledge-enhanced graph inference (KEGI) network, a pioneering method designed for long paragraphs in the industrial domain. This method discerns intricate interactions among entities by constructing a document graph and innovatively integrates knowledge representation into both node construction and path inference through TransR. On the application stratum, BiLSTM-CRF and KEGI are utilized to craft a knowledge graph from a knowledge representation model and Chinese fault reports for a steel production line, specifically SPOnto and SPFRDoc. The F1 value for entity and relation extraction has been enhanced by 2% to 6%. The quality of the extracted knowledge graph complies with the requirements of real-world production environment applications. The results demonstrate that KEGI can profoundly delve into production reports, extracting a wealth of knowledge and patterns, thereby providing a comprehensive solution for production management.

Keywords knowledge graph construction industrial BiLSTM-CRF document-level relation extraction graph inference

Corresponding Author(s): Zhulin HAN

Just Accepted Date: 17 January 2024 Online First Date: 27 February 2024 Issue Date: 13 March 2024

Cite this article:

Zhulin HAN,Jian WANG. Knowledge enhanced graph inference network based entity-relation extraction and knowledge graph construction for industrial domain[J]. Front. Eng, 2024, 11(1): 143-158.

URL:

https://academic.hep.com.cn/fem/EN/10.1007/s42524-023-0273-1
https://academic.hep.com.cn/fem/EN/Y2024/V11/I1/143

Fig.1 Framework of the knowledge graph construction method based on document-level relation extraction.

Fig.2 BiLSTM-CRF network model combined with a dictionary.

Fig.3 Process of KEGI.

Tab.1 Entity category information of SPFRDoc

Tab.2 Relation category information of SPFRDoc

Tab.3 The results of the entity recognition task

Fig.4 Model loss curves during training.

Tab.4 The results of the comparative experiment

Tab.5 The results of the relation extraction task

Fig.5 Comparison of the BERT-RE model and the KEGI model in a specific case.

Tab.6 The results of the ablation experiment

Fig.6 The fault diagnosis knowledge graph of the Chinese steel manufacturing industry.

1	D BahdanauK ChoY Bengio (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint. arXiv:1409.0473
2	P CaoY ChenK LiuJ ZhaoS Liu (2018). Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Brussels: ACL, 182–192
3	F ChristopoulouM MiwaS Ananiadou (2019). Connecting the dots: Document-level neural relation extraction with edge-oriented graphs. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing / 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong: ACL, 4925–4936
4	J Deng, T Wang, Z Wang, J Zhou, L Cheng, (2022). Research on event logic knowledge graph construction method of robot transmission system fault diagnosis. IEEE Access, 10: 17656–17673 https://doi.org/10.1109/ACCESS.2022.3150409
5	J Dong, J Wang, S Chen, (2021). Knowledge graph construction based on knowledge enhanced word embedding model in manufacturing domain. Journal of Intelligent & Fuzzy Systems, 41( 2): 3603–3613 https://doi.org/10.3233/JIFS-210982
6	W Gui, Z Zeng, X Chen, Y Xie, Y Sun, (2020). Knowledge-driven process industry smart manufacturing. Scientia Sinica Informationis, 50( 9): 1345–1360 (in Chinese) https://doi.org/10.1360/SSI-2020-0211
7	Z GuoY ZhangW Lu (2019). Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 241–251
8	X HanT GaoY LinH PengY YangC XiaoZ LiuP LiJ ZhouM Sun (2020). More data, more relations, more context and more openness: A review and outlook for relation extraction. In: Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing. Suzhou: ACL, 745–758
9	L Hu, G Wu, Y Xing, F Wang, (2020). Things2Vec: Semantic modeling in the Internet of Things with graph representation learning. IEEE Internet of Things Journal, 7( 3): 1939–1948 https://doi.org/10.1109/JIOT.2019.2962630
10	Z HuangW XuK Yu (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint. arXiv:1508.01991
11	A Huet, R Pinquié, P Véron, A Mallet, F Segonds, (2021). CACDA: A knowledge graph for a context-aware cognitive design assistant. Computers in Industry, 125: 103377 https://doi.org/10.1016/j.compind.2020.103377
12	S Kamble, A Gunasekaran, S Gawankar, (2018). Sustainable Industry 4.0 framework: A systematic literature review identifying the current trends and future perspectives. Process Safety and Environmental Protection, 117: 408–425 https://doi.org/10.1016/j.psep.2018.05.009
13	T N KipfM Welling (2016). Semi-supervised classification with graph convolutional networks. arXiv preprint. arXiv:1609.02907
14	J LaffertyA McCallumF Pereira (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann Publishers Inc., 282–289
15	G Li, R Pan, J Mao, Y Cao, (2020). Entity recognition of Chinese electronic medical records based on BiLSTM-CRF network and dictionary resources. Journal of Modern Information, 40( 4): 3–12, 58 (in Chinese)
16	J Li, A Sun, J Han, C Li, (2022). A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, 34( 1): 50–70 https://doi.org/10.1109/TKDE.2020.2981314
17	H Lin, Y Liu, W Wang, Y Yue, Z Lin, (2017). Learning entity and relation embeddings for knowledge resolution. Procedia Computer Science, 108: 345–354 https://doi.org/10.1016/j.procs.2017.05.045
18	Y LinT TsaiW ChouK WuT SungW Hsu (2004). A maximum entropy approach to biomedical named entity recognition. In: Proceedings of the 4th International Conference on Data Mining in Bioinformatics. Seattle WA: Springer-Verlag, 56–61
19	M Liu, X Li, J Li, Y Liu, B Zhou, J Bao, (2022). A knowledge graph-based data representation approach for IIoT-enabled cognitive manufacturing. Advanced Engineering Informatics, 51: 101515 https://doi.org/10.1016/j.aei.2021.101515
20	W LiuP ZhouZ ZhaoZ WangQ JuH DengP Wang (2020). K-BERT: Enabling language representation with knowledge graph. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, NY: AAAI, 2901–2908
21	Z Lyu, Y Wu, J Lai, M Yang, C Li, W Zhou, (2023). Knowledge enhanced graph neural networks for explainable recommendation. IEEE Transactions on Knowledge and Data Engineering, 35( 5): 4954–4968 https://doi.org/10.1109/TKDE.2022.3142260
22	G NanZ GuoI SekulicW Lu (2020). Reasoning with latent structure refinement for document-level relation extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). Online: ACL, 1546–1557
23	A QiT SinM FathullahC Lee (2017). The impact of fit manufacturing on green manufacturing: A review. In: Proceedings of the 3rd Electronic and Green Materials International Conference (EGM). Krabi: AIP, 020083
24	H Ren, Z Chen, Z Jiang, C Yang, W Gui, (2021). An industrial multilevel knowledge graph-based local–global monitoring for plant-wide processes. IEEE Transactions on Instrumentation and Measurement, 70: 1–15 https://doi.org/10.1109/TIM.2021.3125110
25	S SahuF ChristopoulouM MiwaS Ananiadou (2019). Inter-sentence relation extraction with document-level graph convolutional neural network. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL). Florence: ACL, 4309–4316
26	H Shi, D Huang, L Wang, M Wu, Y Xu, B Zeng, C Pang, (2021). An information integration approach to spacecraft fault diagnosis. Enterprise Information Systems, 15( 8): 1128–1161 https://doi.org/10.1080/17517575.2019.1709663
27	E StrubellP VergaD BelangerA McCallum (2017). Fast and accurate entity recognition with iterated dilated convolutions. arXiv preprint. arXiv:1702.02098
28	Z Wan, P Ge, X Zhang, G Yin, (2018). Research on equipment manufacturing industry upgrading under intelligent manufacturing. World Sci-Tech R & D, 40( 3): 316–327 (in Chinese)
29	B Wang, B Yi, Z Liu, Y Zhou, Y Zhou, (2021). Evolution and state-of-the-art of intelligent manufacturing from HCPS perspective. Computer Integrated Manufacturing Systems, 27( 10): 2749–2761 (in Chinese)
30	B Wang, J Zang, X Qu, J Dong, Y Zhou, (2018). Research on new-generation intelligent manufacturing based on human-cyber-physical systems. Strategic Study of CAE, 20( 4): 29–34 https://doi.org/10.15302/J-SSCAE-2018.04.006
31	H WangC FockeR SylvesterN MishraW Wang (2019). Fine-tune BERT for DocRED with two-step process. arXiv preprint. arXiv:1909.11898
32	C XiaoY YaoR XieX HanZ LiuM SunF LinL Lin (2020). Denoising relation extraction from document-level distant supervision. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: ACL, 3683–3688
33	B XuQ WangY LyuY ZhuZ Mao (2021). Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction. In: Proceedings of the AAAI Conference on Artificial Intelligence. Online: AAAI, 14149–14157
34	Z Xu, Y Dang, Z Zhang, J Chen, (2020). Typical short-term remedy knowledge mining for product quality problem-solving based on bipartite graph clustering. Computers in Industry, 122: 103277 https://doi.org/10.1016/j.compind.2020.103277
35	Y YaoD YeP LiX HanY LinZ LiuZ LiuL HuangJ ZhouM Sun (2019). DocRED: A large-scale document-level relation extraction dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL). Florence: ACL, 764–777
36	S ZengR XuB ChangL Li (2020). Double graph based reasoning for document-level relation extraction. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: ACL, 1630–1640
37	D Zhang, Z Liu, W Jia, H Liu, J Tan, (2021). A review on knowledge graph and its application prospects to intelligent manufacturing. Journal of Mechanical Engineering, 57( 5): 90–113 (in Chinese) https://doi.org/10.3901/JME.2021.05.090
38	Z ZhangX HanZ LiuX JiangM SunQ Liu (2019). ERNIE: Enhanced language representation with informative entities. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL). Florence: ACL, 1441–1451
39	P Zheng, L Xia, C Li, X Li, B Liu, (2021). Towards Self-X cognitive manufacturing network: An industrial knowledge graph-based multi-agent reinforcement learning approach. Journal of Manufacturing Systems, 61: 16–26 https://doi.org/10.1016/j.jmsy.2021.08.002
40	J Zhou, Y Zhou, B Wang, J Zang, (2019). Human-Cyber-Physical Systems (HCPSs) in the context of new-generation intelligent manufacturing. Engineering, 5( 4): 624–636 https://doi.org/10.1016/j.eng.2019.07.015
41	W ZhouK HuangT MaJ Huang (2021). Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI Conference on Artificial Intelligence. Online: AAAI, 14612–14620
42	Y Zhou, H Huang, H Liu, Z Hao, (2022). Survey on document-level relation extraction. Journal of South China University of Technology (Natural Science Edition), 50( 4): 10–25 (in Chinese)

[1]	Zhao XU, Xiang WANG, Gang WU. Exploring the coupling relationship of industrial agglomeration and low-carbon economy considering spatiotemporal differentiation: An empirical study in China’s construction machinery industry[J]. Front. Eng, 2023, 10(2): 285-299.
[2]	Zhengang GUO, Yingfeng ZHANG, Sichao LIU, Xi Vincent WANG, Lihui WANG. Exploring self-organization and self-adaption for smart manufacturing complex networks[J]. Front. Eng, 2023, 10(2): 206-222.
[3]	Oluwole Alfred OLATUNJI. Causations of failure in megaprojects: A case study of the Ajaokuta Steel Plant project[J]. Front. Eng, 2018, 5(3): 334-346.
[4]	Xiao-jun Liu, Zhen-yu Guo, Chao Ding, Han-liang Fu. Industrial Structure Adjustment of Shaanxi Province under the Limited Water Resources Condition[J]. Front. Eng, 2015, 1(4): 378-384.
[5]	Da-yuan Li,Min Huang. Industrial Greening of the Dongting Lake Eco-economic Zone[J]. Front. Eng, 2015, 2(3): 228-233.
[6]	Wei Chen, Wen Zeng, Ming-de Tong, Jin Yan. Comparative Analysis of the Solid Waste Emissions between the Industrialized and Traditional Residential Building[J]. Front. Eng, 2015, 2(2): 188-192.
[7]	Bin-shi Xu,Pei-jing Shi,Han-dong Zheng,En-zhong Li. Engineering Management Problems of Remanufacturing Industry[J]. Front. Eng, 2015, 2(1): 13-18.
[8]	Mirosɬaw J. Skibniewski. Research Trends in Information Technology Applications in Construction Safety Engineering and Management[J]. Front. Eng, 2014, 1(3): 246-259.
[9]	Li-da Xu. Engineering Informatics: State of the Art and Future Trends[J]. Front. Eng, 2014, 1(3): 270-282.
[10]	Qian Yi. NewIndustrial Revolution and Environmental Protection[J]. Front. Eng, 2014, 1(1): 71-75.

Viewed

Full text

Abstract

Cited

Shared

Discussed