Please wait a minute...
购物车 申请加入邮件组 English
高级检索 图表检索
Frontiers of Structural and Civil Engineering

ISSN 2095-2430

ISSN 2095-2449(Online)

CN 10-1023/X

邮发代号 80-968

2019 Impact Factor: 1.68

热点文章  |  下载排行  |  浏览排行
在线投稿 免费RSS订阅
Frontiers of Structural and Civil Engineering  2024, Vol. 18 Issue (2): 294-308   https://doi.org/10.1007/s11709-024-1045-7
  本期目录
Automated identification of steel weld defects, a convolutional neural network improved machine learning approach
Zhan SHU1, Ao WU1, Yuning SI1, Hanlin DONG2, Dejiang WANG1(), Yifan LI3
1. School of Mechanics and Engineering Science, Shanghai University, Shanghai 200444, China
2. School of Civil Engineering, Shanghai Normal University, Shanghai 201418, China
3. Shanghai PinlanData Technology Co., Ltd., Shanghai 200072, China
 全文: PDF(8992 KB)   HTML
Abstract：

This paper proposes a machine-learning-based methodology to automatically classify different types of steel weld defects, including lack of the fusion, porosity, slag inclusion, and the qualified (no defects) cases. This methodology solves the shortcomings of existing detection methods, such as expensive equipment, complicated operation and inability to detect internal defects. The study first collected percussed data from welded steel members with or without weld defects. Then, three methods, the Mel frequency cepstral coefficients, short-time Fourier transform (STFT), and continuous wavelet transform were implemented and compared to explore the most appropriate features for classification of weld statuses. Classic and convolutional neural network-enhanced algorithms were used to classify, the extracted features. Furthermore, experiments were designed and performed to validate the proposed method. Results showed that STFT achieved higher accuracies (up to 96.63% on average) in the weld status classification. The convolutional neural network-enhanced support vector machine (SVM) outperformed six other algorithms with an average accuracy of 95.8%. In addition, random forest and SVM were efficient approaches with a balanced trade-off between the accuracies and the computational efforts.
Key words： steel weld    machine learning    convolutional neural network    weld defect detection    classification task    percussion
收稿日期: 2022-11-05      出版日期: 2024-06-07
Corresponding Author(s): Dejiang WANG   
 引用本文:   
. [J]. Frontiers of Structural and Civil Engineering, 2024, 18(2): 294-308.
Zhan SHU, Ao WU, Yuning SI, Hanlin DONG, Dejiang WANG, Yifan LI. Automated identification of steel weld defects, a convolutional neural network improved machine learning approach. Front. Struct. Civ. Eng., 2024, 18(2): 294-308.
 链接本文:  
https://academic.hep.com.cn/fsce/CN/10.1007/s11709-024-1045-7
https://academic.hep.com.cn/fsce/CN/Y2024/V18/I2/294
Fig.1  
Fig.2  
Label Specimen index Shape Weld status Dimension Tested signals Generated signals Final signals
Q Q01 T Q 300 × 5 × 8 82 738 820
Q02 T Q 300 × 5 × 8 82 738 820
Q03 T Q 300 × 8 × 8 80 720 800
Q04 L Q 300 × 8 × 10 40 360 400
Q05 T Q 150 × 8 × 10 40 360 400
Q06 L Q 150 × 8 × 10 22 198 220
Q07 T Q 300 × 5 × 8 80 720 800
Q08 L Q 300 × 8 × 10 40 360 400
Q09 T Q 150 × 8 × 10 42 378 420
Total – – – – 508 4572 5080
LF LF01 L LF 300 × 5 × 8 80 720 800
LF02 T LF 300 × 10 × 10 81 729 810
LF03 T LF 150 × 8 × 10 40 360 400
LF04 L LF 150 × 8 × 10 20 180 200
LF05 L LF 300 × 5 × 8 42 378 420
LF06 T LF 300 × 10 × 10 41 369 410
LF07 T LF 150 × 8 × 10 40 360 400
LF08 L LF 150 × 8 × 10 20 180 200
LF09 L LF 300 × 5 × 8 40 360 400
Total – – – – 404 3636 4040
P P01 L P 300 × 6 × 8 41 369 410
P02 L P 300 × 8 × 8 40 360 400
P03 T P 300 × 5 × 8 80 720 800
P04 T P 300 × 5 × 8 83 747 830
P05 T P 300 × 5 × 8 85 765 850
P06 T P 150 × 8 × 10 40 360 400
P07 L P 150 × 8 × 10 25 225 250
P08 T P 300 × 5 × 8 81 729 810
P09 L P 300 × 6 × 8 41 369 410
Total – – – – 516 4644 5160
SI SI01 T SI 300 × 6 × 8 97 873 970
SI02 T SI 150 × 8 × 8 42 378 420
SI03 L SI 300 × 6 × 6 55 495 550
SI04 L SI 150 × 8 × 10 39 351 390
SI05 T SI 300 × 6 × 8 80 720 800
SI06 T SI 150 × 8 × 8 50 450 500
SI07 L SI 300 × 6 × 6 51 459 510
SI08 L SI 150 × 8 × 10 15 135 150
SI09 T SI 300 × 6 × 8 40 360 400
Total – – – – 469 4221 4690
Tab.1  
Fig.3  
Fig.4  
Fig.5  
Fig.6  
Fig.7  
Algorithm Feature
MFCC STFT CWT
HMM (batchsize, 120, 47) (batchsize, 497, 1110) (batchsize, 497, 1110)
RF (batchsize, 5640) (batchsize, 3 × 480 × 640) (batchsize, 3 × 480 × 640)
KNN (batchsize, 5640) (batchsize, 3 × 480 × 640) (batchsize, 3 × 480 × 640)
SVM (batchsize, 5640) (batchsize, 3 × 480 × 640) (batchsize, 3 × 480 × 640)
CNN + RF (batchsize, 1, 120, 47) (batchsize, 3, 480, 640) (batchsize, 3, 480, 640)
CNN + KNN (batchsize, 1, 120, 47) (batchsize, 3, 480, 640) (batchsize, 3, 480, 640)
CNN + SVM (batchsize, 1, 120, 47) (batchsize, 3, 480, 640) (batchsize, 3, 480, 640)
Tab.2  
Layer Kernel size Stride Output shape Other details
Conv2D_1 3 × 3 1 (None, 4, 118, 73) activation = “ReLU”
BatchNorm2D_1 – – (None, 4, 118, 73) –
MaxPooling2D_1 2 × 2 2 (None, 4, 59, 36) –
Conv2D_2 3 × 3 1 (None, 16, 57, 34) activation = “ReLU”
BatchNorm2D_2 – – (None, 16, 57, 34) –
MaxPooling2D_2 2 × 2 2 (None, 16, 28, 17) –
Conv2D_3 3 × 3 1 (None, 64, 26, 15) activation = “ReLU”
BatchNorm2D_3 – – (None, 64, 26, 15) –
MaxPooling2D_3 2 × 2 2 (None, 64, 13, 7) –
Conv2D_4 3 × 3 1 (None, 32, 11, 5) activation = “ReLU”
BatchNorm2D_4 – – (None, 32, 11, 5) –
MaxPooling2D_4 2 × 2 2 (None, 32, 5, 2) –
Adaptive Average Pooling – – (None, 32, 3, 3) –
Dense_1 – – (None, 32) –
Dense_2 – – (None, 4) –
Tab.3  
Layer Kernel size Stride Output shape Other details
Conv2D_1 3 × 3 1 (None, 9, 478, 638) activation = “ReLU”
BatchNorm2D_1 – – (None, 9, 478, 638) –
MaxPooling2D_1 4 × 4 1 (None, 9, 475, 635) –
Conv2D_2 5 × 5 2 (None, 54, 236, 316) activation = “ReLU”
BatchNorm2D_2 – – (None, 54, 236, 316) –
MaxPooling2D_2 4 × 4 2 (None, 54, 117, 157) –
Conv2D_3 5 × 5 2 (None, 216, 57, 77) activation = “ReLU”
BatchNorm2D_3 – – (None, 216, 57, 77) –
MaxPooling2D_3 4 × 4 2 (None, 216, 27, 37) –
Conv2D_4 5 × 5 2 (None, 108, 12, 17) activation = “ReLU”
BatchNorm2D_4 – – (None, 108, 12, 17) –
MaxPooling2D_4 4 × 4 2 (None, 108, 5, 7) –
Adaptive Average Pooling – – (None, 108, 4, 4) –
Dense_1 – – (None, 1728) –
Dense_2 – – (None, 4) –
Tab.4  
Fig.8  
Fig.9  
Fig.10  
Fig.11  
Fig.12  
Fig.13  
Fig.14  
Fig.15  
Data set Feature
MFCC CWT STFT
OD 93.70 68.45 96.53
ED 96.13 86.87 96.73
average 94.92 77.66 96.63
Tab.5  
Classifier HMM RF KNN SVM CNN + RF CNN + KNN CNN + SVM
average accuracy 80.30 91.27 87.09 83.22 95.07 95.38 95.80
Tab.6  
1 G D Connolly, M J S Lowe, S I Rokhlin, J A G Temple, D O Thompson, D E Chimenti. Synthetically focused imaging techniques in simulated austenitic steel welds using an ultrasonic phased array. American Institute of Physics, 2010, 1211(1): 871–878
2 D Afshari, M Sedighi, M R Karimi, Z Barsoum. Prediction of residual stresses in resistance spot weld. Aircraft Engineering, 2016, 88(4): 492–497
3 T Du, J Sun, S Fu, C Zhang, Q Gao. Research on ultrasonic flaw detection of steel weld in spatial grid structure. IOP Conference Series: Materials Science and Engineering, 2017, 216: 012016
4 S Hu, W Wang, M S Alam, K Ke. Life-cycle benefits estimation of self-centering building structures. Engineering Structures, 2023, 284: 115982
5 N Nacereddine, D Ziou, L Hamami. Fusion-based shape descriptor for weld defect radiographic image retrieval. International Journal of Advanced Manufacturing Technology, 2013, 68(9−12): 2815–2832
6 S Sani, M P Ismail, S Mohd, N A Masenwat, T S T Amran, M S M Amin, M R Ahmad. Design and development of PC-based TOFD ultrasonic scanning system for welds inspection. In: proceedings of AIP Conference. New York: AIP Publishing, 2017, 1802: 050015
7 L Zhang, Y Zhang, B Dai, B Chen, Y Li. Welding defect detection based on local image enhancement. IET Image Processing, 2019, 13(13): 2647–2658
8 J Sun, C Li, X Wu, V Palade, W Fang. An effective method of weld defect detection and classification based on machine vision. IEEE Transactions on Industrial Informatics, 2019, 15(12): 6322–6333
9 M Malarvel, H Singh. An autonomous technique for weld defects detection and classification using multi-class support vector machine in X-radiography image. Optik (Stuttgart), 2021, 231(10): 166342
10 Y Li, X Gao, Y Zhang, D You, C Wang. Detection model of invisible weld defects by magneto-optical imaging at rotating magnetic field directions. Optics & Laser Technology, 2020, 121: 105772
11 X Gao, X Zhou, C Wang, N Ma, D You. Skin depth and detection ability of magneto-optical imaging for weld defects in alternating magnetic field. Journal of Manufacturing Systems, 2020, 55: 44–55
12 S Dorafshan, M Maguire, W Collins. Infrared thermography for weld inspection: Feasibility and application. Infrastructures, 2018, 3(4): 45
13 Z Xu, M Wu, W Fan. Sparse-based defect detection of weld feature guided waves with a fusion of shear wave characteristics. Measurement, 2021, 174: 109018
14 W Zeng, F Cai, F Wang, L Miao, F You, F Yao. Finite element simulation of laser-generated ultrasonic waves for quantitative detection of internal defects in welds. Optik (Stuttgart), 2020, 221: 165361
15 J Salamon, J P Bello. Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Processing Letters, 2017, 24(3): 279–283
16 Y Kim, J Sa, Y Chung, D Park, S Lee. Resource-efficient pet dog sound events classification using LSTM-FCN based on time-series data. Sensors (Basel), 2018, 18(11): 4019
17 T Fernando, H Ghaemmaghami, S Denman, S Sridharan, N Hussain, C Fookes. Heart sound segmentation using bidirectional LSTMs with attention. IEEE Journal of Biomedical and Health Informatics, 2020, 24(6): 1601–1609
18 S J Lim, S J Jang, J Y Lim, J H Ko. Classification of snoring sound based on a recurrent neural network. Expert Systems with Applications, 2019, 123: 237–245
19 A Sujono, B Santoso, W Endra. Sound vibration signal processing for detection and identification detonation (knock) to optimize performance Otto engine. AIP Publishing, 2016, LLC: 030003
20 S Bourke, D Nunes, F Stafford, G Hurley, I Graham. Percussion of the chest re-visited: A comparison of the diagnostic value of ausculatory and conventional chest percussion. Irish Journal of Medical Science, 1989, 158(4): 82–84
21 K P Ayodele, O Ogunlade, O J Olugbon, O B Akinwale, L O Kehinde. A medical percussion instrument using a wavelet-based method for archivable output and automatic classification. Computers in Biology and Medicine, 2020, 127: 104100
22 F Wang, G Song. Looseness detection in cup-lock scaffolds using percussion-based method. Automation in Construction, 2020, 118: 103266
23 D Chen, V Montano, L Huo, S Fan, G Song. Detection of subsurface voids in concrete-filled steel tubular (CFST) structure using percussion approach. Construction & Building Materials, 2020, 262: 119761
24 F Wang, G Song, Y Mo. Shear loading detection of through bolts in bridge structures using a percussion-based one-dimensional memory-augmented convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 36(3): 289–301
25 Y Zhou, S Wang, M Zhou, H Chen, C Yuan, Q Kong. Percussion-based bolt looseness identification using vibration-guided sound reconstruction. Structural Control and Health Monitoring, 2022, 29(2): e2876
26 S Hu, S Zhu, M S Alam, W Wang. Machine learning-aided peak and residual displacement-based design method for enhancing seismic performance of steel moment-resisting frames by installing self-centering braces. Engineering Structures, 2022, 271: 114935
27 S Hu, C Qiu, S Zhu. Machine learning-driven performance-based seismic design of hybrid self-centering braced frames with SMA braces and viscous dampers. Smart Materials and Structures, 2022, 31(10): 105024
28 S Hu, W Wang, M S Alam, S Zhu, K Ke. Machine learning-aided peak displacement and floor acceleration-based design of hybrid self-centering braced frames. Journal of Building Engineering, 2023, 72: 106429
29 S Hu, C Qiu, S Zhu. Floor acceleration control of self-centering braced frames using viscous dampers. Journal of Building Engineering, 2023, 74: 105944
30 A Raza, A Mehmood, S Ullah, M Ahmad, GS Choi, BW On. Heartbeat sound signal classification using deep learning. Sensors, 2019, 19(21): 4819
31 P Wang, C S Lim, S Chauhan, J Y A Foo, V Anantharaman. Phonocardiographic signal analysis method using a modified hidden Markov model. Annals of Biomedical Engineering, 2007, 35(3): 367–374
32 Z Zhou. Machine Learning. Beijing: Tsinghua University Publishing House Co., Ltd., 2016
33 J Robert. Manipulate audio with a simple and easy high level interface. 2022. (available at the website of GitHub)
34 P Chagas, L Souza, I Araújo, N Aldeman, A Duarte, M Angelo, W L C dos-Santos, L Oliveira. Classification of glomerular hypercellularity using convolutional features and support vector machine. Artificial Intelligence in Medicine, 2020, 103: 101808
35 Y Ma, Q Xie, Y Liu, S Xiong. A weighted KNN-based automatic image annotation method. Neural Computing & Applications, 2020, 32(11): 6559–6570
36 V NairG E Hinton. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning. Haifa: International Machine Learning Society, 2010, 807–814
37 A KrizhevskyI SutskeverG E Hinton. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 25(2)
38 K He, X Zhang, S Ren, J Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904–1916
[1] FSC-23045-OF-ZS_suppl_1 Download
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed