Digital image correlation-based structural state detection through deep learning

doi:10.1007/s11709-021-0777-x

Front. Struct. Civ. Eng.

2022, Vol. 16

Issue (1) : 45-56 https://doi.org/10.1007/s11709-021-0777-x

RESEARCH ARTICLE

Digital image correlation-based structural state detection through deep learning

Shuai TENG¹, Gongfa CHEN¹(

), Shaodi WANG^1,², Jiqiao ZHANG¹, Xiaoli SUN^1,³

¹. School of Civil and Transportation Engineering, Guangdong University of Technology, Guangzhou 510006, China
². Earthquake Engineering Research & Test Center, Guangzhou University, Guangzhou 510405, China
³. Guangzhou Municipal Engineering Testing Co., Ltd., Guangzhou 510520, China

Download: PDF(35854 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

This paper presents a new approach for automatical classification of structural state through deep learning. In this work, a Convolutional Neural Network (CNN) was designed to fuse both the feature extraction and classification blocks into an intelligent and compact learning system and detect the structural state of a steel frame; the input was a series of vibration signals, and the output was a structural state. The digital image correlation (DIC) technology was utilized to collect vibration information of an actual steel frame, and subsequently, the raw signals, without further pre-processing, were directly utilized as the CNN samples. The results show that CNN can achieve 99% classification accuracy for the research model. Besides, compared with the backpropagation neural network (BPNN), the CNN had an accuracy similar to that of the BPNN, but it only consumes 19% of the training time. The outputs of the convolution and pooling layers were visually displayed and discussed as well. It is demonstrated that: 1) the CNN can extract the structural state information from the vibration signals and classify them; 2) the detection and computational performance of the CNN for the incomplete data are better than that of the BPNN; 3) the CNN has better anti-noise ability.

Keywords structural state detection deep learning digital image correlation vibration signal steel frame

Corresponding Author(s): Gongfa CHEN

Just Accepted Date: 19 November 2021 Online First Date: 04 January 2022 Issue Date: 07 March 2022

Cite this article:

Shuai TENG,Gongfa CHEN,Shaodi WANG, et al. Digital image correlation-based structural state detection through deep learning[J]. Front. Struct. Civ. Eng., 2022, 16(1): 45-56.

URL:

https://academic.hep.com.cn/fsce/EN/10.1007/s11709-021-0777-x
https://academic.hep.com.cn/fsce/EN/Y2022/V16/I1/45

Fig.1 Technological process of structural state detection.

Fig.2 Experimental layout and the model with 355 rods. (a) Front view; (b) side view; (c) ball joint.

Fig.3 Experimental facilities. (a) Hammer; (b) Nikon D5300 camera.

Fig.4 Tracking process of the moving point.

Fig.5 Excitation positions and signal acquisition region.

Fig.6 Intact rods (I-1 and I-2) and damaged rods (D-1 and D-2).

Fig.7 The sample acquisition of the intact structure.

Tab.1 The database of detection stage.

Tab.2 Structural parameters of the CNN

Fig.8 The CNN architecture.

Fig.9 Scenarios of incomplete data. (a) The normal data; (b) node 1 missing data; (c) node 2 missing data.

Fig.10 The displacement-time history curves of the intact structure. N-#: Node of signal acquisition.

Tab.3 The result of the k-fold cross validation

Tab.4 The detection effect of the first stage

Tab.5 The detection effect of the second stage

Tab.6 Detection results of the BPNN

Fig.11 The accuracy and loss of training and validation processes. (a) Training process; (b) validation process.

Tab.7 Detection results of incomplete data by the CNN

Tab.8 Detection results of incomplete data by the BPNN

Tab.9 Testing results under different signal-to-noise ratio

Fig.12 The 3-D images of the samples. (a) The sample of State 1; (b) the sample of State 2.

Fig.13 The convolution and pooling processes of the sample of State 1.

Fig.14 The convolution and pooling processes of the samples of State 2.

Fig.15 The outputs of fully connected layer. (a) The samples of State 1; (b) the samples of State 2.

1	Y Zhang, Y Miyamori, S Mikami, T Saito. Vibration-based structural state identification by a 1-dimensional convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2019, 34( 9): 822– 839 https://doi.org/10.1111/mice.12447
2	H Guo, H Zheng, X Zhuang. Numerical manifold method for vibration analysis of Kirchhoff’s plates of arbitrary geometry. Applied Mathematical Modelling, 2019, 66 : 695– 727 https://doi.org/10.1016/j.apm.2018.10.006
3	X Jiang, H Adeli. Intelligent Infrastructure-Neural Networks, Wavelets, and Chaos Theory. Boca Raton: CRC Press, Taylor & Francis, 2008
4	P Cawley, R D Adams. The location of defects in structures from measurements of natural frequencies. Journal of Strain Analysis for Engineering Design, 1979, 14( 2): 49– 57 https://doi.org/10.1243/03093247V142049
5	A Katunin. Identification of structural damage using S-transform from 1D and 2D mode shapes. Measurement, 2021, 173 : 108656– https://doi.org/10.1016/j.measurement.2020.108656
6	Y Cha, O Buyukozturk. Structural damage detection using modal strain energy and hybrid multiobjective optimization. Computer-Aided Civil and Infrastructure Engineering, 2015, 30( 5): 347– 358 https://doi.org/10.1111/mice.12122
7	A Dutta, S Talukdar. Damage detection in bridges using accurate modal parameters. Finite Elements in Analysis and Design, 2004, 40( 3): 287– 304 https://doi.org/10.1016/S0168-874X(02)00227-5
8	S Al-Jailawi, S Rahmatalla. Transmissibility-based damage detection using angular velocity versus acceleration. Journal of Civil Structural Health Monitoring, 2018, 8( 4): 649– 659 https://doi.org/10.1007/s13349-018-0297-0
9	W Hu, E Caetano, Á Cunha. Structural health monitoring of a stress-ribbon footbridge. Engineering Structures, 2013, 57 : 578– 593 https://doi.org/10.1016/j.engstruct.2012.06.051
10	Y Xia, B Chen, S Weng, Y Q Ni, Y L Xu. Temperature effect on vibration properties of civil structures: A literature review and case studies. Journal of Civil Structural Health Monitoring, 2012, 2( 1): 29– 46 https://doi.org/10.1007/s13349-011-0015-7
11	S H Sung, K Y Koo, H J Jung. Modal flexibility-based damage detection of cantilever beam-type structures using baseline modification. Journal of Sound and Vibration, 2014, 333( 18): 4123– 4138 https://doi.org/10.1016/j.jsv.2014.04.056
12	B Jaishi, W X Ren. Damage detection by finite element model updating using modal flexibility residual. Journal of Sound and Vibration, 2006, 290( 1−2): 369– 387 https://doi.org/10.1016/j.jsv.2005.04.006
13	J Li, B Wu, Q C Zeng, C W Lim. A generalized flexibility matrix based approach for structural damage detection. Journal of Sound and Vibration, 2010, 329( 22): 4583– 4587 https://doi.org/10.1016/j.jsv.2010.05.024
14	Q Lu, G Ren, Y Zhao. Multiple damage location with flexibility curvature and relative frequency change for beam structures. Journal of Sound and Vibration, 2002, 253( 5): 1101– 1114 https://doi.org/10.1006/jsvi.2001.4092
15	J Pal, S Banerjee. A combined modal strain energy and particle swarm optimization for health monitoring of structures. Journal of Civil Structural Health Monitoring, 2015, 5( 4): 353– 363 https://doi.org/10.1007/s13349-015-0106-y
16	H Hu, C Wu. Development of scanning damage index for the damage detection of plate structures using modal strain energy method. Mechanical Systems and Signal Processing, 2009, 23( 2): 274– 287 https://doi.org/10.1016/j.ymssp.2008.05.001
17	A K Pandey, M Biswas, M M Samman. Damage detection from changes in curvature mode shapes. Journal of Sound and Vibration, 1991, 145( 2): 321– 332 https://doi.org/10.1016/0022-460X(91)90595-B
18	C Anitescu, E Atroshchenko, N Alajlan, T Rabczuk. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59( 1): 345– 359 https://doi.org/10.32604/cmc.2019.06641
19	H Guo, X Zhuang, T Rabczuk. A deep collocation method for the bending analysis of Kirchhoff plate. Computers, Materials & Continua, 2019, 59( 2): 433– 456 https://doi.org/10.32604/cmc.2019.06660
20	E Samaniego, C Anitescu, S Goswami, V M Nguyen-Thanh, H Guo, K Hamdia, X Zhuang, T Rabczuk. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362 : 112790– https://doi.org/10.1016/j.cma.2019.112790
21	H Xu, J M Humar. Damage detection in a girder bridge by artificial neural network technique. Computer-Aided Civil and Infrastructure Engineering, 2006, 21( 6): 450– 464 https://doi.org/10.1111/j.1467-8667.2006.00449.x
22	X Yao. Evolutionary Artificial Neural Networks. International Journal of Neural Systems, 1993, 4( 3): 203– 222 https://doi.org/10.1142/S0129065793000171
23	X Zhuang, H Guo, N Alajlan, H Zhu, T Rabczuk. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics. A, Solids, 2021, 87 : 104225– https://doi.org/10.1016/j.euromechsol.2021.104225
24	Y Lecun, Y Bengio. Convolutional Networks for Images, Speech, and Time Series. The Handbook of Brain Theory and Neural Networks. Cambridge: MIT Press, 1995
25	Y Lécun, L Bottou, Y Bengio, P Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86( 11): 2278– 2324 https://doi.org/10.1109/5.726791
26	P J S Cruz, R Salgado. Performance of Vibration-Based Damage Detection Methods in Bridges. Computer-Aided Civil and Infrastructure Engineering, 2009, 24( 1): 62– 79 https://doi.org/10.1111/j.1467-8667.2008.00546.x
27	O Abdeljaber, O Avci, S Kiranyaz, M Gabbouj, D J Inman. Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks. Journal of Sound and Vibration, 2017, 388 : 154– 170 https://doi.org/10.1016/j.jsv.2016.10.043
28	S Teng, G Chen, P Gong, G Liu, F Cui. Structural damage detection using convolutional neural networks combining strain energy and dynamic response. Meccanica, 2020, 55( 4): 945– 959 https://doi.org/10.1007/s11012-019-01052-w
29	S Teng, G Chen, G Liu, J Lv, F Cui. Modal strain energy-based structural damage detection using convolutional neural networks. Applied Sciences-Basel, 2019, 9( 16): 3376– https://doi.org/10.3390/app9163376
30	T Khuc, N Catbas. Completely contactless structural health monitoring of real-life structures using cameras and computer vision. Structural Control and Health Monitoring, 2017, 24( 1): e1852– https://doi.org/10.1002/stc.1852
31	A M Wahbeh, J P Caffrey, S F Masri. A vision-based approach for the direct measurement of displacements in vibrating systems. Smart Materials and Structures, 2003, 12( 5): 785– 794 https://doi.org/10.1088/0964-1726/12/5/016
32	S W Kim, N S Kim. Dynamic characteristics of suspension bridge hanger cables using digital image processing. NDT & E International, 2013, 59 : 25– 33 https://doi.org/10.1016/j.ndteint.2013.05.002
33	T C Chu, W F Ranson, M A Sutton. Applications of digital-image-correlation techniques to experimental mechanics. Experimental Mechanics, 1985, 25( 3): 232– 244 https://doi.org/10.1007/BF02325092
34	B Pan, K Qian, H Xie, A Asundi. Two-dimensional digital image correlation for in-plane displacement and strain measurement: A review. Measurement Science & Technology, 2009, 20( 6): 062001– https://doi.org/10.1088/0957-0233/20/6/062001
35	G Chen, Z Wu, C Gong, J Zhang, X Sun. DIC-based operational modal analysis of bridges. Advances in Civil Engineering, 2021, 2021 : 1– 13
36	G Chen, Q Liang, W Zhong, X Gao, F Cui. Homography-based measurement of bridge vibration using UAV and DIC method. Measurement, 2021, 170 : 108683– https://doi.org/10.1016/j.measurement.2020.108683
37	Y Zhang, Y Yang. Cross-validation for selecting a model selection procedure. Journal of Econometrics, 2015, 187( 1): 95– 112 https://doi.org/10.1016/j.jeconom.2015.02.006
38	X Geng, S Lu, M Jiang, Q Sui, S Lv, H Xiao, Y Jia, L Jia. Researchon FBG-Based CFRP structural damage identification using BP neural network. Photonic Sensors, 2018, 8( 2): 168– 175 https://doi.org/10.1007/s13320-018-0466-0
39	Y J Yan, L Cheng, Z Y Wu, L H Yam. Development in vibration-based structural damage detection technique. Mechanical Systems and Signal Processing, 2007, 21( 5): 2198– 2211 https://doi.org/10.1016/j.ymssp.2006.10.002

[1]	Dong ZHOU, Yicheng YE, Nanyan HU, Weiqi WANG, Xianhua WANG. Crack evolution of soft–hard composite layered rock-like specimens with two fissures under uniaxial compression[J]. Front. Struct. Civ. Eng., 2021, 15(6): 1372-1389.
[2]	Trong NGHIA-NGUYEN, Mamoru KIKUMOTO, Samir KHATIR, Salisa CHAIYAPUT, H. NGUYEN-XUAN, Thanh CUONG-LE. Data-driven approach to solve vertical drain under time-dependent loading[J]. Front. Struct. Civ. Eng., 2021, 15(3): 696-711.
[3]	Alipujiang JIERULA, Tae-Min OH, Shuhong WANG, Joon-Hyun LEE, Hyunwoo KIM, Jong-Won LEE. Detection of damage locations and damage steps in pile foundations using acoustic emissions with deep learning technology[J]. Front. Struct. Civ. Eng., 2021, 15(2): 318-332.
[4]	Harun TANYILDIZI, Abdulkadir ŞENGÜR, Yaman AKBULUT, Murat ŞAHİN. Deep learning model for estimating the mechanical properties of concrete containing silica fume exposed to high temperatures[J]. Front. Struct. Civ. Eng., 2020, 14(6): 1316-1330.
[5]	Mohammad Sadegh ES-HAGHI, Aydin SHISHEGARAN, Timon RABCZUK. Evaluation of a novel Asymmetric Genetic Algorithm to optimize the structural design of 3D regular and irregular steel frames[J]. Front. Struct. Civ. Eng., 2020, 14(5): 1110-1130.
[6]	Yijie HUANG, Xujia HE, Qing WANG, Jianzhuang XIAO. Deformation field and crack analyses of concrete using digital image correlation method[J]. Front. Struct. Civ. Eng., 2019, 13(5): 1183-1199.
[7]	E. D. PASIOU, I. STAVRAKAS, D. TRIANTIS, S. K. KOURKOULIS. Marble epistyles under shear: an experimental study of the role of “Relieving Space”[J]. Front. Struct. Civ. Eng., 2019, 13(4): 767-786.
[8]	Anthony Deloge ARIYANAYAGAM,Mahen MAHENDRAN. Development of realistic design fire time-temperature curves for the testing of cold-formed steel wall systems[J]. Front. Struct. Civ. Eng., 2014, 8(4): 427-447.
[9]	Hui ZHU, Yuching WU. Finite element analysis of creep for plane steel frames in fire[J]. Front Struc Civil Eng, 2012, 6(3): 297-307.
[10]	Xin CHENG, Xianzhong ZHAO, Yiyi CHEN, Zhenyu LI. A model study on affordable steel residential housing in China[J]. Front Struc Civil Eng, 2012, 6(3): 288-296.
[11]	CHEN Yiyi, BIAN Ruoning. Tests on impact effect of partial fracture at steel frame connections[J]. Front. Struct. Civ. Eng., 2008, 2(4): 295-301.

Viewed

Full text

Abstract

Cited

Shared

Discussed