1. State Key Laboratory of Hydroscience and Engineering, Tsinghua University, Beijing 100084, China 2. China Renewable Energy Engineering Institute, Beijing 100120, China 3. State Key Laboratory of Stimulation and Regulation of Water Cycles in River Basins, China Institute of Water Resources and Hydropower Research, Beijing 100038, China 4. Hunan Provincial Water Resources Development & Investment Co., Ltd., Changsha 410007, China
Regular detection and repair for lining cracks are necessary to guarantee the safety and stability of tunnels. The development of computer vision has greatly promoted structural health monitoring. This study proposes a novel encoder–decoder structure, CrackRecNet, for semantic segmentation of lining segment cracks by integrating improved VGG-19 into the U-Net architecture. An image acquisition equipment is designed based on a camera, 3-dimensional printing (3DP) bracket and two laser rangefinders. A tunnel concrete structure crack (TCSC) image data set, containing images collected from a double-shield tunnel boring machines (TBM) tunnel in China, was established. Through data preprocessing operations, such as brightness adjustment, pixel resolution adjustment, flipping, splitting and annotation, 2880 image samples with pixel resolution of 448 × 448 were prepared. The model was implemented by Pytorch in PyCharm processed with 4 NVIDIA TITAN V GPUs. In the experiments, the proposed CrackRecNet showed better prediction performance than U-Net, TernausNet, and ResU-Net. This paper also discusses GPU parallel acceleration effect and the crack maximum width quantification.
M F Lei, L H Liu, C H Shi, Y Tan, Y X Lin, W D Wang. A novel tunnel-lining crack recognition system based on digital image technology. Tunnelling and Underground Space Technology, 2021, 108: 103724 https://doi.org/10.1016/j.tust.2020.103724
2
Y YangL F WangY F ZhangX J Han. Multi-feature fusion based classification algorithm of surface disease image of concrete structure. Journal of Chang’an University: Natural Science Edition, 2021, 41(3): 64−74 (in Chinese)
3
S Zhao, M Shadabfar, D M Zhang, J Y Chen, H W Huang. Deep learning-based classification and instance segmentation of leakage-area and scaling images of shield tunnel linings. Structural Control and Health Monitoring, 2021, 28(6): e2732 https://doi.org/10.1002/stc.2732
4
Y Yao, S T E Tung, B Glisic. Crack detection and characterization techniques—An overview. Structural Control and Health Monitoring, 2014, 21(12): 1387–1413 https://doi.org/10.1002/stc.1655
5
Y R Liu, S K Hou, C Y Li, H W Zhou, F Jin, P X Qin, Q Yang. Study on support time in double-shield TBM tunnel based on self-compacting concrete backfilling material. Tunnelling and Underground Space Technology, 2020, 96: 103212 https://doi.org/10.1016/j.tust.2019.103212
6
C Koch, K Georgieva, V Kasireddy, B Akinci, P Fieguth. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Advanced Engineering Informatics, 2015, 29(2): 196–210 https://doi.org/10.1016/j.aei.2015.01.008
7
J Liu, X Yang, S Lau, X Wang, S Luo, V C S Lee, L Ding. Automated pavement crack detection and segmentation based on two-step convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291–1305 https://doi.org/10.1111/mice.12622
8
J Valença, E Julio. MCrack-Dam: the scale-up of a method to assess cracks on concrete dams by image processing. The case study of Itaipu Dam, at the Brazil–Paraguay border. Journal of Civil Structural Health Monitoring, 2018, 8(5): 857–866 https://doi.org/10.1007/s13349-018-0309-0
9
Y Wang, J Y Zhang, J X Liu, Y Zhang, Z P Chen, C G Li, R B Yan. Research on crack detection algorithm of the concrete bridge based on image processing. Procedia Computer Science, 2019, 154: 610–616 https://doi.org/10.1016/j.procs.2019.06.096
10
L Ying, E Salari. Beamlet transform-based technique for pavement crack detection and classification. Computer-Aided Civil and Infrastructure Engineering, 2010, 25(8): 572–580 https://doi.org/10.1111/j.1467-8667.2010.00674.x
11
I Abdel-Qader, O Abudayyeh, M E Kelly. Analysis of edge-detection techniques for crack identification in bridges. Journal of Computing in Civil Engineering, 2003, 17(4): 255–263 https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255
12
H OliveiraP L Correia. Automatic road crack segmentation using entropy and image dynamic thresholding. In: Proceedings of the 17th European Signal Processing Conference. Glasgow: IEEE, 2009: 622–626
13
Y Xu, D Li, Q Xie, Q Wu, J Wang. Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement, 2021, 178: 109316 https://doi.org/10.1016/j.measurement.2021.109316
14
C Feng, H Zhang, H Wang, S Wang, Y Li. Automatic pixel-level crack detection on dam surface using deep convolutional network. Sensors, 2020, 20(7): 2069 https://doi.org/10.3390/s20072069
15
Y Shi, L Cui, Z Qi, F Meng, Z Chen. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434–3445 https://doi.org/10.1109/TITS.2016.2552248
16
Y NohD Koo Y M KangD ParkD Lee. Automatic crack detection on concrete images using segmentation via fuzzy C-means clustering. In: Proceedings of the International conference on applied system innovation (ICASI). Sapporo: IEEE, 2017: 877–880
17
G Li, X Zhao, K Du, F Ru, Y Zhang. Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine. Automation in Construction, 2017, 78: 51–61 https://doi.org/10.1016/j.autcon.2017.01.019
18
O Russakovsky, J Deng, H Su, J Krause, S Satheesh, S Ma, F F Li. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211–252 https://doi.org/10.1007/s11263-015-0816-y
19
S Park, S Bang, H Kim, H Kim. Patch-based crack detection in black box images using convolutional neural networks. Journal of Computing in Civil Engineering, 2019, 33(3): 04019017 https://doi.org/10.1061/(ASCE)CP.1943-5487.0000831
20
C ZhangC C ChangM Jamshidi. Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model. Structural Health Monitoring, 2021, 20(4): 147592172098543
21
Y J Cha, W Choi, G Suh, S Mahmoudkhani, O Büyüköztürk. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(9): 731–747 https://doi.org/10.1111/mice.12334
J Chen, Y He. A novel U-shaped encoder–decoder network with attention mechanism for detection and evaluation of road cracks at pixel level. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(13): 1721–1736 https://doi.org/10.1111/mice.12826
24
Z Zhou, J Zhang, C Gong. Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2023, 38(17): 2491–2510 https://doi.org/10.1111/mice.13003
25
Y XuY L Fan H Li. Lightweight semantic segmentation of complex structural damage recognition for actual bridges. Structural Health Monitoring, 2023, 22(5): 14759217221147015
26
P ArafinA M BillahA Issa. Deep learning-based concrete defects classification and detection using semantic segmentation. Structural Health Monitoring, 2023: 14759217231168212
27
L Chen, H Yao, J Fu, C T Ng. The classification and localization of crack using lightweight convolutional neural network with CBAM. Engineering Structures, 2023, 275: 115291 https://doi.org/10.1016/j.engstruct.2022.115291
28
L ZhangF YangY D ZhangY J Zhu. Road crack detection using deep convolutional neural network. In: Proceedings of the 2016 IEEE international conference on image processing (ICIP). Phoenix: IEEE, 2016: 3708–3712
29
Q Zou, Z Zhang, Q Li, X Qi, Q Wang, S Wang. Deepcrack: Learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing, 2018, 28(3): 1498–1512 https://doi.org/10.1109/TIP.2018.2878966
30
S Bang, S Park, H Kim, H Kim. Encoder-decoder network for pixel-level road crack detection in black-box images. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8): 713–727 https://doi.org/10.1111/mice.12440
31
D O’Brien, J A Osborne, E Perez-Duenas, R Cunningham, Z L Li. Automated crack classification for the CERN underground tunnel infrastructure using deep learning. Tunnelling and Underground Space Technology, 2023, 131: 104668 https://doi.org/10.1016/j.tust.2022.104668
32
L M Dang, H X Wang, Y F Li, Y Park, C Oh, T N Nguyen, H Moon. Automatic tunnel lining crack evaluation and measurement using deep learning. Tunnelling and Underground Space Technology, 2022, 124: 104472 https://doi.org/10.1016/j.tust.2022.104472
33
S J Feng, Y Feng, X L Zhang, Y H Chen. Deep learning with visual explanations for leakage defect segmentation of metro shield tunnel. Tunnelling and Underground Space Technology, 2023, 136: 105107 https://doi.org/10.1016/j.tust.2023.105107
34
Y D Xue, F Jia, X Y Cai, M Shadabfar, H W Huang. An optimization strategy to improve the deep learning-based recognition model of leakage in shield tunnels. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(3): 386–402 https://doi.org/10.1111/mice.12731
35
P ManjunathaS F MasriA NakanoL C Wellford. CrackDenseLinkNet: A deep convolutional neural network for semantic segmentation of cracks on concrete surface images. Structural Health Monitoring, 2023: 14759217231173305
36
Y Que, Y Dai, X Ji, A K Leung, Z Chen, Z L Jiang, Y C Tang. Automatic classification of asphalt pavement cracks using a novel integrated generative adversarial networks and improved VGG model. Engineering Structures, 2023, 277: 115406 https://doi.org/10.1016/j.engstruct.2022.115406
37
L Guo, R Li, B Jiang, X Shen. Automatic crack distress classification from concrete surface images using a novel deep-width network architecture. Neurocomputing, 2020, 397: 383–392 https://doi.org/10.1016/j.neucom.2019.08.107
38
H Xu, X Su, Y Wang, H Cai, K Cui, X Chen. Automatic bridge crack detection using a convolutional neural network. Applied Sciences, 2019, 9(14): 2867 https://doi.org/10.3390/app9142867
39
J Huyan, W Li, S Tighe, Z C Xu, J Z Zhai. CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection. Structural Control and Health Monitoring, 2020, 27(8): e2551 https://doi.org/10.1002/stc.2551
40
K Zhang, H D Cheng, B Zhang. Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning. Journal of Computing in Civil Engineering, 2018, 32(2): 04018001 https://doi.org/10.1061/(ASCE)CP.1943-5487.0000736
41
S K HouZ G OuP X QinY L WangY R Liu. Image-based crack recognition of tunnel lining using residual U-Net convolutional neural network. In: Proceedings of the IOP Conference Series: Earth and Environmental Science. Jakarta: IOP Publishing, 2021: 072001
42
L Attard, C J Debono, G Valentino, M Di Castro. Tunnel inspection using photogrammetric techniques and image processing: A review. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 144: 180–188 https://doi.org/10.1016/j.isprsjprs.2018.07.010
43
Q Ai, Y Yuan, X L Bi. Acquiring sectional profile of metro tunnels using charge-coupled device cameras. Structure and Infrastructure Engineering, 2016, 12(9): 1065–1075 https://doi.org/10.1080/15732479.2015.1076855
44
S Zhao, D M Zhang, H W Huang. Deep learning-based image instance segmentation for moisture marks of shield tunnel lining. Tunnelling and Underground Space Technology, 2020, 95: 103156 https://doi.org/10.1016/j.tust.2019.103156
45
Q Ai, Y Yuan. Rapid acquisition and identification of structural defects of metro tunnel. Sensors, 2019, 19(19): 4278 https://doi.org/10.3390/s19194278
46
M L Zhou, W Cheng, H W Huang, J Y Chen. A novel approach to automated 3D spalling defects inspection in railway tunnel linings using laser intensity and depth information. Sensors, 2021, 21(17): 5725 https://doi.org/10.3390/s21175725
47
V IglovikovA Shvets. Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation. 2018, arXiv:1801.05746
48
C BenzP DebusH K HaV Rodehorst. Crack segmentation on UAS-based imagery using transfer learning. In: Proceedings of the 2019 International Conference on Image and Vision Computing New Zealand (IVCNZ). Dunedin: IEEE, 2019: 1–6
49
M JoginM S MadhulikaG D DivyaR K MeghanaS Apoorva. Feature extraction using convolution neural networks (CNN) and deep learning. In: Proceedings of the 2018 3rd IEEE international conference on recent trends in electronics, information & communication technology (RTEICT). Bangalore: IEEE, 2018: 2319–2323
50
K SimonyanA Zisserman. Very deep convolutional networks for large-scale image recognition. 2014, arXiv:1409.1556
51
V NairG E Hinton. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10). Madison, WI: ACM, 2010: 807–814
52
Z Swiderska-Chadaj, T de Bel, L Blanchet, A Baidoshvili, D Vossen, J van der Laak, G Litjens. Impact of rescanning and normalization on convolutional neural network performance in multi-center, whole-slide classification of prostate cancer. Scientific Reports, 2020, 10(1): 1–14 https://doi.org/10.1038/s41598-020-71420-0
53
S IoffeC Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. International conference on machine learning. Stockholm: ICML, 2015: 448–456
54
Y WuK He. Group normalization. In: Proceedings of the European conference on computer vision (ECCV), Munich: Springer, 2018: 3–19
55
O RonnebergerP FischerT Brox. U-Net: convolutional Networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munish: Springer, 2015: 234–241
56
L Y Deng. The Cross-Entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation, and machine learning. Technometrics, 2006, 48(1): 147–148 https://doi.org/10.1198/tech.2006.s353
57
Z Qu, J Mei, L Liu, D Y Zhou. Crack detection of concrete pavement with cross-entropy loss function and improved VGG16 network model. IEEE Access : Practical Innovations, Open Solutions, 2020, 8: 54564–54573 https://doi.org/10.1109/ACCESS.2020.2981561
58
C Y Li, S K Hou, Y R Liu, P X Qin, Q Yang. Analysis on the crown convergence deformation of surrounding rock for double-shield TBM tunnel based on advance borehole monitoring and inversion analysis. Tunnelling and Underground Space Technology, 2020, 103: 103513 https://doi.org/10.1016/j.tust.2020.103513
59
W J Wang, C Su. Semi-supervised semantic segmentation network for surface crack detection. Automation in Construction, 2021, 128: 103786 https://doi.org/10.1016/j.autcon.2021.103786
60
M M Bejani, M Ghatee. A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review, 2021, 54(8): 1–48 https://doi.org/10.1007/s10462-021-09975-1
X B Yang, R Chen, F Q Zhang, L Zhang, X J Fan, Q L Ye, L Y Fu. Pixel-level automatic annotation for forest fire image. Engineering Applications of Artificial Intelligence, 2021, 104: 104353 https://doi.org/10.1016/j.engappai.2021.104353
63
F Coelho, J P Neto. A method for regularization of evolutionary polynomial regression. Applied Soft Computing, 2017, 59: 223–228 https://doi.org/10.1016/j.asoc.2017.05.047
64
M Ohsaki, P Wang, K Matsuda, S Katagiri, H Watanabe, A Ralescu. Confusion-matrix-based kernel logistic regression for imbalanced data classification. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(9): 1806–1819 https://doi.org/10.1109/TKDE.2017.2682249
65
Y Ren, J Huang, Z Hong, W Lu, J Yin, L Zou, X Shen. Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction & Building Materials, 2020, 234: 117367 https://doi.org/10.1016/j.conbuildmat.2019.117367
66
M Sokolova, G Lapalme. A systematic analysis of performance measures for classification tasks. Information Processing & Management, 2009, 45(4): 427–437 https://doi.org/10.1016/j.ipm.2009.03.002
67
Z Q Liu, Y Z Wang, X G Hua, H P Zhu, Z W Zhu. Optimization of wind turbine TMD under real wind distribution countering wake effects using GPU acceleration and machine learning technologies. Journal of Wind Engineering and Industrial Aerodynamics, 2021, 208: 104436 https://doi.org/10.1016/j.jweia.2020.104436
68
S Dong, P Zhao, X Lin, D Kaeli. Exploring GPU acceleration of Deep Neural Networks using Block Circulant Matrices. Parallel Computing, 2020, 100: 102701 https://doi.org/10.1016/j.parco.2020.102701
69
J J Wang, Y F Liu, X Nie, Y L Mo. Deep convolutional neural networks for semantic segmentation of cracks. Structural Control and Health Monitoring, 2022, 29(1): e2850 https://doi.org/10.1002/stc.2850
70
Y Rong. The research on the key technology of the crack controlling of reinforced concrete lining of undersea tunnel. Dissertation for the Doctoral Degree. Shanghai: Toingji University, 2007 (in Chinese)
71
X C Yang, H Li, Y T Yu, X C Luo, T Huang, X Yang. Automatic pixel-level crack detection and measurement using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090–1109 https://doi.org/10.1111/mice.12412
72
F SunY K Choi Y YuW Wang. Medial meshes for volume approximation. 2013, arXiv:1308.3917
73
H W Huang, Q T Li, D M Zhang. Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunnelling and Underground Space Technology, 2018, 77: 166–176 https://doi.org/10.1016/j.tust.2018.04.002
