|
|
|
Pixel-level crack segmentation of tunnel lining segments based on an encoder–decoder network |
Shaokang HOU1,2,3, Zhigang OU1,3, Yuequn HUANG1,4, Yaoru LIU1( ) |
1. State Key Laboratory of Hydroscience and Engineering, Tsinghua University, Beijing 100084, China 2. China Renewable Energy Engineering Institute, Beijing 100120, China 3. State Key Laboratory of Stimulation and Regulation of Water Cycles in River Basins, China Institute of Water Resources and Hydropower Research, Beijing 100038, China 4. Hunan Provincial Water Resources Development & Investment Co., Ltd., Changsha 410007, China |
|
|
|
|
Abstract Regular detection and repair for lining cracks are necessary to guarantee the safety and stability of tunnels. The development of computer vision has greatly promoted structural health monitoring. This study proposes a novel encoder–decoder structure, CrackRecNet, for semantic segmentation of lining segment cracks by integrating improved VGG-19 into the U-Net architecture. An image acquisition equipment is designed based on a camera, 3-dimensional printing (3DP) bracket and two laser rangefinders. A tunnel concrete structure crack (TCSC) image data set, containing images collected from a double-shield tunnel boring machines (TBM) tunnel in China, was established. Through data preprocessing operations, such as brightness adjustment, pixel resolution adjustment, flipping, splitting and annotation, 2880 image samples with pixel resolution of 448 × 448 were prepared. The model was implemented by Pytorch in PyCharm processed with 4 NVIDIA TITAN V GPUs. In the experiments, the proposed CrackRecNet showed better prediction performance than U-Net, TernausNet, and ResU-Net. This paper also discusses GPU parallel acceleration effect and the crack maximum width quantification.
|
| Keywords
tunnel lining segment
crack detection
semantic segmentation
convolutional neural network
encoder–decoder structure
|
|
Corresponding Author(s):
Yaoru LIU
|
|
Just Accepted Date: 24 May 2024
Online First Date: 18 June 2024
Issue Date: 26 June 2024
|
|
| 1 |
M F Lei, L H Liu, C H Shi, Y Tan, Y X Lin, W D Wang. A novel tunnel-lining crack recognition system based on digital image technology. Tunnelling and Underground Space Technology, 2021, 108: 103724
https://doi.org/10.1016/j.tust.2020.103724
|
| 2 |
Y YangL F WangY F ZhangX J Han. Multi-feature fusion based classification algorithm of surface disease image of concrete structure. Journal of Chang’an University: Natural Science Edition, 2021, 41(3): 64−74 (in Chinese)
|
| 3 |
S Zhao, M Shadabfar, D M Zhang, J Y Chen, H W Huang. Deep learning-based classification and instance segmentation of leakage-area and scaling images of shield tunnel linings. Structural Control and Health Monitoring, 2021, 28(6): e2732
https://doi.org/10.1002/stc.2732
|
| 4 |
Y Yao, S T E Tung, B Glisic. Crack detection and characterization techniques—An overview. Structural Control and Health Monitoring, 2014, 21(12): 1387–1413
https://doi.org/10.1002/stc.1655
|
| 5 |
Y R Liu, S K Hou, C Y Li, H W Zhou, F Jin, P X Qin, Q Yang. Study on support time in double-shield TBM tunnel based on self-compacting concrete backfilling material. Tunnelling and Underground Space Technology, 2020, 96: 103212
https://doi.org/10.1016/j.tust.2019.103212
|
| 6 |
C Koch, K Georgieva, V Kasireddy, B Akinci, P Fieguth. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Advanced Engineering Informatics, 2015, 29(2): 196–210
https://doi.org/10.1016/j.aei.2015.01.008
|
| 7 |
J Liu, X Yang, S Lau, X Wang, S Luo, V C S Lee, L Ding. Automated pavement crack detection and segmentation based on two-step convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291–1305
https://doi.org/10.1111/mice.12622
|
| 8 |
J Valença, E Julio. MCrack-Dam: the scale-up of a method to assess cracks on concrete dams by image processing. The case study of Itaipu Dam, at the Brazil–Paraguay border. Journal of Civil Structural Health Monitoring, 2018, 8(5): 857–866
https://doi.org/10.1007/s13349-018-0309-0
|
| 9 |
Y Wang, J Y Zhang, J X Liu, Y Zhang, Z P Chen, C G Li, R B Yan. Research on crack detection algorithm of the concrete bridge based on image processing. Procedia Computer Science, 2019, 154: 610–616
https://doi.org/10.1016/j.procs.2019.06.096
|
| 10 |
L Ying, E Salari. Beamlet transform-based technique for pavement crack detection and classification. Computer-Aided Civil and Infrastructure Engineering, 2010, 25(8): 572–580
https://doi.org/10.1111/j.1467-8667.2010.00674.x
|
| 11 |
I Abdel-Qader, O Abudayyeh, M E Kelly. Analysis of edge-detection techniques for crack identification in bridges. Journal of Computing in Civil Engineering, 2003, 17(4): 255–263
https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255
|
| 12 |
H OliveiraP L Correia. Automatic road crack segmentation using entropy and image dynamic thresholding. In: Proceedings of the 17th European Signal Processing Conference. Glasgow: IEEE, 2009: 622–626
|
| 13 |
Y Xu, D Li, Q Xie, Q Wu, J Wang. Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement, 2021, 178: 109316
https://doi.org/10.1016/j.measurement.2021.109316
|
| 14 |
C Feng, H Zhang, H Wang, S Wang, Y Li. Automatic pixel-level crack detection on dam surface using deep convolutional network. Sensors, 2020, 20(7): 2069
https://doi.org/10.3390/s20072069
|
| 15 |
Y Shi, L Cui, Z Qi, F Meng, Z Chen. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434–3445
https://doi.org/10.1109/TITS.2016.2552248
|
| 16 |
Y NohD Koo Y M KangD ParkD Lee. Automatic crack detection on concrete images using segmentation via fuzzy C-means clustering. In: Proceedings of the International conference on applied system innovation (ICASI). Sapporo: IEEE, 2017: 877–880
|
| 17 |
G Li, X Zhao, K Du, F Ru, Y Zhang. Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine. Automation in Construction, 2017, 78: 51–61
https://doi.org/10.1016/j.autcon.2017.01.019
|
| 18 |
O Russakovsky, J Deng, H Su, J Krause, S Satheesh, S Ma, F F Li. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211–252
https://doi.org/10.1007/s11263-015-0816-y
|
| 19 |
S Park, S Bang, H Kim, H Kim. Patch-based crack detection in black box images using convolutional neural networks. Journal of Computing in Civil Engineering, 2019, 33(3): 04019017
https://doi.org/10.1061/(ASCE)CP.1943-5487.0000831
|
| 20 |
C ZhangC C ChangM Jamshidi. Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model. Structural Health Monitoring, 2021, 20(4): 147592172098543
|
| 21 |
Y J Cha, W Choi, G Suh, S Mahmoudkhani, O Büyüköztürk. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(9): 731–747
https://doi.org/10.1111/mice.12334
|
| 22 |
Y LeCun, Y Bengio, G Hinton. Deep learning. Nature, 2015, 521(7553): 436–444
https://doi.org/10.1038/nature14539
|
| 23 |
J Chen, Y He. A novel U-shaped encoder–decoder network with attention mechanism for detection and evaluation of road cracks at pixel level. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(13): 1721–1736
https://doi.org/10.1111/mice.12826
|
| 24 |
Z Zhou, J Zhang, C Gong. Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2023, 38(17): 2491–2510
https://doi.org/10.1111/mice.13003
|
| 25 |
Y XuY L Fan H Li. Lightweight semantic segmentation of complex structural damage recognition for actual bridges. Structural Health Monitoring, 2023, 22(5): 14759217221147015
|
| 26 |
P ArafinA M BillahA Issa. Deep learning-based concrete defects classification and detection using semantic segmentation. Structural Health Monitoring, 2023: 14759217231168212
|
| 27 |
L Chen, H Yao, J Fu, C T Ng. The classification and localization of crack using lightweight convolutional neural network with CBAM. Engineering Structures, 2023, 275: 115291
https://doi.org/10.1016/j.engstruct.2022.115291
|
| 28 |
L ZhangF YangY D ZhangY J Zhu. Road crack detection using deep convolutional neural network. In: Proceedings of the 2016 IEEE international conference on image processing (ICIP). Phoenix: IEEE, 2016: 3708–3712
|
| 29 |
Q Zou, Z Zhang, Q Li, X Qi, Q Wang, S Wang. Deepcrack: Learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing, 2018, 28(3): 1498–1512
https://doi.org/10.1109/TIP.2018.2878966
|
| 30 |
S Bang, S Park, H Kim, H Kim. Encoder-decoder network for pixel-level road crack detection in black-box images. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8): 713–727
https://doi.org/10.1111/mice.12440
|
| 31 |
D O’Brien, J A Osborne, E Perez-Duenas, R Cunningham, Z L Li. Automated crack classification for the CERN underground tunnel infrastructure using deep learning. Tunnelling and Underground Space Technology, 2023, 131: 104668
https://doi.org/10.1016/j.tust.2022.104668
|
| 32 |
L M Dang, H X Wang, Y F Li, Y Park, C Oh, T N Nguyen, H Moon. Automatic tunnel lining crack evaluation and measurement using deep learning. Tunnelling and Underground Space Technology, 2022, 124: 104472
https://doi.org/10.1016/j.tust.2022.104472
|
| 33 |
S J Feng, Y Feng, X L Zhang, Y H Chen. Deep learning with visual explanations for leakage defect segmentation of metro shield tunnel. Tunnelling and Underground Space Technology, 2023, 136: 105107
https://doi.org/10.1016/j.tust.2023.105107
|
| 34 |
Y D Xue, F Jia, X Y Cai, M Shadabfar, H W Huang. An optimization strategy to improve the deep learning-based recognition model of leakage in shield tunnels. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(3): 386–402
https://doi.org/10.1111/mice.12731
|
| 35 |
P ManjunathaS F MasriA NakanoL C Wellford. CrackDenseLinkNet: A deep convolutional neural network for semantic segmentation of cracks on concrete surface images. Structural Health Monitoring, 2023: 14759217231173305
|
| 36 |
Y Que, Y Dai, X Ji, A K Leung, Z Chen, Z L Jiang, Y C Tang. Automatic classification of asphalt pavement cracks using a novel integrated generative adversarial networks and improved VGG model. Engineering Structures, 2023, 277: 115406
https://doi.org/10.1016/j.engstruct.2022.115406
|
| 37 |
L Guo, R Li, B Jiang, X Shen. Automatic crack distress classification from concrete surface images using a novel deep-width network architecture. Neurocomputing, 2020, 397: 383–392
https://doi.org/10.1016/j.neucom.2019.08.107
|
| 38 |
H Xu, X Su, Y Wang, H Cai, K Cui, X Chen. Automatic bridge crack detection using a convolutional neural network. Applied Sciences, 2019, 9(14): 2867
https://doi.org/10.3390/app9142867
|
| 39 |
J Huyan, W Li, S Tighe, Z C Xu, J Z Zhai. CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection. Structural Control and Health Monitoring, 2020, 27(8): e2551
https://doi.org/10.1002/stc.2551
|
| 40 |
K Zhang, H D Cheng, B Zhang. Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning. Journal of Computing in Civil Engineering, 2018, 32(2): 04018001
https://doi.org/10.1061/(ASCE)CP.1943-5487.0000736
|
| 41 |
S K HouZ G OuP X QinY L WangY R Liu. Image-based crack recognition of tunnel lining using residual U-Net convolutional neural network. In: Proceedings of the IOP Conference Series: Earth and Environmental Science. Jakarta: IOP Publishing, 2021: 072001
|
| 42 |
L Attard, C J Debono, G Valentino, M Di Castro. Tunnel inspection using photogrammetric techniques and image processing: A review. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 144: 180–188
https://doi.org/10.1016/j.isprsjprs.2018.07.010
|
| 43 |
Q Ai, Y Yuan, X L Bi. Acquiring sectional profile of metro tunnels using charge-coupled device cameras. Structure and Infrastructure Engineering, 2016, 12(9): 1065–1075
https://doi.org/10.1080/15732479.2015.1076855
|
| 44 |
S Zhao, D M Zhang, H W Huang. Deep learning-based image instance segmentation for moisture marks of shield tunnel lining. Tunnelling and Underground Space Technology, 2020, 95: 103156
https://doi.org/10.1016/j.tust.2019.103156
|
| 45 |
Q Ai, Y Yuan. Rapid acquisition and identification of structural defects of metro tunnel. Sensors, 2019, 19(19): 4278
https://doi.org/10.3390/s19194278
|
| 46 |
M L Zhou, W Cheng, H W Huang, J Y Chen. A novel approach to automated 3D spalling defects inspection in railway tunnel linings using laser intensity and depth information. Sensors, 2021, 21(17): 5725
https://doi.org/10.3390/s21175725
|
| 47 |
V IglovikovA Shvets. Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation. 2018, arXiv:1801.05746
|
| 48 |
C BenzP DebusH K HaV Rodehorst. Crack segmentation on UAS-based imagery using transfer learning. In: Proceedings of the 2019 International Conference on Image and Vision Computing New Zealand (IVCNZ). Dunedin: IEEE, 2019: 1–6
|
| 49 |
M JoginM S MadhulikaG D DivyaR K MeghanaS Apoorva. Feature extraction using convolution neural networks (CNN) and deep learning. In: Proceedings of the 2018 3rd IEEE international conference on recent trends in electronics, information & communication technology (RTEICT). Bangalore: IEEE, 2018: 2319–2323
|
| 50 |
K SimonyanA Zisserman. Very deep convolutional networks for large-scale image recognition. 2014, arXiv:1409.1556
|
| 51 |
V NairG E Hinton. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10). Madison, WI: ACM, 2010: 807–814
|
| 52 |
Z Swiderska-Chadaj, T de Bel, L Blanchet, A Baidoshvili, D Vossen, J van der Laak, G Litjens. Impact of rescanning and normalization on convolutional neural network performance in multi-center, whole-slide classification of prostate cancer. Scientific Reports, 2020, 10(1): 1–14
https://doi.org/10.1038/s41598-020-71420-0
|
| 53 |
S IoffeC Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. International conference on machine learning. Stockholm: ICML, 2015: 448–456
|
| 54 |
Y WuK He. Group normalization. In: Proceedings of the European conference on computer vision (ECCV), Munich: Springer, 2018: 3–19
|
| 55 |
O RonnebergerP FischerT Brox. U-Net: convolutional Networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munish: Springer, 2015: 234–241
|
| 56 |
L Y Deng. The Cross-Entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation, and machine learning. Technometrics, 2006, 48(1): 147–148
https://doi.org/10.1198/tech.2006.s353
|
| 57 |
Z Qu, J Mei, L Liu, D Y Zhou. Crack detection of concrete pavement with cross-entropy loss function and improved VGG16 network model. IEEE Access : Practical Innovations, Open Solutions, 2020, 8: 54564–54573
https://doi.org/10.1109/ACCESS.2020.2981561
|
| 58 |
C Y Li, S K Hou, Y R Liu, P X Qin, Q Yang. Analysis on the crown convergence deformation of surrounding rock for double-shield TBM tunnel based on advance borehole monitoring and inversion analysis. Tunnelling and Underground Space Technology, 2020, 103: 103513
https://doi.org/10.1016/j.tust.2020.103513
|
| 59 |
W J Wang, C Su. Semi-supervised semantic segmentation network for surface crack detection. Automation in Construction, 2021, 128: 103786
https://doi.org/10.1016/j.autcon.2021.103786
|
| 60 |
M M Bejani, M Ghatee. A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review, 2021, 54(8): 1–48
https://doi.org/10.1007/s10462-021-09975-1
|
| 61 |
C Shorten, T M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 2019, 6(1): 1–48
https://doi.org/10.1186/s40537-019-0197-0
|
| 62 |
X B Yang, R Chen, F Q Zhang, L Zhang, X J Fan, Q L Ye, L Y Fu. Pixel-level automatic annotation for forest fire image. Engineering Applications of Artificial Intelligence, 2021, 104: 104353
https://doi.org/10.1016/j.engappai.2021.104353
|
| 63 |
F Coelho, J P Neto. A method for regularization of evolutionary polynomial regression. Applied Soft Computing, 2017, 59: 223–228
https://doi.org/10.1016/j.asoc.2017.05.047
|
| 64 |
M Ohsaki, P Wang, K Matsuda, S Katagiri, H Watanabe, A Ralescu. Confusion-matrix-based kernel logistic regression for imbalanced data classification. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(9): 1806–1819
https://doi.org/10.1109/TKDE.2017.2682249
|
| 65 |
Y Ren, J Huang, Z Hong, W Lu, J Yin, L Zou, X Shen. Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction & Building Materials, 2020, 234: 117367
https://doi.org/10.1016/j.conbuildmat.2019.117367
|
| 66 |
M Sokolova, G Lapalme. A systematic analysis of performance measures for classification tasks. Information Processing & Management, 2009, 45(4): 427–437
https://doi.org/10.1016/j.ipm.2009.03.002
|
| 67 |
Z Q Liu, Y Z Wang, X G Hua, H P Zhu, Z W Zhu. Optimization of wind turbine TMD under real wind distribution countering wake effects using GPU acceleration and machine learning technologies. Journal of Wind Engineering and Industrial Aerodynamics, 2021, 208: 104436
https://doi.org/10.1016/j.jweia.2020.104436
|
| 68 |
S Dong, P Zhao, X Lin, D Kaeli. Exploring GPU acceleration of Deep Neural Networks using Block Circulant Matrices. Parallel Computing, 2020, 100: 102701
https://doi.org/10.1016/j.parco.2020.102701
|
| 69 |
J J Wang, Y F Liu, X Nie, Y L Mo. Deep convolutional neural networks for semantic segmentation of cracks. Structural Control and Health Monitoring, 2022, 29(1): e2850
https://doi.org/10.1002/stc.2850
|
| 70 |
Y Rong. The research on the key technology of the crack controlling of reinforced concrete lining of undersea tunnel. Dissertation for the Doctoral Degree. Shanghai: Toingji University, 2007 (in Chinese)
|
| 71 |
X C Yang, H Li, Y T Yu, X C Luo, T Huang, X Yang. Automatic pixel-level crack detection and measurement using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090–1109
https://doi.org/10.1111/mice.12412
|
| 72 |
F SunY K Choi Y YuW Wang. Medial meshes for volume approximation. 2013, arXiv:1308.3917
|
| 73 |
H W Huang, Q T Li, D M Zhang. Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunnelling and Underground Space Technology, 2018, 77: 166–176
https://doi.org/10.1016/j.tust.2018.04.002
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|