Road sub-surface defect detection based on gprMax forward simulation-sample generation and Swin Transformer-YOLOX

doi:10.1007/s11709-024-1076-0

Frontiers of Structural and Civil Engineering

2024, Vol. 18

Issue (3): 334-349 https://doi.org/10.1007/s11709-024-1076-0

本期目录

Road sub-surface defect detection based on gprMax forward simulation-sample generation and Swin Transformer-YOLOX

Longjian LI¹, Li YANG², Zhongyu HAO², Xiaoli SUN³, Gongfa CHEN¹(

)

¹. School of Civil and Transportation Engineering, Guangdong University of Technology, Guangzhou 510006, China
². JSTI Group Guizhou Engineering Survey and Design Co., Ltd., Guangzhou 510800, China
³. Guangzhou Municipal Engineering Testing Co., Ltd., Guangzhou 510520, China

全文: PDF(11153 KB) HTML

Abstract：

Training samples for deep learning networks are typically obtained through various field experiments, which require significant manpower, resource and time consumption. However, it is possible to utilize simulated data to augment the training samples. In this paper, by comparing the actual experimental model with the simulated model generated by the gprMax [1] forward simulation method, the feasibility of obtaining simulated samples through gprMax simulation is validated. Subsequently, the samples generated by gprMax forward simulation are used for training the network to detect objects in existing real samples. At the same time, aiming at the detection and intelligent recognition of road sub-surface defects, the Swin-YOLOX algorithm is introduced, and the excellence of the detection network, which is improved by augmenting the simulated samples with real samples, is further verified. By comparing the prediction performance of the object detection models, it is observed that the model trained with mixed samples achieved a recall of 94.74% and a mean average precision ( $m A P$ ) of 97.71%, surpassing the model trained only on real samples by 12.95% and 15.64%, respectively. The feasibility and excellence of training the model with mixed samples are confirmed. The potential of using a fusion of simulated and existing real samples instead of repeatedly acquiring new real samples by field experiment is demonstrated by this study, thereby improving detection efficiency, saving resources, and providing a new approach to the problem of multiple interpretations in ground penetrating radar (GPR) data.

Key words： ground penetrating radar gprMax forward modeling sample generation Swin-YOLOX object detection

收稿日期: 2023-07-30 出版日期: 2024-06-12

Corresponding Author(s): Gongfa CHEN

引用本文:

. [J]. Frontiers of Structural and Civil Engineering, 2024, 18(3): 334-349.
Longjian LI, Li YANG, Zhongyu HAO, Xiaoli SUN, Gongfa CHEN. Road sub-surface defect detection based on gprMax forward simulation-sample generation and Swin Transformer-YOLOX. Front. Struct. Civ. Eng., 2024, 18(3): 334-349.

链接本文:

https://academic.hep.com.cn/fsce/CN/10.1007/s11709-024-1076-0
https://academic.hep.com.cn/fsce/CN/Y2024/V18/I3/334

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Tab.1

Fig.9

Fig.10

Fig.11

Fig.12

Tab.2

Fig.13

Fig.14

Tab.3

Tab.4

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Fig.20

Network model	Training model with real samples		Training model with mixed samples
Network model	$m A P$ _0.5	recall	$m A P$ _0.5	recall
Swin-YOLOX	82.07%	81.79%	97.71%	94.74%

Tab.5

Tab.6

Fig.21

1	Z F Wang, J Wang, K F Chen, Z P Li, J Xu, Y Li, Q M Sui. Unsupervised learning method for rebar signal suppression and defect signal reconstruction and detection in ground penetrating radar images. Measurement, 2023, 211: 112652 https://doi.org/10.1016/j.measurement.2023.112652
2	Lai W Wai-Lok, X Dérobert, P Annan. A review of ground penetrating radar application in civil engineering: A 30-year journey from Locating and Testing to Imaging and Diagnosis. NDT & E International, 2018, 96: 58–78 https://doi.org/10.1016/j.ndteint.2017.04.002
3	H Guo, X Zhuang, T Rabczuk. A deep collocation method for the bending analysis of Kirchhoff plate. Computers, Materials & Continua, 2019, 59(2): 433–456 https://doi.org/10.32604/cmc.2019.06660
4	Z H Dong, S B Ye, Y Z Gao, G Y Fang, X J Zhang, Z J Xue, T Zhang. Rapid detection methods for asphalt pavement thicknesses and defects by a vehicle-mounted ground penetrating radar (GPR) system. Sensors, 2016, 16(12): 2067 https://doi.org/10.3390/s16122067
5	Z Tong, J Gao, H T Zhang. Innovative method for recognizing subgrade defects based on a convolutional neural network. Construction & Building Materials, 2018, 169: 69–82 https://doi.org/10.1016/j.conbuildmat.2018.02.081
6	E Samaniego, C Anitescu, S Goswami, V M Nguyen-Thanh, H Guo, K Hamdia, X Zhuang, T Rabczuk. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790 https://doi.org/10.1016/j.cma.2019.112790
7	X Zhuang, H Guo, N Alajlan, H Zhu, T Rabczuk. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics. A, Solids, 2021, 87: 104225 https://doi.org/10.1016/j.euromechsol.2021.104225
8	H Guo, X Zhuang, P Chen, N Alajlan, T Rabczuk. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Engineering with Computers, 2022, 38(6): 5173–5198 https://doi.org/10.1007/s00366-021-01586-2
9	Y P Yue, H Liu, X Meng, Y G Li, Y L Du. Generation of high-precision ground penetrating radar images using improved least square generative adversarial networks. Remote Sensing, 2021, 13(22): 4590 https://doi.org/10.3390/rs13224590
10	W J Ye, C J Liu, Y H Chen, Y J Liu, C M Liu, H H Zhou. Multi-style transfer and fusion of image’s regions based on attention mechanism and instance segmentation. Signal Processing Image Communication, 2023, 110: 116871 https://doi.org/10.1016/j.image.2022.116871
11	H Guo, X Zhuang, P Chen, N Alajlan, T Rabczuk. Analysis of three-dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis. Engineering with Computers, 2022, 38(6): 5423–5444 https://doi.org/10.1007/s00366-022-01633-6
12	H Guo, X Zhuang, X Fu, Y Zhu, T Rabczuk. Physics-informed deep learning for three-dimensional transient heat transfer analysis of functionally graded materials. Computational Mechanics, 2023, 72(3): 513–524 https://doi.org/10.1007/s00466-023-02287-x
13	H Guo, X Zhuang, N Alajlan, T Rabczuk. Physics-informed deep learning for melting heat transfer analysis with model-based transfer learning. Computers & Mathematics with Applications (Oxford, England), 2023, 143: 303–317 https://doi.org/10.1016/j.camwa.2023.05.014
14	R X Gao, H Q Zhu, Q Liao, B L Qu, L T Hu, H R Wang. Detection of coal fire by deep learning using ground penetrating radar. Measurement, 2022, 201: 111585 https://doi.org/10.1016/j.measurement.2022.111585
15	C Warren, A Giannopoulos, I Giannakis. gprMax: Open source software to simulate electromagnetic wave propagation for ground penetrating radar. Computer Physics Communications, 2016, 209: 163–170 https://doi.org/10.1016/j.cpc.2016.08.020
16	J Berenger. Perfectly matched layer for the FDTD solution of wave-structure interaction problems. IEEE Transactions on Antennas and Propagation, 1996, 44(1): 110–117 https://doi.org/10.1109/8.477535
17	J Zhang, X Yang, W G Li, S B Zhang, Y Y Jia. Automatic detection of moisture damages in asphalt pavements from GPR data with deep CNN and IRS method. Automation in Construction, 2020, 113: 103119 https://doi.org/10.1016/j.autcon.2020.103119
18	C Fauchard, X Dérobert, J Cariou, P Côte. GPR performances for thickness calibration on road test sites. NDT & E International, 2003, 36(2): 67–75 https://doi.org/10.1016/S0963-8695(02)00090-7
19	S Khudoyarov, N Kim, J J Lee. Three-dimensional convolutional neural network-based underground object classification using three-dimensional ground penetrating radar data. Structural Health Monitoring, 2020, 19(6): 1884–1893 https://doi.org/10.1177/1475921720902700
20	T Rabczuk, H Ren, X Zhuang. A nonlocal operator method for partial differential equations with application to electromagnetic waveguide problem. Computers, Materials & Continua, 2019, 59(1): 31–55 https://doi.org/10.32604/cmc.2019.04567
21	Y Kane. Numerical solution of initial boundary value problems involving maxwell’s equations in isotropic media. IEEE Transactions on Antennas and Propagation, 1966, 14(3): 302–307 https://doi.org/10.1109/TAP.1966.1138693
22	K S KunzR J Luebbers. The Finite Difference Time Domain Method for Electromagnetics. Leiden: CRC Press, 1993
23	H Ren, X Zhuang, T Rabczuk. A higher order nonlocal operator method for solving partial differential equations. Computer Methods in Applied Mechanics and Engineering, 2020, 367: 113132 https://doi.org/10.1016/j.cma.2020.113132
24	A DosovitskiyL BeyerA KolesnikovD WeissenbornX ZhaiT UnterthinerM DehghaniM MindererG Heigold S GellyJ UszkoreitN Houlsby. An image is worth 16 × 16 words: Transformers for image recognition at scale. 2020, arXiv:2010.11929
25	Z GeS T Liu F WangZ M LiJ Sun. YOLOX: Exceeding YOLO series in 2021. 2021, arXiv:2107.08430
26	W Ji, Q J Liu, C W Huang, R Yang, H L Huang, G H Xu. YOLOX traffic sign detection based on Swin-Transformer. Radio Communications Technology, 2023, 49(3): 547–555
27	G X Wu, Y C Li. Non-maximum suppression for object detection based on the chaotic whale optimization algorithm. Journal of Visual Communication and Image Representation, 2021, 74: 102985 https://doi.org/10.1016/j.jvcir.2020.102985
28	C W Zheng, H Lin. YOLOv5 helmet wearing detection method based on Swin Transformer. Computer Measurement and Control, 2023, 31(3): 15–21
29	A Iqbal, M Sharif. BTS-ST: Swin transformer network for segmentation and classification of multimodality breast cancer images. Knowledge-Based Systems, 2023, 267: 110393 https://doi.org/10.1016/j.knosys.2023.110393
30	H Üzen, M Türkoğlu, B Yanikoglu, D Hanbay. Swin-MFINet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications, 2022, 209: 118269 https://doi.org/10.1016/j.eswa.2022.118269
31	H N Yang, D P Yang. CSwin-PNet: A CNN-Swin Transformer combined pyramid network for breast lesion segmentation in ultrasound images. Expert Systems with Applications, 2023, 213: 119024 https://doi.org/10.1016/j.eswa.2022.119024
32	S Jiang, R N Kong, P C Li, C W lu, S Zhang, M Li. Intelligent detection algorithm of obstacles in front of open-pit mine cars based on Swin Transformer and CNN. Metal Mine, 2023, 5: 228–236
33	S L Lu, X Y Liu, Z X He, X Zhang, W B Liu, M Karkee. Swin-Transformer-YOLOv5 for real-time wine grape bunch detection. Remote Sensing, 2022, 14(22): 5853 https://doi.org/10.3390/rs14225853
34	K Ishitsuka, S Iso, K Onishi, T Matsuoka. Object detection in ground-penetrating radar images using a deep convolutional neural network and image set preparation by migration. International Journal of Geophysics, 2018, 2018: 1–8 https://doi.org/10.1155/2018/9365184

Viewed

Full text

Abstract

Cited

Shared

Discussed