Please wait a minute...
Shopping Cart Email List Chinese
Advanced Search Fig/Tab
Frontiers of Engineering Management

ISSN 2095-7513

ISSN 2096-0255(Online)

CN 10-1205/N

Postal Subscription Code 80-905

Featured Articles  |  Most Downloaded  |  Most Reading
Online Submission Subscribe to Our RSS Content Feed
Front. Eng    2018, Vol. 5 Issue (2) : 227-239    https://doi.org/10.15302/J-FEM-2018071
RESEARCH ARTICLE
Evaluation of computer vision techniques for automated hardhat detection in indoor construction safety applications
Bahaa Eddine MNEYMNEH, Mohamad ABBAS, Hiam KHOURY()
Department of Civil and Environmental Engineering, American University of Beirut, Beirut 1107 2020, Lebanon
 Download: PDF(2225 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Construction is considered among the most dangerous industries and is responsible for a large portion of total worker fatalities. A construction worker has a probability of 1-in-200 of dying on the job during a 45-year career, mainly due to fires, falls, and being struck by or caught between objects. Hence, employers must ensure their workers wear personal protective equipment (PPE), in particular hardhats, if they are at risk of falling, being struck by falling objects, hitting their heads on static objects, or coming in proximity to electrical hazards. However, monitoring the presence and proper use of hardhats becomes inefficient when safety officers must survey large areas and a considerable number of workers. Using images captured from indoor jobsites, this paper evaluates existing computer vision techniques, namely object detection and color-based segmentation tools, used to rapidly detect if workers are wearing hardhats. Experiments are conducted and the results highlight the potential of cascade classifiers, in particular, to accurately, precisely, and rapidly detect hardhats under different scenarios and for repetitive runs, and the potential of color-based segmentation to eliminate false detections.
Keywords construction      safety      personal protective equipment      hardhat      computer vision     
Corresponding Author(s): Hiam KHOURY   
Just Accepted Date: 30 March 2018   Online First Date: 10 May 2018    Issue Date: 28 June 2018
 Cite this article:   
Bahaa Eddine MNEYMNEH,Mohamad ABBAS,Hiam KHOURY. Evaluation of computer vision techniques for automated hardhat detection in indoor construction safety applications[J]. Front. Eng, 2018, 5(2): 227-239.
 URL:  
https://academic.hep.com.cn/fem/EN/10.15302/J-FEM-2018071
https://academic.hep.com.cn/fem/EN/Y2018/V5/I2/227
Fig.1  Number of detected SURF, BRISK, and FAST features in blue (top) and white (bottom) hardhats
Fig.2  Number of detected SURF, BRISK, and FAST features in white hardhat with sticker
Fig.3  Matching features for (a) first hardhat and (b) second hardhat
Fig.4  No detection example- sticker size excessively low
Fig.5  Wrong detection using template matching
Fig.6  High rate of incorrect detections using (a) Haar features and (b) LBP features
Fig.7  Extracted HOG features: (a) blue hardhat and (b) white hardhat
Fig.8  Detected hardhat: (a) front view, (b) side view, and (c) back view
Computational duration Feature detection, extraction, & matching
Medium		 Template matching
Very High		 Cascade classifier
Low
Color invariance Yes No Yes
Orientation invariance No No Yes
Practicality No No Yes
Customizability Yes No Yes
Training database/fingerprinting No No Yes
Tab.1  Comparative summary of object detection techniques
Fig.9  Average Red, Green, and Blue values in images of blue hardhat
Fig.10  Average Hue, Saturation, and Value values in images of blue hardhat
Fig.11  Standard deviation of Hue, Saturation, and Value values in images of blue hardhat
Scenario 1- High contrast against background, variable colors, and orientations
Image ID 1 2 3 4 5 6 7 8 9 10
True number of hardhats 1 1 1 1 1 1 1 2 2 2
Detected- FAR= 0.05 1 1 0 0 1 2 1 2 1 3 t = 2.171s
Detected- FAR= 0.1 1 1 1 1 1 2 1 2 2 3 t = 2.231s
Time statistics- FAR= 0.05
Action Number of Calls Total Time Percentage of Total Time Average Time
Execution of detector 10 19.949 s 91.9% 1.9949 s
Reading image file 10 1.734 s 8.0% 0.1734 s
All other actions - 0.031 s 0.1% 0.0031 s
Total - 21.714 s 100% 2.1714 s
Time statistics- FAR= 0.1
Action Number of Calls Total Time Percentage of Total Time Average Time
Execution of detector 10 20.517 s 91.9% 2.0517 s
Reading image file 10 1.767 s 7.9% 0.1767 s
All other actions - 0.034 s 0.2% 0.0034 s
Total - 22.318 s 100% 2.2318 s
Tab.2  Performance of cascade classifier in Scenario 1
Fig.12  Wrong classification of head region in Image 6- Scenario 1
Scenario 2- Low contrast against background, variable colors, and orientations
Image ID 1 2 3 4 5 6 7 8 9 10
True number of hardhats 1 1 1 1 1 1 1 2 2 2
Detected- FAR= 0.05 0 1 0 1 0 0 1 1 1 1 t = 2.191s
Detected- FAR= 0.1 0 1 1 1 1 0 1 2 2 1 t = 2.196s
Time statistics- FAR= 0.05
Action Number of Calls Total Time Percentage of Total Time Average Time
Execution of detector 10 20.120 s 91.8% 2.0120 s
Reading image file 10 1.765 s 8.1% 0.1765 s
All other actions - 0.030 s 0.1% 0.0030 s
Total - 21.915 s 100% 2.1915 s
Time statistics- FAR= 0.1
Action Number of Calls Total Time Percentage of Total Time Average Time
Execution of detector 10 20.148 s 91.7% 2.0148 s
Reading image file 10 1.778 s 8.1% 0.1778 s
All other actions - 0.036 s 0.2% 0.0036 s
Total - 21.962 s 100% 2.1962 s
Tab.3  Performance of cascade classifier in Scenario 1
Fig.13  Correct identification of both hardhats in Image 8- Scenario 2
Scenario 3- Different image resolutions
Image ID 1 2 3 4 5 6 7 8 9 10
True number of hardhats 1 1 1 1 1 1 1 2 2 2
Detected- FAR= 0.05 1 1 0 0 1 2 0 1 1 3 t = 0.541s
Detected- FAR= 0.1 1 1 1 1 1 2 0 1 2 3 t = 0.568s
Time statistics- FAR= 0.05
Action Number of Calls Total Time Percentage of Total Time Average Time
Execution of detector 10 4.887 s 90.3% 0.4887 s
Reading image file 10 0.486 s 9.0% 0.0486 s
All other actions - 0.040 s 0.7% 0.0040 s
Total - 5.413 s 100% 0.5413 s
Time statistics- FAR= 0.1
Action Number of Calls Total Time Percentage of Total Time Average Time
Execution of detector 10 5.150 s 90.7% 0.5150 s
10 8.6%
All other actions - 0.040 s 0.7% 0.0040 s
Total - 5.681 s 100% 0.5681 s
Tab.4  Performance of cascade classifier in Scenario 3
Fig.14  Hardhat detection results for positive images: (a) cascade object detector and (b) cascade object detector combined with color-based segmentation
1 Abbas M, Mneymneh B E, Khoury H (2016). Use of unmanned aerial vehicles and computer vision in construction safety inspections. In: Proceedings of the 3rd Australasia and South-East Asia Structural Engineering and Construction Conference (ASEA-SEC-3). Kuching, Sarawak, Malaysia
https://doi.org/10.14455/ISEC.res.2016.183
2 Abbas M, Mneymneh B E, Khoury H (2018). Assessing on-site construction personnel hazard perception in a middle eastern developing country: An interactive graphical approach. Safety Science Journal, Elsevier Science, 103: 183–196
https://doi.org/10.1016/j.ssci.2017.10.026
3 Alionte E, Lazar C (2015). A practical implementation of face detection by using matlab cascade object detector. In: Proceedings of Conference on System Theory, Control and Computing. Cheile Gradistei, Romania, 785–790
4 Bajracharya D (2013). Real time pattern recognition in digital video with application to safety in construction sites. Dissertation for the College Degree. Las Vegas: University of Nevada–Las Vegas
5 Brunelli R (2009). Template Matching Techniques in Computer Vision: Theory and Practice. New Jersey: Wiley
6 Chdid D, Oueis R, Khoury H, Asmar D, Elhajj I (2011). Inertial-vision sensor fusion for pedestrian localization. In: Proceedings of IEEE International Conference on Robotics and Biomimetics (ROBIO). 1695–1701
7 Cheng T, Teizer J (2014). Modeling tower crane operator visibility to minimize the risk of limited situational awareness. Journal of Computing in Civil Engineering, 28(3): 04014004
https://doi.org/10.1061/(ASCE)CP.1943-5487.0000282
8 Chi S, Caldas C H, Kim D Y (2009). A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques. Computer-Aided Civil and Infrastructure Engineering, 24(3): 199–211
https://doi.org/10.1111/j.1467-8667.2008.00580.x
9 Cyganek B (2013). Object Detection and Recognition in Digital Images: Theory and Practice. New Jersey: Wiley
10 Dickscheid T, Schindler F, Förstner W (2011). Coding images with local features. International Journal of Computer Vision, 94(2): 154–174
https://doi.org/10.1007/s11263-010-0340-z
11 Dimitrov A, Golparvar-Fard M (2014). Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections. Advanced Engineering Informatics, Elsevier, 28(1): 37–49
https://doi.org/10.1016/j.aei.2013.11.002
12 Ding L Y, Yu H L, Li H, Zhou C, Wu X G, Yu M H (2012). Safety risk identification system for metro construction on the basis of construction drawings. Automation in Construction, Elsevier, 27: 120–137
https://doi.org/10.1016/j.autcon.2012.05.010
13 Du S, Shehata M, Badawy W (2011). Hardhat detection in video sequences based on face features, motion and color information. In: Proceedings of 3rd IEEE International Conference on Computer Research and Development
https://doi.org/10.1109/ICCRD.2011.5763846
14 Fang Q, Li H, Luo X, Ding L, Luo H, Rose T M, An W (2018). Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Automation in Construction, 85: 1–9
https://doi.org/10.1016/j.autcon.2017.09.018
15 Gheisari M, Irizarry J, Walker B N (2014). UAS4SAFETY: The potential of unmanned aerial systems for construction safety applications. Construction Research Congress
https://doi.org/10.1061/9780784413517.184
16 Girshick R (2015). Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision. 1440–1448
17 Gualdi G, Prati A, Cucchiara R (2011). Contextual Information and covariance descriptors for people surveillance: An application for safety of construction workers. EURASIP Journal on Image and Video Processing, 2011(1): 1–16
https://doi.org/10.1155/2011/684819
18 Ham Y, Han K K, Lin J J, Golparvar-Fard M (2016). Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): A review of related works. Visualization in Engineering, 4(1): 1
https://doi.org/10.1186/s40327-015-0029-z
19 Hamledari H, McCabe B, Davari S (2017). Automated computer vision-based detection of components of under construction indoor partition. Automation in Construction, Elsevier, 74: 78–94
https://doi.org/10.1016/j.autcon.2016.11.009
20 He K, Zhang X, Ren S, Sun J (2014). Spatial pyramid pooling in deep convolutional networks for visual recognition. European Conference on Computer Vision, 346–361
21 Health and Safety Executive (2014). Health and safety in construction sector in Great Britain.
22 Kaur N, Banga V K, Kaur A (2013). Image segmentation based on color. In: Proceedings of IJRET. International Journal of Research in Engineering and Technology, 2(11)
https://doi.org/10.1109/ICCCC.2016.7496766
23 Khoury H, Chdid D, Oueis R, Elhajj I, Asmar D (2015). Infrastructureless approach for ubiquitous user location tracking in construction environments. Automation in Construction, 56: 47–66
https://doi.org/10.1016/j.autcon.2015.04.009
24 Khoury H M, Akula M, Kamat V R (2012). Mobile and Pervasive Computing in Construction. Hoboken: John Wiley & Sons
25 Khoury H M, Kamat V R (2009a). Evaluation of position tracking technologies for user localization in indoor construction environments. Automation in Construction, Elsevier Science, 18(4): 444–457
https://doi.org/10.1016/j.autcon.2008.10.011
26 Khoury H M, Kamat V R (2009b). High-precision identification of contextual information in location-aware engineering applications. Advanced Engineering Informatics, Elsevier Science, 23(4): 483–496
https://doi.org/10.1016/j.aei.2009.04.002
27 Kim H, Lee Y, Yim B, Park E, Kim H (2016). On-road object detection using deep neural network. In: Proceedings of IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). IEEE, 1–4
28 Kim Y S, Lee J H, Yoo H S, Lee J B, Jung U S (2009). A performance evaluation of a Stewart platform based Hume concrete pipe manipulator. Automation in Construction, 18(5): 665–676
https://doi.org/10.1016/j.autcon.2009.02.003
29 Kopp A P, Saidi K S, Khoury H M (2010). Evaluation of ultra-wideband technology for use in 3D locating systems. In: Proceedings of the 10th Performance Metrics for Intelligent Systems (PerMIS) Workshop. Baltimore, MD, United States, 7–12
30 Krizhevsky A, Sutskever I, Hinton G E (2012). Imagenet classification with deep convolutional neural networks. In: Proceedings of the International Conference on Neural Information Processing Systems. 1097–1105
31 Memarzadeh M, Golparvar-Fard M, Niebles J C (2013). Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors. Automation in Construction, 32: 24–37
https://doi.org/10.1016/j.autcon.2012.12.002
32 Mikolajczyk K, Tuytelaars T (2009). Encyclopedia of Biometrics. New York: Springer US
33 Mneymneh B E, Abbas M, Khoury H (2016). A UAV-based image processing system for identifying and visualizing construction hazardous areas. In: Proceedings of 16th International Conference on Construction Applications of Virtual Reality. Hong Kong Occupational Safety and Health Administration
34 Commonly Used Statistics., 2016–12–19
35 Oueiss R, Chdid D, Khoury H M, Asmar D, Elhajj I (2012). Infrastructureless inspection process automation in construction. International Society for Gerontechnology, 11(2): 92–98
36 Park MW, Elsafty N, Zhu Z (2015). Hardhat-wearing detection for enhancing on-site safety of construction workers. Journal of Construction Engineering and Management, 141(9)
https://doi.org/org/10.1061/(ASCE)CO.1943-7862.0000974
37 Park M W, Koch C, Brilakis I (2012). Three-dimensional tracking of construction resources using an on-site camera system. Journal of Computing in Civil Engineering, 26(4): 541–549
https://doi.org/10.1061/(ASCE)CP.1943-5487.0000168
38 Peng Q, Luo W, Hong G, Feng M, Xia Y, Yu L, Hao X, Wang X, Li M (2016). Pedestrian detection for transformer substation based on gaussian mixture model and YOLO. In: Proceedings of International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). IEEE, 562–565
39 Ren S, He K, Girshick R, Sun J (2015). Faster r-cnn: towards real-time object detection with region proposal networks. In: Proceedings of International Conference on Neural Information Processing Systems. 91–99
40 Rubaiyat A H M, Toma T T, Kalantari-Khandani M, Rahman S A, Chen L, Ye Y, Pan C S (2016). Automatic detection of helmet uses for construction safety. In: Proceedings of IEEE/WIC/ACM International Conference on Web Intelligence Workshops. 135–142
41 Seo J, Han S, Lee S, Kim H (2015). Computer vision techniques for construction safety and health monitoring. Advanced Engineering Informatics, Elsevier, 29(2): 239–251
https://doi.org/10.1016/j.aei.2015.02.001
42 Shrestha K, Sherestha P P, Bajracharya D, Yfantis E A (2015). Hard-hat detection for construction safety visualization. Journal of Construction Engineering, 1–8
43 Skibniewski M J (2014). Information technology applications in construction safety assurance. Journal of Civil Engineering and Management, 20(6): 778–794
https://doi.org/10.3846/13923730.2014.987693
44 Chae S,Yoshida T, (2010). Application of RFID technology to prevention of collision accident with heavy equipment. Automation in Construction, 19(3): 368–374
https://doi.org/10.1016/j.autcon.2009.12.008
45 Yu J, Amores J, Sebe N, Tian Q (2006). A new study on distance metrics as similarity measurement. In: Proceedings of IEEE International Conference on Multimedia and Expo
46 Zhou Y, Nejati H, Do T T, Cheung N M, Cheah L (2016). Image-based vehicle analysis using deep neural network: A systematic study. In: Proceedings of IEEE International Conference on Digital Signal Processing
[1] Jun DU, Fangwen WENG. Construction management and technology innovation for main projects of Quanzhou Bay Bridge[J]. Front. Eng, 2021, 8(1): 151-155.
[2] Mingyue LI, Zhuoling MA, Xi TANG. Owner-dominated building information modeling and lean construction in a megaproject[J]. Front. Eng, 2021, 8(1): 60-71.
[3] Xiaoxiao XU, Patrick X. W. ZOU. System dynamics analytical modeling approach for construction project management research: A critical review and future directions[J]. Front. Eng, 2021, 8(1): 17-31.
[4] Gjorgji SHEMOV, Borja GARCIA de SOTO, Hoda ALKHZAIMI. Blockchain applied to the construction supply chain: A case study with threat model[J]. Front. Eng, 2020, 7(4): 564-577.
[5] Algan TEZEL, Eleni PAPADONIKOLAKI, Ibrahim YITMEN, Per HILLETOFTH. Preparing construction supply chains for blockchain technology: An investigation of its potential and future directions[J]. Front. Eng, 2020, 7(4): 547-563.
[6] Botao ZHONG, Haitao WU, Lieyun DING, Hanbin LUO, Ying LUO, Xing PAN. Hyperledger fabric-based consortium blockchain for construction quality information management[J]. Front. Eng, 2020, 7(4): 512-527.
[7] Yongkui LI, Qing YANG, Beverly PASIAN, Yan ZHANG. Project management maturity in construction consulting services: Case of Expo in China[J]. Front. Eng, 2020, 7(3): 384-395.
[8] Roshan POUDEL, Borja GARCIA de SOTO, Eder MARTINEZ. Last Planner System and Scrum: Comparative analysis and suggestions for adjustments[J]. Front. Eng, 2020, 7(3): 359-372.
[9] Ibrahim Yahaya WUNI, Geoffrey Qiping SHEN, Bon-Gang HWANG. Risks of modular integrated construction: A review and future research directions[J]. Front. Eng, 2020, 7(1): 63-80.
[10] Junqiang ZHANG, Honglong ZHENG, Wenyuan HE, Weihe HUANG. West-East Gas Pipeline Project[J]. Front. Eng, 2020, 7(1): 163-167.
[11] Conrad BOTON, Éric ST-PIERRE, Gabriel LEFEBVRE. Should medium-sized contractors still implement home-made information technologies on construction sites?[J]. Front. Eng, 2020, 7(1): 142-158.
[12] Chunfang LU, Junfei LIU, Yanhong LIU, Yuming LIU. Intelligent construction technology of railway engineering in China[J]. Front. Eng, 2019, 6(4): 503-516.
[13] Sarah M. L. HUBBARD, Bryan HUBBARD. A review of sustainability metrics for the construction and operation of airport and roadway infrastructure[J]. Front. Eng, 2019, 6(3): 433-452.
[14] Ming LU, Nicolas DIAZ, Monjurul HASAN. Proposing a “lean and green” framework for equipment cost analysis in construction[J]. Front. Eng, 2019, 6(3): 384-394.
[15] Houchen CAO, Yang Miang GOH. Analyzing construction safety through time series methods[J]. Front. Eng, 2019, 6(2): 262-274.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed