Improving accuracy of automatic optical inspection with machine learning

doi:10.1007/s11704-021-0244-9

Front. Comput. Sci.

2022, Vol. 16

Issue (1) : 161310 https://doi.org/10.1007/s11704-021-0244-9

RESEARCH ARTICLE

Improving accuracy of automatic optical inspection with machine learning

Xinyu TONG¹, Ziao YU¹, Xiaohua TIAN¹(

), Houdong GE², Xinbing WANG¹

¹. Department of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
². Ambit Microsystems, Shanghai 201600, China

Download: PDF(5290 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Electronic devices require the printed circuit board (PCB) to support the whole structure, but the assembly of PCBs suffers from welding problem of the electronic components such as surface mounted devices (SMDs) resistors. The automated optical inspection (AOI) machine, widely used in industrial production, can take the image of PCBs and examine the welding issue. However, the AOI machine could commit false negative errors and dedicated technicians have to be employed to pick out those misjudged PCBs. This paper proposes a machine learning based method to improve the accuracy of AOI. In particular, we propose an adjacent pixel RGB value based method to pre-process the image from the AOI machine and build a customized deep learning model to classify the image. We present a practical scheme including two machine learning procedures to mitigate AOI errors.We conduct experiments with the real dataset from a production line for three months, the experimental results show that our method can reduce the rate of misjudgment from 0.3%–0.5% to 0.02%–0.03%, which is meaningful for thousands of PCBs each containing thousands of electronic components in practice.

Keywords automated optical inspection industrial internet of things machine learning image classification

Corresponding Author(s): Xiaohua TIAN

Just Accepted Date: 11 March 2021 Issue Date: 03 November 2021

Cite this article:

Xinyu TONG,Ziao YU,Xiaohua TIAN, et al. Improving accuracy of automatic optical inspection with machine learning[J]. Front. Comput. Sci., 2022, 16(1): 161310.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-021-0244-9
https://academic.hep.com.cn/fcs/EN/Y2022/V16/I1/161310

1	G R Blackwell. Electronic Systems Maintenance Handbook. 2nd ed. CRC Press, 2002
2	X Huang, S Zhu, X Huang, B Su, C Ou, W Zhou. Detection of plated through hole defects in printed circuit board with X-ray. In: Proceedings of the 16th IEEE Intnational Conferenceon on Electronic Packaging Technology. 2015, 1296–1301 https://doi.org/10.1109/ICEPT.2015.7236817
3	N E B Alaoui, P Tounsi, A Boyer, A Viard. Detecting PCB assembly defects using infrared thermal signatures. In: Proceedings of International Conference “Mixed Design of Integrated Circuits and Systems”. 2019, 345–349
4	S Härter, T Klinger, J Franke, D Beer. Comprehensive correlation of inline inspection data for the evaluation of defects in heterogeneous electronic assemblies. In: Proceedings of Pan Pacific Microelectronics Symposium. 2016, 1–6 https://doi.org/10.1109/PanPacific.2016.7428408
5	K P Wen, W M Wu, C Y Huang. Automatic optical inspection system and operating method thereof. U.S. Patent 10,438,340. 2019
6	JM Runji, C Lin. Automatic optical inspection aided augmented realitybased PCBA inspection: a development. In: Proceedings of IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology. 2019, 667–671 https://doi.org/10.1109/JEEIT.2019.8717385
7	W Wang, S Chen, L Chen, W Chang. A machine vision based automatic optical inspection system for measuring drilling quality of printed circuit boards. IEEE Access, 2017, 5: 10817–10833 https://doi.org/10.1109/ACCESS.2016.2631658
8	G Qiang, Z Shanshan, Z Yang, C Mao. Detection method of PCB component based on automatic optical stitching algorithm. Circuit World, 2015, 41(4): 133–136 https://doi.org/10.1108/CW-09-2014-0039
9	C Shorten, TM Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 2019, 6(1):1–48 https://doi.org/10.1186/s40537-019-0197-0
10	B Lengerich, E P Xing, R Caruana. On dropout, overfitting, and interaction effects in deep neural networks. 2020, arXiv preprint arXiv: 2007.00823
11	X Liu, J Zhang, S Jiang, Y Yang, K Li, J Cao, J Liu. Accurate localization of tagged objects using mobile RFID-augmented robots. IEEE Transactions on Mobile Computing, 2021, 20(4): 1273–1284 https://doi.org/10.1109/TMC.2019.2962129
12	X Liu, S Chen, J Liu, W Qu, F Xiao, A X Liu, J Liu. Fast and accurate detection of unknown tags for RFID systems-hash collisions are desirable. IEEE/ACM Transactions on Networking, 2020, 28(1):126–139 https://doi.org/10.1109/TNET.2019.2957239
13	X Tong, K Liu, X Tian, L Fu, X Wang. Fineloc: a fine-grained selfcalibrating wireless indoor localization system. IEEE Transactions on Mobile Computing, 2018, 18(9): 2077–2090 https://doi.org/10.1109/TMC.2018.2871206
14	T B Du, G H Shen, Z Q Huang, Y S Yu, D X Wu. Automatic traceability link recovery via active learning. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1–9 https://doi.org/10.1631/FITEE.1900222
15	J H Huang, X G Di, A Y Chen. A novel convolutional neural network method for crowd counting. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1150–1160 https://doi.org/10.1631/FITEE.1900282
16	E Alreshidi. Smart sustainable agriculture (SSA) solution underpinned by internet of things (IoT) and artificial intelligence (AI). 2019, arXiv preprint arXiv: 1906.03106 https://doi.org/10.14569/IJACSA.2019.0100513
17	S G Tzafestas. Synergy of IoT and AI in modern society: the robotics and automation case. Robotics & Automation Engineering Journal, 2018, 31(5): 1–15 https://doi.org/10.19080/RAEJ.2018.03.555621
18	L Xiao, X Wan, X Lu, Y Zhang, D Wu. IoT security techniques based on machine learning: how do IoT devices use AI to enhance security? IEEE Signal Processing Magazine, 2018, 35(5): 41–49 https://doi.org/10.1109/MSP.2018.2825478
19	Y Meidan, M Bohadana, A Shabtai, J D Guarnizo, M Ochoa, N O Tippenhauer, Y Elovici. ProfilIoT: a machine learning approach for IoT device identification based on network traffic analysis. In: Proceedings of the Symposium on Applied Computing. 2017, 506–509 https://doi.org/10.1145/3019612.3019878
20	W Njima, I Ahriz, R Zayani, M Terre, R Bouallegueet. Deep CNN for indoor localization in IoT-sensor systems. Journal of Sensors, 2019, 19(14): 3127–3132 https://doi.org/10.3390/s19143127
21	J Canedo, A Skjellum. Using machine learning to secure IoT systems. In: Proceedings of the 14th Annual Conference on Privacy, Security and Trust. 2016, 219–222 https://doi.org/10.1109/PST.2016.7906930
22	S Wang, T Tuor, T Salonidis, K K Leung, C Makaya, T He, K Chan. When edge meets learning: adaptive control for resource-constrained distributed machine learning. In: Proceedings of the IEEE Conference on Computer Communications. 2018, 63–71 https://doi.org/10.1109/INFOCOM.2018.8486403
23	L Xiao, X Wan, X Lu, Y Zhang, D Wu. IoT security techniques based on machine learning: how do IoT devices use AI to enhance security? IEEE Signal Processing Magazine, 2018, 35(5): 41–49 https://doi.org/10.1109/MSP.2018.2825478
24	R Pramudita, F I Hariadi. Development of techniques to determine object shifts for PCB board assembly automatic optical inspection. In: Proceedings of the International Symposium on Electronics and Smart Devices. 2018, 1–4 https://doi.org/10.1109/ISESD.2018.8605458
25	F Wu, S Li, Y Zhao. A self-adaptive study method for multi-parameters thresholds in AOI system. In: Proceedings of the 11th World Congress on Intelligent Control and Automation. 2014, 5256–5259 https://doi.org/10.1109/WCICA.2014.7053610
26	X Jia, T Wang, Y Li, J Liu, Y Zhang. AOI planning method based on genetic algorithm. In: Proceedings of International Conference on Mechatronics and Automation. 2019, 1801–1805 https://doi.org/10.1109/ICMA.2019.8816530
27	T Takacs, L Vajta. Novel outlier filtering method for AOI image databases. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2012, 2(4): 700–709 https://doi.org/10.1109/TCPMT.2012.2184765
28	V Chaudhary, I R Dave, K P Upla. Automatic visual inspection of printed circuit board for defect detection and classification. In: Proceedings of the International Conference on Wireless Communications, Signal Processing and Networking. 2017, 732–737 https://doi.org/10.1109/WiSPNET.2017.8299858
29	J Tsai, C Lin, C Chang, J Chou. Optimized positional compensation parameters for exposure machine for flexible printed circuit board. IEEE Transactions on Industrial Informatics, 2015, 11(6): 1366–1377 https://doi.org/10.1109/TII.2015.2489578
30	P Mohammadi, Z J Wang. Machine learning for quality prediction in abrasion-resistant material manufacturing process. In: Proceedings of the 2016 IEEE Canadian Conference on Electrical and Computer Engineering. 2016, 1–4 https://doi.org/10.1109/CCECE.2016.7726783
31	I Sartzetakis, K K Christodoulopoulos, EM Varvarigos. Accurate quality of transmission estimation with machine learning. IEEE/OSA Journal of Optical Communications and Networking, 2019, 11(3): 140–150 https://doi.org/10.1364/JOCN.11.000140
32	S Von Enzberg, A Al-Hamadi. A multiresolution approach to modelbased 3-D surface quality inspection. IEEE Transactions on Industrial Informatics, 2016, 12(4): 1498–1507 https://doi.org/10.1109/TII.2016.2585982
33	LM B Alonzo, F B Chioson, H S Co, N T Bugtai, R G Baldovino. A machine learning approach for coconut sugar quality assessment and prediction. In: Proceedings of the 10th IEEE International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management. 2018, 1–4 https://doi.org/10.1109/HNICEM.2018.8666315
34	E Sultanow, A Ullrich, S Konopik, G Vladova. Machine learning based static code analysis for software quality assurance. In: Proceedings of the 13th International Conference on Digital Information Management. 2018, 156–161 https://doi.org/10.1109/ICDIM.2018.8847079
35	X Li, W Zhang, Q Ding, X Li. Diagnosing rotating machines with weakly supervised data using deep transfer learning. IEEE Transactions on Industrial Informatics, 2019, 16(3): 1688–1697 https://doi.org/10.1109/TII.2019.2927590

[1]

Article highlights

Download

[1]	Zhen SONG, Yu GU, Zhigang WANG, Ge YU. DRPS: efficient disk-resident parameter servers for distributed machine learning[J]. Front. Comput. Sci., 2022, 16(4): 164321-.
[2]	Yu OU, Lang LI. Side-channel analysis attacks based on deep learning network[J]. Front. Comput. Sci., 2022, 16(2): 162303-.
[3]	Suyu MEI. A framework combines supervised learning and dense subgraphs discovery to predict protein complexes[J]. Front. Comput. Sci., 2022, 16(1): 161901-.
[4]	Yi REN, Ning XU, Miaogen LING, Xin GENG. Label distribution for multimodal machine learning[J]. Front. Comput. Sci., 2022, 16(1): 161306-.
[5]	Xia-an BI, Yiming XIE, Hao WU, Luyun XU. Identification of differential brain regions in MCI progression via clustering-evolutionary weighted SVM ensemble algorithm[J]. Front. Comput. Sci., 2021, 15(6): 156903-.
[6]	Xiaobing SUN, Tianchi ZHOU, Rongcun WANG, Yucong DUAN, Lili BO, Jianming CHANG. Experience report: investigating bug fixes in machine learning frameworks/libraries[J]. Front. Comput. Sci., 2021, 15(6): 156212-.
[7]	Yan-Ping SUN, Min-Ling ZHANG. Compositional metric learning for multi-label classification[J]. Front. Comput. Sci., 2021, 15(5): 155320-.
[8]	Jian SUN, Pu-Feng DU. Predicting protein subchloroplast locations: the 10th anniversary[J]. Front. Comput. Sci., 2021, 15(2): 152901-.
[9]	Syed Farooq ALI, Muhammad Aamir KHAN, Ahmed Sohail ASLAM. Fingerprint matching, spoof and liveness detection: classification and literature review[J]. Front. Comput. Sci., 2021, 15(1): 151310-.
[10]	Xu-Ying LIU, Sheng-Tao WANG, Min-Ling ZHANG. Transfer synthetic over-sampling for class-imbalance learning with limited minority class data[J]. Front. Comput. Sci., 2019, 13(5): 996-1009.
[11]	Yu-Feng LI, De-Ming LIANG. Safe semi-supervised learning: a brief introduction[J]. Front. Comput. Sci., 2019, 13(4): 669-676.
[12]	Wenhao ZHENG, Hongyu ZHOU, Ming LI, Jianxin WU. CodeAttention: translating source code to comments by exploiting the code constructs[J]. Front. Comput. Sci., 2019, 13(3): 565-578.
[13]	Hao SHAO. Query by diverse committee in transfer active learning[J]. Front. Comput. Sci., 2019, 13(2): 280-291.
[14]	Qingying SUN, Zhongqing WANG, Shoushan LI, Qiaoming ZHU, Guodong ZHOU. Stance detection via sentiment information and neural network model[J]. Front. Comput. Sci., 2019, 13(1): 127-138.
[15]	Ruochen HUANG, Xin WEI, Liang ZHOU, Chaoping LV, Hao MENG, Jiefeng JIN. A survey of data-driven approach on multimedia QoE evaluation[J]. Front. Comput. Sci., 2018, 12(6): 1060-1075.

Viewed

Full text

Abstract

Cited

Shared

Discussed