1. Department of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China 2. Ambit Microsystems, Shanghai 201600, China
Electronic devices require the printed circuit board (PCB) to support the whole structure, but the assembly of PCBs suffers from welding problem of the electronic components such as surface mounted devices (SMDs) resistors. The automated optical inspection (AOI) machine, widely used in industrial production, can take the image of PCBs and examine the welding issue. However, the AOI machine could commit false negative errors and dedicated technicians have to be employed to pick out those misjudged PCBs. This paper proposes a machine learning based method to improve the accuracy of AOI. In particular, we propose an adjacent pixel RGB value based method to pre-process the image from the AOI machine and build a customized deep learning model to classify the image. We present a practical scheme including two machine learning procedures to mitigate AOI errors.We conduct experiments with the real dataset from a production line for three months, the experimental results show that our method can reduce the rate of misjudgment from 0.3%–0.5% to 0.02%–0.03%, which is meaningful for thousands of PCBs each containing thousands of electronic components in practice.
G R Blackwell. Electronic Systems Maintenance Handbook. 2nd ed. CRC Press, 2002
2
X Huang, S Zhu, X Huang, B Su, C Ou, W Zhou. Detection of plated through hole defects in printed circuit board with X-ray. In: Proceedings of the 16th IEEE Intnational Conferenceon on Electronic Packaging Technology. 2015, 1296–1301 https://doi.org/10.1109/ICEPT.2015.7236817
3
N E B Alaoui, P Tounsi, A Boyer, A Viard. Detecting PCB assembly defects using infrared thermal signatures. In: Proceedings of International Conference “Mixed Design of Integrated Circuits and Systems”. 2019, 345–349
4
S Härter, T Klinger, J Franke, D Beer. Comprehensive correlation of inline inspection data for the evaluation of defects in heterogeneous electronic assemblies. In: Proceedings of Pan Pacific Microelectronics Symposium. 2016, 1–6 https://doi.org/10.1109/PanPacific.2016.7428408
5
K P Wen, W M Wu, C Y Huang. Automatic optical inspection system and operating method thereof. U.S. Patent 10,438,340. 2019
6
JM Runji, C Lin. Automatic optical inspection aided augmented realitybased PCBA inspection: a development. In: Proceedings of IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology. 2019, 667–671 https://doi.org/10.1109/JEEIT.2019.8717385
7
W Wang, S Chen, L Chen, W Chang. A machine vision based automatic optical inspection system for measuring drilling quality of printed circuit boards. IEEE Access, 2017, 5: 10817–10833 https://doi.org/10.1109/ACCESS.2016.2631658
8
G Qiang, Z Shanshan, Z Yang, C Mao. Detection method of PCB component based on automatic optical stitching algorithm. Circuit World, 2015, 41(4): 133–136 https://doi.org/10.1108/CW-09-2014-0039
B Lengerich, E P Xing, R Caruana. On dropout, overfitting, and interaction effects in deep neural networks. 2020, arXiv preprint arXiv: 2007.00823
11
X Liu, J Zhang, S Jiang, Y Yang, K Li, J Cao, J Liu. Accurate localization of tagged objects using mobile RFID-augmented robots. IEEE Transactions on Mobile Computing, 2021, 20(4): 1273–1284 https://doi.org/10.1109/TMC.2019.2962129
12
X Liu, S Chen, J Liu, W Qu, F Xiao, A X Liu, J Liu. Fast and accurate detection of unknown tags for RFID systems-hash collisions are desirable. IEEE/ACM Transactions on Networking, 2020, 28(1):126–139 https://doi.org/10.1109/TNET.2019.2957239
13
X Tong, K Liu, X Tian, L Fu, X Wang. Fineloc: a fine-grained selfcalibrating wireless indoor localization system. IEEE Transactions on Mobile Computing, 2018, 18(9): 2077–2090 https://doi.org/10.1109/TMC.2018.2871206
14
T B Du, G H Shen, Z Q Huang, Y S Yu, D X Wu. Automatic traceability link recovery via active learning. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1–9 https://doi.org/10.1631/FITEE.1900222
15
J H Huang, X G Di, A Y Chen. A novel convolutional neural network method for crowd counting. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1150–1160 https://doi.org/10.1631/FITEE.1900282
16
E Alreshidi. Smart sustainable agriculture (SSA) solution underpinned by internet of things (IoT) and artificial intelligence (AI). 2019, arXiv preprint arXiv: 1906.03106 https://doi.org/10.14569/IJACSA.2019.0100513
17
S G Tzafestas. Synergy of IoT and AI in modern society: the robotics and automation case. Robotics & Automation Engineering Journal, 2018, 31(5): 1–15 https://doi.org/10.19080/RAEJ.2018.03.555621
18
L Xiao, X Wan, X Lu, Y Zhang, D Wu. IoT security techniques based on machine learning: how do IoT devices use AI to enhance security? IEEE Signal Processing Magazine, 2018, 35(5): 41–49 https://doi.org/10.1109/MSP.2018.2825478
19
Y Meidan, M Bohadana, A Shabtai, J D Guarnizo, M Ochoa, N O Tippenhauer, Y Elovici. ProfilIoT: a machine learning approach for IoT device identification based on network traffic analysis. In: Proceedings of the Symposium on Applied Computing. 2017, 506–509 https://doi.org/10.1145/3019612.3019878
20
W Njima, I Ahriz, R Zayani, M Terre, R Bouallegueet. Deep CNN for indoor localization in IoT-sensor systems. Journal of Sensors, 2019, 19(14): 3127–3132 https://doi.org/10.3390/s19143127
21
J Canedo, A Skjellum. Using machine learning to secure IoT systems. In: Proceedings of the 14th Annual Conference on Privacy, Security and Trust. 2016, 219–222 https://doi.org/10.1109/PST.2016.7906930
22
S Wang, T Tuor, T Salonidis, K K Leung, C Makaya, T He, K Chan. When edge meets learning: adaptive control for resource-constrained distributed machine learning. In: Proceedings of the IEEE Conference on Computer Communications. 2018, 63–71 https://doi.org/10.1109/INFOCOM.2018.8486403
23
L Xiao, X Wan, X Lu, Y Zhang, D Wu. IoT security techniques based on machine learning: how do IoT devices use AI to enhance security? IEEE Signal Processing Magazine, 2018, 35(5): 41–49 https://doi.org/10.1109/MSP.2018.2825478
24
R Pramudita, F I Hariadi. Development of techniques to determine object shifts for PCB board assembly automatic optical inspection. In: Proceedings of the International Symposium on Electronics and Smart Devices. 2018, 1–4 https://doi.org/10.1109/ISESD.2018.8605458
25
F Wu, S Li, Y Zhao. A self-adaptive study method for multi-parameters thresholds in AOI system. In: Proceedings of the 11th World Congress on Intelligent Control and Automation. 2014, 5256–5259 https://doi.org/10.1109/WCICA.2014.7053610
26
X Jia, T Wang, Y Li, J Liu, Y Zhang. AOI planning method based on genetic algorithm. In: Proceedings of International Conference on Mechatronics and Automation. 2019, 1801–1805 https://doi.org/10.1109/ICMA.2019.8816530
27
T Takacs, L Vajta. Novel outlier filtering method for AOI image databases. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2012, 2(4): 700–709 https://doi.org/10.1109/TCPMT.2012.2184765
28
V Chaudhary, I R Dave, K P Upla. Automatic visual inspection of printed circuit board for defect detection and classification. In: Proceedings of the International Conference on Wireless Communications, Signal Processing and Networking. 2017, 732–737 https://doi.org/10.1109/WiSPNET.2017.8299858
29
J Tsai, C Lin, C Chang, J Chou. Optimized positional compensation parameters for exposure machine for flexible printed circuit board. IEEE Transactions on Industrial Informatics, 2015, 11(6): 1366–1377 https://doi.org/10.1109/TII.2015.2489578
30
P Mohammadi, Z J Wang. Machine learning for quality prediction in abrasion-resistant material manufacturing process. In: Proceedings of the 2016 IEEE Canadian Conference on Electrical and Computer Engineering. 2016, 1–4 https://doi.org/10.1109/CCECE.2016.7726783
31
I Sartzetakis, K K Christodoulopoulos, EM Varvarigos. Accurate quality of transmission estimation with machine learning. IEEE/OSA Journal of Optical Communications and Networking, 2019, 11(3): 140–150 https://doi.org/10.1364/JOCN.11.000140
32
S Von Enzberg, A Al-Hamadi. A multiresolution approach to modelbased 3-D surface quality inspection. IEEE Transactions on Industrial Informatics, 2016, 12(4): 1498–1507 https://doi.org/10.1109/TII.2016.2585982
33
LM B Alonzo, F B Chioson, H S Co, N T Bugtai, R G Baldovino. A machine learning approach for coconut sugar quality assessment and prediction. In: Proceedings of the 10th IEEE International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management. 2018, 1–4 https://doi.org/10.1109/HNICEM.2018.8666315
34
E Sultanow, A Ullrich, S Konopik, G Vladova. Machine learning based static code analysis for software quality assurance. In: Proceedings of the 13th International Conference on Digital Information Management. 2018, 156–161 https://doi.org/10.1109/ICDIM.2018.8847079
35
X Li, W Zhang, Q Ding, X Li. Diagnosing rotating machines with weakly supervised data using deep transfer learning. IEEE Transactions on Industrial Informatics, 2019, 16(3): 1688–1697 https://doi.org/10.1109/TII.2019.2927590
