Automated Detection of Solder Splashes on Electronic Boards Using Advanced Machine Learning

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Nov 28 2023

In an article published in the journal Nature, researchers explored the application of cutting-edge machine-learning algorithms for the automated detection of solder splashes on electronic boards. The focus was on enhancing the reliability and lifespan of electronic components, crucial for contemporary manufacturing processes.

Background

The longevity and reliability of electronic boards hinge on the precision of the manufacturing process, particularly in soldering, a fundamental method for connecting circuit components. Solder splashes, common in power module production, can lead to variations in selected electrical parameters. This study addressed this challenge by introducing machine learning, specifically deep neural networks, into the automated detection of solder splashes. The object of interest, solder splash, presents challenges due to its small area and properties similar to its surroundings in terms of texture and color.

The research compared seven object detection models, focusing on You Only Look Once (YOLO) and Faster Region-Based Convolutional Neural Network (R-CNN) architectures. The results highlighted the efficacy of a custom-trained YOLOv8n detection model with 1.9 million parameters, demonstrating a low detection speed of 90 ms and a mean average precision of 96.6%. These findings suggested that integrating deep neural networks can facilitate early detection of solder splashes, potentially leading to increased productivity and cost savings in electronic board manufacturing. The study emphasized the shift from manual inspection to advanced machine learning techniques in the pursuit of more efficient and reliable manufacturing processes.

Materials and Methods

The study employed advanced machine learning models for automated visual inspection of Printed Circuit Boards (PCB), focusing on solder splash detection. Notably, the convolutional neural networks (CNNs) are integral to this research, with YOLOv8 and Faster R-CNN among the state-of-the-art methods for PCB defect detection.

• Related Work: Recent works, such as those by Xiong and Glučina, have highlighted YOLOv8's outstanding accuracy of 97%, with smaller YOLO versions proving effective for PCB inspection. Adibhatla et al. utilized Tiny-YOLOv2, achieving 98% accuracy for batch size 32, emphasizing YOLO's suitability for real-time detection. Liao modified YOLOv4 to detect various PCB surface defects, achieving a remarkable 98.6% mean average precision.
  Faster R-CNN, employed by Hu et al., demonstrated efficacy in detecting small defects on PCBs, showcasing its versatility in classifying various PCB defect types. The study emphasized the trade-off between detection accuracy and real-time processing, noting the suitability of deep neural networks for large target detection.
  The YOLO model, known for its detection speed and accuracy, is detailed, with YOLOv5 and YOLOv8 versions highlighted. YOLOv5, optimized for real-time applications, achieved state-of-the-art performance with a 50.5% average precision on the COCO dataset.
• Searching of Model Parameters (Adam Algorithm): The Adam optimization algorithm was explained, combining the strengths of AdaGrad and RMSProp. The equations governing the optimization process and updating of model parameters were detailed, emphasizing its effectiveness in minimizing error functions in neural networks.
• PCB Image Acquisition: The dataset creation involved high-resolution image capture using a Basler camera in a real manufacturing environment. A specialized tiling algorithm was introduced, optimizing the division of high-resolution images into smaller tiles to enhance YOLO training efficiency.
• Dataset Augmentation: The dataset was augmented to 24,000 tiles using LabVIEW, incorporating variations in light, blurring, and noise to simulate real manufacturing conditions. The dataset was extended and refined for training, focusing on images containing solder splashes.

Experimental Results

The experiments utilized Python 3.9 and PyTorch 1.13 for software development, executed on an AMD Ryzen 5600X CPU, NVIDIA GeForce RTX 3060 GPU, and 32 GB RAM on Windows 10. Seven object detection algorithms, five YOLO models, and two Faster RCNN models were compared for automated solder splash detection on PCBs.

Model Comparison: The YOLOv8n model emerged as the preferred choice, striking a balance between detection speed and accuracy. The computational complexity, detection speed, and mean average precision (mAP) scores for various models were evaluated. YOLOv8n demonstrated superiority, localizing small solder splashes with a precision score of over 96.6%.

Performance Metrics: Each model underwent ten learning runs with different pseudorandom number generator seed settings, and the best result was recorded. Precision and recall scores were used as key metrics. YOLOv8n exhibited high precision and recall, with fewer than 4% false positives and below 5% false negatives.

Learning Hyperparameters: The Ultralytics implementation of YOLOv5 and YOLOv8 involved adjusting hyperparameters to enhance detection performance. Parameters like learning rate and the proportion of background images in the dataset were optimized for improved results.

Learning Curves: Learning curves for YOLOv8n illustrated a decreasing trend in box loss and objectness loss, indicating the model's accuracy in localizing solder splash centers. Training was halted early based on increased objectness loss in the validation set, demonstrating good generalization potential.

Precision-Recall Curve: The precision-recall curve for YOLOv8n showcased its performance across different confidence thresholds, emphasizing its ability to balance precision and recall.

Discussion

The research aimed at detecting and localizing a specific PCB defect, solder splashes, using machine learning. The YOLOv8n algorithm accurately identified solder splashes on power electronics PCB boards. The challenge was the presence of numerous objects with similar texture, color, and shape in the high-resolution PCB images.

In contrast to existing work, the originality of this research was in the small size of the solder splashes relative to PCB dimensions and the high image resolution. This posed a unique challenge for deep neural networks. The study compared widely used object detection methods, Faster RCNN and YOLO, with seven different versions tested.

The YOLOv8n model emerged as the optimal choice, achieving the highest detection performance with a reasonable detection speed (90 milliseconds) and a moderate number of network parameters. YOLOv5n demonstrated faster detection (60 milliseconds) but at the expense of slightly lower mAP. In comparison, Faster RCNN with ResNet50-FPN showed competitive recall (96.3%) and mAP (95%) but with a considerably slower detection speed (690 milliseconds) and higher data storage requirements.

Considering the trade-off between detection speed, data storage, and performance, the YOLOv8n model was favored for automated solder splash detection. This decision was crucial for real-world applications, especially in serial manufacturing, where a balance between accuracy and efficiency is paramount. The study thus contributed valuable insights into selecting an appropriate model for detecting small defects in complex images, offering practical implications for industrial applications.

Conclusion

In conclusion, this study advocated utilizing machine learning, specifically the YOLOv8n detection model, to identify a specific PCB defect—solder splashes. The model proved effective with 1.9 million parameters, a rapid 90 ms detection speed, and an outstanding 96.6% mAP. Integrating automated optical inspection promises early PCB defect detection, potentially heightening productivity and reducing costs. Consistent lighting and color conditions were crucial, with Semikron-Danfoss maintaining normalized conditions.

Future research could explore dataset enlargement and increased image resolution for enhanced detection. Detecting various PCB defects remains a potential avenue for further investigation, highlighting the study's practical applications.