Autonomous Welding Advancement: YOLOv5 Algorithm and RealSense Depth Camera Integration

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Dec 7 2023

In an article published in the journal Nature, researchers developed an innovative robot welding guidance system by integrating an improved You Only Look Once (YOLO)v5 algorithm with a RealSense Depth Camera. This system enhanced the autonomy of welding robots by addressing the limitations of traditional laser vision sensors, allowing for precise weld groove detection and autonomous welding operations.

Background

In industrial welding, widespread manual labor results in high intensity and low efficiency. The increasing use of welding robots addresses these challenges, but scenarios with randomly placed workpieces pose difficulties. Laser vision sensors (LVS) are employed for weld tracking, yet their limited field of view hinders comprehensive positioning. Existing machine vision methods assist in weld seam identification, relying on graphical features and stable object states.

Deep learning, specifically YOLOv5, has shown promise in weld seam detection. However, previous methods have limitations in considering large and complex work areas and often provide 2D image coordinates rather than real-world coordinates. To overcome these shortcomings, researchers of the present study proposed an advanced YOLOv5 algorithm with a Coordinate Attention (CA) module, enhancing focus on weld groove features in intricate environments. Additionally, integrating a RealSense Depth Camera enabled real-time inspection of 3D coordinates, offering a more comprehensive understanding of the weld groove's position in a global environment.

Materials and Methods

The welding robot guidance system, composed of an improved YOLOv5 algorithm and a RealSense depth camera, was designed to enhance welding automation. It involved a host computer processing image data and transmitting 3D coordinates to guide the welding robot. First, a YOLOv5 model was trained for weld groove recognition. The depth camera captured RGB images for input to the YOLOv5 algorithm, providing weld groove type and location. A CA module, enhancing feature extraction in cluttered backgrounds, was added to YOLOv5 to focus more on weld grooves. This CA module efficiently embedded location details into the attention mechanism, intensifying feature extraction for weld grooves. The system utilized the YOLOv5s model, balancing performance and computational efficiency, with the CA module improving weld groove detection.

A RealSense D435i depth camera was used for accurate weld groove positioning based on camera calibration. It established relationships among world, camera, image, and pixel plane coordinate systems. The camera calibration, performed using Intel RealSense Dynamic Calibrator software and a calibration board, obtained intrinsic and extrinsic camera parameters, enhancing subsequent image analysis accuracy.

The system integrated the improved YOLOv5 output with depth camera localization, identifying the weld groove's center point on the workpiece surface. By correlating the RGB image with the depth point cloud, it converted the weld groove's position to spatial coordinates in the camera system. This process allowed the precise determination of the weld groove's 3D coordinates, which is critical for guiding the welding robot.

Overall, the system's synergy between the YOLOv5 algorithm's weld groove detection and the RealSense depth camera's accurate positioning enabled autonomous weld groove identification and robot guidance, addressing complexities in industrial welding environments. The method optimized detection accuracy while ensuring computational efficiency, which is vital for real-world implementation in settings with computing constraints.

Experiment and Analysis

The experiment evaluated the performance of the improved YOLOv5 algorithm for V-groove weld groove identification and guiding in robotic welding. A dataset of 4000 V-groove plate images, obtained with an Intel RealSense D435i camera, underwent augmentation for the representation of diverse working conditions.

The improved YOLOv5 algorithm, incorporating the CA module, demonstrated enhanced accuracy, with a notable increase in mean average precision (mAP) from 82.3% to 90.8%. Real-time performance, crucial for welding tasks, exhibited an impressive 20 frames per second (FPS), meeting the demands of practical production scenarios. Comparative analyses with other models, such as Faster Region-based Convolutional Neural Network (R-CNN) and Detection Transformer (DETR), highlighted the superiority of the proposed method in terms of accuracy and real-time capabilities.

Furthermore, the experiment deployed the algorithm on a robotic system, combining the improved YOLOv5 with a RealSense depth camera. The robotic system accurately identified and positioned V-groove weld grooves at different distances, demonstrating its effectiveness in practical applications. The guiding experiments yielded precise results, with absolute errors in the X and Y directions within 2 mm and in the Z direction within 3 mm.

Error percentages were effectively controlled within 2% for various distances, showcasing the system's robust performance. The algorithm's detection speed reached 20 FPS on the test platform, ensuring real-time weld groove identification. Overall, the experiment validated the algorithm's accuracy, real-time capabilities, and suitability for guiding welding robots in diverse scenarios, marking a significant advancement in welding automation and precision.

Conclusion

To summarize, the proposed robot welding guidance system, integrating an enhanced YOLOv5 algorithm with a RealSense depth camera, offered a solution to the manual setup of weld seam tracking sensors. The improved object detection, combined with depth camera positioning, achieved real-time, accurate V-weld groove detection. The system enhanced welding robot automation, eliminating the need for preset scanning trajectories. Future research would explore the extension to various weld types and further optimization for computational efficiency on constrained platforms.