Overview of the Project In this collaborative project with Reachbot, our aim is to develop a defect detection system for videos captured by their welding robot. This system will employ AI algorithms to identify welding defects, such as porosity, weld spatter, arc-strikes, and defective cross-sectional profiles, from video and laser evidence. The implementation of AI-based defect detection is crucial in industries with high production rates, as it ensures consistent and accurate results, reduces human errors, and, consequently, leads to potential cost savings. Moreover, these AI systems can function in real-time, detecting defects promptly, which mitigates the propagation of issues and reduces repair costs.
Project Objective**:** The primary goal of this project is to develop an AI algorithm capable of detecting weld defects using camera images and cross-sectional profiles from a 2D laser scanner. The process encompasses gathering diverse data, labeling defects, training a machine learning model, validating its performance, and integrating it for real-time detection. Ultimately, this project aims to improve welding quality and safety.
Challenges Implementing AI-based weld defect detection presents various challenges, such as the need for a diverse dataset, precise labeling, machine learning model development, validation, and real-time integration. The project must address these challenges to achieve the overarching goal.
Reachbot Data: The project is based on data from the welds of a robotic welder. The data includes both color video in 1920x1080px mp4 format and laser profile video in 1240x1392px mp4 format. The project adopts both classification and object detection for modeling.🤖
Video Preprocessing: To effectively preprocess the video data, frames were extracted at 2Hz and 5Hz intervals, resulting in thousands of images. However, the closely spaced frames created near-identical images. As a solution, frame capture was reduced to approximately one image every three seconds for creating the training set. Nevertheless, this posed challenges as the robot's speed could change or it might stop altogether. Manual checks were performed to remove near-duplicates.🎥
Synthetic data**:** Data Preprocessing included the acquisition and preprocessing of training data as well as the generation of synthetic data. Initially, due to limited images, efforts were made to find public sources for welding camera images. Synthetic images were generated using techniques like GANs and Stable Diffusion.
Labeling and Annotation: The labeling process involved creating a classification system (ontology) for weld faults, and significant time was invested in labeling the images. Later in the project, laser scanner videos were received and labeled using the same ontology, based on partner-provided documents. We utilized Labelbox, an image labeling platform, to annotate images extracted from a video for our project. The project involved both classification and object detection tasks, aiming to develop a model capable of accurately identifying welding defects and features. The ontology we followed included labels such as 'spatter', 'porosity', 'burn_through', and 'cracks'. By labeling the images, we provided the necessary ground truth data for training the model. We created a pipeline, as depicted in the figure.✍️
Dataset: The most recent dataset from Reachbot consists of 1,649 images. This dataset was used for training and testing, with 80% used for training and 20% for testing. It includes a range of classes, classifications, and annotations, as shown in the table below:📸
| Classes | Classifications | Annotations |
|---|---|---|
| spatter | 751 | 9658 |
| irregular_bead | 458 | 1348 |
| slag | 408 | 1370 |
| start_stop_overlap | 275 | 491 |
| porosity_burn_through | 321 | 694 |
Before classification, the images go through a series of preprocessing steps for enhancing their quality. These operations include: a. Blur: The images are subjected to blurring to reduce noise and achieve a smoother, more regular appearance. b. Gradation: Gradation techniques are applied to improve contrast and emphasize image details. c. OTSU Threshold: OTSU thresholding is employed to convert the images into a binary format. Pixels are categorized as black or white based on a calculated threshold.