Revolutionizing Construction Waste Sorting: A Machine Vision Approach

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Jan 19 2024

In a paper published in the journal PLOS ONE, researchers addressed the challenges posed by the surge in urbanization, which brought both convenience and increased construction solid waste. They developed a machine vision (MV) detection algorithm for sorting solid waste, overcoming the limitations of scarce datasets from field conditions and high manual annotation costs. Diverse sets of red, green, blue, and depth (RGB-D) images were automatically collected, for instance, labeling through an acquisition and detection platform.

The researchers proposed a rapid-generation method for the synthetic construction of solid waste datasets and automatic annotation methods grounded in semi-supervised self-training and RGB-D fusion edge detection. Validation in two working conditions demonstrated the effectiveness of these approaches, with the generated dataset achieving higher F1 scores than manually labeled datasets, highlighting the success of the proposed methods in addressing construction solid waste detection challenges.

Related Work

Previous research has highlighted the escalating issue of construction solid waste in China, with only 10% adequately disposed of. While deep learning (DL), particularly instance segmentation, holds promise for waste sorting, challenges include the labor-intensive manual labeling of datasets and the difficulty adapting to changing sorting conditions. Synthetic datasets, especially those employing the copy-paste technique, offer a more efficient alternative for data annotation.

Despite the effectiveness demonstrated in various studies, the debate persists on the necessity of contextual information for accurate segmentation. Moreover, existing instance segmentation approaches often rely on weakly supervised annotations, hindering their adaptability to new conditions.

Automated Construction Waste Annotation

The research focuses on the automatic annotation of construction waste datasets, utilizing 298 samples comprising four waste categories: bricks, concrete, wood, and rubber. These samples, collected in Quanzhou City, Fujian Province, present a realistic mix of construction solid waste. The team constructed an acquisition and detection platform featuring RGB and depth imaging modules on a conveyor belt, ensuring efficient data collection. Additionally, the method introduces two automatic annotation techniques based on semi-supervised self-training and RGB-D fusion edge detection.

The study proposes a rapid dataset generation method using copy-paste, aiming to augment the initial dataset automatically. It involves acquiring waste instances, performing data augmentation, capturing background images, generating distribution points, and pasting instances onto background images.

By leveraging matrix operations, the method efficiently produces diverse datasets, overcoming the limitations of manually arranged sparse cases. The postprocessing step addresses label interference in densely distributed images, ensuring accurate contours for synthetic stacked datasets. Introducing two automatic labeling methods for real stacked datasets aims to reduce manual labeling efforts. 

The first method employs semi-supervised self-training, utilizing a model trained on rapidly generated datasets to predict accurate condition data. Researchers convert the predicted results into annotated contours and labels, followed by manual correction and continuous model retraining. The second method combines RGB-D fusion edge detection with rapid-generation datasets to annotate real stacked datasets automatically. This approach, designed to handle natural shadows, heights, and edges, enhances the model's adaptability to working conditions.

The methods outlined in the research encompass experimental setup, rapid generation of synthetic datasets, and automatic labeling techniques for real working condition datasets. The integration of automatic annotation approaches demonstrates an effort to streamline the dataset augmentation and labeling process, aiming for efficiency and accuracy in training instance segmentation models for construction waste sorting.

Efficient Construction Waste Segmentation

The study aimed to validate the effectiveness of a proposed method for construction waste sorting. The researchers actively generated datasets using a rapid generation method, partitioning them into training and testing sets. They utilized the mask region-based convolutional neural network (mask R-CNN) instance segmentation algorithm, incorporating a residual network (ResNet101) and a feature pyramid network (FPN) for training and evaluation. The experiment assessed the algorithm's performance using precision, recall, and F1-score metrics. The researchers executed all experiments following a consistent training methodology on a DL workstation with specific configurations.

The training process utilized the Tensorflow 1.15 framework to initialize the model with pre-trained weights from the everyday objects in context (COCO) dataset. The training parameters included a learning rate of 0.001, a batch size of four, and an image size of 512x512.

The researchers collected datasets under two working conditions: simple (a) and complicated (b). They automated the generation of training sets (aauto and bauto) and utilized a rapid generation method (ACP and BCP) to augment these datasets. Researchers utilized manually annotated test sets (atest and btest) to evaluate the model's performance. The study highlighted the importance of simulating real-world conditions through dataset augmentation.

The evaluation metrics included precision, recall, and F1-score. The experiment compared annotation performance between manually labeled datasets and automatically generated datasets. The study revealed that the proposed rapid generation method significantly reduced labeling time while enhancing annotation accuracy. The instance segmentation performance was assessed under simple and complex conditions, showcasing the model's adaptability to varying working conditions. The results demonstrated that the rapid generation method outperformed manually labeled datasets, emphasizing its efficacy in handling changing operational scenarios.

Conclusion

To sum up, this paper introduces a rapid dataset generation method for stacked construction solid waste datasets, offering two automatic annotation techniques. The proposed platform automates RGB-D image collection and dataset generation, significantly reducing time and workload. Semi-supervised learning and edge detection methods enable quick annotation of real stacked datasets without manual efforts.

Experimental results demonstrate the effectiveness of achieving high F1 scores under simple and complex working conditions. Despite these advantages, limitations include a limited sample variety, potential challenges in extreme scenarios, and the need for human verification despite reduced manual efforts.