UAV-Based Individual Chinese Cabbage Weight Prediction Using Multi-Temporal Data

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Nov 21 2023

In an article published in the journal Nature, researchers introduced a novel method for predicting individual Chinese cabbage harvest weight using unmanned aerial vehicles (UAVs) and multi-temporal features.

Background

Current challenges in vegetable production, exacerbated by environmental factors, necessitate efficient phenotyping methods. Traditional manual phenotyping is labor-intensive, prompting a shift to remote sensing. UAVs equipped with various sensors enable large-scale, non-destructive data collection. While these technologies have shown promise in canopy-level yield predictions, applications at the individual plant level, crucial for crops like Chinese cabbage, are limited.

Existing studies on UAV-based phenotyping often focus on canopy-level assessments, overlooking the specific needs of crops sold individually by weight. Chinese cabbage, a significant East Asian crop, relies on individual plant weight for pricing. Previous research explored weight prediction using either multi-spectral (MS) or Red-Green-Blue (RGB) imagery but involved manual data extraction.

The present study introduced an innovative approach, automating the detection of individual Chinese cabbage plants in the field using YOLOv5, an object detection algorithm. Subsequently, machine learning models leveraged multi-temporal features for predicting individual plant weights.

By evaluating the effectiveness of this approach, the study filled gaps in existing research by demonstrating the feasibility of predicting Chinese cabbage weights up to 53 days before harvest, achieving a Root Mean Squared Error (RMSE) of 560 g/plant. Using multi-temporal features enhanced the timeliness and accuracy of predictions, addressing the limitations of previous methods and providing valuable insights for precision agriculture and crop management.

Results

The study employed UAVs equipped with RGB and MS cameras to collect data on 1196 Chinese cabbage plants over 26 time points (TPs). Object-detection algorithms successfully identified 95% of the plants, allowing for the extraction of features for weight prediction. Three feature levels—RGB and MS imagery, vegetation indices (VIs), and 3D point cloud data—were analyzed using feature selection methods, identifying seven crucial features.

The key findings were as follows:

• Individual Plant Detection: Object-detection algorithms successfully identified 95% of Chinese cabbage plants, enabling the extraction of features for weight prediction.
• Feature Selection: Seven features were identified as crucial for weight prediction, encompassing RGB and MS imagery, VIs, and 3D point cloud data.
• Regression Models: Partial Least Squared (PLS), lasso, ridge, Support Vector Machine(SVM)-linear, and Random Forest (RF) regression models were trained, with combined feature levels yielding the best prediction results (RMSE = 541 g/plant).
• Multi-Temporal Data: Multi-temporal data significantly improved weight prediction accuracy, with the SVM-linear model using data between the first TP and eighth TP achieving the best cross-validation results (RMSE = 464 g/plant).
• Optimal Resolutions: The 7-day multi-temporal resolution provided reliable predictions (RMSE = 457 g/plant), outperforming other resolutions.
• Early Prediction: Accurate weight predictions were achieved up to 53 days before harvest, with the SVM-linear model exhibiting an RMSE of 436 g/plant.

Materials and Methodology

The pipeline implemented in this study was divided into four main steps:

1. Data Acquisition:

• Plant Materials and Cultivation: Chinese cabbage cultivars were bought and their fixed lines and F2 populations were cultivated in a Tokyo field.
• UAV Imagery Acquisition: RGB and MS imagery were captured using DJI Phantom 4 RTK and P4 Multispectral UAVs. Ground control points and network RTK Service enhanced spatial accuracy.
• Individual Plant Weight Measurement: Weights were measured 102-103 days post-transplanting, with fresh weights recorded after cutting.

2. Individual Plant Detection:

• YOLOv5 Algorithm: Implemented for detecting individual plants in RGB orthomosaic images.
• Training Dataset Preparation: Images were divided into tiles, manually labeled on the RoboFlow platform, and augmented for training.

3. Individual Plant Image Processing:

• Plant-Height Image: 3D point cloud data was transformed into plant-height images after interpolating and subtracting reference surface values.
• MS Co-Registration: MS imagery was registered to RGB resolution for combined analysis.
• Image Layer Stacking: RGB, MS, and plant height images were stacked for each time point.
• VIs: Seven relevant VIs were calculated from individual plant images.
• Noise Filtering: Filters were applied to remove non-vegetation pixels, smaller contours, and brightness/shadow effects.
• Image Centering: Images were centered by computing a convex hull polygon from the first TP RGB image.

4. Weight Prediction:

• Regression Models: PLS, lasso, ridge, SVM-linear, and RF models were trained and optimized using a five-fold cross-validation approach.
• Data Splitting: Individual plant data were split into training (90%) and testing (10%) datasets for evaluation.
• Feature Selection: Lasso regression, exhaustive search, sequential feature selection (SFS), and RF were used for selecting relevant features at three information levels.
• Evaluation Metrics: RMSE was used to assess individual weight prediction accuracy.

Discussion

This study introduced a novel approach for characterizing individual Chinese cabbage plants using UAV-based data, surpassing the limitations of previous methods. Leveraging an object detection algorithm, 95% of plants were automatically detected, enabling precise individual-level analysis. The study identified three levels of UAV information (RGB-MS, VI, and 3D point cloud) and determined that models trained with a reduced set of features (seven) achieve weight prediction accuracy comparable to models with all features.

3D point-cloud features, particularly volume, demonstrated the highest prediction accuracy. Combining information from multiple UAV sources significantly improved weight prediction accuracy. The evaluation of multi-temporal data highlighted the advantage of using sequences over single time-point features, with lower multi-temporal resolutions proving effective. Notably, the study achieved high accuracy in predicting individual plant weights as early as 21 days post-transplanting, showcasing the potential of multi-sourcing and multi-temporal UAV-based data for precise and early crop weight prediction.

In conclusion, this study successfully predicted individual Chinese cabbage weights using an object detection model and multi-temporal UAV-based data, offering insights for precision agriculture and crop management, with implications for farmers and breeders.