Deep Neural Network Improves Tea Leaf Pose Estimation

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Aug 8 2024

In an article published in the journal Computers and Electronics in Agriculture, researchers introduced TeaPoseNet, a deep neural network for estimating the pose of tea leaves, focusing on the Yinghong No.9 variety. They detailed the network's training on a specific dataset, comparison with other pose estimation methods, and the impact of the tea keypoint similarity non-maximum suppression algorithm (TKS_NMS) on improving pose recognition accuracy by 16.33%. The algorithm demonstrated high-performance metrics and a quick processing speed, marking a novel application of pose estimation in tea leaf analysis.

Background

Tea leaves, a globally consumed economic crop, exhibit a wide range of types and qualities that are intricately linked to their morphology, structure, and color. Accurate pose recognition of tea leaves is crucial for quality assessment, grading, and maturity monitoring. Previous research has made strides in tea leaf detection using deep learning models and computer vision, such as enhanced you-only-look-once (YOLO) algorithms and improved image processing techniques. However, challenges persist in reliably estimating tea leaf poses due to positioning errors and variations in growth poses.

To address these gaps, this study introduced TeaPoseNet, a deep neural network designed specifically for tea leaf pose estimation. This novel approach integrated a custom algorithm to accurately identify key points and analyze pose attributes like length, angle, and structure. TeaPoseNet utilized a newly constructed public dataset for one-bud-one-leaf pose estimation and demonstrated significant improvements, including a 16.33% enhancement in endpoint error accuracy through TKS_NMS. This paper thus provided a substantial advancement in the precision of tea leaf pose recognition.

Methodology for Tea Leaf Pose Estimation

The authors focused on developing an advanced method for estimating the poses of one-bud-one-leaf tea leaves. Initially, a schematic diagram was created identifying key points on tea leaves, which were annotated using Labelme software to aid in model training. The dataset, collected between March and April 2023 using a Realsense435 camera in Guangdong Province, included images under various conditions and was augmented for robustness.

The pose estimation network was built upon residual network (ResNet)-50, a powerful ResNet pre-trained on ImageNet. This network extracted features from tea leaf images, which were then processed through deconvolution layers to estimate pose key points. Enhancements included the use of TKS_NMS, an improvement on soft NMS (SOFT_NMS), to refine key point predictions by evaluating TKS and eliminating less accurate predictions based on a threshold.

Evaluation metrics such as the area under curve (AUC), endpoint error (EPE), normalized mean error (NME), percentage of correct key points (PCK), and frame per second (FPS) were employed to assess the algorithm's performance. These metrics measured the accuracy of key point predictions, the Euclidean distance between true and predicted points, and the overall detection performance, ensuring a thorough evaluation of the proposed pose estimation method.

Analysis and Findings

The authors evaluated the TeaPoseNet algorithm for tea leaf pose estimation using a high-performance computer setup with an i7-3960X central processing unit (CPU), 16 gigabytes (GB) random access memory (RAM), and an NVIDIA giga texel shader extreme (GTX) 1080TI graphics processing unit (GPU), running on Windows 10 with Python 3.8, PyTorch 1.12.1, and compute unified device architecture (CUDA) 11.3.1. The algorithm was trained over 300 epochs using the Adam optimizer.

The ablation study revealed significant improvements with the addition of the TKS_NMS module. TeaPoseNet with TKS_NMS demonstrated a 16.33% improvement in EPE accuracy and a 5.19% enhancement in normal NME accuracy, though at the expense of a 5.48% decrease in detection speed. Comparatively, other algorithms like ResNet-101 and high-resolution network (HRNet) also showed performance gains, particularly in EPE and NME metrics, with HRNet-W32 achieving the highest improvements.

The effect of image size on algorithm performance was assessed, indicating that larger image sizes led to increased computational complexity and decreased accuracy. Specifically, using larger images resulted in a drop of 5.10% in PCK and a notable decrease in FPS.

The visual analysis highlighted that TeaPoseNet performed robustly under high illumination but struggled with low and medium lighting conditions. The heatmap analysis of key points showed initial progress in accuracy but indicated that lower thresholds might be needed to address noise in feature maps.

Robustness testing on three public tea leaf datasets revealed limitations in the algorithm's adaptability to varied conditions and types. Future work will address these by incorporating temporal analysis and depth information and expanding to a broader range of tea varieties.

Conclusion

In conclusion, the researchers developed TeaPoseNet, a deep neural network for tea leaf pose estimation, focusing on the Yinghong No.9 variety. Trained on a custom dataset, TeaPoseNet, with the TKS_NMS algorithm, improved pose recognition accuracy by 16.33%. The system achieved high-performance metrics, although with a slight reduction in processing speed.

The study highlighted that TeaPoseNet performed well under high illumination but faced challenges with varying lighting conditions and larger image sizes. Future work will explore incorporating temporal models and depth information and expanding the dataset to enhance the algorithm's robustness and generalizability.