Sustainability in Fisheries: Real-Time Nephrops Counting with Edge Computing

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.May 2 2024

In an article published in the journal Nature, researchers focused on developing a real-time underwater video processing system to count Nephrops norvegicus (Nephrops) individuals entering a trawl in demersal trawl fisheries. By utilizing object detection and tracking methods on an edge device, the system aimed to provide catch composition and rates in real time.

The study evaluated various you-only-look-once (YOLO) models and frame-skipping approaches to achieve high processing speed and accurate counting, ultimately improving the sustainability of the fishery.

Background

Demersal trawling, a conventional fishing method, involves towing nets along the seafloor to catch bottom-dwelling fish and shellfish like cod, haddock, plaice, and Nephrops. Despite its global use, demersal trawling is controversial due to its seafloor impact and bycatch issues.

Efforts to mitigate these impacts have focused on improving gear selectivity and reducing benthic disturbance, but a significant challenge remains, which is the lack of real-time catch information during trawling operations. This hampers efficient fishing and sustainability efforts. Recent studies have explored using underwater cameras to collect in-trawl data, but processing this data in real-time onboard vessels presents a challenge.

While deep learning models offer promise for automated video processing, their computational demands often hinder real-time performance. Cloud-based solutions introduce network delays, making edge devices—an alternative solution—more attractive. This paper addressed these challenges by developing a real-time underwater video processing system to count Nephrops individuals in trawls.

Leveraging state-of-the-art object detection models and an edge computing device, the authors aimed to achieve near real-time processing speeds while accurately estimating catch counts. By evaluating different model settings and frame-skipping strategies, the researchers sought to optimize performance and overcome hardware limitations. Ultimately, this research filled a critical gap  by demonstrating the feasibility of real-time catch reporting in demersal trawl fisheries, paving the way for more sustainable and efficient fishing practices.

Methodology for Real-Time Nephrops Detection in Trawl Fisheries

Utilizing datasets generated from underwater stereo cameras on Nephrops fishing grounds, the researchers explored the feasibility of real-time catch monitoring. The dataset comprised 4044 images selected for Nephrops presence, with 1000 additional images created via augmentation for training. For testing, five diverse videos without overlapping Nephrops instances from the training set were chosen. The cameras recorded at 60 frames per second (FPS), with videos processed from the right camera only, consistent with previous work.

Two recent YOLO models, YOLOv7 and YOLOv8, were employed for Nephrops detection, each offering distinct architectural advantages. YOLOv7 integrated novel computational units like extended efficient layer aggregation network (E-ELAN) and auxiliary prediction heads, while YOLOv8 adopted a cross-stage partial network and eliminated anchor boxes, enhancing speed and accuracy.

Furthermore, adaptive frame skipping strategies were implemented to balance processing speed and counting accuracy, considering variations in Nephrops distribution across videos. Tracking Nephrops detections was achieved using the simple online and real-time tracking (SORT) algorithm, with specific criteria to identify true catch items. The study conducted extensive experiments varying model settings, including optimizers, batch sizes, and input dimensions, to optimize training performance.

Moreover, the impact of power consumption on processing speed was investigated, crucial for practical onboard deployment. Experimental setups were executed on a Tesla A100 graphics processing unit (GPU) for training and NVIDIA Jetson AGX Orin for testing, ensuring compatibility with edge computing hardware. Performance metrics included correct count rate and F-score for counting accuracy, alongside FPS for processing speed evaluation. Overall, this research contributed to advancing real-time catch monitoring in demersal trawl fisheries, addressing key challenges through a comprehensive methodology integrating cutting-edge deep learning models and edge computing technologies.

Performance Analysis and Optimization Strategies

The authors provided a comprehensive analysis of the methodology's performance in real-time Nephrops detection, considering diverse training and testing settings. The overview presented comparisons of models trained under various conditions, focusing on counting accuracy and processing speed. Models trained with Adam optimizer, 416-pixel image dimension, and 32-image batches exhibited superior counting performance, particularly YOLOv8n, YOLOv8s, YOLOv8m, and YOLOv7-Tiny models, known for their faster processing speeds.

Selected models demonstrated high accuracy in Nephrops counting across different test videos. Frame skipping strategies were explored to enhance processing speed, with adaptive frame skipping proving effective by dynamically adjusting frame processing based on Nephrops presence. Additionally, constraints on power consumption impacted processing speed, with a reduction observed when limiting the edge device to 50 watts.

The rate of processed frames significantly influenced overall processing speed, particularly evident in adaptive frame-skipping scenarios where frame skipping varies dynamically. These findings offered valuable insights into optimizing real-time Nephrops detection in demersal trawl fisheries, bridging the gap between efficient catch monitoring and sustainable fishing practices.

Exploring Optimized Configurations for Real-time Nephrops Counting

The researchers extensively examined various training and testing configurations for a real-time Nephrops counting algorithm, showcasing its potential application in commercial fishing vessels. By evaluating different object detection models and settings, they identified optimal combinations for counting accuracy and processing speed. Notably, models trained with Adam optimizer, 416-pixel image dimensions, and a batch size of 32 demonstrated superior performance.

Additionally, the introduction of frame-skipping strategies enhanced processing speed, albeit with a slight decrease in counting accuracy. Adaptive frame skipping emerged as the most promising approach, allowing for significant speed improvements while maintaining counting precision. The authors also highlighted the importance of real-time catch information in enhancing fishing sustainability. They proposed future enhancements to the algorithm, including multi-species detection capabilities and optimization for reduced computational needs.

Conclusion

In conclusion, the researchers pioneered a real-time underwater video processing system for Nephrops counting in demersal trawl fisheries, leveraging advanced object detection models and edge computing. Through rigorous experimentation, optimal model configurations and frame-skipping strategies were identified, balancing processing speed and accuracy.

The findings underscored the potential for sustainable fishing practices through informed catch monitoring. By addressing critical challenges in real-time data processing, this research laid the groundwork for enhanced efficiency and transparency in demersal trawl fisheries.