Deep Learning-Based Manatee Crowd Counting Using Anisotropic Gaussian Kernel for Low-Quality Images

By Dr. Sampath LonkaReviewed by Susha Cheriyedath, M.Sc.Nov 17 2023

Recent strides in artificial intelligence (AI) have extended its applications across various domains. Notably, AI's role in ecological preservation, particularly in safeguarding endangered species, has burgeoned. In a recent publication in the journal Scientific Reports, researchers proposed a deep neural network to aid in conserving manatees, an endangered species facing threats from human activities.

Background

Anthropogenic environmental impacts, encompassing deforestation, overpopulation, poaching, overfishing, and climate change, exert profound repercussions on wildlife. The resulting dispersion, while advantageous for certain invasive species, predominantly threatens natural ecosystems. Wildlife populations, including manatees, face alarming declines, with extinction rates surpassing the norm.

Manatees, belonging to four distinct global species—Trichechus inunguis, Trichechus pygmaeus, Trichechus senegalensis, and Trichechus manatus—encounter diverse challenges due to human-related threats. Variations in geographical distribution and habitat preferences make tracking manatees a formidable task due to their diverse behaviors. The recent decline in manatee populations necessitates innovative conservation strategies.

Traditional counting approaches are rendered insufficient due to the group or solitary existence, low population density, and evasive behavior of manatees. Accurate real-time data on manatee numbers and gathering patterns are imperative for informed ecological management decisions. Moreover, their dependence on seagrass in shallow waters heightens their susceptibility to boat collisions.

Various counting methods have emerged, focusing on detection-based, regression-based, and density estimation-based approaches. Convolutional Neural Network (CNN) techniques have asserted dominance in diverse counting tasks, particularly in density estimation. While prior methodologies mainly counted people, recent efforts extend to counting cars, animals, and cells. Detection-based approaches initially targeted people but struggled in congested scenes.

Crowd counting, vital in complex environments, has gained attention across domains like yield estimation, phenotyping, and wildlife monitoring. The current study proposes a specialized deep neural network for counting submerged animals, specifically manatees, in outdoor open-water environments using low-quality overhead webcam images. The unique challenges, including occlusion and the elongated shape of manatees, demand a tailored methodology.

Methodology for manatee counting

Researchers introduced a novel approach for manatee counting using anisotropic Gaussian kernel (AGK)-based crowd counting. The methodology involves kernel density-based counting and the proposition of a manatee-customized crowd counting framework. The foundational network adheres to recent trends, employing a density estimation framework.

Inputting an image represented as a matrix, the deep neural network is trained using the ground-truth density map, integrating normalized two-dimensional (2D) Gaussians at manatee locations. The kernel function, crucial in machine learning, undergoes discussion, with examples such as the Gaussian kernel, multivariate Student kernel, and Laplacian kernel.

The kernel density map generation process involves comparing each pixel to labeled points, employing a kernel density estimator (KDE). Subsequently, the density map is converted to counting through element-wise summation of density values. The transition to manatee-specific counting introduces line-segment labels as an alternative to point labels. This approach tackles the oval shape of manatees, utilizing Gaussian kernels for density map generation.

Kernel-based method AGK is proposed for enhanced accuracy, adjusting kernel parameters based on line-segment length and Full Width at Half Maximum (FWHM). This modification allows the AGK to capture the width of manatees' bodies effectively. The overall framework incorporates the Congested Scene Recognition Network (CSRNet) for learning to predict density maps. The workflow entails annotation and training, involving the generation of AGK density maps, followed by deep neural network training. The prediction phase utilizes the trained neural network for counting manatees in test images.

Experiments and results

Benchmark Data: The proposed framework's performance with low-resolution images is validated using manatee surveillance video clips from "Save the Manatee Club" at Blue Spring State Park. After generating images with FFmpeg and removing duplicates, a testbed of 784 images with varying manatee counts is obtained. To address image similarity, Visual Geometry Group (VGG)-16 extracts layer-wise features, and a method calculates differences between images using Euclidean distance.

Manatee Annotations: Line labels, drawn from tail to head, are employed to annotate images. Two endpoints of each line segment are saved into a JSON file, serving as point labels.

Parameter Settings: Multiple baseline methods' performance is validated on the Manatee dataset using three labels. The mean square error (MSE) loss measures pixel differences between ground truth and predicted density maps. Mean absolute error (MAE) and root mean square error (RMSE) are used to evaluate algorithm performance.

Baseline Models: Multi-column CNN, scale aggregation network (SANet), VGGNet, CSRNet, and multi-level attention refined U-Net (MARUNet) serve as baseline models for counting problems, trained with different types of density maps.

Results: The proposed AGK exhibits superior performance, closely resembling original maps in high-density scenarios. However, AGK may not universally outperform other models, showing variations across different density levels and model structures. The AGK-based method is applied to wheat head counting, demonstrating effectiveness in estimating density. CSRNet-anisotropy outperforms Yolo-v5, emphasizing overall quantity estimation rather than precise localization.

Conclusion

In summary, researchers introduced a deep neural network-based approach for counting manatee aggregations in low-quality images, addressing challenges such as reflection and occlusion. Line-label annotation and AGK are utilized, outperforming traditional methods. Transitioning from dot to line labeling improves wheat head counting, suggesting broader applications for convex-shaped objects in diverse contexts. Future research may explore counting manatees in complex underwater backgrounds, considering their movement for enhanced accuracy.