Computer Vision for Object Tracking: A Review

By Muhammad OsamaReviewed by Susha Cheriyedath, M.Sc.Jun 4 2024

In a recent publication in the journal Computers, researchers explored recent literature on object-tracking methods, datasets, sensors, and applications. Their review aimed to guide engineers and developers in selecting the appropriate equipment, datasets, and techniques for their specific object-tracking needs.

Background

Object tracking is crucial in computer vision, enabling machines to interact with dynamic environments and perform tasks. It involves estimating an object's state, such as position, orientation, shape, appearance, motion, and identity, over time using sensor data like images, videos, depth maps, and point clouds.

Challenges include occlusion, illumination changes, background clutter, camera motion, object deformation, scale variation, and pose change. Therefore, effective object-tracking methods must address these challenges with suitable sensors, datasets, algorithms, and evaluation metrics.

About the Review

In this article, the authors conducted a systematic literature review of object-tracking methods in computer vision from 2013 to 2023. They searched for relevant papers in major journals and conferences using keywords such as “object tracking,” “computer vision,” “sensor,” “dataset,” and “application.” After applying their inclusion and exclusion criteria, which focused on the relevance, quality, and impact of the research, they selected 75 papers for analysis. The analysis of the selected papers covered the following aspects:

• Sensor equipment: The types and configurations of sensors used for object tracking were examined, including monocular, stereo, depth-based, or hybrid cameras. The survey considered the advantages and challenges associated with each type of sensor, such as cost, complexity, and accuracy in different environments and applications.
• Datasets: The sources and characteristics of data used for object tracking were reviewed. This included whether datasets were public or private, synthetic or real, contained single or multiple objects, and were application-specific or general. The authors also evaluated the size, diversity, and annotation quality of the datasets, which are crucial for training and benchmarking tracking algorithms.
• Approaches and methods: The strategies and techniques used for object detection, localization, and tracking were categorized and analyzed. This included traditional image processing methods, various deep learning architectures, data association techniques, filtering algorithms, and optimization strategies. The review also considered the computational efficiency and real-time performance of these methods.
• Applications: The domains and scenarios where object tracking is applied were explored, such as autonomous driving, robotics, medical diagnosis, and human-computer interaction. The survey highlighted specific use cases within each domain, demonstrating the practical significance and impact of object-tracking technologies.

The researchers also provided a comprehensive taxonomy of the literature, outlining the strengths and limitations of different approaches. This taxonomy serves as a valuable resource for selecting appropriate equipment, methods, and applications based on specific requirements.

Furthermore, they identified key research gaps and future directions for object tracking, such as improving the diversity and quality of datasets, developing end-to-end deep learning models, addressing challenges like occlusion and long-term tracking, and integrating multiple sensors and modalities to enhance robustness and accuracy.

Results

The authors found that object-tracking methods evolved significantly over the last decade, driven by deep learning advancements and the availability of annotated datasets. Different types of sensors offer distinct advantages and disadvantages, depending on the application and problem constraints.

Various approaches and methods involved trade-offs between accuracy, robustness, and efficiency, with no single method performing well in all scenarios. They suggested tailoring and adapting object-tracking methods to specific application requirements and challenges.

Key outcomes of the literature review were:

• Monocular cameras: Widely used for their low cost and accessibility, but lacking depth information and sensitivity to occlusion and scale variation.
• Depth-based cameras: Cameras like stereo and red, green, blue, and depth (RGB-D) provided depth information and 3D localization but were more expensive with limited range and resolution.
• Hybrid sensors: Combining sensors such as camera-radar, camera-LiDAR, or camera-IMU provided complementary data for high-risk applications like autonomous vehicles and drones but required complex calibration and fusion algorithms.
• Public datasets: These datasets were useful for benchmarking but had limitations and did not cover all scenarios.
• Image processing methods: Techniques like feature matching, morphological operations, and marker-based detection were simple and fast but not robust or accurate enough for complex, dynamic scenes.
• Deep learning methods: Techniques like convolutional neural networks, recurrent neural networks, and attention mechanisms were powerful and flexible but required large datasets and were computationally expensive.
• Data association methods: Methods like k-nearest neighbor, Hungarian algorithm, and multiple hypothesis tracking were essential for multiple objects tracking but suffered from identity switches and false positives/negatives due to occlusion and noise.
• Filtering and optimization methods: Techniques like the Kalman filter, particle filter, and genetic algorithm helped estimate and predict object states but had high computational complexity and did not capture nonlinear and multimodal dynamics.

Applications

Object tracking has a wide range of applications in various fields, such as autonomous driving, robotics, medical diagnosis, and human-computer interaction. In autonomous driving, object tracking helps vehicles navigate, avoid collisions, and follow traffic rules. In robotics, it enables interaction with humans and objects, task performance, and coordination. In medical diagnosis, it assists in disease detection, treatment, and monitoring. In human-computer interaction, it facilitates communication and interaction.

Conclusion

In summary, the review demonstrated that object tracking is a fundamental and challenging problem in computer vision with numerous applications and implications. The researchers provided a taxonomy of the different approaches and techniques used for object tracking, highlighting their strengths and limitations. Moving forward, they suggested improving the diversity and quality of datasets, developing end-to-end deep learning models, addressing occlusion and long-term tracking, and integrating multiple sensors and modalities.