Real-time UAV Patrolling Video Positioning: Enhanced Algorithms Using DOM Rectification

By Samudrapom DamReviewed by Susha Cheriyedath, M.Sc.Dec 13 2023

In an article recently published in the journal Scientific Reports, researchers proposed a series of visual positioning algorithms for unmanned aerial vehicle (UAV) patrolling video sequence images based on digital orthophoto map (DOM) rectification.

Background

The technological advancement of multi sensors has enabled UAVs to locate and identify key targets in geological disaster-prone areas or crucial monitoring areas by capturing video sequence images. UAVs are increasingly being used for field patrol inspection due to their high efficiency and low cost. UAVs equipped with a position and orientation system (POS) unit and an image sensor can be used for field patrol inspection in real-time.

In the real-time patrolling video sequence images captured by UAVs, accessing the precise interest point location is important for eliminating and discovering hidden safety hazards. However, simultaneously realizing precise, dynamic, and robust positioning of UAV’s patrolling video sequence images remains a significant challenge.

Currently, four different methods, including the optical flow method, image feature matching method, binocular vision method, and photogrammetry method, can be utilized for UAV patrolling video sequence image positioning. However, no method can effectively and accurately solve the absolute positioning of UAV’s patrolling video sequence images in real-time.

The proposed approach

In this study, researchers proposed a visual positioning model, including a precise polynomial-rectifying algorithm and a robust block-matching algorithm, to realize real-time positioning of UAV’s patrolling video sequence images based on DOM rectification.

The block-matching algorithm was utilized to obtain the best matching area for UAV’s video sequence image on DOM, while the precise polynomial-rectifying algorithm was used to calculate the precise rectification parameters of mapping UAV’s video sequence image to the best matching area obtained using the block-matching algorithm.

DOM is a pre-acquired digital orthophoto map covering the UAV’s entire patrolling region. Initially, a robust block-matching algorithm was constructed to obtain rough positioning of UAV’s video patrolling video sequence images. The block-matching algorithm was divided into five steps, including scaling datum-DOM, block-matching roughly based on gradient magnitude, block-matching robustly, block-matching roughly based on RGB, and extracting block-matched-DOM.

Researchers obtained the block-matched-DOM through these five steps, and the rough positioning of UAV’s patrolling video sequence images was attained by assigning the geodetic coordinates of every pixel in block-matched-DOM to pixels at the same position in UAV’s patrolling video sequence images. Subsequently, the precise polynomial-rectifying algorithm was constructed to achieve accurate UAV’s patrolling video sequence image positioning.

The precise polynomial-rectifying algorithm was also divided into five steps, including the construction of polynomials of the video frame and block-matched-DOM, the construction of precise rectifying equations, the construction of differential-difference polynomials, the construction of optimal estimation model, and calculating the geodetic coordinates of the interest points in the video frame.

Researchers obtained accurate rectification parameters through these five steps. They realized the accurate positioning of UAV’s patrolling video sequence images using the accurate rectification parameters to calculate the geodetic coordinates of every pixel in UAV patrolling video sequence images. They performed three practical experiments to analyze and verify the proposed visual positioning algorithms.

Experimental evaluation and findings

The practical experiments designed for this study were three videos and two region-DOMs. Among them, three videos were shot by three sorties fly of UAV in various areas, including high relief amplitude area, river area, and town area, while two region-DOMs possessed different spatial resolutions, with one of the two region-DOMs had a higher spatial resolution and the other one had a higher spatial resolution.

Experiment results demonstrated that all proposed visual positioning algorithms were feasible, effective, and fast. The algorithms steadily achieved positioning of every UAV’s patrolling video sequence image within 1 s with about 2.5 m level accuracy even when the surface-specific features, spatial resolution, topographic relief, and illumination condition were significantly different between UAV and DOM’s patrolling video sequence images.

The average video frame positioning deviations using the robust block-matching algorithm was 5.653 m, and the average video frame positioning deviations using the precise polynomial-rectifying algorithm was 2.681 m, which indicated the effectiveness of using the precise polynomial-rectifying algorithm to significantly increase the video frame positioning accuracy.

Additionally, using region-DOM of high spatial resolution substantially improved the positioning accuracy of video frames. The red homologous points located on high-rise buildings and mountains had a lower positioning accuracy, while points located on low-rise buildings and roads had a higher positioning accuracy.

Moreover, the average positioning accuracy in the river area and town area was slightly higher than the average positioning accuracy in the high-relief amplitude area. The average time of calculating all pixels’ accurate geodetic coordinates in a video frame, the average time of calculating the optimal estimation, and the average time of extracting the block-matched-DOM were 0.076 s, 0.19 s, and 0.148 s, respectively. Thus, the total positioning time of a UAV’s patrolling video frame was less than 1 s.