Sky High Insights: Estimating Building Heights with DLIPHE Algorithm and Google Street View

By Dr. Sampath LonkaReviewed by Susha Cheriyedath, M.Sc.Jul 10 2023

In a paper published in the journal Mathematics, researchers proposed the deep learning and image processing-based height estimator (DLIPHE) algorithm, combining deep learning and image processing, to estimate building heights using static Google Street View images. By employing semantic segmentation and advanced image processing techniques, DLIPHE identifies buildings and extracts their contours, facilitating real-time and automatic height estimation for urban aerial devices (UADs) when selecting communication paths.

Background

Advancements in technology, particularly the introduction of 5G, have propelled the development of 6G networks. Although only a few technology giants have implemented 5G on a global scale, its scalability and flexibility offer extensive connectivity possibilities. The future of 6G networks may involve integrating artificial intelligence into intelligent transportation systems (ITS), enabling vehicle-to-vehicle (V2V), and vehicle-to-roadside (V2R) communications through high-frequency bands like mmWave.

To support these advancements, vehicles come equipped with advanced sensing technologies, including Internet of Things (IoT) devices, consumer electronics, and intelligent software. However, these technologies have diverse requirements, including low latency, high bandwidth, and increased capacity. Utilizing high-frequency bands presents challenges, as obstacles such as buildings, trees, and clouds can absorb signals. Diffraction and reflection further contribute to signal losses. Future 6G networks aim to integrate satellite, aerial, and terrestrial networks to address these limitations.

The present research focuses on aerial network scenarios in 6G, specifically electric vertical take-off and landing aircraft (eVTOL) and unmanned aerial vehicles (UAVs), referred to as UADs. Flying at around 120 meters, UADs also operate in high-frequency bands and may face diffraction losses when flying over buildings. Improving communication paths for UADs requires understanding and estimating the heights of these buildings.

The contributions of this research include the development of the DLIPHE algorithm, which

• Utilizes publicly available static Google Street View images to estimate building heights.
• Employs semantic segmentation and advanced image processing techniques to identify buildings and extract their contours from Street View images.

Related work

This section provides an overview of related works on diffraction loss and building height estimation.

Methods for estimating diffraction loss provide mathematical models but hinder real-time and automatic estimation due to the requirement of manual inputs, such as building heights.

High-resolution optical imagery methods utilize synthetic aperture radar (SAR) and satellite images to estimate building heights, but they are costly, computationally demanding, and rely on extensive data collection, limiting scalability and real-time usability.

Street Scene Imagery methods offer cost-effective and scalable building height estimation using deep learning on street-view data. These methods explore various approaches, including building corners, rooflines, and camera calibration, but the limited availability of deep data may constrain them.

​​​​Other approaches involve convolutional-deconvolutional networks on monocular remote sensing imagery, but they require heavyweight algorithms and machine learning for edge detection. 

Proposed methodology

This section provides an overview of the proposed DLIPHE technique, including the system model and diffraction model.

System model: This model focuses on potential applications for ITS equipped with consumer communication devices using mmWave for V2V or V2R communications in outdoor environments. The proposed method involves deploying UADs that fly over buildings, along with vehicles and consumer devices that communicate with UADs in non-line-of-sight (NLoS) scenarios. UADs transmit data to dedicated devices on the 28 GHz spectrum and operate in urban environments. The focus revolves around UADs that serve as transmitters and receivers, analyzing diffraction loss caused by buildings as obstacles. The model utilizes complex Fresnel integrals to estimate diffraction losses based on the height of the building.

​​​​​The proposed DLIPHE algorithm involves the following steps: downloading building images using the Google Street View API, applying semantic segmentation for building identification, extracting building heights using image processing techniques, obtaining building footprint data from the OpenStreetMap API, and estimating building height using a pinhole camera projection framework. The algorithm utilizes deep learning and image processing techniques to estimate building heights in a lightweight and real-time manner.

Experiment methodology and results

The dataset used in the experiment consisted of images obtained from the Google Street View API and building footprint data from OpenStreetMap. Two subsets of buildings were considered: low-rise and high-rise. The low-rise dataset comprised small buildings captured with a street-oriented camera, while the high-rise dataset included buildings taller than 200 meters with an upward-looking view. The DLIPHE method was employed for height estimation, and the results were evaluated on both datasets.

The accuracy of the methodology was assessed by measuring relative error. Buildings with a relative error of less than 2% were classified as accurate, resulting in an accuracy rate of 39%. Similar evaluations were performed for relative errors of less than 4%, 7%, and 10%, leading to accuracies of 83%, 96%, and 96%, respectively. Additionally, the normalized error was used to evaluate the efficiency of the DLIPHE results.

The analysis of errors revealed challenging scenarios, including buildings obstructed by vegetation in rural areas, overlapping buildings in dense urban areas, and slanted high-rise buildings. These challenges could impact the accuracy of height estimation. However, adjusting camera parameters and capturing images from different angles or distances using the Google Street View API can help mitigate these challenges.

Conclusion

In summary, the researchers studied diffraction losses for UAVs and eVTOLs hovering over buildings. A building height estimation technique called DLIPHE is proposed, utilizing deep learning and image processing techniques via the Street View API. The methodology achieves good accuracy on low-rise and high-rise buildings, with low relative errors in high-rise buildings. Future research will explore automation and designing communication paths based on building height estimation.