Deep Learning-based Automation of Cephalometric Landmarking

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Mar 25 2024

In an article published in the journal Nature, researchers proposed an automated cephalometric annotation method using deep learning-based approaches for three-dimensional (3D) facial stereophotogrammetry.

By comparing the precision of automated landmarks with manual annotation and semi-automated methods, the study demonstrated the effectiveness of the proposed approach. Achieving precise and consistent landmark annotation, the method enabled quantitative analysis of large datasets, facilitating applications in diagnosis, follow-up, and virtual surgical planning.

Background

Advancements in imaging technology, particularly in 3D imaging, have significantly impacted fields such as genetics, orthodontics, craniomaxillofacial surgery, and plastic surgery. Cone-beam computed tomography (CBCT) has emerged as a prominent technique for high-resolution multiplanar reconstruction of the craniofacial skeleton and facial soft tissue, albeit with the drawback of ionizing radiation.

In contrast, 3D stereophotogrammetry offers a radiation-free method for capturing detailed representations of craniofacial soft tissue. Cephalometric analysis plays a crucial role in extracting information about landmark positions and distances from 3D stereo-photographs to aid clinical diagnosis. However, manual landmarking remains a time-consuming and subjective process prone to observer variability.

Previous studies have explored the automation of hard-tissue landmark extraction using deep learning algorithms but have been limited in their application to soft-tissue landmarks. This paper addressed this gap by introducing an automated approach for extracting soft-tissue facial landmarks from 3D photographs using deep learning techniques.

By leveraging deep learning algorithms, the proposed method aimed to improve the accuracy and efficiency of landmark identification, offering potential advancements in soft-tissue analysis. The authors contributed by integrating deep learning into soft-tissue landmarking methodologies, potentially enhancing the precision and reliability of cephalometric analysis in clinical practice.

Method

The authors aimed to automate the process of cephalometric landmark annotation on 3D facial photographs, enhancing efficiency and reducing observer variability. A dataset of 3188 3D facial images was collected, manually annotated with 10 cephalometric landmarks. An automated landmarking workflow was developed, utilizing two DiffusionNet models for rough and refined landmark prediction.

The workflow involved preprocessing, DiffusionNet configuration, training, and post-processing steps. During rough prediction, meshes were downsampled, and semantic segmentation tasks were performed to predict landmark locations roughly. Subsequently, meshes were realigned, and facial regions were segmented to standardize data. A second semantic segmentation task was then executed to refine landmark predictions. Statistical analyses were conducted to assess the precision of automated landmarking compared to manual annotation and semi-automated methods.

The automated workflow achieved success in 98.6% of test cases, with a mean precision of 1.69 ± 1.15 mm comparable to manual annotation variability. The researchers demonstrated the feasibility and accuracy of deep learning-based automated landmark annotation on 3D facial photographs, offering potential benefits in clinical practice. The developed workflow provided a standardized and efficient method for cephalometric analysis, facilitating diagnosis, follow-up, and virtual surgical planning.

Results

The research involved the analysis of 2897 3D facial photographs after excluding 291 due to various criteria, with the most common exclusion being the lack of texture information. Significant differences in age and gender were observed between the two source databases. However, no statistically significant variations were found between the training and test datasets. Intra-observer and inter-observer variability in manual annotation ranged from 0.94 ± 0.71 mm to 1.31 ± 0.91 mm, respectively.

The initial DiffusionNet exhibited an average precision of 2.66 ± 2.37 mm, while the complete workflow achieved a precision of 1.69 ± 1.15 mm. Notably, 98.6% of the test data were successfully processed by the workflow, with precision within 2 mm for 69% of refined predicted landmarks. The semi-automated MeshMonk method showed an average precision of 1.97 ± 1.34 mm for the 10 landmarks.

Comparatively, the DiffusionNet-based method demonstrated significantly better precision for certain landmarks and worse for others. Overall, the authors showcased the effectiveness of the developed workflow in automating landmark annotation on 3D facial photographs with high precision, offering potential clinical applications in diagnosis and surgical planning.

Discussion

The developed workflow demonstrated high precision and consistency, closely matching the inter-observer variability observed in manual annotation. While previous studies focused on hard-tissue landmarking or utilized non-deep learning methods for soft-tissue landmarking, the researchers innovatively applied DiffusionNet-based deep learning, achieving notable precision. Although limitations existed regarding hardware and texture information, the workflow's adaptability to different imaging systems warranted further investigation.

Challenges related to symmetry detection and image quality were addressed, enhancing the workflow's robustness. Despite being evaluated on 10 landmarks, the method's potential for broader anatomical regions and diverse research objectives underscored its versatility. Applications spanned from facial deformity analysis to virtual surgical planning, leveraging its suitability for large-scale dataset analysis and potential integration into real-time movement analysis systems. Overall, the authors presented a promising advancement in automated soft-tissue landmarking, with implications for various clinical and research domains, pending further validation and refinement.

Conclusion

In conclusion, the authors introduced a deep learning-based automated method for soft-tissue landmark extraction from 3D facial photographs. The developed workflow demonstrated precise and consistent landmark annotation, comparable to manual and semi-automatic methods. By leveraging deep learning algorithms, the approach offered efficiency and reliability in cephalometric analysis, with potential applications in clinical diagnosis, follow-up, and virtual surgical planning.

Despite limitations, such as hardware constraints and texture information, the method's adaptability to different imaging systems suggested promising prospects. Overall, automated landmarking methods presented a valuable tool for analyzing large datasets and advancing research in various fields, including orthodontics and craniofacial surgery.