In an article published in the journal npj Precision Oncology, researchers from Stony Brook University, USA, and the University of Edinburgh, UK, introduced an open-source software ecosystem called WSInfer that enables the sharing and reusability of deep learning models for digital pathology. They discussed the capability of WSInfer to apply patch-based classification models to entire slide images and visualize the outcomes through a bioimage analysis platform named QuPath.
Study: WSInfer: Empowering Digital Pathology with Open-Source Deep Learning. Image credit: Komsan Loonprom/Shutterstock
Background
Digital pathology is the field of using high-resolution images of tissue specimens called whole slide images (WSIs) for diagnosis and prognosis of diseases such as cancer, infectious diseases, autoimmune disorders, and neurological conditions. Deep learning, a branch of artificial intelligence (AI), has shown great potential to revolutionize digital pathology by automating tasks such as identifying tumor regions, genomic aberrations, lymphocytic infiltrates, and other clinically relevant phenotypes.
However, many of the existing deep learning models for digital pathology are not readily available or usable by the research community, which limits their validation and generalization across different datasets and applications. Moreover, the lack of user-friendly tools and interfaces makes it difficult for pathologists, the main domain experts, to explore and evaluate the models on their data.
About the Research
In the present paper, the authors proposed WSInfer, a collection of software tools developed to streamline the deployment of trained patch-based deep learning classification models on WSIs. A patch-based classification is a common approach in digital pathology, where a WSI is divided into smaller patches and each patch is assigned a label or a probability by a deep learning model. The output of patch classification is typically a spatial map of the WSI, which can provide insights into the tissue composition, morphology, and heterogeneity and helps to differentiate patches into several tissue components, such as stroma, tumor, lymphocytes, or Gleason grades. WSInfer consists of the following three main components:
- WSInfer inference runtime: A command-line tool and a Python package that performs patch extraction, model inference, and output aggregation on a directory of WSIs using TorchScript models.
- QuPath WSInfer extension: A Java-based plugin that integrates WSInfer into QuPath and allows users to select a model and a region for visualizing, annotating, and analyzing the results interactively.
- WSInfer Model Zoo: A repository of trained patch classification models hosted on Hugging Face Hub, a platform for sharing and discovering machine learning models. Each model in the Zoo is accompanied by a model card, a document that provides metadata and instructions for the model, and a configuration file that specifies the model parameters and requirements.
The study demonstrates WSInfer's potential in applying different models from the Model Zoo to WSIs from various cancer types, generating spatial maps of tumors and tumor-infiltrating lymphocytes (TILs). These biomarkers are crucial for cancer prognosis and treatment response. The researchers utilized several patch classification models for different pathology applications, including breast tumor detection, colorectal tissue phenotyping, and lymph node metastasis screening, integrating them into the WSInfer Model Zoo. They integrated WSInfer with other QuPath features, such as tile merging, cell segmentation, and data export to create sophisticated analysis pipelines.
Research Findings
The outcomes showed that the proposed technique is an efficient, scalable, open-source, reusable, cross-platform, and cross-language software ecosystem. It supports a variety of patch classification models and can be used by different types of users, including pathologists, computational researchers, and data scientists.
The authors evaluated the performance of WSInfer in different environments and scenarios. They measured the running time of the WSInfer inference runtime on two systems with different graphics processing unit (GPUs) and found that it took an average of 23-29 seconds (s) per WSI for breast tumor classification. Additionally, they tested the QuPath WSInfer extension on the same task and found that it took 40 s per WSI with a GPU and 6 minutes 37 s per WSI with a central processing unit (CPU).
The study highlighted that WSInfer can be used for various applications in digital pathology, such as screening tissue images for cancer metastasis, identifying and measuring tumor regions, predicting microsatellite instability status and genomic aberrations, and creating spatial maps of tissue phenotypes. It can also be utilized for nucleus detection, pixel classification, and slide-level inference.
Conclusion
In summary, this novel software ecosystem is innovative and supports the sharing and reuse of deep learning models for digital pathology. It uses patch-based classification models and provides tools for applying them to WSIs and visualizing the results in QuPath. The study demonstrated the efficiency and usability of WSInfer and discussed its potential applications and implications.
The authors acknowledged the limitations and challenges of their approach, such as the need for model validation, verification, and ethical considerations and the difficulty of model generalization across scanners, cohorts, and laboratories. They suggested future directions, such as incorporating more models, supporting more tasks, and promoting interoperability with other software. Moreover, they invited the digital pathology community to contribute to WSInfer and use it for research purposes.
Advanced Deep Learning Models for Plant Disease Detection