Smartphone Raman Spectrometer with CNN for Precise Drug Classification

By Samudrapom DamReviewed by Susha Cheriyedath, M.Sc.Aug 31 2023

In an article recently published in the journal Nature Communications, researchers proposed a novel approach for precise drug classification by combining Raman spectroscopy with the convolutional neural network (CNN).

Background

Recently, several efforts have been made to convert smartphone cameras into optical spectrometers for different applications, such as mobile food inspection and healthcare. In this approach, the smartphone image sensor detects the optical signals from an object, such as Raman emissions, fluorescence, and reflectance, and then the communication chip and application processor (AP) of the smartphone collectively perform cloud-linked/on-device analysis to evaluate the chemical/physical conditions or identify the specimens.

Most studies on smartphone spectrometers utilize gratings assembled in external optics modules as dispersion components. Although gratings in spectrometers can effectively disperse optical signals with high spectral resolution, it is extremely difficult to minimize the form factor of gratings to fit them into smartphones.

The feasibility of using mini spectrometers, such as silicone nanowires integrated into the charge-coupled detector (CCD), metasurfaces, quantum dots, and photonic crystals, as a replacement for conventional grating has been investigated in several studies. However, the findings of these studies have significant limitations, specifically in capturing the high and weak spectral resolution needed for Raman signatures.

Counterfeit drugs are increasingly becoming a significant risk to public health with the increasing number of online pharmacies and expanding drug supply chain, which can provide blind spots for the distribution of substandard or counterfeit drugs in the market.

Existing smartphone applications, such as Drug Info, Pill Identifier, and DrugID, can differentiate the drug models and types by comparing the red, green, and blue (RGB) images of the drug pill with the United States Food and Drug Administration (USFDA) database or by entering the etched marks, color, shape, and/or name of the drugs.

However, the drug identification accuracy of these applications is insufficient owing to the similar appearance of many drugs or the absence of the drug in the database. Machine learning (ML)-assisted Raman spectroscopy has been used in a few studies to classify drugs as Raman spectrum can offer crucial drug-related information.

For instance, partial least squares discriminate analysis (PLS-DA), principal component analysis (PCA), and CNN was used with Raman spectroscopy to classify pharmaceutical ingredients, identify newly emerging psychoactive substances, and detect illicit drugs, respectively.

Combining Raman spectroscopy with CNN

In this paper, researchers investigated the feasibility of using a smartphone Raman spectrometer to classify drugs by chemical components. The Raman spectrometer was composed of a complementary metal-oxide-semiconductor (CMOS) image sensor of the Samsung Galaxy Note 9 smartphone with a periodic bandpass filter array and an external compact Raman module, which can capture two-dimensional (2D) Raman spectral intensity map/spectral barcode.

Researchers experimentally evaluated 54 commonly utilized drugs for hypertension, hyperlipidemia, and diabetes, nutritional supplements, and painkillers, which are frequently available in almost identical colors, sizes, and shapes. These include varsartan, losartan, amlodipine, metformin, glimepride, simavastatin, rosuvastatin, atorvastatin, vitamin B6 and vitamin C, and tylenol.

Subsequently, researchers identified the spectral barcodes of drugs using a CNN embedded in the Samsung Galaxy Note 9 smartphone, as every drug spectral barcode contains distinct Raman signatures of the material. The identification accuracy can be improved by information fusion between conventional RGB images captured by the smartphone camera and spectral barcode, which is a major advantage of spectral barcode-based classification.

Additionally, the spectral barcode classification allows the identification of chemical components of unfamiliar drugs when other drugs with similar chemical components are present in the database. CNN with a residual network (ResNet) structure was used for spectral barcode classification, and only a single residual block was used as a spectral barcode has 120 degrees of freedom.

The CNN for RGB images used the visual geometry group network (VGGNet) architecture for color and shape classification. Additionally, a classification algorithm in series was designed for drug brand name identification, with nine CNNs for brand name identification and one CNN for major component classification. A randomly selected RGB image and spectral barcode were tested 1000 times to determine the accuracy of the combined CNNs for RGB images and spectral barcodes.

Significance of the study

The smartphone Raman spectrometer displayed a sufficiently high Q factor as a portable spectrometer with high power consumption efficiency, even with a lower signal-to-noise ratio (SNR) and spectral resolution compared to the spectrometers available commercially with CCD and grating due to the intrinsic properties of CMOS image sensor and bandpass filter arrays.

Only collection optics and external excitation were required to collect and excite Raman signals from the specimen without any additional electronic board connecting to the smartphone, which demonstrated the versatility and compactness of the smartphone spectrometer with a minimized external module.

Moreover, integrating artificial intelligence (AI) capability with the smartphone spectrometer significantly increased the effectiveness of the spectrometer. 79.5% accuracy and 99.0% accuracy were achieved for brand name and major component classifications, respectively, using the smartphone Raman spectrometer for drug classification by spectral barcodes containing weak Raman signals. The brand name identification accuracy increased to 83.2% using the CNN of RGB images as an auxiliary classification tool.