Enhancing Security Scanning with Infrared Thermography and CNNs

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Apr 23 2024

In an article published in the journal Nature, researchers investigated the efficacy of utilizing a convolutional neural network (CNN) to classify infrared (IR) images for security scanning. They explored IR thermography for non-invasive concealed object detection, employing residual network (ResNet)-50 as the CNN model.

Various image pre-processing techniques were compared, aiming to enhance classification accuracy. Results indicated that preprocessing methods such as k-means and fuzzy-c clustering improved performance by emphasizing relevant features, potentially enhancing security scanning effectiveness.

Background

Security operations face ongoing challenges in detecting concealed weapons and explosive devices, particularly in densely populated venues like airports and stadiums. Traditional methods, such as manual checks, are impractical for large crowds. Thus, there's a critical need for effective stand-off and walk-through scanners that can swiftly identify potential threats while minimizing harm to civilians.

Historically, electromagnetic (EM) waves have been instrumental in non-contact object detection for security purposes. X-ray detectors, while effective, pose health risks due to ionizing radiation. Longer wavelength EM waves like millimeter waves (MMW) and terahertz (THz) have shown promise, offering penetration capabilities through clothing. IR thermography presents another avenue for security scanning.

While less explored in concealed object detection due to its limited penetration, IR technology offers advantages such as higher resolution and privacy preservation. However, existing approaches often rely on MMW or THz due to their superior penetration, leaving a gap in leveraging IR for this purpose. This paper addressed this gap by proposing a novel approach using CNNs for concealed object detection with IR thermography. Leveraging transfer learning, a pre-trained CNN model was fine-tuned using IR images to predict the presence of concealed objects beneath clothing layers.

By employing CNNs, the study aimed to overcome the limitations of IR systems in making consistent and accurate decisions, particularly in scenarios involving layered clothing. Through experimentation with various pre-processing techniques and evaluation via receiver operating characteristic (ROC) curves, the paper presented a comprehensive framework for enhancing security scanning effectiveness using IR technology.

Refining Security Scanning Methods

The authors utilized a FLIR A6750sc mid-wavelength IR camera to gather thermal data for concealed object detection. Simulated concealed weapons, such as neoprene rubber blocks and metal bearings in clay, were employed alongside various types of outer clothing, including layered windproof garments and hoodies. To enhance classification accuracy, four image pre-processing methods were explored: raw data, cropped region-of-interest (ROI), k-means clustering, and fuzzy-c clustering.

The ROI images were generated by cropping subjects from raw images based on a skin temperature threshold, effectively removing background information. k-means and fuzzy-c clustering techniques were applied to reduce information complexity and simplify image representation. A ResNet-50 CNN model was adopted for image classification, leveraging its effectiveness in extracting features from images. Transfer learning was employed to fine-tune the pre-trained model using IR images of subjects with and without concealed objects. This approach mitigated the challenge of limited labeled IR datasets by utilizing knowledge from the extensive ImageNet dataset.

During training, data augmentation techniques were applied to diversify the dataset and prevent overfitting. Following model training, performance evaluation was conducted using an internal holdout/test set of 200 images. Model performance was assessed using ROC curves, providing insight into the true positive rate (TPR) and false positive rate (FPR). The area-under-the-curve (AUC) of the ROC curve served as a performance metric, with values above 0.9 indicating outstanding performance. Overall, the researchers demonstrated the effectiveness of CNNs in classifying IR images for concealed object detection, offering the potential for enhancing security scanning protocols.

Refinement and Evaluation

The authors delved into the results and discussions stemming from the application of various pre-processing methods to enhance security scanning using CNNs and IR thermography. Four different techniques were employed: raw data, cropped ROI, k-means clustering, and fuzzy-c clustering. These methods aimed to refine the input data for the CNN model by reducing background noise and emphasizing relevant features.

Results showcased the effectiveness of pre-processing, with k-means and fuzzy-c clustered images outperforming raw and ROI-cropped images in terms of classification accuracy. Analysis of ROC curves revealed AUC values of 0.8923, 0.9256, 0.9485, and 0.9669 for the raw, ROI, k-means, and fuzzy-c models, respectively. Notably, the post-processed images exhibited improved performance, suggesting the importance of background removal and feature enhancement in object detection.

Furthermore, the researchers highlighted the potential for advanced segmentation techniques or two-dimensional (2D) clustering to further enhance model performance. The researchers also touched upon the challenges posed by complex clothing materials, where differing emissivity levels could create temperature differentials that mimic concealed objects. Addressing this challenge required a robust dataset encompassing a variety of clothing materials for comprehensive model training. Overall, the findings underscored the significance of pre-processing methods in optimizing CNN-based security scanning with IR thermography, paving the way for more effective concealed object detection in diverse real-world scenarios.

Conclusion

In conclusion, the researchers refined pre-processing techniques to optimize classification accuracy. Utilizing ResNet-50 and transfer learning, they fine-tuned models with IR images, achieving AUC values of 0.8923 to 0.9669. K-means and fuzzy-c clustering notably improved performance. Results highlighted the importance of background removal and feature enhancement. Challenges with complex clothing materials were acknowledged, suggesting the need for diverse datasets. Overall, the authors demonstrated the efficacy of CNNs in automated concealed object detection, advancing security scanning protocols.