Optical Meta-Imager Accelerates Machine Vision

By Soham NandiReviewed by Susha Cheriyedath, M.Sc.Jan 10 2024

In an article published in the journal Nature, researchers introduced a meta-imager that collaborated with a digital back end to offload computationally intensive convolution operations into high-speed, low-power optics. Leveraging metasurfaces for angle and polarization multiplexing, the system created multiple information channels enabling both positive and negative valued convolution operations simultaneously. The meta-imager was applied to object classification, achieving high accuracy in handwritten digits and fashion images.

Background

The rise of digital neural networks and extensive training datasets has revolutionized machine learning applications, notably in image analysis, speech recognition, and machine vision. However, the quest for heightened performance often leads to larger computing requirements and increased energy consumption, which is particularly challenging for real-time decision-making in resource-constrained scenarios. In the context of machine vision, particularly in autonomous systems, there is a crucial need for compact, lightweight, and power-efficient imagers and processors.

Optics has been explored as a potential solution to accelerate computational operations and improve energy efficiency. Traditional Fourier-based optical computation, while capable, faces challenges such as increased system size. Coherent sources, although advantageous, limit freedom in convolution operations. Existing optical diffractive neural networks offer an alternative but are more suited for back-end processing.

This study introduced a meta-imager, serving as a front-end multichannel convolutional accelerator for incoherent light, addressing gaps in existing optical computation approaches. Leveraging metasurfaces, the platform achieved compactness and a broader range of optical properties. The meta-imager employed angular and polarization multiplexing to enable parallel multichannel convolution, demonstrating both positively and negatively valued kernels for incoherent illumination.

A second metasurface corrector widened the field of view (FOV). Through experimental validation, the platform showcased high-accuracy classification of datasets, significantly offloading operations from the digital platform to the front-end optics. This breakthrough filled a crucial gap, providing an architecture capable of generating multiple independent convolution channels essential for efficient machine vision systems.

Angular and polarization multiplexing

The researchers introduced a meta-optic designed for the optical replication of convolutional layers in digital neural networks. In digital networks, convolution involves matrix multiplication and forming feature maps. The meta-optic created an optical analog to the digital kernel by engineering the point spread function with N×N focal spots, each with varying weights.

Positively and negatively valued kernel weights were achieved through circular polarization, decoded by a polarization-sensitive camera. The bilayer-metasurface architecture split the incident signal into angular channels, encoding positive and negative kernel values. This approach simplified design, allowing direct implementation of digital kernel designs with optics, achieving multiple independent feature maps with one aperture.

Meta-optic design

The meta-optic design began with optimizing a coma-free two-metasurface lens over a ±10° angular range. Angular multiplexing was applied to form focal-spot arrays for convolution kernels and a wider FOV could be achieved by cascading metasurfaces. The meta-optic's complex-amplitude profile was engineered using angular signals, and a phase-only metasurface was designed to overcome reflection loss limitations.

An optimization platform based on the angular spectrum propagation method and stochastic gradient descent solver was used. The phase-only approximation effectively avoided complex-amplitude function loss, achieving a high theoretical diffraction efficiency of 84.3%.

Efficient object classification

The authors presented a hybrid neural network for image classification, integrating a meta-optic with a shallow convolutional neural network. The meta-optic, designed using two metasurfaces, employed angular and polarization multiplexing for parallel multichannel convolution. The polarization-selective metasurface achieved independent control of left- and right-circular polarizations, while the second metasurface provided polarization-insensitive phase control.

The neural network, comprising optical convolution followed by digital processing, demonstrated successful classification of Modified National Institute of Standards and Technology (MNIST) and Fashion MNIST datasets. Fabrication and characterization confirmed the meta-optic's accuracy and robustness, with 94% of operations offloaded to enhance classification speed.

The approach held promise for efficient object classification in machine vision applications. The system's scalability was explored, showing consistent theoretical accuracy up to an areal density drop below 2 μm. The meta-imager computing unit density was highlighted, offering a promising perspective for efficient optical computing compared to traditional architectures.

Conclusion

In conclusion, the meta-imager served as a proof of concept for a convolutional front end, offering an alternative to traditional imaging optics in machine vision. Enabled by the meta-optic, it facilitated the offloading of more operations into the front-end optics, allowing for negatively valued kernels and multichannel convolution.

The architecture supported incoherent illumination, a wide FOV, and potential integration with chip-based photonics for ultrafast and low-latency processing. While suitable for lightweight neural networks, future advancements, such as larger kernels and multifunctional processing, could enhance its effectiveness in various applications beyond machine vision.