Auto Tiny Classifiers: Efficient Hardware Solutions for Tabular Data

By Muhammad OsamaReviewed by Susha Cheriyedath, M.Sc.May 2 2024

In a recent article published in the journal Nature Electronics, researchers from the UK introduced a novel methodology called auto tiny classifiers. This approach automatically generates classifier circuits tailored for tabular data.

In particular, the innovation can offer comparable prediction performance to conventional machine learning techniques while using substantially fewer hardware resources and power. Additionally, the research demonstrated the practical application of the presented circuits as flexible integrated circuits, presenting a comparison with two established machine learning baseline models.

Background

Tabular data is utilized across various fields, including recommender systems, medical diagnosis, and smart packaging. However, these datasets pose challenges due to their heterogeneous nature, which involves the combination of numerical and categorical data with weak correlations among features. This complexity presents obstacles for deep learning architectures, which excel at capturing spatial or semantic relationships typically found in image or speech data.

In machine learning development, a prevalent approach is to optimize performance during model training and then reduce the memory and area footprint of the trained model for deployment across diverse platforms, including processing cores, graphics processing units, microcontrollers, or custom hardware accelerators.

However, achieving a balance between performance and resource efficiency becomes difficult as machine learning models grow larger and more intricate. Moreover, implementing machine learning in hardware requires additional steps like translation, verification, and optimization, which may introduce errors and overheads into the process.

About the Research

The paper introduced auto tiny classifiers, a novel method for automatically generating classifier circuits directly from tabular data. Unlike traditional approaches, this method does not rely on pre-defined machine learning models or hardware circuits. These classifier circuits consist of only a few hundred logic gates yet achieve prediction accuracy comparable to state-of-the-art machine learning techniques such as gradient-boosted decision trees and deep neural networks.

The study employs an evolutionary algorithm to explore the logic gate space and generate classifier circuits that maximize training prediction accuracy. This algorithm emulates natural Darwinian evolution, with circuit fitness evaluated based on balanced accuracy. Termination occurs when validation accuracy fails to improve by a specified threshold within a generation window.

To validate their methodology, the researchers assessed it across 33 different tabular datasets, primarily sourced from OpenML, University of California, Irvine (UCI), and Kaggle repositories. Comparative evaluations were conducted against Google's TabNet architecture, AutoGluon (an AutoML system developed by Amazon), and other foundational machine learning models. Additionally, the authors designed tiny classifiers and baseline models in hardware, targeting both conventional silicon technology and flexible integrated circuits (FlexICs).

Findings

The outcomes revealed that across all datasets, AutoGluon XGBoost exhibited the highest average prediction accuracy at 81%, while tiny classifiers followed closely with a mean accuracy of 78%, marking the second-highest overall. Additionally, the analysis demonstrated that tiny classifiers boasted a low variance in the accuracy distribution, indicating their robustness to variations.

Synthesizing the tiny classifiers and baseline models using synopsis design compiler and targeting open 45 nm process design kit (PDK) silicon technology, the authors unveiled compelling insights. Specifically, they found that tiny classifier circuits consumed between 0.04-0.97 mW, with gate counts ranging from 11 to 426 NAND2-equivalent gates.

In contrast, multilayer perceptron’s (MLP) power consumption ranged from 34 to 38 mW, marking an 86–118 times increase over tiny classifiers, while area size was approximately 171 and 278 times larger for blood and led datasets, respectively, compared to tiny classifiers. Similarly, XGBoost exhibited approximately 3.9- and 8.0-times higher power consumption and 8.0- and 18.0-times larger area than tiny classifiers for blood and led datasets, respectively.

Furthermore, the authors implemented tiny classifiers and XGBoost as flexible chips using Pragmatic's 0.8 μm FlexIC metal-oxide thin-film transistor process. Their findings showcased the superior performance of tiny classifiers, which could be clocked 2-3 times faster, were 10-75 times smaller, and consumed lower power compared to XGBoost. Moreover, tiny classifiers exhibited a sixfold higher yield than XGBoost chips, suggesting lower production costs.

The new methodology could be used in a variety of applications, such as triggering circuits within system on chips, smart packages equipped with FlexICs, and near-sensor computing systems where inference is performed at the source. It is not limited to tabular data, and could be extended to other forms of data, such as time-series data, by using recurrent-graph-based genetic programming.

Conclusion

To sum up, an effective and adaptable novel approach was designed for automatically producing classifier circuits for tabular data. The technique could offer comparable prediction performance to conventional machine learning techniques while using fewer resources. Moreover, it showed that the methodology can generate low-cost and flexible chips that can be integrated into various applications.

Moving forward, the researchers acknowledged limitations and challenges and suggested directions for future work. They recommended exploring other fitness functions, such as the number of gates or power consumption, and using multi-objective graph-based genetic programming to search for the Pareto-optimal front of solutions and characterize the trade-off between objectives.