Bibliographic Details
| Title: |
Revisiting the Inference Time Optimization of TinyML for ARM-Based Microcontrollers. |
| Authors: |
Lee, Chan-Kyu, Ohk, Seung-Ryeol, Kim, Young-Jin |
| Source: |
Electronics (2079-9292); Jul2026, Vol. 15 Issue 14, p2997, 16p |
| Subject Terms: |
ARM microprocessors, Software libraries (Computer programming), Mathematical optimization, Machine learning, Indexing |
| Abstract: |
As AI research advances, various performance optimization techniques have been studied for TinyML models on microcontrollers with very limited resources. In TinyML frameworks such as TFLM (TensorFlow Lite Micro) and NNOM (Neural Network on Microcontrollers), their execution engines mainly depend on CMSIS-NN for inference acceleration, which is an ARM's back-end library to execute optimized kernel functions for performance optimization. But we often find that CMSIS-NN is not invincible for inference time optimization on TinyML. In this paper, we examine TinyML frameworks and their CMSIS-NN libraries and consider how to improve CMSIS-NN in terms of runtime. Then, we propose the D2I technique to reduce the overhead of memory operations that occur while performing the Im2col procedure within the convolution function, which takes most of the inference time in CMSIS-NN. The proposed technique creates a necessary index table, finds the location of the input with the corresponding index, and performs direct operations between filters and inputs. Thus, it can quite mitigate data copy operations in Im2col with a small additional amount of memory compared to Im2col. In extensive experiments using an Arduino nano 33 BLE board with Cortex-M4 and an STM32F746G-DISCO board with Cortex-M7, D2I was found to achieve about 16.3% and 14.5% inference time improvements against the Im2col in TFLM's and NNOM's CMSIS-NNs, respectively, for the SqueezeNet model. And the additional memory usage was shown to be identically 11.52 kB. [ABSTRACT FROM AUTHOR] |
|
Copyright of Electronics (2079-9292) is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| Database: |
Complementary Index |