Academic Journal
Revisiting the Inference Time Optimization of TinyML for ARM-Based Microcontrollers.
| Τίτλος: | Revisiting the Inference Time Optimization of TinyML for ARM-Based Microcontrollers. |
|---|---|
| Συγγραφείς: | Lee, Chan-Kyu, Ohk, Seung-Ryeol, Kim, Young-Jin |
| Πηγή: | Electronics (2079-9292); Jul2026, Vol. 15 Issue 14, p2997, 16p |
| Θεματικοί όροι: | ARM microprocessors, Software libraries (Computer programming), Mathematical optimization, Machine learning, Indexing |
| Περίληψη: | As AI research advances, various performance optimization techniques have been studied for TinyML models on microcontrollers with very limited resources. In TinyML frameworks such as TFLM (TensorFlow Lite Micro) and NNOM (Neural Network on Microcontrollers), their execution engines mainly depend on CMSIS-NN for inference acceleration, which is an ARM's back-end library to execute optimized kernel functions for performance optimization. But we often find that CMSIS-NN is not invincible for inference time optimization on TinyML. In this paper, we examine TinyML frameworks and their CMSIS-NN libraries and consider how to improve CMSIS-NN in terms of runtime. Then, we propose the D2I technique to reduce the overhead of memory operations that occur while performing the Im2col procedure within the convolution function, which takes most of the inference time in CMSIS-NN. The proposed technique creates a necessary index table, finds the location of the input with the corresponding index, and performs direct operations between filters and inputs. Thus, it can quite mitigate data copy operations in Im2col with a small additional amount of memory compared to Im2col. In extensive experiments using an Arduino nano 33 BLE board with Cortex-M4 and an STM32F746G-DISCO board with Cortex-M7, D2I was found to achieve about 16.3% and 14.5% inference time improvements against the Im2col in TFLM's and NNOM's CMSIS-NNs, respectively, for the SqueezeNet model. And the additional memory usage was shown to be identically 11.52 kB. [ABSTRACT FROM AUTHOR] |
| Copyright of Electronics (2079-9292) is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Βάση Δεδομένων: | Complementary Index |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:edb&genre=article&issn=20799292&ISBN=&volume=15&issue=14&date=20260715&spage=2997&pages=2997-3012&title=Electronics (2079-9292)&atitle=Revisiting%20the%20Inference%20Time%20Optimization%20of%20TinyML%20for%20ARM-Based%20Microcontrollers.&aulast=Lee%2C%20Chan-Kyu&id=DOI:10.3390/electronics15142997 Name: Full Text Finder (for New FTF UI) (ns324271) Category: fullText Text: Full Text Finder MouseOverText: Full Text Finder |
|---|---|
| Header | DbId: edb DbLabel: Complementary Index An: 195811874 RelevancyScore: 1082 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 1082.42224121094 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Revisiting the Inference Time Optimization of TinyML for ARM-Based Microcontrollers. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Lee%2C+Chan-Kyu%22">Lee, Chan-Kyu</searchLink><br /><searchLink fieldCode="AR" term="%22Ohk%2C+Seung-Ryeol%22">Ohk, Seung-Ryeol</searchLink><br /><searchLink fieldCode="AR" term="%22Kim%2C+Young-Jin%22">Kim, Young-Jin</searchLink> – Name: TitleSource Label: Source Group: Src Data: Electronics (2079-9292); Jul2026, Vol. 15 Issue 14, p2997, 16p – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22ARM+microprocessors%22">ARM microprocessors</searchLink><br /><searchLink fieldCode="DE" term="%22Software+libraries+%28Computer+programming%29%22">Software libraries (Computer programming)</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+optimization%22">Mathematical optimization</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Indexing%22">Indexing</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: As AI research advances, various performance optimization techniques have been studied for TinyML models on microcontrollers with very limited resources. In TinyML frameworks such as TFLM (TensorFlow Lite Micro) and NNOM (Neural Network on Microcontrollers), their execution engines mainly depend on CMSIS-NN for inference acceleration, which is an ARM's back-end library to execute optimized kernel functions for performance optimization. But we often find that CMSIS-NN is not invincible for inference time optimization on TinyML. In this paper, we examine TinyML frameworks and their CMSIS-NN libraries and consider how to improve CMSIS-NN in terms of runtime. Then, we propose the D2I technique to reduce the overhead of memory operations that occur while performing the Im2col procedure within the convolution function, which takes most of the inference time in CMSIS-NN. The proposed technique creates a necessary index table, finds the location of the input with the corresponding index, and performs direct operations between filters and inputs. Thus, it can quite mitigate data copy operations in Im2col with a small additional amount of memory compared to Im2col. In extensive experiments using an Arduino nano 33 BLE board with Cortex-M4 and an STM32F746G-DISCO board with Cortex-M7, D2I was found to achieve about 16.3% and 14.5% inference time improvements against the Im2col in TFLM's and NNOM's CMSIS-NNs, respectively, for the SqueezeNet model. And the additional memory usage was shown to be identically 11.52 kB. [ABSTRACT FROM AUTHOR] – Name: Abstract Label: Group: Ab Data: <i>Copyright of Electronics (2079-9292) is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edb&AN=195811874 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.3390/electronics15142997 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 16 StartPage: 2997 Subjects: – SubjectFull: ARM microprocessors Type: general – SubjectFull: Software libraries (Computer programming) Type: general – SubjectFull: Mathematical optimization Type: general – SubjectFull: Machine learning Type: general – SubjectFull: Indexing Type: general Titles: – TitleFull: Revisiting the Inference Time Optimization of TinyML for ARM-Based Microcontrollers. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Lee, Chan-Kyu – PersonEntity: Name: NameFull: Ohk, Seung-Ryeol – PersonEntity: Name: NameFull: Kim, Young-Jin IsPartOfRelationships: – BibEntity: Dates: – D: 15 M: 07 Text: Jul2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 20799292 Numbering: – Type: volume Value: 15 – Type: issue Value: 14 Titles: – TitleFull: Electronics (2079-9292) Type: main |
| ResultId | 1 |