Summary

This is a tracking issue for integrating ExecuTorch with the CMSIS ecosystem to support Cortex-M microcontroller development. CMSIS (Cortex Microcontroller Software Interface Standard) provides a standardized approach for embedded development, offering improved portability, code reuse, and simplified development workflows for ARM Cortex-M based systems.

Why CMSIS for Cortex-M Development?

Benefits of CMSIS-Pack Distribution

CMSIS-Pack offers several advantages for distributing ExecuTorch in the embedded ecosystem:

• Standardized Distribution: CMSIS-Pack provides a unified format for software components, device drivers, and middleware, making it easy for developers to discover, install, and manage dependencies.
• Tool Integration: Compatible with major embedded development tools including Keil MDK, Keil Studio, IAR Embedded Workbench, and VS Code with appropriate extensions.
• Dependency Management: Automatic resolution of dependencies and version management, similar to modern package managers but tailored for embedded systems.
• Hardware Abstraction: Provides consistent APIs across different microcontroller families and vendors.
• Performance Optimization: CMSIS-NN kernels offer 4.6X improvement in runtime/throughput and 4.9X improvement in energy efficiency for neural network inference on Cortex-M processors.
• Cross-Platform Support: Enables consistent development across different toolchains and operating systems.

Current Status

ExecuTorch CMSIS-Pack Distribution

• Create ExecuTorch CMSIS-Pack: Similar to LiteRT (formerly TensorFlow Lite Micro) distribution model

  • LiteRT provides three acceleration variants: Software Reference, CMSIS-NN optimized, and Ethos-U acceleration
  • ExecuTorch should follow similar pattern with multiple performance tiers?
  • Pack should include runtime libraries, header files, documentation, and project templates

• Pack Dependencies: Define required CMSIS components

  • CMSIS-DSP for digital signal processing functions
  • CMSIS-NN for optimized neural network kernels
  • CMSIS-RTOS2 for real-time operating system support - Optional
  • Device Family Packs for target microcontrollers

Reference Application Development

• Create Reference Application: Develop a comprehensive example targeting CMSIS ecosystem boards

  • Target devices from https://www.keil.arm.com/devices/ (12000+ supported devices)
  • Support major microcontroller families: STM32, NXP, Infineon, Nuvoton, Texas Instruments
  • Include complete project templates for popular development boards
  • Provide performance benchmarks across different device categories

• Board Support: Ensure compatibility with popular development boards (Ethos-U first)

  • STM32 Discovery and Nucleo boards
  • NXP Freedom and LPCXpresso boards
  • Infineon XMC and PSoC boards
  • Alif Ensemble E7/E8 boards
  • Support for both evaluation boards and custom hardware

• Memory Management: Optimize for Cortex-M memory constraints

  • Utilize Kernel Library Selective Build
  • Static memory allocation patterns
  • Minimal heap usage

Cross-Platform Development Support

• Windows Support: Currently experimental in ExecuTorch v0.6+

Disclaimers

• Toolchain Requirements: Windows development requires Clang-CL, MSVC is not supported
• Backend Limitations: Initial focus on Ethos-U NPU, CMSIS-NN CPU backend support to be evaluated separately once available
• Device Support: Priority on devices featuring an Arm NPU.

Dependencies

This work depends on:

• Stable ExecuTorch Windows support (Official Windows Support #13781)
• CMSIS-NN integration improvements ((CMSIS-NN Integration) Testing strategy #13902)

cc @freddan80 @per @zingo @oscarandersson8218 @digantdesai