Until now, developers who use PyTorch for model training could not easily deploy them directly to resource-constrained edge devices – as the models which the PyTorch framework spits out are data-heavy and require some serious compute power to run. That’s fine for the cloud: not so good for edge AI.
But naturally, manufacturers of products that perform inference at the edge, on devices such as microcontrollers, want to take advantage of the PyTorch skills and knowledge available across the AI developer community.
So Meta has developed a fix: the ExecuTorch Runtime, a quantization extension which scales down models developed in PyTorch for the constrained processor and memory resources available in edge AI systems.
ExecuTorch builds a bridge from PyTorch to the edge, but edge devices need to meet it halfway with compatible hardware. And here’s the good news: the Ensemble family of MCUs and fusion processors now support ExecuTorch.
This makes the PyTorch framework a viable choice for developers of AI applications across hot product categories. Think smart glasses with real-time language translation, personal health monitors which analyze ECG patterns locally to preserve patient privacy, or educational robots which adapt their teaching style to individual students – all running AI models without constant cloud connectivity.
And for engineering managers, this means faster time-to-market because they can use the abundant existing PyTorch expertise rather than relying on the much smaller number of developers who know how to use embedded-specific frameworks.
Lighting up the path to the future of edge AI
OEMs targeting the edge AI market have come to expect innovation firsts from Alif: the first MCU with an integrated NPU (Arm® Ethos™-U55), the first capable of running transformer networks for generative AI at the edge. Now it’s leading the way once again with support for PyTorch at the embedded edge.
What’s next? While we can’t reveal specifics yet, Alif’s roadmap continues to focus on eliminating the barriers between AI innovation and edge deployment on MCUs. For developers and engineering leaders looking to future-proof their edge AI strategy, PyTorch support is just the beginning.