The field of edge computing is one of the fastest-growing sectors of the computing industry. Recent data indicates that the global edge computing market was valued at $16.45 billion in 2023, with expectations to grow at a CAGR of 36.9% through 2030. With such growth on the horizon, it is no surprise that the rising demand for artificial intelligence (AI) at the edge continues to completely change the way the industry is approaching computing.
The demand for AI at the edge continues to push the edge computing
industry forward. (Source: Adobe Stock)
By processing data directly on the device, Edge devices can perform real-time decision-making significantly faster than devices that rely on pushing all gathered data to the cloud for analysis. Local processing, if done efficiently, also reduces the overall system power consumption, which is crucial for battery powered devices.
When it comes to delivering efficient edge AI compute on microcontrollers, few technologies are more powerful and important than integrated neural processing units (NPUs). Alif Semiconductor has a long standing technology partnership with Arm®, and has integrated their Ethos-U55 NPUs in the Ensemble as well as the Balletto product families.
With industry-leading processing capabilities, power efficiency, and dedicated hardware for AI tasks, the Ethos technology is already a key enabler of the future of edge AI. In their recent announcement of the Ethos-U85, in which Alif was named as an early adopter, Arm shared that they expect Ethos-U85 to be their highest-performing and most efficient Ethos NPU to date.
Inside the Arm Ethos-U85
As it has been designed to support transformer-based models as well as more conventional Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), the Ethos-U85 will enable an even more diverse set of AI applications, from natural language processing to advanced image recognition tasks, to be executed directly on the device.
Cortex-M architecture of Ethos-U85 NPU. (Image Source: Arm).
The U85 also supports new sparsity levels of 2/4, effectively doubling throughput for specific neural network computations. This feature, combined with its enhanced weight compression capabilities, helps in reducing the memory footprint significantly—up to 70% model size reduction—thereby enabling the execution of larger networks or speeding up existing ones.
Ethos-U85 for Edge AI
All of these specifications are exciting, but how exactly do these features benefit edge AI in the real world? To better understand that, let’s consider a real-world application such as a smart security camera system for residential and commercial properties.
High-Performance Processing Capabilities
Integrated NPUs, provides the computational power necessary to perform detailed image analysis in real time. High-performance edge computing allows these cameras to utilize sophisticated algorithms for object detection, facial recognition, and anomaly detection without experiencing the delays inherent in cloud processing.
By processing data directly on the device, smart security cameras equipped with the Ethos NPUs can perform real-time decision-making. This immediacy is crucial for security applications where reducing response times by even a few seconds can prevent theft or damage. Local processing also eliminates the need for continuous video streaming to a central server, reducing bandwidth usage and minimizing exposure to data breaches.
Energy Efficiency
Edge devices often operate on limited power supplies or size-constrained batteries, but for a security camera to be effective, it can’t experience downtime. The improved battery efficiency an Ethos NPU delivers ensures that smart security cameras can run longer on the same battery capacity or with less energy consumption when hardwired. That way, security cameras can provide occupants with continuous monitoring without fear of dead batteries.
Advanced AI Model Support
The support for advanced AI models, particularly transformer-based models, enables smart cameras to understand complex scenarios better than ever before. For instance, transformer models can analyze sequential frames to understand the context of movements or changes within the camera’s field of view, such as identifying if an object left behind in a public area poses a threat, or if an individual’s actions around a restricted area warrant a security alert.
Alif Semiconductor’s Future with Ethos
In the Alif Ensemble and Balletto families, the Ethos NPU is integrated seamlessly with the Cortex-M55 processors, and benchmark data shows that it is able to uplift inference performance and improve power efficiency by two orders of magnitude for popular open source models.
Alif is already designing next generation products with Ethos-U85 on board. In fact, because we recognize the potential of Ethos-U85, we became one of the first partners to license the technology from Arm upon its release (much like our approach with the previous Ethos-U55). The Ethos-U85 is the ideal solution for the performance required by Alif’s next generation of microcontrollers and fusion processors.
Ethos-U85 will extend our microcontrollers ability to process cutting edge AI/ML workloads with increased memory bandwidth, allowing ML models to operate significantly faster! Keep an eye out for future releases from Alif that are built around the Arm Ethos-U85 NPU.