When it comes to computing, size matters. Hardware dimensions typically line up with computers’ capabilities, with big tasks in artificial intelligence (AI) such as machine learning algorithms usually needing more extensive and bulky computing servers. However, emerging technologies could one day put supercomputing in the palms of our hands; specifically, through our edge devices.
Ranging from our smartphones to internet-enabled home appliances, edge devices are devices that can act as interfaces between networks and the real world to collect and communicate information, said Zhehui Wang, a Research Scientist at A*STAR’s Institute of High Performance Computing (IHPC).
Edge devices have the potential to run sophisticated computing functions such as conventional neural networks (CNNs), but their small batteries and consumer-level hardware can hold them back. “If they’re running ‘hungry’ computing processes like those involved in traditional CNNs, they’ll run out of power quickly,” explained Wang, whose team works on innovative approaches towards small but mighty next-generation edge devices.
The researchers developed EDCompress, a CNN model compressor that adapts and optimises CNN processes to be more energy efficient when used in compact, battery-powered hardware. “Think of it as creating a zipped file on your computer,” said Wang. “EDCompress analyses the hardware features of the device running the CNN model, and shrinks the model’s architecture and energy requirements to suit.”
Using reinforcement learning techniques—similar to how a chess AI is ‘taught’ optimal strategies by playing thousands of games—Wang’s group designed EDCompress to determine the best combination of quantisation (making data values less precise) and pruning (removing less important parameters) that would speed up CNN processes, reduce their energy consumption, and still enable them to generate reasonably accurate and precise results.
“CNNs are well-suited for tasks such as classifying, detecting and segmenting images and objects. If edge devices could run CNNs on their own, they could do these tasks offline, without relying on a back-and-forth with external networks,” Wang added. “This would improve data privacy, reduce server workloads and speed up data processing.”
Validation experiments using EDCompress on three existing CNN architectures proved successful, with the team demonstrating remarkable energy efficiency boosts without sacrificing accuracy. They found that EDCompress provided between 17 and 26 times more energy efficiency, and was intuitive, automatically picking the optimal dataflow to conserve energy consumption.
“With EDCompress, we can reach lower latencies that let applications run more closely to real-time on edge devices while lowering energy and hardware costs,” commented Tao Luo, a fellow Research Scientist on the IHPC team. “There’s a wide range of possibilities to explore with it. Imagine a smartwatch that could monitor your heart without needing frequent recharging, or a smartphone that could identify objects in real-time.”
Applications that demand more intelligence, computing power and advanced services at the network edge, such as medical wearables, could benefit greatly from EDCompress’ capabilities. Wang added that EDCompress also contributes to a ‘green AI’ future, driving down carbon emissions associated with advanced computing. The team hopes to collaborate with other research institutes and external partners to further develop its application in real-use spaces.
The A*STAR researchers contributing to this research are from the Institute of High Performance Computing (IHPC).
