Highlights

In brief

EDCompress makes convolutional neural networks more energy efficient, enabling small, battery-powered edge devices to use powerful computing abilities without relying on large central computers.

© Unsplash

Giving handheld devices a computing edge

13 Dec 2022

A*STAR computer scientists develop a compression method to shrink energy and hardware requirements for complex computational platforms on everyday devices.

When it comes to computing, size matters. Hardware dimensions typically line up with computers’ capabilities, with big tasks in artificial intelligence (AI) such as machine learning algorithms usually needing more extensive and bulky computing servers. However, emerging technologies could one day put supercomputing in the palms of our hands; specifically, through our edge devices.

Ranging from our smartphones to internet-enabled home appliances, edge devices are devices that can act as interfaces between networks and the real world to collect and communicate information, said Zhehui Wang, a Research Scientist at A*STAR’s Institute of High Performance Computing (IHPC).

Edge devices have the potential to run sophisticated computing functions such as conventional neural networks (CNNs), but their small batteries and consumer-level hardware can hold them back. “If they’re running ‘hungry’ computing processes like those involved in traditional CNNs, they’ll run out of power quickly,” explained Wang, whose team works on innovative approaches towards small but mighty next-generation edge devices.

The researchers developed EDCompress, a CNN model compressor that adapts and optimises CNN processes to be more energy efficient when used in compact, battery-powered hardware. “Think of it as creating a zipped file on your computer,” said Wang. “EDCompress analyses the hardware features of the device running the CNN model, and shrinks the model’s architecture and energy requirements to suit.”

Using reinforcement learning techniques—similar to how a chess AI is ‘taught’ optimal strategies by playing thousands of games—Wang’s group designed EDCompress to determine the best combination of quantisation (making data values less precise) and pruning (removing less important parameters) that would speed up CNN processes, reduce their energy consumption, and still enable them to generate reasonably accurate and precise results.

“CNNs are well-suited for tasks such as classifying, detecting and segmenting images and objects. If edge devices could run CNNs on their own, they could do these tasks offline, without relying on a back-and-forth with external networks,” Wang added. “This would improve data privacy, reduce server workloads and speed up data processing.”

Validation experiments using EDCompress on three existing CNN architectures proved successful, with the team demonstrating remarkable energy efficiency boosts without sacrificing accuracy. They found that EDCompress provided between 17 and 26 times more energy efficiency, and was intuitive, automatically picking the optimal dataflow to conserve energy consumption.

“With EDCompress, we can reach lower latencies that let applications run more closely to real-time on edge devices while lowering energy and hardware costs,” commented Tao Luo, a fellow Research Scientist on the IHPC team. “There’s a wide range of possibilities to explore with it. Imagine a smartwatch that could monitor your heart without needing frequent recharging, or a smartphone that could identify objects in real-time.”

Applications that demand more intelligence, computing power and advanced services at the network edge, such as medical wearables, could benefit greatly from EDCompress’ capabilities. Wang added that EDCompress also contributes to a ‘green AI’ future, driving down carbon emissions associated with advanced computing. The team hopes to collaborate with other research institutes and external partners to further develop its application in real-use spaces.

The A*STAR researchers contributing to this research are from the Institute of High Performance Computing (IHPC).

Want to stay up to date with breakthroughs from A*STAR? Follow us on Twitter and LinkedIn!

References

Wang, Z., Luo, T., Goh, R.S.M. and Zhou, J.T. EDCompress: Energy-Aware Model Compression for Dataflows. IEEE Transactions on Neural Networks and Learning Systems 1-13 (2022). | article

About the Researchers

Zhehui Wang is a Research Scientist with the Computing & Intelligence Department, Institute of High-Performance Computing (IHPC), A*STAR. He received his BS in Electrical Engineering from Fudan University, China, in 2010, and PhD degree in Electronic and Computer Engineering from Hong Kong University of Science and Technology, Hong Kong, in 2016. He is currently leading the hardware-aware AI area of IHPC. His research interests cover both hardware and software areas, including efficient AI deployment, AI on emerging technologies, hardware-software co-design, and high-performance computing.
Tao Luo received his PhD from the School of Computer Science and Engineering, Nanyang Technological University, Singapore, in 2017. Luo is currently a Research Scientist and Group Manager at A*STAR’s Institute of High Performance Computing (IHPC). He has led many research and industrial projects and published numerous papers in prestigious conferences and journals. His current research interests include high performance computing, green and efficient artificial intelligence (AI), quantum computing and hardware-software co-exploration.
View articles

Rick Siow Mong Goh

Director, Computing and Intelligence (CI) Department

Institute of High Performance Computing (IHPC)
Rick Siow Mong Goh received his PhD degree in electrical and computer engineering from the National University of Singapore in 2006. He is currently the Director of the Computing and Intelligence (CI) Department at A*STAR's Institute of High Performance Computing (IHPC). He leads a team of over 80 scientists in performing world-leading scientific research, developing technology to commercialisation, and engaging and collaborating with industry. His research interests include artificial intelligence (AI), high performance computing, blockchain and federated learning.
Joey Tianyi Zhou is currently a senior scientist, investigator and group manager with A*STAR's Centre for Frontier AI Research (CFAR), Singapore. He also holds an adjunct faculty position at the National University of Singapore (NUS). Before he joined IHPC, he was a senior research engineer with SONY US Research Center in San Jose, USA. Zhou received his PhD in computer science from Nanyang Technological University (NTU), Singapore. His current interests focus mainly on machine learning with limited resources and their applications to natural language processing and computer vision tasks.

This article was made for A*STAR Research by Wildtype Media Group