Imagine running late for an important meeting, and your car refuses to start despite having a full gas tank and no warning signs flashing on the dash. If only there had been a way to anticipate that the vehicle’s engine was close to its end.
Artificial intelligence can make these predictions, at least in an industrial setting. Many companies use deep learning algorithms to estimate when machines are likely to start winding down, helping avoid emergencies due to unexpected failures. However, because these algorithms are so complex, they need to be run on advanced computing systems that are often housed off-site, thus limiting their use in real-time decision making.
A team of researchers led by Qing Xu, a Research Engineer at A*STAR’s Institute for Infocomm Research (I2R), was interested in streamlining these computational platforms to make them more accessible for everyday, on-the-job use.
The experts turned to knowledge distillation, a method by which a larger, more complicated computing system called the ‘teacher’ transfers its knowledge to a smaller, more economical ‘student’ system. Upon distillation, the ‘student’ learns to copy the outputs of the ‘teacher’ using less disk space, allowing advanced calculations to be performed by regular computer hardware.
The problem is that this distillation process is not fool-proof—compressing elaborate equations compromises the algorithm’s predictive accuracy. To overcome this challenge, the team created a novel framework for knowledge distillation that they named KDnet-RUL.
Designed to be fast and take up minimal storage space, KDnet-RUL can retain its teacher’s accuracy through a specialized two-factor approach. First, a generative adversarial network facilitates the actual knowledge transfer from the original, highly complex prediction algorithm to a basic convolutional neural network. The ‘student’ then passes through several cycles of learning-during-teaching knowledge distillation to improve its accuracy at forecasting a machine’s remaining lifespan.
For KDnet-RUL’s test run, the researchers used C-MAPSS, a public dataset that simulates how turbofan engines degrade over time. They found that KDnet-RUL was just as effective as its ‘teacher’ network at estimating when these engines would fail. In some instances, the ‘student’ even had more accurate estimates than the ‘teacher’ and could deliver its lifespan predictions more rapidly.
“These findings provide a possible model compression solution that addresses an actual industry requirement of deploying a powerful but cumbersome network into resource-limited edge devices,” Xu said.
“In the future, we will consider a more realistic and challenging scenario where the data for training and testing may come from different distributions,” added Xu, stating that future iterations of KDnet-RUL may be able to apply a model derived from one machine to predict the lifespan of another.
The A*STAR-affiliated researchers contributing to this research are from the Institute for Infocomm Research (I2R).
