In brief

Multiview machine learning networks can help analyse large, diverse streams of data more accurately and efficiently than traditional models.


How to harmonise disparate datasets

6 Oct 2022

A new machine learning model processes datasets from different sources simultaneously, paving the way for major advancements in medicine and beyond.

From self-driving cars to ‘what to watch next’ streaming recommendations, artificial intelligence (AI) is slowly transforming how we live, work, and play. Experts say that AI trained using machine learning (ML) still has a wealth of untapped potential. However, the challenge is that real-world data is often too complex and multi-dimensional to be used as training material for many of today’s ML platforms.

The doctor-patient scenario illustrates the challenges of using complex real-world data, said Hongyuan Zhu, a Research Scientist at A*STAR’s Institute for Infocomm Research (I2R). “Medical records can include text that describes the patient’s symptoms alongside images of diagnostic scans, such as CT scans of tumours,” explained Zhu. “These varying sensors often create different data formats or types. Each source therefore builds a different perspective or ‘view’ of the same patient’s medical history."

In theory, an AI that could efficiently analyse these multiple and varied views could help doctors make faster and more accurate healthcare decisions. However, in practice, training AI to do so involves a number of technical hurdles.

“In the real world, as hospitals gradually introduce new diagnostic sensors, new views are added to medical records over time,” Zhu said. “Conventional ML models—which depend on having all views available from the start—need to be entirely retrained to handle new views, which is time-consuming.”

As a solution for extracting insights from increasingly mixed data sources, Zhu and colleagues looked to a branch of ML called multiview learning. Multiview learning can make sense of large, diverse datasets much more accurately and efficiently than the narrower perspective offered by traditional single-view analyses.

The research team developed one of the first independent semi-supervised view-specific networks (ISVNs) capable of consolidating streams of data from individual sources before processing and analysing the dataset as a whole.

The team’s new network was designed to bypass the training limitations of first-generation algorithms, said Zhu. “While co-existing under one system, our ISVNs work independently so that newly incorporated models can function optimally without retraining old ones from scratch,” he added.

The researchers’ ISVN platform proved to be a game-changer for working with multiview datasets. Zhu’s team validated it against a panel of 13 other ML approaches and found their ISVN was the most flexible and scalable, training itself to handle add-on datasets with ease. The team’s ISVN was also ultraefficient, requiring fewer computational resources such as Graphics Processing Unit (GPU) memory and less training time.

Many industries, including the medical sector, stand to benefit from this exciting development. For example, ISVNs could automate the data processing and analysis of ultrasound, computed tomography (CT) and magnetic resonance imaging (MRI) scans, leading to fewer manual tasks for doctors and better outcomes for patients.

Following the study’s success, the researchers are currently collaborating with other expert groups to optimise and enhance the performance of their models.

The A*STAR-affiliated researchers contributing to this research are from the Institute for Infocomm Research (I2R) and the Institute of High Performance Computing (IHPC).

Want to stay up to date with breakthroughs from A*STAR? Follow us on Twitter and LinkedIn!


Hu, P., Peng, X., Zhu, H., Zhen, L., Lin, J., et al. Deep Semisupervised Multiview Learning With Increasing Views. IEEE Transactions on Cybernetics (2021) │ article

About the Researcher

Hongyuan Zhu received a PhD degree from Nanyang Technological University, Singapore in 2015 for his research into multi-modal deep learning. He is currently a Research Scientist at the A*STAR Institute for Infocomm Research (I2R), and on secondment at the A*STAR Institute for High Performance Computing (IHPC). He currently leads the Multi-modal Perceptual & Reasoning Team focusing on multi-modal scene perception, learning and reasoning. Zhu’s research has been published in many conference proceedings, including the Conference on Computer Vision and Pattern Recognition (CVPR), and he has served as the guest editor of IET Image Processing 2019.

This article was made for A*STAR Research by Wildtype Media Group