generated digits

[Source: rdn-consulting]





Deep learning 2.0


» New inductive biases
» Memory-enabled AI
» Learning to reason
» Learning with less labels
» Social reinforcement learning
» Value-aligned learning



» Understanding human behaviours in video
» Visual question answering and dialog
» AI for automated software engineering
» Finding structure of care
» AI for structural biology
» Exploring molecular and materials spaces

As an approach to general intelligence, we study new ways for differentiable learning to reason with minimal supervision, towards System 2 capability. Deep Learning achieves the goals through compositional neural networks, iterative estimation, and differentiable programming. Our research program draws certain inspiration from cognitive neuroscience, fused with rigorous probabilistic inference. The ultimate long-term goal is devise a unified cognitive architecture that guides the learning and reasoning across scales in space-time.

The research program has three broad aims:

» To understand intelligence from computational and cognitive perspectives.
» To design intelligent machines that are competent, scalable and robust.
» To solve important data-rich problems across living, physical and digital domains.

Sub-area: New inductive biases

Successes in machine learning depend critically on having good priors on inductive biases. In deep learning, the strongest prior thus far has been neural architectures built on a small set of operators (signal filtering, convolution, recurrence, gating, memory and attention). We derive modular networks for regular data such as matrix and tensor as well as new data such as graphs and relations. We draw our architectural inspiration from neuroscience including the columnar structure of the neocortex for distributed processing, the thalamus structure for information routing, working memory for problem solving, and episodic memory for integrating information over time.

Column networks

Column Networks, as inspired by the cortical columns, to solve multi-relational learning.

Sub-area: Memory-enabled AI

Deep neural networks excel at function approximation and pattern recognition but fall short on manipulating complex, highly dependent systems, rapid contextualisation in new settings, retaining previously acquired skills, and holding long conversations. These limitations are possibly due to the lack of an explicit notion of memory. We design new kinds of memory with more robust handling of variability, less memorization, and stored programs. The memory serves as a central component in a grand unified cognitive architecture that naturally supports learning, reasoning, rapid contextualisation and imagination.

Variational memory encoder decoder

Variational Memory Encoder-Decoder, as applied for generating a diverse and coherent dialog.

Sub-area: Learning to reason

We are concerned about learning the capability to deduce new knowledge from previously acquired knowledge in response to a query. Such behaviours can be demonstrated naturally using a symbolic system with a rich set inferential tools, given that the symbols can be grounded to the sensory world. Deep learning contributes to the bottom-up learning of such a reasoning system by resolving the symbol grounding problem. Our research aims to build neural architectures that can learn to exhibit high-level reasoning functionalities, e.g., answering new questions over space-time in a compositional and progressive fashion.

A system for Video QA

A system for Video Question Answering that implements the dual-process theory of reasoning.

Sub-area: Learning with less labels

Learning with a few explicit labels is the hallmark of human intelligence. Leveraging unlabelled data, either through existing datasets, or through self-exploration, will be critical to the next AI generation. We investigate the following sub-areas. Representation learning: Learning starts with representation of latent factors in the data which are invariant to small changes and insensitive of noise. Generative models: The ability to model high-dimensional world and to imagine the future is fundamental to AI. We investigate fundamental issues of deep generative models including stability, generalisation and catastrophic forgetting in Generative Adversarial Networks, as well as disentanglement in Variational Auto-Encoders. Continual learning: We design new learning algorithms that adapt continually as new tasks are introduced, even if the task change is not explicitly marked.

2D Boltzmann machine

A Boltzmann machine for recommender system.

Sub-area: Social reinforcement learning

We leverage deep neural networks to enable multi-agent systems to perceive the world, act on it, interact with others, build theory of mind, imagine the future and receive feedbacks. Equipped with deep nets for perception, memory, statistical relational learning, and reasoning capabilities, we aim to bring multi-agent reinforcement learning to a new level.

ToMAGA system.

A system of multi-agents equipped with social psychology.

Sub-area:    Value-aligned learning

The rapid advancement of AI raises new ethical challenges which pose great risks to humanity if unsolved. We aim to invent new machine learning algorithms that teach machine to align values with humans. We provide a computational definition of value and derive a generic value regularisation and optimisation framework. Our framework is demonstrated via (a) a conversational system that enables natural human–computer interaction through which human values can be regularised in batch or in an online fashion; and (b) a value-centric multi-agent learning system that enables agents to learn values of others through exchanging of value-laden expressions. This project aims to transform machine learning from being performance-driven to aligning with human values.

Ongoing Projects:

·       Human behaviour understanding in video

This project aims at a deep understanding of human behaviours seen through (fixed and moving) videos in various indoor and outdoor contexts. We build new models of trajectories and social interactions, and predict actions and intention.

Partners: iCetana

Anomaly detection with skeleton trajectories

Detecting anomalies in video using skeleton trajectories (last row).

·       Visual question answering and dialog

We study the new cognitive capability of a system to answer new natural questions about an image or a video. This is a powerful way to demonstrate the reasoning capacity, which involves linguistic, visual processing and high-level symbols manipulation skills. In visual dialog, we build a system having a natural multi-turn chat with human about a visual object.

Answering question about video

Answering questions about a video.

·       AI for automated software engineering

We design new deep neural architectures to read the code, fix the bugs, synthesize programs, translate between languages, automate the programming process, understand developer and support team management.

Partners: University of Wollongong, Samsung.

AI system for code vulerability detection and fixing

A system for instant vulnerability warning and suggesting fix patches in code.

·       Finding structure of care

This research aims at designing neural architectures for representation, clustering and prediction both at the patient and the cohort levels, based on the electronic medical records and genomics data. The long-term goals include acquiring and reasoning about established medical knowledge; having a meaningful dialog with patients; recommending the personalized course of medical actions; and supporting doctors and hospital managers to improve their precision and efficiency.

Partners: Barwon Health, University of Sydney, Northshore Hospital, Institute for Health Transformation at Deakin University.

Deepr: A system for health risk prediction

Deepr - a deep neural net for scanning medical records, detecting risk motifs and predicting future risks.

·       AI for structural biology

This research aims at designing neural architectures for representation of -omics and structured biological data. We map genotype-phenotype, answer any genomic queries for a given sequence, predict protein-target interactions, estimate protein folding, design drugs, and learn to generate DNA/RNA and protein. The long-term goals also include acquiring, organizing and reasoning about established biology knowledge.

Partners: TBA.

·       Exploring the molecular and materials space

We use deep learning to characterise the chemical space, replace expensive physical computation and experiments, predict molecular properties, molecular-molecular interactions and chemical reactions, and generate drug molecules given a set of desirable bioactivity properties. In materials design, we design new tools for understanding the structure and characteristics of materials, searching for new alloys, and generating molecules & crystals.

Partners: Institute of Frontier Materials at Deakin, Japan Institute of Advanced Science and Technology.

Relational Dynamic Memory Network

Relational Dynamic Memory Network, a model for detecting interactions among molecules.