generated digits

[Source: rdn-consulting]

Home   


 

 

 

 

 

Deep and representation learning

Deep learning is learning through multiple steps of data transformation in a compositional fashion. It enables end-to-end learning from raw data to tasks of interests. We target the following sub-problems:

» Representation learning

The learning starts with representation. This is because raw data may lie on hidden manifolds and contain noise and thus it may not be appropriate for tasks at hand. The goal of representation learning is to discover latent factors in the data which are invariant to small changes and insensitive of noise. Learning curve for the later stages will be much easier (e.g., better linearity and pre-conditioning) and the final performance will be improved (e.g., due to noise reduction and invariance promotion).

» Architecture engineering

A major part of modern deep learning is architecture engineering, i.e., designing a neural network that best fits the problems at hand, and at the same time, enables faster learning.

  • Statistical relational learning [Published in: ASE'15, AAAI'17]
  • Distance metrics [Published in ICME'13]
  • Motifs in medical records [Deepr]
  • Irregular timing [DeepCare,ICU]
  • Flows with interventions [DeepCare]
  • Compositional multiscale [DeepSoft]
  • Nested sequences [HSCRF]
  • Rank and permutation [Published in BDM]
  • Smart gates (Recurrent highway nets, GRU) [Published in ICPR'16]
  • Time-gap and intervention-induced gates [DeepCare]
  • Multi-layered sequences [Adaboost.MRF].
» Non-cognitive apps

Deep learning is currently flooded with work in vision, speech and NLP, the areas where humans perform well without any difficulties. But the world isn't just about see, listen and read. The world is also about living, learning, working, and staying safe. This is where our research comes to play.
  • Healthcare analytics [DeepCare, Deepr]
  • Software analytics [DeepSoft, Stacked Inference, DL-RNN]
  • Cybersecurity [MAD]
  • Process analytics 
  • Complex survey analysis [Mv.RBM, TBM]
  • Recommendation [OBM]
  • Choice and decision [NCBE]
  • Medical imaging [Tv.RBM]
  • Wearable computing




Preprints
  1. On Size Fit Many: Column Bundle for Multi-X Learning, Trang Pham, Truyen Tran, Svetha Venkatesh. arXiv preprint arXiv: 1702.07021.
  2. Multilevel Anomaly Detection for Mixed Data, Kien Do, Truyen Tran, Svetha Venkatesh, arXiv preprint arXiv: 1610.06249.
  3. A deep learning model for estimating story points, M Choetkiertikul, HK Dam, T Tran, T Pham, A Ghose, T Menzies, arXiv preprint arXiv: 1609.00489
  4. Learning deep representation of multityped objects and tasksTruyen Tran, D. Phung, and S. Venkatesh, arXiv preprint arXiv: 1603.01359.

Publications

  1. Graph Classification via Deep Learning with Virtual Nodes Trang Pham, Truyen Tran, Hoa Dam, Svetha Venkatesh, Third Representation Learning for Graphs Workshop (ReLiG 2017).
  2. Deep Learning to Attend to Risk in ICU, Phuoc Nguyen, Truyen Tran, Svetha Venkatesh, IJCAI'17 Workshop on Knowledge Discovery in Healthcare II: Towards Learning Healthcare Systems (KDH 2017).
  3. Learning Recurrent Matrix Representation, Kien Do, Truyen Tran, Svetha Venkatesh. Third Representation Learning for Graphs Workshop (ReLiG 2017), also: arXiv preprint arXiv: 1703.01454.
  4. Hierarchical semi-Markov conditional random fields for deep recursive sequential data, Truyen Tran, Dinh Phung, Hung Bui, Svetha Venkatesh,  Artificial Intelligence, Volume 246, May 2017, Pages 53–85. (Extension of the NIPS'08 paper).
  5. Predicting healthcare trajectories from medical records: A deep learning approach,Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh, Journal of Biomedical Informatics, April 2017, DOI: 10.1016/j.jbi.2017.04.001. [Tech report PDF].
  6. Deepr: A Convolutional Net for Medical Records, Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, Svetha Venkatesh,  IEEE Journal of Biomedical and Health Informaticsvol. 21, no. 1, pp. 22–30, Jan. 2017, Doi: 10.1109/JBHI.2016.2633963.
  7. Column Networks for Collective Classification, T Pham, T Tran, D Phung, S Venkatesh, AAAI'17
  8. Outlier Detection on Mixed-Type Data: An Energy-based Approach, K Do, T Tran, D Phung, S Venkatesh,International Conference on Advanced Data Mining and Applications (ADMA 2016).
  9. Stabilizing Linear Prediction Models using Autoencoder, Shivapratap Gopakumara, Truyen Tran, Dinh Phung, Svetha Venkatesh, International Conference on Advanced Data Mining and Applications (ADMA 2016).
  10. A deep language model for software code, Hoa Khanh Dam, Truyen Tran and Trang Pham, FSE NL+SE 2016.
  11. DeepSoft: A vision for a deep model of software, Hoa Khanh Dam, Truyen Tran, John Grundy and Aditya Ghose, FSE VaR 2016.
  12. Faster Training of Very Deep Networks Via p-Norm Gates, Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh, ICPR'16.
  13. DeepCare: A Deep Dynamic Memory Model for Predictive Medicine, Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh, PAKDD'16, Auckland, NZ, April 2016. 
  14. Neural Choice by Elimination via Highway Networks, Truyen Tran, Dinh Phung and Svetha Venkatesh,  PAKDD workshop on Biologically Inspired Techniques for Data Mining (BDM'16), April 19-22 2016, Auckland, NZ.
  15. Graph-induced restricted Boltzmann machines for document modeling, Tu D. Nguyen, Truyen Tran, D. Phung, and S. Venkatesh, Information Sciences. doi:10.1016/j.ins.2015.08.023.
  16. Predicting delays in software projects using networked classification, Morakot Choetikertikul, Hoa Khanh Dam, Truyen Tran, Aditya Ghose, 30th IEEE/ACM International Conference on Automated Software Engineering, November 9–13, 2015 Lincoln, Nebraska, USA.
  17. Learning vector representation of medical objects via EMR-driven nonnegative restricted Boltzmann machines (e-NRBM), Truyen Tran, Tu D. Nguyen, D. Phung, and S. Venkatesh, Journal of Biomedical Informatics, 2015, pii: S1532-0464(15)00014-3. doi: 10.1016/j.jbi.2015.01.012. 
  18. Tensor-variate Restricted Boltzmann Machines, Tu D. Nguyen, Truyen Tran, D. Phung, and S. Venkatesh, AAAI 2015. 
  19. Thurstonian Boltzmann machines: Learning from multiple inequalities, Truyen Tran, D. Phung, and S. Venkatesh, In Proc. of 30th International Conference in Machine Learning (ICML’13), Atlanta, USA, June, 2013.
  20. Learning parts-based representations with Nonnegative Restricted Boltzmann Machine, Tu D. Nguyen, Truyen Tran, D. Phung, and S. Venkatesh, Journal of Machine Learning Research (JMLR) Workshop and Conference Proceedings, Vol. 29, Proc. of 5th Asian Conference on Machine Learning, Nov 2013.
  21. Latent patient profile modelling and applications with Mixed-Variate Restricted Boltzmann Machine, Tu D. Nguyen, Truyen Tran, D. Phung, and S. Venkatesh,  In Proc. of 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD’13), Gold Coast, Australia, April 2013.
  22. Learning sparse latent representation and distance metric for image retrieval, Tu D. Nguyen, Truyen Tran, D. Phung, and S. Venkatesh, In Proc. of IEEE International Conference on Multimedia and Expo (ICME), San Jose, California, USA, July 2013.
  23. Learning from Ordered Sets and Applications in Collaborative Ranking, Truyen Tran, Dinh Phung and Svetha Venkatesh, in Proc. of. the 4th Asian Conference on Machine Learning (ACML2012), Singapore, Nov 2012.
  24. Cumulative Restricted Boltzmann Machines for Ordinal Matrix Data Analysis, Truyen Tran, Dinh Phung and Svetha Venkatesh, in Proc. of. the 4th Asian Conference on Machine Learning (ACML2012), Singapore, Nov 2012.
  25. Embedded Restricted Boltzmann Machines for Fusion of Mixed Data Types and Applications in Social Measurements Analysis, Truyen Tran, Dinh Phung, Svetha Venkatesh, in Proc. of 15-th International Conference on Information Fusion (FUSION-12), Singapore, July 2012.
  26. A Sequential Decision Approach to Ordinal Preferences in Recommender Systems, Truyen Tran, Dinh Phung, Svetha Venkatesh, in Proc. of 25-th Conference on Artificial Intelligence (AAAI-12), Toronto, Canada, July 2012.
  27. Learning Boltzmann Distance Metric for Face RecognitionTruyen Tran, Dinh Phung, Svetha Venkatesh, in Proc. of IEEE International Conference on Multimedia & Expo (ICME-12), Melbourne, Australia, July 2012.
  28. Mixed-Variate Restricted Boltzmann Machines, Truyen Tran, Dinh Phung and Svetha Venkatesh, in Proc. of. the 3rd Asian Conference on Machine Learning (ACML2011), Taoyuan, Taiwan, Nov 2011.
  29. Nonnegative Shared Subspace Learning and Its Application to Social Media Retrieval, S. Gupta, D. Phung, B. Adams, Tran The Truyen Proc. of 16th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 25-28 Jul, Washington DC, 2010. and Svetha Venkatesh, In
  30. Ordinal Boltzmann Machines for Collaborative Filtering. Truyen Tran, Dinh Q. Phung and Svetha Venkatesh. In Proc. of 25th Conference on Uncertainty in Artificial Intelligence, June, 2009, Montreal, Canada. Runner-up for the best paper award.
  31. MCMC for Hierarchical Semi-Markov Conditional Random Fields, Truyen Tran, Dinh Q. Phung, Svetha Venkatesh and Hung H. Bui. In NIPS'09 Workshop on Deep Learning for Speech Recognition and Related Applications. December, 2009, Whistler, BC, Canada
  32. Hierarchical Semi-Markov Conditional Random Fields for Recursive Sequential Data, Truyen Tran, Dinh Q. Phung, Hung H. Bui, and Svetha Venkatesh. In Proc. of 21st Annual Conference on Neural Information Processing Systems, Dec 2008, Vancouver, Canada. [See technical report and thesis for more details and extensions.]
  33. AdaBoost.MRF: Boosted Markov random forests and application to multilevel activity recognition, Truyen Tran, Dinh Quoc Phung, Hung Hai Bui, and Svetha Venkatesh. In Proc. of  IEEE Conference on Computer Vision and Pattern Recognition, volume Volume 2, pages 1686-1693, New York, USA, June 2006.