Deep Graph Learning: Foundations, Advances and Applications
Many real data come in the form of non-grid objects, i.e. graphs, from social networks to molecules. Adaptation of deep learning from grid-alike data (e.g. images) to graphs has recently received unprecedented attention from both machine learning and data mining communities, leading to a new cross-domain field---Deep Graph Learning (DGL). Instead of painstaking feature engineering, DGL aims to learn informative representations of graphs in an end-to-end manner. It has exhibited remarkable success in various tasks, such as node/graph classification, link prediction, etc.
In this tutorial, we aim to provide a comprehensive introduction to deep graph learning. We first introduce the theoretical foundations on deep graph learning with a focus on describing various Graph Neural Network Models (GNNs). We then cover the key achievements of DGL in recent years. Specifically, we discuss the four topics: 1) training deep GNNs; 2) robustness of GNNs; 3) scalability of GNNs; and 4) self-supervised and unsupervised learning of GNNs. Finally, we will introduce the applications of DGL towards various domains, including but not limited to drug discovery, computer vision, medical image analysis, social network analysis, natural language processing and recommendation.
- Yu Rong, Tencent AI Lab, email@example.com
- Tingyang Xu, Tencent AI Lab, firstname.lastname@example.org
- Junzhou Huang, Tencent AI Lab and UTA, email@example.com
- Wenbing Huang, Tsinghua University, firstname.lastname@example.org
- Hong Cheng, The Chinese University of Hong Kong, email@example.com
- Yao Ma, Michigan State University, firstname.lastname@example.org
- Yiqi Wang, Michigan State University, email@example.com
- Tyler Derr, Vanderbilt University, firstname.lastname@example.org
- Lingfei Wu, IBM Research AI, email@example.com
- Tengfei Ma, IBM Research AI,firstname.lastname@example.org
This tutorial will contain two themes. Here is the schedule.
- 08:00 am - 08:10 am: Opening (Junzhou Huang)
Theme I: Foundations and Applications (08:10 am – 00:15 pm)
- 08:10 am – 09:00 am: Introduction to Graphs and Graph Neural Networks (Yao Ma)
- 09:00 am – 09:40 am: Robustness of Graph Neural Networks (Yiqi Wang)
- 09:40 am – 10:00 am: Break
- 10:00 am – 10:40 am: Self-Supervised Learning for Graph Neural Network I (Tyler Derr)
- 10:40 am – 11:20 am: Scalable Learning for Graph Neural Networks & Healthcare (Tengfei Ma )
- 11:20 am – 00:15 pm: Graph Structure Learning & NLP (Lingfei Wu)
Theme II: Advances and Applications (01:00 pm – 05:00 pm)
- 01:00 pm – 01:30 pm: Brief History of Graph Neural Networks (Yu Rong)
- 01:30 pm – 02:00 pm: Expressivity of GNNs (Wenbing Huang)
- 02:00 pm – 02:45 pm: Training Deep GNNs (Tingyang Xu)
- 02:45 pm – 03:10 pm: Break
- 03:15 pm – 03:45 pm: Scalability of GNNs (Yu Rong)
- 03:45 pm – 04:15 pm: Self/Un-Supervised Learning of GNNs (Wenbing Huang)
- 04:15 pm – 04:35 pm: GNN in Social Networks (Tingyang Xu)
- 04:35 pm – 04:55 pm: GNN in Medical Imaging & Future Directions (Junzhou Huang)
- 04:55 pm – 05:00 pm: Q&A
Theme I: Foundations and Applications
Theme II: Advances and Applications
Yu Rong is a Senior researcher of Machine Learning Center in Tencent AI Lab. He received this B.E. degree from Sun Yat-sen University, Guangzhou, China in 2012 and the Ph.D. degree from The Chinese University of Hong Kong in 2016. He joined Tencent AI Lab in June 2017. In Tencent AI Lab, he is working on building the large-scale graph learning framework and applying the deep graph learning model to various applications, such as ADMET prediction and malicious detection. His main research interests include social network analysis, graph neural networks, and large-scale graph systems, with particular focus on the design and efficient training for deep and complex graph learning models. He has published several papers on data mining, machine learning top conferences KDD, WWW, NeurIPS, ICLR, CVPR, ICCV, etc. He has served as a reviewer for KDD, WWW, CVPR, CIKM, WSDM, SDM and other journals such as VLDBJ and TKDE.
Wenbing Huang is now an assistant researcher at Tsinghua University, awarded as the Shuimu Tsinghua Scholar in 2019 and Tencent Rhino-Bird Visiting Scholar in 2020. He received his Ph.D. degree of computer science and technology from Tsinghua University in 2017 and the bachelor’s degree of mathematics from Beihang University in 2012. His current research mainly lies in the areas of machine learning, computer vision, and robotics, with particular focus on learning on irregular structures including graphs and videos. He has published about 30 peer-reviewed top-tier conference and journal papers, including the Proceedings of NeurIPS, ICLR, ICML, CVPR, etc. He has won the Champion of Autonomous Grasp Challenges in IROS 2016, 2019. He will co-organize the workshop “Human-centric multimedia analysis” in ACMMM 2020. He served as the session chair of “Video: Events, Activities and Surveillance” in IJCAI 2019, the PC member of IJCAI 2019-2020, AAAI 2019-2020, and the reviewer of TPAMI, IJCV, TIP, Neurocomputing, NeurIPS 2019-2020, CVPR 2019-2020, ICML 2019-2020, ICCV 2019, ECCV 2020, AISTATS 2019-2020. He was selected in the list of Top Reviewers in NeurIPS 2019.
Tingyang Xu is a Senior researcher of Machine Learning Center in Tencent AI Lab. He obtained the Ph.D. degree from The University of Connecticut in 2017 and joined Tencent AI Lab in July 2017. In Tencent AI Lab, he is working on deep graph learning, graph generations and applying the deep graph learning model to various applications, such as molecular generation and rumor detection. His main research interests include social network analysis, graph neural networks, and graph generations, with particular focus on design deep and complex graph learning models for molecular generations. He has published several papers on data mining, machine learning top conferences KDD, WWW, NeurIPS, ICLR, CVPR, ICML, etc. He has served as a reviewer for ICML, NeurIPS, KDD, WWW, AAAI, and other journals such as TKDE.
Hong Cheng is an Associate Professor in the Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong. She received the PhD degree from the University of Illinois at Urbana-Champaign in 2008. Her research interests include data mining, database systems, and machine learning. She received research paper awards at ICDE’07, SIGKDD’06, and SIGKDD’05, and the certificate of recognition for the 2009 SIGKDD Doctoral Dissertation Award. She received the 2010 Vice-Chancellor’s Exemplary Teaching Award at the Chinese University of Hong Kong.
Junzhou Huang is an Associate Professor in the Computer Science and Engineering department at the University of Texas at Arlington. He also served as the director of machine learning center in Tencent AI Lab. He received the B.E. degree from Huazhong University of Science and Technology, Wuhan, China, the M.S. degree from the Institute of Automation, Chinese Academy of Sciences, Beijing, China, and the Ph.D. degree in Computer Science at Rutgers, The State University of New Jersey. His major research interests include machine learning, computer vision and imaging informatics. He was selected as one of the 10 emerging leaders in multimedia and signal processing by the IBM T.J. Watson Research Center in 2010. His work won the MICCAI Young Scientist Award 2010, the FIMH Best Paper Award 2011, the MICCAI Young Scientist Award Finalist 2011, the STMI Best Paper Award 2012, the NIPS Best Reviewer Award 2013, the MICCAI Best Student Paper Award Finalist 2014 and the MICCAI Best Student Paper Award 2015. He received the NSF CAREER Award in 2016. He enjoys to develop efficient algorithms with nice theoretical guarantees to solve practical problems involved large scale data.
Yao Ma is a Ph.D. student of Computer Science and Engineering at Michigan State University. His research interests include network embedding and graph neural networks for graph representation learning. He was the leading presenter for the tutorial of "Graph Neural Networks: methods and applications" at AAAI2020 that is the most well-received tutorial with more than 400 audience. Updated information can be found at http://cse.msu.edu/~mayao4/.
Yiqi Wang is a Ph.D. student in the Computer Science and Engineering Department at Michigan State University. She is working on graph neural networks including fundamental algorithms, robustness and applications. She is one of the key contributors to the survey and empirical study on adversarial attacks and defenses on graphs with the developed repository. Updated information can be found at http://cse.msu.edu/wangy206 .
Tyler Derr is an Assistant Professor at Vanderbilt University in the Electrical Engineering and Computer Science department. He received his PhD in Computer Science from Michigan State University in 2020. His research is in network analysis and representation learning. He has published and serves as a program committee member at the top conferences in these domains and co-organized the Deep Graph Learning workshop at IEEE BigData’19. He received the Best Reviewer Award at ICWSM’19 and Best Student Poster Award at SDM’19. More details can be found at http://www.TylerDerr.com.
Lingfei Wu is a research staff member at IBM Research and is leading a research team (10+ RSMs) for developing novel Graph Neural Networks for many AL/ML/NLP tasks, which leads to the \#1 AI Challenge Project in IBM Research and multiple IBM Awards including IBM Outstanding Technical Achievement Award and IBM Invention Achievement Awards. He has published more than 60 top-ranked conference and journal papers and is a co-inventor of more than 25 filed US patents. He was the recipient of the Best Paper Award and Best Student Paper Award of several conferences such as AAAI workshop on DLGMA’20, IEEE ICC’19 and KDD workshop on DLG’19. His research has been featured in numerous media outlets, including NatureNews, YahooNews, Venturebeat, and TechTalks. He has co-organized 10+ conferences and workshops, including IEEE BigData’19, IEEE BigData’18, Workshops of Deep Learning on Graphs (with IJCAI’20, AAAI’20, KDD’19, and IEEE BigData’19). He has currently served as Associate Editor of ACM TKDD, and regularly served as a SPC/PC member of the following major AI/ML/NLP conferences including KDD, IJCAI, AAAI, NIPS, ICML, ICLR, and ACL. More details can be found via his personal website: https://sites.google.com/a/email.wm.edu/teddy-lfwu/home.
Tengfei Ma is a research staff member of IBM Research AI. Prior to moving to the IBM T. J. Watson Research Center in 2016, he received his Ph.D. from the University of Tokyo and joined IBM Research Tokyo in 2015. His research interests have spanned a number of different topics in machine learning and natural language processing. Recently his research is mainly focused on graph neural networks and their applications in healthcare and natural language processing; and he has published a series of papers about this topic in top conferences such as NeurIPS, ICLR, IJCAI, AAAI. More details can be found via his personal website: http://www.matengfei.com
In this tutorial, we focus on the emerging challenges of current DGL methods, and present the potential solutions to address them, including those approaches developed on our own. Our tutorial could share the main ideas among researchers, with the aim of pushing the boundary of DGL research. Meanwhile, we have involved the presentation of the applications in various domains. For instance, in terms of drug discovery, DGL has been applied in molecular property prediction @li2018adaptive, virtual screening @lim2019predicting, etc. This line of research no doubt helps chemist to discover the new drugs more efficiently, encouraging large positive potential social impacts. We believe that the more we understand DGL, the better we can solve these challenging problems and produce the great values for the society.