Michael Bronstein is a Professor at ICL, Department of Computing. He has served as a professor at USI Lugano, Switzerland since 2010 and held visiting positions at Stanford, Harvard, MIT, TUM, and Tel Aviv University. Michael received his PhD with distinction from the Technion (Israel Institute of Technology) in 2007. His main expertise is in theoretical and computational geometric methods for machine learning and data science, and his research encompasses a broad spectrum of applications ranging from computer vision and pattern recognition to geometry processing, computer graphics, and biomedicine.
Title: Graph Neural Networks through the lens of algebraic topology, differential geometry, and PDEs.
Abstract: The message-passing paradigm has been the "battle horse" of deep learning on graphs for several years, making graph neural networks a big success in a wide range of applications, from particle physics to protein design. From a theoretical viewpoint, it established the link to the Weisfeiler-Lehman hierarchy, allowing to analyse the expressive power of GNNs. I argue that the very "graph-centric" mindset of current graph deep learning schemes may hinder future progress in the field. As an alternative, I propose physics-inspired "continuous" learning models that open up a new trove of tools from the fields of differential geometry, algebraic topology, and differential equations so far largely unexplored in graph ML.
Danai Koutra is a Associate Director at Michigan Institute for Data Science and a Morris Wellman Associate Professor at University of Michigan, Ann Arbor. Her research in large-scale data mining and machine learning (ML) focuses on principled, interpretable, and scalable methods for discovering and summarizing the unknown unknowns in the world's data by leveraging the inherent connections within them. These connections are naturally modeled in networks or graphs, which in turn span every facet of our lives: email communication networks, knowledge graphs for web search, social networks, coauthorship graphs, brain networks, artificial neural networks, and more.
Title: Beyond Homophily in Graph Neural Networks .
Abstract: Graph learning, which leverages the complex relationships between entities in non-Euclidean data, has a wide range of high-impact applications, including recommendation systems, bioinformatics, fraud detection, and more. Graph neural networks (GNNs) have become one of the most popular graph deep learning models, achieving state-of-the-art results for semi-supervised classification, in which the goal is to infer the unknown labels of the nodes given partially labeled networks with node features. While many different GNN models have been proposed, most of them perform best in graphs that exhibit the property of homophily, sometimes referred to as the idea that “birds of a feather flock together”, in which linked nodes often belong to the same class or have similar features. However, in the real world, there are also many settings where “opposites attract”, leading to networks that exhibit heterophily, in which linked nodes tend to be from different classes. Many of the most popular GNNs fail to generalize well to networks with heterophily (low homophily). In this talk, I will present my group’s work on identifying effective GNN designs and introducing new architectures that can improve performance in heterophilous networks. I will also discuss connections between heterophily and other well-known challenges of GNNs, including oversmoothing and robustness.
Marinka Zitnik is an Assistant Professor of Biomedical Informatics at Harvard Medical School. She investigates machine learning for science and medicine. Her methods leverage biomedical data at the scale of billions of interactions among millions of entities, blend machine learning with statistics and data science, and infuse biomedical knowledge into deep learning. Problems she investigates are motivated by network biology and medicine, genomics, drug discovery, and health.
Title: Graph-Guided Networks for Complex Time Series.
Abstract: In many domains, including healthcare, biology, and climate science, time series are irregularly sampled with varying time intervals between successive readouts and different subsets of variables (sensors) observed at different time points. For example, sensors' observations might not be aligned, time intervals among adjacent readouts can vary across sensors, different samples can have varying numbers of readouts recorded at different times. While machine learning methods usually assume fully observable and fixed-size inputs, irregularly sampled time series raise considerable challenges. In this talk, I will introduce Raindrop, a graph neural network that embeds irregularly sampled and multivariate time series while also learning the dynamics of sensors purely from observational data. Raindrop represents every sample as a separate graph, where nodes indicate sensors, and time-varying dependencies between sensors are modeled by a novel message passing operator. I will describe applications of Raindrop to classify time series and interpret temporal dynamics on three healthcare and human activity datasets. Raindrop shows superiority on multiple setups, including challenging leave-sensor-out settings.
Xin Luna Dong is the Head Scientist at Facebook AR/VR Assistant. Prior to joining Facebook, she was a Senior Principal Scientist at Amazon, leading the efforts of constructing Amazon Product Knowledge Graph, and before that one of the major contributors to the Google Knowledge Vault project, and has led the Knowledge-based Trust project, which is called the “Google Truth Machine” by Washington’s Post. She has co-authored books "Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases" and “Big Data Integration”, was awarded ACM Distinguished Member, and VLDB Early Career Research Contribution Award for “Advancing the state of the art of knowledge fusion”. She serves in the VLDB endowment and PVLDB advisory committee, and is a PC co-chair for KDD ADS 2022, WSDM 2022, and VLDB 2021.
Title: Zero to One Billion: The Path to a Rich Product Knowledge Graph
Abstract: Knowledge graphs have been used to support a wide range of applications and enhance search results for multiple major search engines, such as Google and Bing. To better serve customers in eCommerce, we aim to build a product knowledge graph with authoritative knowledge for products. The thousands of product verticals we need to model, the vast number of data sources we need to extract knowledge from, the huge volume of new products we need to handle every day, and the various applications in Search, Discovery, Personalization, Voice, that we wish to support, all present big challenges in constructing such a graph.
In this talk we describe our efforts for knowledge collection for products of thousands of types. We describe how we nail down the most important first step for delivering the data business: training high-precision models that generate accurate data. We then describe how we scale up the models with learning from limited labels, and how we increase the yields with multi-modal models and web extraction. We share the many learnings and lessons in building this product graph and applying it to support customer-facing applications..
Stephan Günnemann is a professor for Data Analytics and Machine Learning at Technical University of Munich and an acting director of the Munich Data Science Institute. He conducts research in the area of machine learning and data analytics. His main research focuses on how to make machine learning techniques reliable, thus, enabling their safe and robust use in various application domains. He is particularly interested in studying machine learning methods targeting complex data domains such as graphs/networks and temporal data.
Title: Graph Neural Networks for Molecules.
Abstract: Effectively predicting molecular interactions has the potential to accelerate molecular dynamics by multiple orders of magnitude and thus revolutionize chemical simulations. Graph neural networks (GNNs) have recently shown great successes for this task. However, they still appear very limited from a theoretical perspective, since regular GNNs cannot distinguish certain types of graphs. In this talk, I present recent advances in GNNs for molcules specifically incorporating the aspect of directionality, considering empirical performance and theoretical expressiveness. Moreover, I will give an outlook to ab-initio methods which avoid gathering expensive training data.
Pan Li is an Assistant Professor at Purdue University. Before that, he was a postdoc in the SNAP group at Stanford, where he worked with Prof. Jure Leskovec. His research focuses on computational and machine learning methods for data analysis, especially graph or network structured data. This combines interests in graph theory, machine learning, computational science and statistics.
Title: New Results on the Foundations of Graph Neural Networks.
Abstract: In this talk, we will introduce several new foundational results of graph neural networks. We may cover only a part of them given the time.
First, we connect information theory to graph neural network interpretation. The goal of GNN-based interpretation is to find the subgraphs of the input graphs that are most indicative to the prediction task labels. We disclose the drawbacks of posthoc interpretation methods that have been typically discussed in the community. We demonstrate that the information bottleneck principle is the only needed prior assumption needed for GNN interpretation, where previously-used sparsity and connectivity constraints may bias the interpretation results. Our theory naturally yields a novel architecture termed graph stochastic attention mechanism which can inherently provide reliable interpretation results.
Second, we introduce how to properly leverage traditional network embedding approaches in graph neural network models, so as to improve GNN expressive power while keeping the model generalizable across different networks (i.e., the inductive property). This is a work to respond to the previous criticism that using traditional network embedding may lose the inductive property unless using random augmentation. We address this problem in both theory and practice.
Third, if we have more time, we will introduce a novel graph auto-encoder. Previous GAE/VGAE and many others self-supervised GNN models are only applied to proximity-driven node-level tasks. However, our novel graph auto-encoder can also be applied to structure-driven node-level tasks. Our key idea is to leverage optimal transport to reconstruct neighbor nodes representations.
Xia Hu is an Associate Professor in Computer Science at Rice University. He is currently directing the DATA Lab, with his students and collaborators, they strive to develop automated and interpretable data mining and machine learning algorithms with theoretical properties to better discover actionable patterns from large-scale, networked, dynamic and sparse data. Their research is directly motivated by, and contributes to, applications in social informatics, health informatics and information security. Their work has been featured in Various News Media, such as MIT Tech Review, ACM TechNews, New Scientist, Fast Company, Economic Times. Their research is generously supported by federal agencies such as DARPA (XAI, D3M and NGS2) and NSF (CAREER, III, SaTC, CRII, S&AS), and industrial sponsors such as Adobe, Apple, Alibaba, Google, LinkedIn and JP Morgan. He was the General Co-Chair for WSDM 2020.
Title: Provable Benign Training of Graph Neural Networks.
Abstract: Graph neural networks (GNNs), which learn the node representations by recursively aggregating neighborhoods, have become prominent computation tools in graph structured data. However, the training of GNNs is notoriously challenging due to over-smoothing issue and poor scalability. The over-smoothing (i.e., smoothed node representations with recursive aggregation) prevents the exploration of high-order neighborhoods. Even worse, the storage of node representations results in high memory usage on large-scale graphs. To bridge the gaps towards stable and scalable training, we optimize GNNs from three coherent perspectives: (1) Differentiable group normalization, an architectural component to relieve the over-smoothing; (2) Dirichlet energy constrained learning, a principle to guide the design of deep GNNs; and (3) Extreme activation compression, a systematic tool to reduce memory. In practice, we could train deep GNNs up to 64 layers to achieve state-of-the-art performances on real-world datasets, and reduce the memory footprint of node representations by up to 32X. Furthermore, we provide the comprehensive theoretical analysis to understand how our optimizations guarantee the benign training of GNNs.
Renjie Liao is a Visiting Faculty Researcher at Google Brain, working with Geoffrey Hinton and David Fleet. He's also affiliated with Vector Institute. He will join UBC ECE and CS (associated) as an assistant professor in Jan. 2022. He's interested in machine learning and its interplay with vision, self-driving, language, and other areas. In the long run, He is keen to build learning systems that excel at integrating perception and reasoning. Recently, He's been focusing on deep learning for structures, e.g., graphs.
Title: On the Generalization of Graph Neural Networks.
Abstract: Graph neural networks (GNNs) are increasingly popular in handling graph-structured data. Although substantial progress has been achieved in the expressiveness/capacity of GNNs, we do not know much about the generalization ability of GNNs from the statistical learning theory perspective. In this talk, I will introduce our recent work that derives generalization bounds for the two prominent classes of GNNs, namely graph convolutional networks (GCNs) and message passing GNNs (MPGNNs), via a PAC-Bayesian approach. Our result reveals that the maximum node degree and spectral norm of the weights govern the generalization bounds of both models. For MPGNNs, our PAC-Bayes bound improves over the Rademacher complexity-based bound, showing a tighter dependency on the maximum node degree and the maximum hidden dimension..