UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs

However, graph learning has predominantly focused on single-graph models, tailored to specific tasks or datasets, lacking the ability to transfer learned knowledge to different domains. This limitation stems from the inherent complexity and diversity of graph structures, along with the different feature and label spaces specific to graph data. In this paper, we recognize text as an effective unifying medium and employ Text-Attributed Graphs (TAGs) to leverage this potential. We present our UniGraph1 framework, designed to learn a foundation model for TAGs, which is capable of generalizing to unseen graphs and tasks across diverse domains. Unlike single-graph models that use pre-computed node features of varying dimensions as input, our approach leverages textual features for unifying node representations, even for graphs such as molecular graphs that do not naturally have textual features. We propose a novel cascaded architecture of Language Models (LMs) and Graph Neural Networks (GNNs) as backbone networks. Additionally, we propose the first pre-training algorithm specifically designed for large-scale self-supervised learning on TAGs, based on Masked Graph Modeling. We introduce graph instruction tuning using Large Language Models (LLMs) to enable zero-shot prediction ability. Our comprehensive experiments across various graph learning tasks and domains demonstrate the model’s effectiveness in self-supervised representation learning on unseen graphs, few-shot in-context transfer, and zero-shot transfer, even surpassing or matching the performance of GNNs that have undergone supervised training on target datasets.

4.1 Unifying Graphs and Tasks from Diverse Domains

Graphs from different domains often have different applications, corresponding to different tasks. Graph learning tasks can generally be divided into node, edge, and graph-level tasks, each focusing on different parts of the graph. The key to using one model to handle any task on any graph lies in finding a universal function acting as a versatile mapping tool, adaptable to different graph learning tasks. In this paper, we utilize the concept of Anchor Node(s) and their contextual subgraph(s) to construct this universal function. The unification of node, edge, and graph-level tasks can be achieved through a general contextual subgraphs processing and Anchor Nodes embedding refinement operation, denoted by h = 𝑔(X,A), where h represents the output vector representation for a node, an edge or a graph, X denotes the set of contextual subgraphs, and A signifies the set of Anchor Node(s).