Similar articles

Abstract: continual learning (CL) has emerged as a crucial paradigm for learning from sequential data while retaining previous knowledge. continual graph learning (CGL), characterized by dynamically evolving graphs from streaming data, presents distinct challenges that demand efficient algorithms to prevent catastrophic forgetting. The first challenge stems from the interdependencies between different graph data, in which previous graphs influence new data distributions. The second challenge is handling large graphs in an efficient manner. To address these challenges, we propose an efficient continual graph learner (E-CGL) in this paper. We address the interdependence issue by demonstrating the effectiveness of replay strategies and introducing a combined sampling approach that considers both node importance and diversity. To improve efficiency, E-CGL leverages a simple yet effective multi-layer perceptron (MLP) model that shares weights with a graph neural network (GNN) during training, thereby accelerating computation by circumventing the expensive message-passing process. Our method achieves state-of-the-art results on four CGL datasets under two settings, while significantly lowering the catastrophic forgetting value to an average of -1.1%. Additionally, E-CGL achieves the training and inference speedup by an average of 15.83× and 4.89×, respectively, across four datasets. These results indicate that E-CGL not only effectively manages correlations between different graph data during continual training but also enhances efficiency in large-scale CGL.

E-CGL：一个高效的图连续学习器

[1]Aljundi R, Babiloni F, Elhoseiny M, et al., 2018. Memory aware synapses: learning what (not) to forget. Proc 15^th European Conf on Computer Vision, p.144-161.

[2]Bojchevski A, Günnemann S, 2018. Deep Gaussian embedding of graphs: unsupervised inductive learning via ranking. 6^th Int Conf on Learning Representations.

[3]Buzzega P, Boschini M, Porrello A, et al., 2020. Dark experience for general continual learning: a strong, simple baseline. Proc 34^th Int Conf on Neural Information Processing Systems, Article 1335.

[4]Choi S, Kim W, Kim S, et al., 2024. DSLR: diversity enhancement and structure learning for rehearsal-based graph continual learning. Proc ACM Web Conf, p.733-744.

[5]Fan ZH, Wei ZY, Chen JJ, et al., 2022. A unified continuous learning framework for multi-modal knowledge discovery and pre-training. https://arxiv.org/abs/2206.05555

[6]Gao XY, Chen T, Zang YL, et al., 2024. Graph condensation for inductive node representation learning. IEEE 40^th Int Conf on Data Engineering, p.3056-3069.

[7]Hamilton WL, Ying R, Leskovec J, 2017. Inductive representation learning on large graphs. Proc 31^st Int Conf on Neural Information Processing Systems, p.1025-1035.

[8]Han XT, Zhao T, Liu YZ, et al., 2023. MLPInit: embarrassingly simple GNN training acceleration with MLP initialization. The 11^th Int Conf on Learning Representations.

[9]Han XX, Feng Z, Ning Y, 2024. A topology-aware graph coarsening framework for continual graph learning. Proc 38^th Int Conf on Neural Information Processing Systems, Article 4212.

[10]Hsu CC, Lai YA, Chen WH, et al., 2017. Unsupervised ranking using graph structures and node attributes. Proc 10^th ACM Int Conf on Web Search and Data Mining, p.771-779.

[11]Hu WH, Fey M, Zitnik M, et al., 2020. Open graph benchmark: datasets for machine learning on graphs. Proc 34^th Int Conf on Neural Information Processing Systems, Article 1855.

[12]Hu XT, Tang KH, Miao CY, et al., 2021. Distilling causal effect of data in class-incremental learning. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3956-3965.

[13]Kim S, Yun S, Kang J, 2022. DyGRAIN: an incremental learning framework for dynamic graphs. Proc 31^st Int Joint Conf on Artificial Intelligence, p.3157-3163.

[14]Kipf TN, Welling M, 2017. Semi-supervised classification with graph convolutional networks. 5^th Int Conf on Learning Representations, p.3157-3163.

[15]Kirkpatrick J, Pascanu R, Rabinowitz N, et al., 2017. Overcoming catastrophic forgetting in neural networks. Proc Natl Acad Sci USA, 114(13):3521-3526.

[16]Li ZZ, Hoiem D, 2018. Learning without forgetting. IEEE Trans Patt Anal Mach Intell, 40(12):2935-2947.

[17]Liu HH, Yang YD, Wang XC, 2021. Overcoming catastrophic forgetting in graph neural networks. Proc AAAI Conf on Artificial Intelligence, p.8653-8661.

[18]Liu YL, Qiu RH, Huang Z, 2023. CaT: balanced continual graph learning with graph condensation. IEEE Int Conf on Data Mining, p.1157-1162.

[19]Lopez-Paz D, Ranzato M, 2017. Gradient episodic memory for continual learning. Proc 31^st Int Conf on Neural Information Processing Systems, p.6470-6479.

[20]Ni ZX, Wei LH, Tang SL, et al., 2023. Continual vision-language representation learning with off-diagonal information. Proc 40^th Int Conf on Machine Learning, Article 1087.

[21]Niepert M, Ahmed M, Kutzkov K, 2016. Learning convolutional neural networks for graphs. Proc 33^rd Int Conf on Machine Learning, p.2014-2023.

[22]Page L, Brin S, Motwani R, et al., 1999. The PageRank Citation Ranking: Bringing Order to the Web. Technical Report, SIDL-WP-1999-0120. http://ilpubs.stanford.edu:8090/422/

[23]Rahimi A, Recht B, 2007. Random features for large-scale kernel machines. Proc 21^st Int Conf on Neural Information Processing Systems, p.1177-1184.

[24]Rakaraddi A, Siew Kei L, Pratama M, et al., 2022. Reinforced continual learning for graphs. Proc 31^st ACM Int Conf on Information & Knowledge Management, p.1666-1674.

[25]Rebuffi SA, Kolesnikov A, Sperl G, et al., 2017. iCaRL: incremental classifier and representation learning. IEEE Conf on Computer Vision and Pattern Recognition, p.5533-5542.

[26]Srinivasan T, Chang TY, Alva LP, et al., 2022. CLiMB: a continual learning benchmark for vision-and-language tasks. Proc 36^th Int Conf on Neural Information Processing Systems, Article 2135.

[27]Su JW, Zou DF, Wu C, 2025. On the limitation and experience replay for GNNs in continual learning. Proc 3^rd Conf on Lifelong Learning Agents, p.342-366.

[28]Thrun S, 1994. A lifelong learning perspective for mobile robot control. Proc IEEE/RSJ Int Conf on Intelligent Robots and Systems, p.22-30.

[29]Veličković P, Cucurull G, Casanova A, et al., 2018. Graph attention networks.

[30]Wang JS, Song GJ, Wu Y, et al., 2020. Streaming graph neural networks via continual learning. Proc 29^th ACM Int Conf on Information & Knowledge Management, p.1515-1524.

[31]Wang ZZ, Sun YY, Zhang XK, et al., 2025. Continual learning with high-order experience replay for dynamic network embedding. Patt Recogn, 159:111093.

[32]Wu F, Zhang TY, de Souza AHJr, et al., 2019. Simplifying graph convolutional networks. Proc 36^th Int Conf on Machine Learning, p.6861-6871.

[33]Xu K, Hu WH, Leskovec J, et al., 2019. How powerful are graph neural networks? 7^th Int Conf on Learning Representations.

[34]Yang CX, Wu QT, Wang JH, et al., 2023. Graph neural networks are inherently good generalizers: insights by bridging GNNs and MLPs.

[35]Yuan Q, Guan SU, Ni P, et al., 2023. Continual graph learning: a survey. https://arxiv.org/abs/2301.12230

[36]Zhang XK, Song DJ, Tao DC, 2022a. CGLB: benchmark tasks for continual graph learning. Proc 36^th Int Conf on Neural Information Processing Systems, Article 945.

[37]Zhang XK, Song DJ, Tao DC, 2022b. Sparsified subgraph memory for continual graph representation learning. IEEE Int Conf on Data Mining, p.1335-1340.

[38]Zhou F, Cao CT, 2021. Overcoming catastrophic forgetting in graph neural networks with experience replay. Proc 35^th AAAI Conf on Artificial Intelligence, p.4714-4722.

[39]Zhu J, Yan YJ, Zhao LX, et al., 2020. Beyond homophily in graph neural networks: current limitations and effective designs. Proc 34^th Int Conf on Neural Information Processing Systems, Article 653.