Cargando…
Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark
In this work, we study the phenomenon of catastrophic forgetting in the graph representation learning scenario. The primary objective of the analysis is to understand whether classical continual learning techniques for flat and sequential data have a tangible impact on performances when applied to g...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8855050/ https://www.ncbi.nlm.nih.gov/pubmed/35187476 http://dx.doi.org/10.3389/frai.2022.824655 |
_version_ | 1784653571250192384 |
---|---|
author | Carta, Antonio Cossu, Andrea Errica, Federico Bacciu, Davide |
author_facet | Carta, Antonio Cossu, Andrea Errica, Federico Bacciu, Davide |
author_sort | Carta, Antonio |
collection | PubMed |
description | In this work, we study the phenomenon of catastrophic forgetting in the graph representation learning scenario. The primary objective of the analysis is to understand whether classical continual learning techniques for flat and sequential data have a tangible impact on performances when applied to graph data. To do so, we experiment with a structure-agnostic model and a deep graph network in a robust and controlled environment on three different datasets. The benchmark is complemented by an investigation on the effect of structure-preserving regularization techniques on catastrophic forgetting. We find that replay is the most effective strategy in so far, which also benefits the most from the use of regularization. Our findings suggest interesting future research at the intersection of the continual and graph representation learning fields. Finally, we provide researchers with a flexible software framework to reproduce our results and carry out further experiments. |
format | Online Article Text |
id | pubmed-8855050 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-88550502022-02-19 Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark Carta, Antonio Cossu, Andrea Errica, Federico Bacciu, Davide Front Artif Intell Artificial Intelligence In this work, we study the phenomenon of catastrophic forgetting in the graph representation learning scenario. The primary objective of the analysis is to understand whether classical continual learning techniques for flat and sequential data have a tangible impact on performances when applied to graph data. To do so, we experiment with a structure-agnostic model and a deep graph network in a robust and controlled environment on three different datasets. The benchmark is complemented by an investigation on the effect of structure-preserving regularization techniques on catastrophic forgetting. We find that replay is the most effective strategy in so far, which also benefits the most from the use of regularization. Our findings suggest interesting future research at the intersection of the continual and graph representation learning fields. Finally, we provide researchers with a flexible software framework to reproduce our results and carry out further experiments. Frontiers Media S.A. 2022-02-04 /pmc/articles/PMC8855050/ /pubmed/35187476 http://dx.doi.org/10.3389/frai.2022.824655 Text en Copyright © 2022 Carta, Cossu, Errica and Bacciu. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Artificial Intelligence Carta, Antonio Cossu, Andrea Errica, Federico Bacciu, Davide Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark |
title | Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark |
title_full | Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark |
title_fullStr | Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark |
title_full_unstemmed | Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark |
title_short | Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark |
title_sort | catastrophic forgetting in deep graph networks: a graph classification benchmark |
topic | Artificial Intelligence |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8855050/ https://www.ncbi.nlm.nih.gov/pubmed/35187476 http://dx.doi.org/10.3389/frai.2022.824655 |
work_keys_str_mv | AT cartaantonio catastrophicforgettingindeepgraphnetworksagraphclassificationbenchmark AT cossuandrea catastrophicforgettingindeepgraphnetworksagraphclassificationbenchmark AT erricafederico catastrophicforgettingindeepgraphnetworksagraphclassificationbenchmark AT bacciudavide catastrophicforgettingindeepgraphnetworksagraphclassificationbenchmark |