Cargando…
In-domain versus out-of-domain transfer learning in plankton image classification
Plankton microorganisms play a huge role in the aquatic food web. Recently, it has been proposed to use plankton as a biosensor, since they can react to even minimal perturbations of the aquatic environment with specific physiological changes, which may lead to alterations in morphology and behavior...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10300094/ https://www.ncbi.nlm.nih.gov/pubmed/37369770 http://dx.doi.org/10.1038/s41598-023-37627-7 |
_version_ | 1785064512689274880 |
---|---|
author | Maracani, Andrea Pastore, Vito Paolo Natale, Lorenzo Rosasco, Lorenzo Odone, Francesca |
author_facet | Maracani, Andrea Pastore, Vito Paolo Natale, Lorenzo Rosasco, Lorenzo Odone, Francesca |
author_sort | Maracani, Andrea |
collection | PubMed |
description | Plankton microorganisms play a huge role in the aquatic food web. Recently, it has been proposed to use plankton as a biosensor, since they can react to even minimal perturbations of the aquatic environment with specific physiological changes, which may lead to alterations in morphology and behavior. Nowadays, the development of high-resolution in-situ automatic acquisition systems allows the research community to obtain a large amount of plankton image data. Fundamental examples are the ZooScan and Woods Hole Oceanographic Institution (WHOI) datasets, comprising up to millions of plankton images. However, obtaining unbiased annotations is expensive both in terms of time and resources, and in-situ acquired datasets generally suffer from severe imbalance, with only a few images available for several species. Transfer learning is a popular solution to these challenges, with ImageNet1K being the most-used source dataset for pre-training. On the other hand, datasets like the ZooScan and the WHOI may represent a valuable opportunity to compare out-of-domain and large-scale plankton in-domain source datasets, in terms of performance for the task at hand.In this paper, we design three transfer learning pipelines for plankton image classification, with the aim of comparing in-domain and out-of-domain transfer learning on three popular benchmark plankton datasets. The general framework consists in fine-tuning a pre-trained model on a plankton target dataset. In the first pipeline, the model is pre-trained from scratch on a large-scale plankton dataset, in the second, it is pre-trained on large-scale natural image datasets (ImageNet1K or ImageNet22K), while in the third, a two-stage fine-tuning is implemented (ImageNet [Formula: see text] large-scale plankton dataset [Formula: see text] target plankton dataset). Our results show that an out-of-domain ImageNet22K pre-training outperforms the plankton in-domain ones, with an average boost in test accuracy of around 6%. In the next part of this work, we adopt three ImageNet22k pre-trained Vision Transformers and one ConvNeXt, obtaining results on par (or slightly superior) with the state-of-the-art, corresponding to the usage of CNN models ensembles, with a single model. Finally, we design and test an ensemble of our Vision Transformers and the ConvNeXt, outperforming the state-of-the-art existing works on plankton image classification on the three target datasets. To support scientific community contribution and further research, our implemented code is open-source and available at https://github.com/Malga-Vision/plankton_transfer. |
format | Online Article Text |
id | pubmed-10300094 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-103000942023-06-29 In-domain versus out-of-domain transfer learning in plankton image classification Maracani, Andrea Pastore, Vito Paolo Natale, Lorenzo Rosasco, Lorenzo Odone, Francesca Sci Rep Article Plankton microorganisms play a huge role in the aquatic food web. Recently, it has been proposed to use plankton as a biosensor, since they can react to even minimal perturbations of the aquatic environment with specific physiological changes, which may lead to alterations in morphology and behavior. Nowadays, the development of high-resolution in-situ automatic acquisition systems allows the research community to obtain a large amount of plankton image data. Fundamental examples are the ZooScan and Woods Hole Oceanographic Institution (WHOI) datasets, comprising up to millions of plankton images. However, obtaining unbiased annotations is expensive both in terms of time and resources, and in-situ acquired datasets generally suffer from severe imbalance, with only a few images available for several species. Transfer learning is a popular solution to these challenges, with ImageNet1K being the most-used source dataset for pre-training. On the other hand, datasets like the ZooScan and the WHOI may represent a valuable opportunity to compare out-of-domain and large-scale plankton in-domain source datasets, in terms of performance for the task at hand.In this paper, we design three transfer learning pipelines for plankton image classification, with the aim of comparing in-domain and out-of-domain transfer learning on three popular benchmark plankton datasets. The general framework consists in fine-tuning a pre-trained model on a plankton target dataset. In the first pipeline, the model is pre-trained from scratch on a large-scale plankton dataset, in the second, it is pre-trained on large-scale natural image datasets (ImageNet1K or ImageNet22K), while in the third, a two-stage fine-tuning is implemented (ImageNet [Formula: see text] large-scale plankton dataset [Formula: see text] target plankton dataset). Our results show that an out-of-domain ImageNet22K pre-training outperforms the plankton in-domain ones, with an average boost in test accuracy of around 6%. In the next part of this work, we adopt three ImageNet22k pre-trained Vision Transformers and one ConvNeXt, obtaining results on par (or slightly superior) with the state-of-the-art, corresponding to the usage of CNN models ensembles, with a single model. Finally, we design and test an ensemble of our Vision Transformers and the ConvNeXt, outperforming the state-of-the-art existing works on plankton image classification on the three target datasets. To support scientific community contribution and further research, our implemented code is open-source and available at https://github.com/Malga-Vision/plankton_transfer. Nature Publishing Group UK 2023-06-27 /pmc/articles/PMC10300094/ /pubmed/37369770 http://dx.doi.org/10.1038/s41598-023-37627-7 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Maracani, Andrea Pastore, Vito Paolo Natale, Lorenzo Rosasco, Lorenzo Odone, Francesca In-domain versus out-of-domain transfer learning in plankton image classification |
title | In-domain versus out-of-domain transfer learning in plankton image classification |
title_full | In-domain versus out-of-domain transfer learning in plankton image classification |
title_fullStr | In-domain versus out-of-domain transfer learning in plankton image classification |
title_full_unstemmed | In-domain versus out-of-domain transfer learning in plankton image classification |
title_short | In-domain versus out-of-domain transfer learning in plankton image classification |
title_sort | in-domain versus out-of-domain transfer learning in plankton image classification |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10300094/ https://www.ncbi.nlm.nih.gov/pubmed/37369770 http://dx.doi.org/10.1038/s41598-023-37627-7 |
work_keys_str_mv | AT maracaniandrea indomainversusoutofdomaintransferlearninginplanktonimageclassification AT pastorevitopaolo indomainversusoutofdomaintransferlearninginplanktonimageclassification AT natalelorenzo indomainversusoutofdomaintransferlearninginplanktonimageclassification AT rosascolorenzo indomainversusoutofdomaintransferlearninginplanktonimageclassification AT odonefrancesca indomainversusoutofdomaintransferlearninginplanktonimageclassification |