Cargando…
Multimodal Federated Learning: A Survey
Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heter...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10422520/ https://www.ncbi.nlm.nih.gov/pubmed/37571768 http://dx.doi.org/10.3390/s23156986 |
_version_ | 1785089230789148672 |
---|---|
author | Che, Liwei Wang, Jiaqi Zhou, Yao Ma, Fenglong |
author_facet | Che, Liwei Wang, Jiaqi Zhou, Yao Ma, Fenglong |
author_sort | Che, Liwei |
collection | PubMed |
description | Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been conducted to improve system performance utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic of multimodal federated learning (MFL) and present a literature review on the state-of-art MFL methods. Furthermore, we categorize multimodal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research. |
format | Online Article Text |
id | pubmed-10422520 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-104225202023-08-13 Multimodal Federated Learning: A Survey Che, Liwei Wang, Jiaqi Zhou, Yao Ma, Fenglong Sensors (Basel) Review Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been conducted to improve system performance utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic of multimodal federated learning (MFL) and present a literature review on the state-of-art MFL methods. Furthermore, we categorize multimodal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research. MDPI 2023-08-06 /pmc/articles/PMC10422520/ /pubmed/37571768 http://dx.doi.org/10.3390/s23156986 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Review Che, Liwei Wang, Jiaqi Zhou, Yao Ma, Fenglong Multimodal Federated Learning: A Survey |
title | Multimodal Federated Learning: A Survey |
title_full | Multimodal Federated Learning: A Survey |
title_fullStr | Multimodal Federated Learning: A Survey |
title_full_unstemmed | Multimodal Federated Learning: A Survey |
title_short | Multimodal Federated Learning: A Survey |
title_sort | multimodal federated learning: a survey |
topic | Review |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10422520/ https://www.ncbi.nlm.nih.gov/pubmed/37571768 http://dx.doi.org/10.3390/s23156986 |
work_keys_str_mv | AT cheliwei multimodalfederatedlearningasurvey AT wangjiaqi multimodalfederatedlearningasurvey AT zhouyao multimodalfederatedlearningasurvey AT mafenglong multimodalfederatedlearningasurvey |