Towards accelerating model parallelism in distributed deep learning systems

Modern deep neural networks cannot be often trained on a single GPU due to large model size and large data size. Model parallelism splits a model for multiple GPUs, but making it scalable and seamless is challenging due to different information sharing among GPUs with communication overhead. Specifi...

Descripción completa

Detalles Bibliográficos
Autores principales: Choi, Hyeonseong, Lee, Byung Hyun, Chun, Se Young, Lee, Jaehwan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2023
Materias:
Research Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10621816/
https://www.ncbi.nlm.nih.gov/pubmed/37917655
http://dx.doi.org/10.1371/journal.pone.0293338

Internet

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10621816/
https://www.ncbi.nlm.nih.gov/pubmed/37917655
http://dx.doi.org/10.1371/journal.pone.0293338

Ejemplares similares