Swarm: A federated cloud framework for large-scale variant analysis

Genomic data analysis across multiple cloud platforms is an ongoing challenge, especially when large amounts of data are involved. Here, we present Swarm, a framework for federated computation that promotes minimal data motion and facilitates crosstalk between genomic datasets stored on various clou...

Descripción completa

Detalles Bibliográficos
Autores principales: Bahmani, Amir, Ferriter, Kyle, Krishnan, Vandhana, Alavi, Arash, Alavi, Amir, Tsao, Philip S., Snyder, Michael P., Pan, Cuiping
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2021
Materias:
Research Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8143397/
https://www.ncbi.nlm.nih.gov/pubmed/33979321
http://dx.doi.org/10.1371/journal.pcbi.1008977
_version_ 1783696744593227776
author Bahmani, Amir
Ferriter, Kyle
Krishnan, Vandhana
Alavi, Arash
Alavi, Amir
Tsao, Philip S.
Snyder, Michael P.
Pan, Cuiping
author_facet Bahmani, Amir
Ferriter, Kyle
Krishnan, Vandhana
Alavi, Arash
Alavi, Amir
Tsao, Philip S.
Snyder, Michael P.
Pan, Cuiping
author_sort Bahmani, Amir
collection PubMed
description Genomic data analysis across multiple cloud platforms is an ongoing challenge, especially when large amounts of data are involved. Here, we present Swarm, a framework for federated computation that promotes minimal data motion and facilitates crosstalk between genomic datasets stored on various cloud platforms. We demonstrate its utility via common inquiries of genomic variants across BigQuery in the Google Cloud Platform (GCP), Athena in the Amazon Web Services (AWS), Apache Presto and MySQL. Compared to single-cloud platforms, the Swarm framework significantly reduced computational costs, run-time delays and risks of security breach and privacy violation.
format Online
Article
Text
id pubmed-8143397
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-81433972021-06-07 Swarm: A federated cloud framework for large-scale variant analysis Bahmani, Amir Ferriter, Kyle Krishnan, Vandhana Alavi, Arash Alavi, Amir Tsao, Philip S. Snyder, Michael P. Pan, Cuiping PLoS Comput Biol Research Article Genomic data analysis across multiple cloud platforms is an ongoing challenge, especially when large amounts of data are involved. Here, we present Swarm, a framework for federated computation that promotes minimal data motion and facilitates crosstalk between genomic datasets stored on various cloud platforms. We demonstrate its utility via common inquiries of genomic variants across BigQuery in the Google Cloud Platform (GCP), Athena in the Amazon Web Services (AWS), Apache Presto and MySQL. Compared to single-cloud platforms, the Swarm framework significantly reduced computational costs, run-time delays and risks of security breach and privacy violation. Public Library of Science 2021-05-12 /pmc/articles/PMC8143397/ /pubmed/33979321 http://dx.doi.org/10.1371/journal.pcbi.1008977 Text en https://creativecommons.org/publicdomain/zero/1.0/This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 (https://creativecommons.org/publicdomain/zero/1.0/) public domain dedication.
spellingShingle Research Article
Bahmani, Amir
Ferriter, Kyle
Krishnan, Vandhana
Alavi, Arash
Alavi, Amir
Tsao, Philip S.
Snyder, Michael P.
Pan, Cuiping
Swarm: A federated cloud framework for large-scale variant analysis
title Swarm: A federated cloud framework for large-scale variant analysis
title_full Swarm: A federated cloud framework for large-scale variant analysis
title_fullStr Swarm: A federated cloud framework for large-scale variant analysis
title_full_unstemmed Swarm: A federated cloud framework for large-scale variant analysis
title_short Swarm: A federated cloud framework for large-scale variant analysis
title_sort swarm: a federated cloud framework for large-scale variant analysis
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8143397/
https://www.ncbi.nlm.nih.gov/pubmed/33979321
http://dx.doi.org/10.1371/journal.pcbi.1008977
work_keys_str_mv AT bahmaniamir swarmafederatedcloudframeworkforlargescalevariantanalysis
AT ferriterkyle swarmafederatedcloudframeworkforlargescalevariantanalysis
AT krishnanvandhana swarmafederatedcloudframeworkforlargescalevariantanalysis
AT alaviarash swarmafederatedcloudframeworkforlargescalevariantanalysis
AT alaviamir swarmafederatedcloudframeworkforlargescalevariantanalysis
AT tsaophilips swarmafederatedcloudframeworkforlargescalevariantanalysis
AT snydermichaelp swarmafederatedcloudframeworkforlargescalevariantanalysis
AT pancuiping swarmafederatedcloudframeworkforlargescalevariantanalysis

Ejemplares similares