Trellis for efficient data and task management in the VA Million Veteran Program

Biomedical studies have become larger in size and yielded large quantities of data, yet efficient data processing remains a challenge. Here we present Trellis, a cloud-based data and task management framework that completely automates the process from data ingestion to result presentation, while tra...

Descripción completa

Detalles Bibliográficos
Autores principales: Ross, Paul Billing, Song, Jina, Tsao, Philip S., Pan, Cuiping
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2021
Materias:
Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8636485/
https://www.ncbi.nlm.nih.gov/pubmed/34853358
http://dx.doi.org/10.1038/s41598-021-02569-5
_version_ 1784608538586251264
author Ross, Paul Billing
Song, Jina
Tsao, Philip S.
Pan, Cuiping
author_facet Ross, Paul Billing
Song, Jina
Tsao, Philip S.
Pan, Cuiping
author_sort Ross, Paul Billing
collection PubMed
description Biomedical studies have become larger in size and yielded large quantities of data, yet efficient data processing remains a challenge. Here we present Trellis, a cloud-based data and task management framework that completely automates the process from data ingestion to result presentation, while tracking data lineage, facilitating information query, and supporting fault-tolerance and scalability. Using a graph database to coordinate the state of the data processing workflows and a scalable microservice architecture to perform bioinformatics tasks, Trellis has enabled efficient variant calling on 100,000 human genomes collected in the VA Million Veteran Program.
format Online
Article
Text
id pubmed-8636485
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-86364852021-12-03 Trellis for efficient data and task management in the VA Million Veteran Program Ross, Paul Billing Song, Jina Tsao, Philip S. Pan, Cuiping Sci Rep Article Biomedical studies have become larger in size and yielded large quantities of data, yet efficient data processing remains a challenge. Here we present Trellis, a cloud-based data and task management framework that completely automates the process from data ingestion to result presentation, while tracking data lineage, facilitating information query, and supporting fault-tolerance and scalability. Using a graph database to coordinate the state of the data processing workflows and a scalable microservice architecture to perform bioinformatics tasks, Trellis has enabled efficient variant calling on 100,000 human genomes collected in the VA Million Veteran Program. Nature Publishing Group UK 2021-12-01 /pmc/articles/PMC8636485/ /pubmed/34853358 http://dx.doi.org/10.1038/s41598-021-02569-5 Text en © This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply 2021 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Ross, Paul Billing
Song, Jina
Tsao, Philip S.
Pan, Cuiping
Trellis for efficient data and task management in the VA Million Veteran Program
title Trellis for efficient data and task management in the VA Million Veteran Program
title_full Trellis for efficient data and task management in the VA Million Veteran Program
title_fullStr Trellis for efficient data and task management in the VA Million Veteran Program
title_full_unstemmed Trellis for efficient data and task management in the VA Million Veteran Program
title_short Trellis for efficient data and task management in the VA Million Veteran Program
title_sort trellis for efficient data and task management in the va million veteran program
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8636485/
https://www.ncbi.nlm.nih.gov/pubmed/34853358
http://dx.doi.org/10.1038/s41598-021-02569-5
work_keys_str_mv AT rosspaulbilling trellisforefficientdataandtaskmanagementinthevamillionveteranprogram
AT songjina trellisforefficientdataandtaskmanagementinthevamillionveteranprogram
AT tsaophilips trellisforefficientdataandtaskmanagementinthevamillionveteranprogram
AT pancuiping trellisforefficientdataandtaskmanagementinthevamillionveteranprogram

Ejemplares similares