Cargando…

JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing

Complex computational pipelines are becoming a staple of modern scientific research. Often these pipelines are resource intensive and require days of computing time. In such cases, it makes sense to run them over high performance computing (HPC) clusters where they can take advantage of the aggregat...

Descripción completa

Detalles Bibliográficos
Autores principales: Brown, David K., Penkler, David L., Musyoka, Thommas M., Bishop, Özlem Tastan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2015
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4539224/
https://www.ncbi.nlm.nih.gov/pubmed/26280450
http://dx.doi.org/10.1371/journal.pone.0134273
_version_ 1782386086581895168
author Brown, David K.
Penkler, David L.
Musyoka, Thommas M.
Bishop, Özlem Tastan
author_facet Brown, David K.
Penkler, David L.
Musyoka, Thommas M.
Bishop, Özlem Tastan
author_sort Brown, David K.
collection PubMed
description Complex computational pipelines are becoming a staple of modern scientific research. Often these pipelines are resource intensive and require days of computing time. In such cases, it makes sense to run them over high performance computing (HPC) clusters where they can take advantage of the aggregated resources of many powerful computers. In addition to this, researchers often want to integrate their workflows into their own web servers. In these cases, software is needed to manage the submission of jobs from the web interface to the cluster and then return the results once the job has finished executing. We have developed the Job Management System (JMS), a workflow management system and web interface for high performance computing (HPC). JMS provides users with a user-friendly web interface for creating complex workflows with multiple stages. It integrates this workflow functionality with the resource manager, a tool that is used to control and manage batch jobs on HPC clusters. As such, JMS combines workflow management functionality with cluster administration functionality. In addition, JMS provides developer tools including a code editor and the ability to version tools and scripts. JMS can be used by researchers from any field to build and run complex computational pipelines and provides functionality to include these pipelines in external interfaces. JMS is currently being used to house a number of bioinformatics pipelines at the Research Unit in Bioinformatics (RUBi) at Rhodes University. JMS is an open-source project and is freely available at https://github.com/RUBi-ZA/JMS.
format Online
Article
Text
id pubmed-4539224
institution National Center for Biotechnology Information
language English
publishDate 2015
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-45392242015-08-24 JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing Brown, David K. Penkler, David L. Musyoka, Thommas M. Bishop, Özlem Tastan PLoS One Research Article Complex computational pipelines are becoming a staple of modern scientific research. Often these pipelines are resource intensive and require days of computing time. In such cases, it makes sense to run them over high performance computing (HPC) clusters where they can take advantage of the aggregated resources of many powerful computers. In addition to this, researchers often want to integrate their workflows into their own web servers. In these cases, software is needed to manage the submission of jobs from the web interface to the cluster and then return the results once the job has finished executing. We have developed the Job Management System (JMS), a workflow management system and web interface for high performance computing (HPC). JMS provides users with a user-friendly web interface for creating complex workflows with multiple stages. It integrates this workflow functionality with the resource manager, a tool that is used to control and manage batch jobs on HPC clusters. As such, JMS combines workflow management functionality with cluster administration functionality. In addition, JMS provides developer tools including a code editor and the ability to version tools and scripts. JMS can be used by researchers from any field to build and run complex computational pipelines and provides functionality to include these pipelines in external interfaces. JMS is currently being used to house a number of bioinformatics pipelines at the Research Unit in Bioinformatics (RUBi) at Rhodes University. JMS is an open-source project and is freely available at https://github.com/RUBi-ZA/JMS. Public Library of Science 2015-08-17 /pmc/articles/PMC4539224/ /pubmed/26280450 http://dx.doi.org/10.1371/journal.pone.0134273 Text en © 2015 Brown et al http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited.
spellingShingle Research Article
Brown, David K.
Penkler, David L.
Musyoka, Thommas M.
Bishop, Özlem Tastan
JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing
title JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing
title_full JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing
title_fullStr JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing
title_full_unstemmed JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing
title_short JMS: An Open Source Workflow Management System and Web-Based Cluster Front-End for High Performance Computing
title_sort jms: an open source workflow management system and web-based cluster front-end for high performance computing
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4539224/
https://www.ncbi.nlm.nih.gov/pubmed/26280450
http://dx.doi.org/10.1371/journal.pone.0134273
work_keys_str_mv AT browndavidk jmsanopensourceworkflowmanagementsystemandwebbasedclusterfrontendforhighperformancecomputing
AT penklerdavidl jmsanopensourceworkflowmanagementsystemandwebbasedclusterfrontendforhighperformancecomputing
AT musyokathommasm jmsanopensourceworkflowmanagementsystemandwebbasedclusterfrontendforhighperformancecomputing
AT bishopozlemtastan jmsanopensourceworkflowmanagementsystemandwebbasedclusterfrontendforhighperformancecomputing