Cargando…

First steps into the cloud: Using Amazon data storage and computing with Python notebooks

With the oncoming age of big data, biologists are encountering more use cases for cloud-based computing to streamline data processing and storage. Unfortunately, cloud platforms are difficult to learn, and there are few resources for biologists to demystify them. We have developed a guide for experi...

Descripción completa

Detalles Bibliográficos
Autores principales: Pollak, Daniel J., Chawla, Gautam, Andreev, Andrey, Prober, David A.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9910747/
https://www.ncbi.nlm.nih.gov/pubmed/36757918
http://dx.doi.org/10.1371/journal.pone.0278316
_version_ 1784884851335233536
author Pollak, Daniel J.
Chawla, Gautam
Andreev, Andrey
Prober, David A.
author_facet Pollak, Daniel J.
Chawla, Gautam
Andreev, Andrey
Prober, David A.
author_sort Pollak, Daniel J.
collection PubMed
description With the oncoming age of big data, biologists are encountering more use cases for cloud-based computing to streamline data processing and storage. Unfortunately, cloud platforms are difficult to learn, and there are few resources for biologists to demystify them. We have developed a guide for experimental biologists to set up cloud processing on Amazon Web Services to cheaply outsource data processing and storage. Here we provide a guide for setting up a computing environment in the cloud and showcase examples of using Python and Julia programming languages. We present example calcium imaging data in the zebrafish brain and corresponding analysis using suite2p software. Tools for budget and user management are further discussed in the attached protocol. Using this guide, researchers with limited coding experience can get started with cloud-based computing or move existing coding infrastructure into the cloud environment.
format Online
Article
Text
id pubmed-9910747
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-99107472023-02-10 First steps into the cloud: Using Amazon data storage and computing with Python notebooks Pollak, Daniel J. Chawla, Gautam Andreev, Andrey Prober, David A. PLoS One Lab Protocol With the oncoming age of big data, biologists are encountering more use cases for cloud-based computing to streamline data processing and storage. Unfortunately, cloud platforms are difficult to learn, and there are few resources for biologists to demystify them. We have developed a guide for experimental biologists to set up cloud processing on Amazon Web Services to cheaply outsource data processing and storage. Here we provide a guide for setting up a computing environment in the cloud and showcase examples of using Python and Julia programming languages. We present example calcium imaging data in the zebrafish brain and corresponding analysis using suite2p software. Tools for budget and user management are further discussed in the attached protocol. Using this guide, researchers with limited coding experience can get started with cloud-based computing or move existing coding infrastructure into the cloud environment. Public Library of Science 2023-02-09 /pmc/articles/PMC9910747/ /pubmed/36757918 http://dx.doi.org/10.1371/journal.pone.0278316 Text en © 2023 Pollak et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Lab Protocol
Pollak, Daniel J.
Chawla, Gautam
Andreev, Andrey
Prober, David A.
First steps into the cloud: Using Amazon data storage and computing with Python notebooks
title First steps into the cloud: Using Amazon data storage and computing with Python notebooks
title_full First steps into the cloud: Using Amazon data storage and computing with Python notebooks
title_fullStr First steps into the cloud: Using Amazon data storage and computing with Python notebooks
title_full_unstemmed First steps into the cloud: Using Amazon data storage and computing with Python notebooks
title_short First steps into the cloud: Using Amazon data storage and computing with Python notebooks
title_sort first steps into the cloud: using amazon data storage and computing with python notebooks
topic Lab Protocol
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9910747/
https://www.ncbi.nlm.nih.gov/pubmed/36757918
http://dx.doi.org/10.1371/journal.pone.0278316
work_keys_str_mv AT pollakdanielj firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks
AT chawlagautam firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks
AT andreevandrey firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks
AT proberdavida firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks