Cargando…
First steps into the cloud: Using Amazon data storage and computing with Python notebooks
With the oncoming age of big data, biologists are encountering more use cases for cloud-based computing to streamline data processing and storage. Unfortunately, cloud platforms are difficult to learn, and there are few resources for biologists to demystify them. We have developed a guide for experi...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9910747/ https://www.ncbi.nlm.nih.gov/pubmed/36757918 http://dx.doi.org/10.1371/journal.pone.0278316 |
_version_ | 1784884851335233536 |
---|---|
author | Pollak, Daniel J. Chawla, Gautam Andreev, Andrey Prober, David A. |
author_facet | Pollak, Daniel J. Chawla, Gautam Andreev, Andrey Prober, David A. |
author_sort | Pollak, Daniel J. |
collection | PubMed |
description | With the oncoming age of big data, biologists are encountering more use cases for cloud-based computing to streamline data processing and storage. Unfortunately, cloud platforms are difficult to learn, and there are few resources for biologists to demystify them. We have developed a guide for experimental biologists to set up cloud processing on Amazon Web Services to cheaply outsource data processing and storage. Here we provide a guide for setting up a computing environment in the cloud and showcase examples of using Python and Julia programming languages. We present example calcium imaging data in the zebrafish brain and corresponding analysis using suite2p software. Tools for budget and user management are further discussed in the attached protocol. Using this guide, researchers with limited coding experience can get started with cloud-based computing or move existing coding infrastructure into the cloud environment. |
format | Online Article Text |
id | pubmed-9910747 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-99107472023-02-10 First steps into the cloud: Using Amazon data storage and computing with Python notebooks Pollak, Daniel J. Chawla, Gautam Andreev, Andrey Prober, David A. PLoS One Lab Protocol With the oncoming age of big data, biologists are encountering more use cases for cloud-based computing to streamline data processing and storage. Unfortunately, cloud platforms are difficult to learn, and there are few resources for biologists to demystify them. We have developed a guide for experimental biologists to set up cloud processing on Amazon Web Services to cheaply outsource data processing and storage. Here we provide a guide for setting up a computing environment in the cloud and showcase examples of using Python and Julia programming languages. We present example calcium imaging data in the zebrafish brain and corresponding analysis using suite2p software. Tools for budget and user management are further discussed in the attached protocol. Using this guide, researchers with limited coding experience can get started with cloud-based computing or move existing coding infrastructure into the cloud environment. Public Library of Science 2023-02-09 /pmc/articles/PMC9910747/ /pubmed/36757918 http://dx.doi.org/10.1371/journal.pone.0278316 Text en © 2023 Pollak et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Lab Protocol Pollak, Daniel J. Chawla, Gautam Andreev, Andrey Prober, David A. First steps into the cloud: Using Amazon data storage and computing with Python notebooks |
title | First steps into the cloud: Using Amazon data storage and computing with Python notebooks |
title_full | First steps into the cloud: Using Amazon data storage and computing with Python notebooks |
title_fullStr | First steps into the cloud: Using Amazon data storage and computing with Python notebooks |
title_full_unstemmed | First steps into the cloud: Using Amazon data storage and computing with Python notebooks |
title_short | First steps into the cloud: Using Amazon data storage and computing with Python notebooks |
title_sort | first steps into the cloud: using amazon data storage and computing with python notebooks |
topic | Lab Protocol |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9910747/ https://www.ncbi.nlm.nih.gov/pubmed/36757918 http://dx.doi.org/10.1371/journal.pone.0278316 |
work_keys_str_mv | AT pollakdanielj firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks AT chawlagautam firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks AT andreevandrey firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks AT proberdavida firststepsintothecloudusingamazondatastorageandcomputingwithpythonnotebooks |