Cargando…

Scalability Issues for Remote Sensing Infrastructure: A Case Study

For the past decade, a team of University of Calgary researchers has operated a large “sensor Web” to collect, analyze, and share scientific data from remote measurement instruments across northern Canada. This sensor Web receives real-time data streams from over a thousand Internet-connected sensor...

Descripción completa

Detalles Bibliográficos
Autores principales: Liu, Yang, Picard, Sean, Williamson, Carey
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5469347/
https://www.ncbi.nlm.nih.gov/pubmed/28468262
http://dx.doi.org/10.3390/s17050994
_version_ 1783243566744600576
author Liu, Yang
Picard, Sean
Williamson, Carey
author_facet Liu, Yang
Picard, Sean
Williamson, Carey
author_sort Liu, Yang
collection PubMed
description For the past decade, a team of University of Calgary researchers has operated a large “sensor Web” to collect, analyze, and share scientific data from remote measurement instruments across northern Canada. This sensor Web receives real-time data streams from over a thousand Internet-connected sensors, with a particular emphasis on environmental data (e.g., space weather, auroral phenomena, atmospheric imaging). Through research collaborations, we had the opportunity to evaluate the performance and scalability of their remote sensing infrastructure. This article reports the lessons learned from our study, which considered both data collection and data dissemination aspects of their system. On the data collection front, we used benchmarking techniques to identify and fix a performance bottleneck in the system’s memory management for TCP data streams, while also improving system efficiency on multi-core architectures. On the data dissemination front, we used passive and active network traffic measurements to identify and reduce excessive network traffic from the Web robots and JavaScript techniques used for data sharing. While our results are from one specific sensor Web system, the lessons learned may apply to other scientific Web sites with remote sensing infrastructure.
format Online
Article
Text
id pubmed-5469347
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-54693472017-06-16 Scalability Issues for Remote Sensing Infrastructure: A Case Study Liu, Yang Picard, Sean Williamson, Carey Sensors (Basel) Article For the past decade, a team of University of Calgary researchers has operated a large “sensor Web” to collect, analyze, and share scientific data from remote measurement instruments across northern Canada. This sensor Web receives real-time data streams from over a thousand Internet-connected sensors, with a particular emphasis on environmental data (e.g., space weather, auroral phenomena, atmospheric imaging). Through research collaborations, we had the opportunity to evaluate the performance and scalability of their remote sensing infrastructure. This article reports the lessons learned from our study, which considered both data collection and data dissemination aspects of their system. On the data collection front, we used benchmarking techniques to identify and fix a performance bottleneck in the system’s memory management for TCP data streams, while also improving system efficiency on multi-core architectures. On the data dissemination front, we used passive and active network traffic measurements to identify and reduce excessive network traffic from the Web robots and JavaScript techniques used for data sharing. While our results are from one specific sensor Web system, the lessons learned may apply to other scientific Web sites with remote sensing infrastructure. MDPI 2017-04-29 /pmc/articles/PMC5469347/ /pubmed/28468262 http://dx.doi.org/10.3390/s17050994 Text en © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Liu, Yang
Picard, Sean
Williamson, Carey
Scalability Issues for Remote Sensing Infrastructure: A Case Study
title Scalability Issues for Remote Sensing Infrastructure: A Case Study
title_full Scalability Issues for Remote Sensing Infrastructure: A Case Study
title_fullStr Scalability Issues for Remote Sensing Infrastructure: A Case Study
title_full_unstemmed Scalability Issues for Remote Sensing Infrastructure: A Case Study
title_short Scalability Issues for Remote Sensing Infrastructure: A Case Study
title_sort scalability issues for remote sensing infrastructure: a case study
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5469347/
https://www.ncbi.nlm.nih.gov/pubmed/28468262
http://dx.doi.org/10.3390/s17050994
work_keys_str_mv AT liuyang scalabilityissuesforremotesensinginfrastructureacasestudy
AT picardsean scalabilityissuesforremotesensinginfrastructureacasestudy
AT williamsoncarey scalabilityissuesforremotesensinginfrastructureacasestudy