Cargando…
Scalability Issues for Remote Sensing Infrastructure: A Case Study
For the past decade, a team of University of Calgary researchers has operated a large “sensor Web” to collect, analyze, and share scientific data from remote measurement instruments across northern Canada. This sensor Web receives real-time data streams from over a thousand Internet-connected sensor...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5469347/ https://www.ncbi.nlm.nih.gov/pubmed/28468262 http://dx.doi.org/10.3390/s17050994 |
_version_ | 1783243566744600576 |
---|---|
author | Liu, Yang Picard, Sean Williamson, Carey |
author_facet | Liu, Yang Picard, Sean Williamson, Carey |
author_sort | Liu, Yang |
collection | PubMed |
description | For the past decade, a team of University of Calgary researchers has operated a large “sensor Web” to collect, analyze, and share scientific data from remote measurement instruments across northern Canada. This sensor Web receives real-time data streams from over a thousand Internet-connected sensors, with a particular emphasis on environmental data (e.g., space weather, auroral phenomena, atmospheric imaging). Through research collaborations, we had the opportunity to evaluate the performance and scalability of their remote sensing infrastructure. This article reports the lessons learned from our study, which considered both data collection and data dissemination aspects of their system. On the data collection front, we used benchmarking techniques to identify and fix a performance bottleneck in the system’s memory management for TCP data streams, while also improving system efficiency on multi-core architectures. On the data dissemination front, we used passive and active network traffic measurements to identify and reduce excessive network traffic from the Web robots and JavaScript techniques used for data sharing. While our results are from one specific sensor Web system, the lessons learned may apply to other scientific Web sites with remote sensing infrastructure. |
format | Online Article Text |
id | pubmed-5469347 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-54693472017-06-16 Scalability Issues for Remote Sensing Infrastructure: A Case Study Liu, Yang Picard, Sean Williamson, Carey Sensors (Basel) Article For the past decade, a team of University of Calgary researchers has operated a large “sensor Web” to collect, analyze, and share scientific data from remote measurement instruments across northern Canada. This sensor Web receives real-time data streams from over a thousand Internet-connected sensors, with a particular emphasis on environmental data (e.g., space weather, auroral phenomena, atmospheric imaging). Through research collaborations, we had the opportunity to evaluate the performance and scalability of their remote sensing infrastructure. This article reports the lessons learned from our study, which considered both data collection and data dissemination aspects of their system. On the data collection front, we used benchmarking techniques to identify and fix a performance bottleneck in the system’s memory management for TCP data streams, while also improving system efficiency on multi-core architectures. On the data dissemination front, we used passive and active network traffic measurements to identify and reduce excessive network traffic from the Web robots and JavaScript techniques used for data sharing. While our results are from one specific sensor Web system, the lessons learned may apply to other scientific Web sites with remote sensing infrastructure. MDPI 2017-04-29 /pmc/articles/PMC5469347/ /pubmed/28468262 http://dx.doi.org/10.3390/s17050994 Text en © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Liu, Yang Picard, Sean Williamson, Carey Scalability Issues for Remote Sensing Infrastructure: A Case Study |
title | Scalability Issues for Remote Sensing Infrastructure: A Case Study |
title_full | Scalability Issues for Remote Sensing Infrastructure: A Case Study |
title_fullStr | Scalability Issues for Remote Sensing Infrastructure: A Case Study |
title_full_unstemmed | Scalability Issues for Remote Sensing Infrastructure: A Case Study |
title_short | Scalability Issues for Remote Sensing Infrastructure: A Case Study |
title_sort | scalability issues for remote sensing infrastructure: a case study |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5469347/ https://www.ncbi.nlm.nih.gov/pubmed/28468262 http://dx.doi.org/10.3390/s17050994 |
work_keys_str_mv | AT liuyang scalabilityissuesforremotesensinginfrastructureacasestudy AT picardsean scalabilityissuesforremotesensinginfrastructureacasestudy AT williamsoncarey scalabilityissuesforremotesensinginfrastructureacasestudy |