Cargando…
A Crowdsourcing Framework for Medical Data Sets
Crowdsourcing services like Amazon Mechanical Turk allow researchers to ask questions to crowds of workers and quickly receive high quality labeled responses. However, crowds drawn from the general public are not suitable for labeling sensitive and complex data sets, such as medical records, due to...
Autores principales: | , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
American Medical Informatics Association
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5961774/ https://www.ncbi.nlm.nih.gov/pubmed/29888085 |
_version_ | 1783324777379790848 |
---|---|
author | Ye, Cheng Coco, Joseph Epishova, Anna Hajaj, Chen Bogardus, Henry Novak, Laurie Denny, Joshua Vorobeychik, Yevgeniy Lasko, Thomas Malin, Bradley Fabbri, Daniel |
author_facet | Ye, Cheng Coco, Joseph Epishova, Anna Hajaj, Chen Bogardus, Henry Novak, Laurie Denny, Joshua Vorobeychik, Yevgeniy Lasko, Thomas Malin, Bradley Fabbri, Daniel |
author_sort | Ye, Cheng |
collection | PubMed |
description | Crowdsourcing services like Amazon Mechanical Turk allow researchers to ask questions to crowds of workers and quickly receive high quality labeled responses. However, crowds drawn from the general public are not suitable for labeling sensitive and complex data sets, such as medical records, due to various concerns. Major challenges in building and deploying a crowdsourcing system for medical data include, but are not limited to: managing access rights to sensitive data and ensuring data privacy controls are enforced; identifying workers with the necessary expertise to analyze complex information; and efficiently retrieving relevant information in massive data sets. In this paper, we introduce a crowdsourcing framework to support the annotation of medical data sets. We further demonstrate a workflow for crowdsourcing clinical chart reviews including (1) the design and decomposition of research questions; (2) the architecture for storing and displaying sensitive data; and (3) the development of tools to support crowd workers in quickly analyzing information from complex data sets. |
format | Online Article Text |
id | pubmed-5961774 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | American Medical Informatics Association |
record_format | MEDLINE/PubMed |
spelling | pubmed-59617742018-06-08 A Crowdsourcing Framework for Medical Data Sets Ye, Cheng Coco, Joseph Epishova, Anna Hajaj, Chen Bogardus, Henry Novak, Laurie Denny, Joshua Vorobeychik, Yevgeniy Lasko, Thomas Malin, Bradley Fabbri, Daniel AMIA Jt Summits Transl Sci Proc Articles Crowdsourcing services like Amazon Mechanical Turk allow researchers to ask questions to crowds of workers and quickly receive high quality labeled responses. However, crowds drawn from the general public are not suitable for labeling sensitive and complex data sets, such as medical records, due to various concerns. Major challenges in building and deploying a crowdsourcing system for medical data include, but are not limited to: managing access rights to sensitive data and ensuring data privacy controls are enforced; identifying workers with the necessary expertise to analyze complex information; and efficiently retrieving relevant information in massive data sets. In this paper, we introduce a crowdsourcing framework to support the annotation of medical data sets. We further demonstrate a workflow for crowdsourcing clinical chart reviews including (1) the design and decomposition of research questions; (2) the architecture for storing and displaying sensitive data; and (3) the development of tools to support crowd workers in quickly analyzing information from complex data sets. American Medical Informatics Association 2018-05-18 /pmc/articles/PMC5961774/ /pubmed/29888085 Text en ©2018 AMIA - All rights reserved. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose |
spellingShingle | Articles Ye, Cheng Coco, Joseph Epishova, Anna Hajaj, Chen Bogardus, Henry Novak, Laurie Denny, Joshua Vorobeychik, Yevgeniy Lasko, Thomas Malin, Bradley Fabbri, Daniel A Crowdsourcing Framework for Medical Data Sets |
title | A Crowdsourcing Framework for Medical Data Sets |
title_full | A Crowdsourcing Framework for Medical Data Sets |
title_fullStr | A Crowdsourcing Framework for Medical Data Sets |
title_full_unstemmed | A Crowdsourcing Framework for Medical Data Sets |
title_short | A Crowdsourcing Framework for Medical Data Sets |
title_sort | crowdsourcing framework for medical data sets |
topic | Articles |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5961774/ https://www.ncbi.nlm.nih.gov/pubmed/29888085 |
work_keys_str_mv | AT yecheng acrowdsourcingframeworkformedicaldatasets AT cocojoseph acrowdsourcingframeworkformedicaldatasets AT epishovaanna acrowdsourcingframeworkformedicaldatasets AT hajajchen acrowdsourcingframeworkformedicaldatasets AT bogardushenry acrowdsourcingframeworkformedicaldatasets AT novaklaurie acrowdsourcingframeworkformedicaldatasets AT dennyjoshua acrowdsourcingframeworkformedicaldatasets AT vorobeychikyevgeniy acrowdsourcingframeworkformedicaldatasets AT laskothomas acrowdsourcingframeworkformedicaldatasets AT malinbradley acrowdsourcingframeworkformedicaldatasets AT fabbridaniel acrowdsourcingframeworkformedicaldatasets AT yecheng crowdsourcingframeworkformedicaldatasets AT cocojoseph crowdsourcingframeworkformedicaldatasets AT epishovaanna crowdsourcingframeworkformedicaldatasets AT hajajchen crowdsourcingframeworkformedicaldatasets AT bogardushenry crowdsourcingframeworkformedicaldatasets AT novaklaurie crowdsourcingframeworkformedicaldatasets AT dennyjoshua crowdsourcingframeworkformedicaldatasets AT vorobeychikyevgeniy crowdsourcingframeworkformedicaldatasets AT laskothomas crowdsourcingframeworkformedicaldatasets AT malinbradley crowdsourcingframeworkformedicaldatasets AT fabbridaniel crowdsourcingframeworkformedicaldatasets |