Cargando…

A Crowdsourcing Framework for Medical Data Sets

Crowdsourcing services like Amazon Mechanical Turk allow researchers to ask questions to crowds of workers and quickly receive high quality labeled responses. However, crowds drawn from the general public are not suitable for labeling sensitive and complex data sets, such as medical records, due to...

Descripción completa

Detalles Bibliográficos
Autores principales: Ye, Cheng, Coco, Joseph, Epishova, Anna, Hajaj, Chen, Bogardus, Henry, Novak, Laurie, Denny, Joshua, Vorobeychik, Yevgeniy, Lasko, Thomas, Malin, Bradley, Fabbri, Daniel
Formato: Online Artículo Texto
Lenguaje:English
Publicado: American Medical Informatics Association 2018
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5961774/
https://www.ncbi.nlm.nih.gov/pubmed/29888085
_version_ 1783324777379790848
author Ye, Cheng
Coco, Joseph
Epishova, Anna
Hajaj, Chen
Bogardus, Henry
Novak, Laurie
Denny, Joshua
Vorobeychik, Yevgeniy
Lasko, Thomas
Malin, Bradley
Fabbri, Daniel
author_facet Ye, Cheng
Coco, Joseph
Epishova, Anna
Hajaj, Chen
Bogardus, Henry
Novak, Laurie
Denny, Joshua
Vorobeychik, Yevgeniy
Lasko, Thomas
Malin, Bradley
Fabbri, Daniel
author_sort Ye, Cheng
collection PubMed
description Crowdsourcing services like Amazon Mechanical Turk allow researchers to ask questions to crowds of workers and quickly receive high quality labeled responses. However, crowds drawn from the general public are not suitable for labeling sensitive and complex data sets, such as medical records, due to various concerns. Major challenges in building and deploying a crowdsourcing system for medical data include, but are not limited to: managing access rights to sensitive data and ensuring data privacy controls are enforced; identifying workers with the necessary expertise to analyze complex information; and efficiently retrieving relevant information in massive data sets. In this paper, we introduce a crowdsourcing framework to support the annotation of medical data sets. We further demonstrate a workflow for crowdsourcing clinical chart reviews including (1) the design and decomposition of research questions; (2) the architecture for storing and displaying sensitive data; and (3) the development of tools to support crowd workers in quickly analyzing information from complex data sets.
format Online
Article
Text
id pubmed-5961774
institution National Center for Biotechnology Information
language English
publishDate 2018
publisher American Medical Informatics Association
record_format MEDLINE/PubMed
spelling pubmed-59617742018-06-08 A Crowdsourcing Framework for Medical Data Sets Ye, Cheng Coco, Joseph Epishova, Anna Hajaj, Chen Bogardus, Henry Novak, Laurie Denny, Joshua Vorobeychik, Yevgeniy Lasko, Thomas Malin, Bradley Fabbri, Daniel AMIA Jt Summits Transl Sci Proc Articles Crowdsourcing services like Amazon Mechanical Turk allow researchers to ask questions to crowds of workers and quickly receive high quality labeled responses. However, crowds drawn from the general public are not suitable for labeling sensitive and complex data sets, such as medical records, due to various concerns. Major challenges in building and deploying a crowdsourcing system for medical data include, but are not limited to: managing access rights to sensitive data and ensuring data privacy controls are enforced; identifying workers with the necessary expertise to analyze complex information; and efficiently retrieving relevant information in massive data sets. In this paper, we introduce a crowdsourcing framework to support the annotation of medical data sets. We further demonstrate a workflow for crowdsourcing clinical chart reviews including (1) the design and decomposition of research questions; (2) the architecture for storing and displaying sensitive data; and (3) the development of tools to support crowd workers in quickly analyzing information from complex data sets. American Medical Informatics Association 2018-05-18 /pmc/articles/PMC5961774/ /pubmed/29888085 Text en ©2018 AMIA - All rights reserved. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose
spellingShingle Articles
Ye, Cheng
Coco, Joseph
Epishova, Anna
Hajaj, Chen
Bogardus, Henry
Novak, Laurie
Denny, Joshua
Vorobeychik, Yevgeniy
Lasko, Thomas
Malin, Bradley
Fabbri, Daniel
A Crowdsourcing Framework for Medical Data Sets
title A Crowdsourcing Framework for Medical Data Sets
title_full A Crowdsourcing Framework for Medical Data Sets
title_fullStr A Crowdsourcing Framework for Medical Data Sets
title_full_unstemmed A Crowdsourcing Framework for Medical Data Sets
title_short A Crowdsourcing Framework for Medical Data Sets
title_sort crowdsourcing framework for medical data sets
topic Articles
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5961774/
https://www.ncbi.nlm.nih.gov/pubmed/29888085
work_keys_str_mv AT yecheng acrowdsourcingframeworkformedicaldatasets
AT cocojoseph acrowdsourcingframeworkformedicaldatasets
AT epishovaanna acrowdsourcingframeworkformedicaldatasets
AT hajajchen acrowdsourcingframeworkformedicaldatasets
AT bogardushenry acrowdsourcingframeworkformedicaldatasets
AT novaklaurie acrowdsourcingframeworkformedicaldatasets
AT dennyjoshua acrowdsourcingframeworkformedicaldatasets
AT vorobeychikyevgeniy acrowdsourcingframeworkformedicaldatasets
AT laskothomas acrowdsourcingframeworkformedicaldatasets
AT malinbradley acrowdsourcingframeworkformedicaldatasets
AT fabbridaniel acrowdsourcingframeworkformedicaldatasets
AT yecheng crowdsourcingframeworkformedicaldatasets
AT cocojoseph crowdsourcingframeworkformedicaldatasets
AT epishovaanna crowdsourcingframeworkformedicaldatasets
AT hajajchen crowdsourcingframeworkformedicaldatasets
AT bogardushenry crowdsourcingframeworkformedicaldatasets
AT novaklaurie crowdsourcingframeworkformedicaldatasets
AT dennyjoshua crowdsourcingframeworkformedicaldatasets
AT vorobeychikyevgeniy crowdsourcingframeworkformedicaldatasets
AT laskothomas crowdsourcingframeworkformedicaldatasets
AT malinbradley crowdsourcingframeworkformedicaldatasets
AT fabbridaniel crowdsourcingframeworkformedicaldatasets