In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workers

We propose a novel three-stage FIND-RESOLVE-LABEL workflow for crowdsourced annotation to reduce ambiguity in task instructions and, thus, improve annotation quality. Stage 1 (FIND) asks the crowd to find examples whose correct label seems ambiguous given task instructions. Workers are also asked to...

Descripción completa

Detalles Bibliográficos
Autores principales: Pradhan, Vivek Krishna, Schaekermann, Mike, Lease, Matthew
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Artificial Intelligence
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9159300/
https://www.ncbi.nlm.nih.gov/pubmed/35664506
http://dx.doi.org/10.3389/frai.2022.828187

Internet

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9159300/
https://www.ncbi.nlm.nih.gov/pubmed/35664506
http://dx.doi.org/10.3389/frai.2022.828187

Ejemplares similares