Cargando…
Proposal for a Five-Step Method to Elicit Expert Judgment
Elicitation is a commonly used tool to extract viable information from experts. The information that is held by the expert is extracted and a probabilistic representation of this knowledge is constructed. A promising avenue in psychological research is to incorporated experts’ prior knowledge in the...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5723340/ https://www.ncbi.nlm.nih.gov/pubmed/29259569 http://dx.doi.org/10.3389/fpsyg.2017.02110 |
_version_ | 1783285191578484736 |
---|---|
author | Veen, Duco Stoel, Diederick Zondervan-Zwijnenburg, Mariëlle van de Schoot, Rens |
author_facet | Veen, Duco Stoel, Diederick Zondervan-Zwijnenburg, Mariëlle van de Schoot, Rens |
author_sort | Veen, Duco |
collection | PubMed |
description | Elicitation is a commonly used tool to extract viable information from experts. The information that is held by the expert is extracted and a probabilistic representation of this knowledge is constructed. A promising avenue in psychological research is to incorporated experts’ prior knowledge in the statistical analysis. Systematic reviews on elicitation literature however suggest that it might be inappropriate to directly obtain distributional representations from experts. The literature qualifies experts’ performance on estimating elements of a distribution as unsatisfactory, thus reliably specifying the essential elements of the parameters of interest in one elicitation step seems implausible. Providing feedback within the elicitation process can enhance the quality of the elicitation and interactive software can be used to facilitate the feedback. Therefore, we propose to decompose the elicitation procedure into smaller steps with adjustable outcomes. We represent the tacit knowledge of experts as a location parameter and their uncertainty concerning this knowledge by a scale and shape parameter. Using a feedback procedure, experts can accept the representation of their beliefs or adjust their input. We propose a Five-Step Method which consists of (1) Eliciting the location parameter using the trial roulette method. (2) Provide feedback on the location parameter and ask for confirmation or adjustment. (3) Elicit the scale and shape parameter. (4) Provide feedback on the scale and shape parameter and ask for confirmation or adjustment. (5) Use the elicited and calibrated probability distribution in a statistical analysis and update it with data or to compute a prior-data conflict within a Bayesian framework. User feasibility and internal validity for the Five-Step Method are investigated using three elicitation studies. |
format | Online Article Text |
id | pubmed-5723340 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-57233402017-12-19 Proposal for a Five-Step Method to Elicit Expert Judgment Veen, Duco Stoel, Diederick Zondervan-Zwijnenburg, Mariëlle van de Schoot, Rens Front Psychol Psychology Elicitation is a commonly used tool to extract viable information from experts. The information that is held by the expert is extracted and a probabilistic representation of this knowledge is constructed. A promising avenue in psychological research is to incorporated experts’ prior knowledge in the statistical analysis. Systematic reviews on elicitation literature however suggest that it might be inappropriate to directly obtain distributional representations from experts. The literature qualifies experts’ performance on estimating elements of a distribution as unsatisfactory, thus reliably specifying the essential elements of the parameters of interest in one elicitation step seems implausible. Providing feedback within the elicitation process can enhance the quality of the elicitation and interactive software can be used to facilitate the feedback. Therefore, we propose to decompose the elicitation procedure into smaller steps with adjustable outcomes. We represent the tacit knowledge of experts as a location parameter and their uncertainty concerning this knowledge by a scale and shape parameter. Using a feedback procedure, experts can accept the representation of their beliefs or adjust their input. We propose a Five-Step Method which consists of (1) Eliciting the location parameter using the trial roulette method. (2) Provide feedback on the location parameter and ask for confirmation or adjustment. (3) Elicit the scale and shape parameter. (4) Provide feedback on the scale and shape parameter and ask for confirmation or adjustment. (5) Use the elicited and calibrated probability distribution in a statistical analysis and update it with data or to compute a prior-data conflict within a Bayesian framework. User feasibility and internal validity for the Five-Step Method are investigated using three elicitation studies. Frontiers Media S.A. 2017-12-05 /pmc/articles/PMC5723340/ /pubmed/29259569 http://dx.doi.org/10.3389/fpsyg.2017.02110 Text en Copyright © 2017 Veen, Stoel, Zondervan-Zwijnenburg and van de Schoot. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Psychology Veen, Duco Stoel, Diederick Zondervan-Zwijnenburg, Mariëlle van de Schoot, Rens Proposal for a Five-Step Method to Elicit Expert Judgment |
title | Proposal for a Five-Step Method to Elicit Expert Judgment |
title_full | Proposal for a Five-Step Method to Elicit Expert Judgment |
title_fullStr | Proposal for a Five-Step Method to Elicit Expert Judgment |
title_full_unstemmed | Proposal for a Five-Step Method to Elicit Expert Judgment |
title_short | Proposal for a Five-Step Method to Elicit Expert Judgment |
title_sort | proposal for a five-step method to elicit expert judgment |
topic | Psychology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5723340/ https://www.ncbi.nlm.nih.gov/pubmed/29259569 http://dx.doi.org/10.3389/fpsyg.2017.02110 |
work_keys_str_mv | AT veenduco proposalforafivestepmethodtoelicitexpertjudgment AT stoeldiederick proposalforafivestepmethodtoelicitexpertjudgment AT zondervanzwijnenburgmarielle proposalforafivestepmethodtoelicitexpertjudgment AT vandeschootrens proposalforafivestepmethodtoelicitexpertjudgment |