Cargando…

Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles

The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from “Strongly Disagree” to “Strongly Agree”. Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e....

Descripción completa

Detalles Bibliográficos
Autores principales: Quirk, Victoria L., Kern, Justin L.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10672242/
https://www.ncbi.nlm.nih.gov/pubmed/37998715
http://dx.doi.org/10.3390/jintelligence11110216
_version_ 1785149505362984960
author Quirk, Victoria L.
Kern, Justin L.
author_facet Quirk, Victoria L.
Kern, Justin L.
author_sort Quirk, Victoria L.
collection PubMed
description The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from “Strongly Disagree” to “Strongly Agree”. Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). In this study, we compare the selection validity and adverse impact consequences of noise traits on selection when scores are estimated using a generalized partial credit model (GPCM) or an IRTree model. First, we present a simulation which demonstrates that when noise traits do exist, the selection decisions made based on the IRTree model estimated scores have higher accuracy rates and have less instances of adverse impact based on extreme response style group membership when compared to the GPCM. Both models performed similarly when there was no influence of noise traits on the responses. Second, we present an application using data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found that the IRTree model had a better fit, but a high agreement rate between the model decisions resulted in virtually identical impact ratios between the models. We offer considerations for applications of the IRTree model and future directions for research.
format Online
Article
Text
id pubmed-10672242
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-106722422023-11-17 Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles Quirk, Victoria L. Kern, Justin L. J Intell Article The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from “Strongly Disagree” to “Strongly Agree”. Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). In this study, we compare the selection validity and adverse impact consequences of noise traits on selection when scores are estimated using a generalized partial credit model (GPCM) or an IRTree model. First, we present a simulation which demonstrates that when noise traits do exist, the selection decisions made based on the IRTree model estimated scores have higher accuracy rates and have less instances of adverse impact based on extreme response style group membership when compared to the GPCM. Both models performed similarly when there was no influence of noise traits on the responses. Second, we present an application using data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found that the IRTree model had a better fit, but a high agreement rate between the model decisions resulted in virtually identical impact ratios between the models. We offer considerations for applications of the IRTree model and future directions for research. MDPI 2023-11-17 /pmc/articles/PMC10672242/ /pubmed/37998715 http://dx.doi.org/10.3390/jintelligence11110216 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Quirk, Victoria L.
Kern, Justin L.
Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles
title Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles
title_full Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles
title_fullStr Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles
title_full_unstemmed Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles
title_short Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles
title_sort using irtree models to promote selection validity in the presence of extreme response styles
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10672242/
https://www.ncbi.nlm.nih.gov/pubmed/37998715
http://dx.doi.org/10.3390/jintelligence11110216
work_keys_str_mv AT quirkvictorial usingirtreemodelstopromoteselectionvalidityinthepresenceofextremeresponsestyles
AT kernjustinl usingirtreemodelstopromoteselectionvalidityinthepresenceofextremeresponsestyles