Cargando…
Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles
The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from “Strongly Disagree” to “Strongly Agree”. Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e....
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10672242/ https://www.ncbi.nlm.nih.gov/pubmed/37998715 http://dx.doi.org/10.3390/jintelligence11110216 |
_version_ | 1785149505362984960 |
---|---|
author | Quirk, Victoria L. Kern, Justin L. |
author_facet | Quirk, Victoria L. Kern, Justin L. |
author_sort | Quirk, Victoria L. |
collection | PubMed |
description | The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from “Strongly Disagree” to “Strongly Agree”. Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). In this study, we compare the selection validity and adverse impact consequences of noise traits on selection when scores are estimated using a generalized partial credit model (GPCM) or an IRTree model. First, we present a simulation which demonstrates that when noise traits do exist, the selection decisions made based on the IRTree model estimated scores have higher accuracy rates and have less instances of adverse impact based on extreme response style group membership when compared to the GPCM. Both models performed similarly when there was no influence of noise traits on the responses. Second, we present an application using data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found that the IRTree model had a better fit, but a high agreement rate between the model decisions resulted in virtually identical impact ratios between the models. We offer considerations for applications of the IRTree model and future directions for research. |
format | Online Article Text |
id | pubmed-10672242 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-106722422023-11-17 Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles Quirk, Victoria L. Kern, Justin L. J Intell Article The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from “Strongly Disagree” to “Strongly Agree”. Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). In this study, we compare the selection validity and adverse impact consequences of noise traits on selection when scores are estimated using a generalized partial credit model (GPCM) or an IRTree model. First, we present a simulation which demonstrates that when noise traits do exist, the selection decisions made based on the IRTree model estimated scores have higher accuracy rates and have less instances of adverse impact based on extreme response style group membership when compared to the GPCM. Both models performed similarly when there was no influence of noise traits on the responses. Second, we present an application using data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found that the IRTree model had a better fit, but a high agreement rate between the model decisions resulted in virtually identical impact ratios between the models. We offer considerations for applications of the IRTree model and future directions for research. MDPI 2023-11-17 /pmc/articles/PMC10672242/ /pubmed/37998715 http://dx.doi.org/10.3390/jintelligence11110216 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Quirk, Victoria L. Kern, Justin L. Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles |
title | Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles |
title_full | Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles |
title_fullStr | Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles |
title_full_unstemmed | Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles |
title_short | Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles |
title_sort | using irtree models to promote selection validity in the presence of extreme response styles |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10672242/ https://www.ncbi.nlm.nih.gov/pubmed/37998715 http://dx.doi.org/10.3390/jintelligence11110216 |
work_keys_str_mv | AT quirkvictorial usingirtreemodelstopromoteselectionvalidityinthepresenceofextremeresponsestyles AT kernjustinl usingirtreemodelstopromoteselectionvalidityinthepresenceofextremeresponsestyles |