Cargando…

The promise of open survey questions—The validation of text-based job satisfaction measures

Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psych...

Descripción completa

Detalles Bibliográficos
Autores principales: Wijngaards, Indy, Burger, Martijn, van Exel, Job
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6932814/
https://www.ncbi.nlm.nih.gov/pubmed/31877156
http://dx.doi.org/10.1371/journal.pone.0226408
_version_ 1783483084717424640
author Wijngaards, Indy
Burger, Martijn
van Exel, Job
author_facet Wijngaards, Indy
Burger, Martijn
van Exel, Job
author_sort Wijngaards, Indy
collection PubMed
description Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psychological constructs. In our study, we demonstrated the potential of CATA methods for the construction of text-based job satisfaction measures based on responses to a completely open and semi-open question. To do this, we employed three sentiment analysis techniques: Linguistic Inquiry and Word Count 2015, SentimentR and SentiStrength, and quantified the forms of measurement error they introduced: specific factor error, algorithm error and transient error. We conducted an initial test of the text-based measures’ validity, assessing their convergence with closed-question job satisfaction measures. We adopted a time-lagged survey design (N(wave 1) = 996; N(wave 2) = 116) to test our hypotheses. In line with our hypotheses, we found that specific factor error is higher in the open question text-based measure than in the semi-open question text-based measure. As expected, algorithm error was substantial for both the open and semi-open question text-based measures. Transient error in the text-based measures was higher than expected, as it generally exceeded the transient error in the human-coded and the closed job satisfaction question measures. Our initial test of convergent and discriminant validity indicated that the semi-open question text-based measure is especially suitable for measuring job satisfaction. Our article ends with a discussion of limitations and an agenda for future research.
format Online
Article
Text
id pubmed-6932814
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-69328142020-01-07 The promise of open survey questions—The validation of text-based job satisfaction measures Wijngaards, Indy Burger, Martijn van Exel, Job PLoS One Research Article Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psychological constructs. In our study, we demonstrated the potential of CATA methods for the construction of text-based job satisfaction measures based on responses to a completely open and semi-open question. To do this, we employed three sentiment analysis techniques: Linguistic Inquiry and Word Count 2015, SentimentR and SentiStrength, and quantified the forms of measurement error they introduced: specific factor error, algorithm error and transient error. We conducted an initial test of the text-based measures’ validity, assessing their convergence with closed-question job satisfaction measures. We adopted a time-lagged survey design (N(wave 1) = 996; N(wave 2) = 116) to test our hypotheses. In line with our hypotheses, we found that specific factor error is higher in the open question text-based measure than in the semi-open question text-based measure. As expected, algorithm error was substantial for both the open and semi-open question text-based measures. Transient error in the text-based measures was higher than expected, as it generally exceeded the transient error in the human-coded and the closed job satisfaction question measures. Our initial test of convergent and discriminant validity indicated that the semi-open question text-based measure is especially suitable for measuring job satisfaction. Our article ends with a discussion of limitations and an agenda for future research. Public Library of Science 2019-12-26 /pmc/articles/PMC6932814/ /pubmed/31877156 http://dx.doi.org/10.1371/journal.pone.0226408 Text en © 2019 Wijngaards et al http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Wijngaards, Indy
Burger, Martijn
van Exel, Job
The promise of open survey questions—The validation of text-based job satisfaction measures
title The promise of open survey questions—The validation of text-based job satisfaction measures
title_full The promise of open survey questions—The validation of text-based job satisfaction measures
title_fullStr The promise of open survey questions—The validation of text-based job satisfaction measures
title_full_unstemmed The promise of open survey questions—The validation of text-based job satisfaction measures
title_short The promise of open survey questions—The validation of text-based job satisfaction measures
title_sort promise of open survey questions—the validation of text-based job satisfaction measures
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6932814/
https://www.ncbi.nlm.nih.gov/pubmed/31877156
http://dx.doi.org/10.1371/journal.pone.0226408
work_keys_str_mv AT wijngaardsindy thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures
AT burgermartijn thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures
AT vanexeljob thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures
AT wijngaardsindy promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures
AT burgermartijn promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures
AT vanexeljob promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures