Cargando…
The promise of open survey questions—The validation of text-based job satisfaction measures
Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psych...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6932814/ https://www.ncbi.nlm.nih.gov/pubmed/31877156 http://dx.doi.org/10.1371/journal.pone.0226408 |
_version_ | 1783483084717424640 |
---|---|
author | Wijngaards, Indy Burger, Martijn van Exel, Job |
author_facet | Wijngaards, Indy Burger, Martijn van Exel, Job |
author_sort | Wijngaards, Indy |
collection | PubMed |
description | Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psychological constructs. In our study, we demonstrated the potential of CATA methods for the construction of text-based job satisfaction measures based on responses to a completely open and semi-open question. To do this, we employed three sentiment analysis techniques: Linguistic Inquiry and Word Count 2015, SentimentR and SentiStrength, and quantified the forms of measurement error they introduced: specific factor error, algorithm error and transient error. We conducted an initial test of the text-based measures’ validity, assessing their convergence with closed-question job satisfaction measures. We adopted a time-lagged survey design (N(wave 1) = 996; N(wave 2) = 116) to test our hypotheses. In line with our hypotheses, we found that specific factor error is higher in the open question text-based measure than in the semi-open question text-based measure. As expected, algorithm error was substantial for both the open and semi-open question text-based measures. Transient error in the text-based measures was higher than expected, as it generally exceeded the transient error in the human-coded and the closed job satisfaction question measures. Our initial test of convergent and discriminant validity indicated that the semi-open question text-based measure is especially suitable for measuring job satisfaction. Our article ends with a discussion of limitations and an agenda for future research. |
format | Online Article Text |
id | pubmed-6932814 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-69328142020-01-07 The promise of open survey questions—The validation of text-based job satisfaction measures Wijngaards, Indy Burger, Martijn van Exel, Job PLoS One Research Article Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psychological constructs. In our study, we demonstrated the potential of CATA methods for the construction of text-based job satisfaction measures based on responses to a completely open and semi-open question. To do this, we employed three sentiment analysis techniques: Linguistic Inquiry and Word Count 2015, SentimentR and SentiStrength, and quantified the forms of measurement error they introduced: specific factor error, algorithm error and transient error. We conducted an initial test of the text-based measures’ validity, assessing their convergence with closed-question job satisfaction measures. We adopted a time-lagged survey design (N(wave 1) = 996; N(wave 2) = 116) to test our hypotheses. In line with our hypotheses, we found that specific factor error is higher in the open question text-based measure than in the semi-open question text-based measure. As expected, algorithm error was substantial for both the open and semi-open question text-based measures. Transient error in the text-based measures was higher than expected, as it generally exceeded the transient error in the human-coded and the closed job satisfaction question measures. Our initial test of convergent and discriminant validity indicated that the semi-open question text-based measure is especially suitable for measuring job satisfaction. Our article ends with a discussion of limitations and an agenda for future research. Public Library of Science 2019-12-26 /pmc/articles/PMC6932814/ /pubmed/31877156 http://dx.doi.org/10.1371/journal.pone.0226408 Text en © 2019 Wijngaards et al http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Wijngaards, Indy Burger, Martijn van Exel, Job The promise of open survey questions—The validation of text-based job satisfaction measures |
title | The promise of open survey questions—The validation of text-based job satisfaction measures |
title_full | The promise of open survey questions—The validation of text-based job satisfaction measures |
title_fullStr | The promise of open survey questions—The validation of text-based job satisfaction measures |
title_full_unstemmed | The promise of open survey questions—The validation of text-based job satisfaction measures |
title_short | The promise of open survey questions—The validation of text-based job satisfaction measures |
title_sort | promise of open survey questions—the validation of text-based job satisfaction measures |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6932814/ https://www.ncbi.nlm.nih.gov/pubmed/31877156 http://dx.doi.org/10.1371/journal.pone.0226408 |
work_keys_str_mv | AT wijngaardsindy thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT burgermartijn thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT vanexeljob thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT wijngaardsindy promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT burgermartijn promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT vanexeljob promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures |