Cargando…
Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour
Some disciplines in the social sciences rely heavily on collecting survey responses to detect empirical relationships among variables. We explored whether these relationships were a priori predictable from the semantic properties of the survey items, using language processing algorithms which are no...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2014
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4153608/ https://www.ncbi.nlm.nih.gov/pubmed/25184672 http://dx.doi.org/10.1371/journal.pone.0106361 |
_version_ | 1782333308068167680 |
---|---|
author | Arnulf, Jan Ketil Larsen, Kai Rune Martinsen, Øyvind Lund Bong, Chih How |
author_facet | Arnulf, Jan Ketil Larsen, Kai Rune Martinsen, Øyvind Lund Bong, Chih How |
author_sort | Arnulf, Jan Ketil |
collection | PubMed |
description | Some disciplines in the social sciences rely heavily on collecting survey responses to detect empirical relationships among variables. We explored whether these relationships were a priori predictable from the semantic properties of the survey items, using language processing algorithms which are now available as new research methods. Language processing algorithms were used to calculate the semantic similarity among all items in state-of-the-art surveys from Organisational Behaviour research. These surveys covered areas such as transformational leadership, work motivation and work outcomes. This information was used to explain and predict the response patterns from real subjects. Semantic algorithms explained 60–86% of the variance in the response patterns and allowed remarkably precise prediction of survey responses from humans, except in a personality test. Even the relationships between independent and their purported dependent variables were accurately predicted. This raises concern about the empirical nature of data collected through some surveys if results are already given a priori through the way subjects are being asked. Survey response patterns seem heavily determined by semantics. Language algorithms may suggest these prior to administering a survey. This study suggests that semantic algorithms are becoming new tools for the social sciences, opening perspectives on survey responses that prevalent psychometric theory cannot explain. |
format | Online Article Text |
id | pubmed-4153608 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2014 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-41536082014-09-05 Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour Arnulf, Jan Ketil Larsen, Kai Rune Martinsen, Øyvind Lund Bong, Chih How PLoS One Research Article Some disciplines in the social sciences rely heavily on collecting survey responses to detect empirical relationships among variables. We explored whether these relationships were a priori predictable from the semantic properties of the survey items, using language processing algorithms which are now available as new research methods. Language processing algorithms were used to calculate the semantic similarity among all items in state-of-the-art surveys from Organisational Behaviour research. These surveys covered areas such as transformational leadership, work motivation and work outcomes. This information was used to explain and predict the response patterns from real subjects. Semantic algorithms explained 60–86% of the variance in the response patterns and allowed remarkably precise prediction of survey responses from humans, except in a personality test. Even the relationships between independent and their purported dependent variables were accurately predicted. This raises concern about the empirical nature of data collected through some surveys if results are already given a priori through the way subjects are being asked. Survey response patterns seem heavily determined by semantics. Language algorithms may suggest these prior to administering a survey. This study suggests that semantic algorithms are becoming new tools for the social sciences, opening perspectives on survey responses that prevalent psychometric theory cannot explain. Public Library of Science 2014-09-03 /pmc/articles/PMC4153608/ /pubmed/25184672 http://dx.doi.org/10.1371/journal.pone.0106361 Text en © 2014 Arnulf et al http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited. |
spellingShingle | Research Article Arnulf, Jan Ketil Larsen, Kai Rune Martinsen, Øyvind Lund Bong, Chih How Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour |
title | Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour |
title_full | Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour |
title_fullStr | Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour |
title_full_unstemmed | Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour |
title_short | Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour |
title_sort | predicting survey responses: how and why semantics shape survey statistics on organizational behaviour |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4153608/ https://www.ncbi.nlm.nih.gov/pubmed/25184672 http://dx.doi.org/10.1371/journal.pone.0106361 |
work_keys_str_mv | AT arnulfjanketil predictingsurveyresponseshowandwhysemanticsshapesurveystatisticsonorganizationalbehaviour AT larsenkairune predictingsurveyresponseshowandwhysemanticsshapesurveystatisticsonorganizationalbehaviour AT martinsenøyvindlund predictingsurveyresponseshowandwhysemanticsshapesurveystatisticsonorganizationalbehaviour AT bongchihhow predictingsurveyresponseshowandwhysemanticsshapesurveystatisticsonorganizationalbehaviour |