A Brief Measure of Interpersonal Interaction for 2-Player Serious Games: Questionnaire Validation

BACKGROUND: Competitive and cooperative serious games have become increasingly popular in areas such as rehabilitation and education and have several potential advantages over single-player games. However, they are not suitable for everyone, and the user experience in competitive and cooperative serious games depends on many factors. One important factor is the verbal interaction between players, but the effect of this factor has not been extensively studied because of the lack of a validated measurement tool. OBJECTIVE: This paper aimed to validate a brief questionnaire that measures the verbal interaction between 2 players of a serious game. The questionnaire consists of 8 questions pertaining to the amount of conversation, its valence (positive or negative emotion), and its game relatedness. METHODS: The questionnaire was validated with 30 pairs of participants who played a competitive serious game for 10 min while being recorded with cameras. The questionnaire was filled out by both participants, an in-person observer, and 2 members of our research group who watched the videos. Results from these raters were used to develop questionnaire instructions, and the finalized questionnaire was given to 2 additional raters who were trained on 5 videos and then rated the other 25 videos independently. RESULTS: The questionnaire’s interrater reliability is excellent for the amount of conversation and its game relatedness (intraclass correlation coefficients [ICCs] above 0.9). Interrater reliability is fair to good for conversation valence (ICCs between 0.4 and 0.7). We believe that the lower interrater reliability for valence is primarily because of a limited spread of valence values in our sample. Furthermore, questionnaire ratings were significantly correlated with players’ personality characteristics (eg, amount of conversation was correlated with extraversion) and pressure/tension experienced in the competitive game. CONCLUSIONS: The validated questionnaire has the potential to be a useful tool for studying user experience in competitive and cooperative serious games. Furthermore, it could be adapted for other applications such as entertainment games. However, it has only been validated with unimpaired university students in a 2-player competitive serious game and should next be validated with different target populations (eg, stroke survivors) and different game designs (eg, cooperative games).