From Motion to Emotion: Accelerometer Data Predict Subjective Experience of Music

Music is often discussed to be emotional because it reflects expressive movements in audible form. Thus, a valid approach to measure musical emotion could be to assess movement stimulated by music. In two experiments we evaluated the discriminative power of mobile-device generated acceleration data produced by free movement during music listening for the prediction of ratings on the Geneva Emotion Music Scales (GEMS-9). The quality of prediction for different dimensions of GEMS varied between experiments for tenderness (R(1)(2)(first experiment) = 0.50, R(2)(2)(second experiment) = 0.39), nostalgia (R(1)(2) = 0.42, R(2)(2) = 0.30), wonder (R(1)(2) = 0.25, R(2)(2) = 0.34), sadness (R(1)(2) = 0.24, R(2)(2) = 0.35), peacefulness (R(1)(2) = 0.20, R(2)(2) = 0.35) and joy (R(1)(2) = 0.19, R(2)(2) = 0.33) and transcendence (R(1)(2) = 0.14, R(2)(2) = 0.00). For others like power (R(1)(2) = 0.42, R(2)(2) = 0.49) and tension (R(1)(2) = 0.28, R(2)(2) = 0.27) results could be almost reproduced. Furthermore, we extracted two principle components from GEMS ratings, one representing arousal and the other one valence of the experienced feeling. Both qualities, arousal and valence, could be predicted by acceleration data, indicating, that they provide information on the quantity and quality of experience. On the one hand, these findings show how music-evoked movement patterns relate to music-evoked feelings. On the other hand, they contribute to integrate findings from the field of embodied music cognition into music recommender systems.