Validating Psychometric Questionnaires Using Experience-Sampling Data: The Case of Nightmare Distress

Nightmares are a comparatively frequent phenomenon. They are often accompanied by emotional distress and gain clinical relevance when recurrent. To assess how much distress nightmares cause the individual, the Nightmare Distress Questionnaire (NDQ, Belicki, 1992) is probably the most often used measure. However, its validity is still disputed. To analyze the validity of the proposed three NDQ subscales in more detail, we conducted an experience sampling study, gathering data either in real-time or short retrospective timeframes over the course of 22 days twice per day (N = 92 participants). The measurements were implemented via a mobile app using participants’ own smartphones. Besides the dream quality, we assessed concepts on a daily basis that past research found to be related to dreams. These included critical life events, alcohol consumption, eating behavior, and well-being. We found that only the subscales “general nightmare distress” and “impact on sleep” showed convergent as well as divergent validity. The validity of the subscale “impact on daily reality perception” is unclear. If at all, this subscale is rather indirectly associated with nightmare distress. Furthermore, all of the NDQ items did not differentiate between a bad dream and a nightmare, which suggests that the NDQ might rather be a measure of negative dreams in general and not nightmares in particular. Based on the present experience sampling design, we propose to advance the validation process by further possibilities, such as an item-level, person-level, and multi-level approach. This approach seems to be especially fruitful for concepts which are not very salient (e.g., laughter), can hardly be remembered retrospectively (e.g., dream content), or are potentially threatened by recall biases (e.g., alcohol consumption).