Using a retrospective pretest instead of a conventional pretest is replacing biases: a qualitative study of cognitive processes underlying responses to thentest items

BACKGROUND: The thentest design aims to detect and control for recalibration response shift. This design assumes (1) more consistency in the content of the cognitive processes underlying patients’ quality of life (QoL) between posttest and thentest assessments than between posttest and pretest assessments; and (2) consistency in the time frame and description of functioning referenced at pretest and thentest. Our objective is to utilize cognitive interviewing to qualitatively examine both assumptions. METHODS: We conducted think-aloud interviews with 24 patients with cancer prior to and after radiotherapy to elicit cognitive processes underlying their assessment of seven EORTC QLQ-C30 items at pretest, posttest and thentest. We used an analytic scheme based on the cognitive process models of Tourangeau et al. and Rapkin and Schwartz that yielded five cognitive processes. We subsequently used this input for quantitative analysis of count data. RESULTS: Contrary to expectation, the number of dissimilar cognitive processes between posttest and thentest was generally larger than between pretest and posttest across patients. Further, patients considered a range of time frames when answering the thentest questions. Moreover, patients’ description at the thentest of their pretest functioning was often not similar to that which was noted at pretest. Items referring to trouble taking a short walk, overall health and QoL were most often violating the assumptions. CONCLUSIONS: Both assumptions underlying the thentest design appear not to be supported by the patients’ cognitive processes. Replacing the conventional pretest–posttest design with the thentest design may simply be replacing one set of biases with another. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1007/s11136-015-1175-4) contains supplementary material, which is available to authorized users.