Creating idiometric short-form measures of cognitive appraisal: balancing theory and pragmatics

BACKGROUND: The Rapkin and Schwartz appraisal theory and measure provided a path toward documenting response-shift effects and describing individual differences in ways of thinking about quality of life (QOL) that distinguished people in different circumstances. Recent work developed and validated the QOL Appraisal Profile(version 2) (QOLAP(v2)), an 85-item measure that taps response-shift-detection domains of Frame of Reference, Standards of Comparison, Sampling of Experience, and Combinatory Algorithm. Recent theoretical work proposed that appraisal measurement constitutes a new class of measurement (idiometric), distinct from psychometric and clinimetric. To validate an idiometric measure, one would document that its items reflect different circumstances and population characteristics, and explain variance in QOL. The present work sought to develop idiometric short-forms of the QOLAP(v2) item bank by examining which items were most informative, retaining the appraisal-domain structure. METHODS: This secondary analysis (n = 1481) included chronically-ill patients and their caregivers from a longitudinal web-based survey (mean follow-up 16.6 months). Data included the QOLAP(v2), the Center for Disease Control Healthy Days Core Module, the PROMIS-10 Global Health, and demographic/medical variables. Appraisal items were measured at baseline (relevant to understanding cognitive appraisal processes); and with change scores (sensitive to response-shift effects). Multivariate analysis of covariance examined what demographic and health-status change variables were reflected by each of 85 appraisal items (in five sets), as dependent variables, and other demographic/medical variables. Multiple linear regression examined how appraisal items explained variance in global physical- and mental-health change, after covariate adjustment. A tally summarized item performance across all five sets of cross-sectional and longitudinal analyses. RESULTS: The vast majority (i.e., 80%) of the QOLAP(v2) items performed well across the analyses presented. Using a relatively strict criterion of explaining meaningful variance across 60% of analyses, one would retain 68 items. A more lenient criterion (40%) would retain 71. CONCLUSIONS: The present study provides heuristics to support investigators’ creating ‘discretionary’ QOLAP(v2) short-forms to fit their study aim and amplifying individual differences in the cognitive processes underlying QOL. This approach enables adapting the measure to the study population, as per the expectation that respondent populations differ in the predominant cognitive processes used. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s41687-021-00317-x.