Optimising the validity and completion of adherence diaries: a multiple case study and randomised crossover trial

BACKGROUND: Diaries are the most commonly used adherence measurement method in home-based rehabilitation trials, yet their completion and validity varies widely between trials. We aimed to: (1) generate theory to explain this variation, (2) create an optimised diary and (3) evaluate the optimised diary’s validity. METHODS: Stage 1. Development: using a multiple case study approach, we collected trialist interviews (n = 7), trial publications (n = 16) and diaries (n = 7) from seven purposively sampled UK rehabilitation trials. We explored return rates, diary designs and trialists’ ideas as to what affected diary completion and validity. Using explanatory case study analysis, we developed a diary optimisation model. Stage 2. Evaluation: we compared a diary optimised according to several model components to one nonoptimised according to the same components in a randomised AB/BA crossover trial. Healthy adults aged 60+ years without mobility impairments undertook a home-based 8-week walking programme. They recorded walking duration and frequency for 4 weeks per diary. We hypothesised that the optimised diary would possess greater validity for self-reported adherence to walking duration (criterion: the Activpal accelerometer), assessed during each diary’s final week. Participants were blinded to the hypothesis. Secondary outcomes included test-retest reliability and acceptability. Ethical approval was granted from Glasgow Caledonian University. RESULTS: Thirty-two out of 33 participants completed the study. Diaries did not significantly differ in validity, reliability or acceptability. Both diaries agreed closely with the Activpal when assessing duration adherence at a group level, however, inter and intraindividual variation in validity was high (mean difference (95 % limits of agreement (LOA): limits of agreement plot the difference between measurements collected using two different methods against their mean and thus assess the extent to which the two measures agree with each other)) optimised diary = 3.09 % (−103.3 to 109.5 %), nonoptimised diary = −0.34 % (−131.1 to 130.5 %), p = 0.732). We found similarly wide LOA for percentage of days adhered to and percentage of walks taken, whilst frequency adherence was underestimated. Participants rated both diaries as low-burden and equal numbers favoured each diary or were neutral. Preference appeared to impact minimally upon validity. CONCLUSION: Group-level adherence diary data are likely to be valid. However, individual diary data lack validity, which raises concerns if using this data in calculations such as predicting functional outcomes. Different diary designs are likely interchangeable, though unanticipated high variation meant that this study was underpowered. TRIAL REGISTRATION: The trial was not eligible for registration in a clinical trial database as diary measurement property outcomes, not clinical health outcomes of participants, were assessed. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13063-016-1615-7) contains supplementary material, which is available to authorized users.