Development and validation of a generic questionnaire for the implementation of complex medical interventions

Introduction: The implementation of complex medical interventions in daily practice is often fraught with difficulties. According to the iterative phase model proposed by the British Medical Research Council (MRC), the development, implementation and evaluation of complex interventions should be theory-driven. A conceptual model that seems to be a promising framework is the Theory of planned behaviour (TPB). In our study we aimed to develop and validate a generic and multifaceted questionnaire based on the TPB to detect physicians’ willingness to implement complex medical interventions and the factors influencing this willingness. Methods: The questionnaire was developed according to the literature and was informed by previous qualitative research of our department. It was validated on the example of an electronic library of decision aids, arriba-lib. The sample consisted of 181 General Practitioners (GPs) who received a training regarding arriba-lib and subsequently filled in the questionnaire, assessing the TPB variables attitude, subjective norm, perceived behaviour control and intention. Follow-up assessments were conducted after two (assessing retest reliability) and eight weeks (assessing target behaviour). We performed a confirmatory factor analysis investigating the factorial structure of our questionnaire according to the TPB. Beside the calculation of the questionnaire’s psychometric properties we conducted a structural equation model and an ordinal regression to predict actual behaviour regarding the installation and application of arriba-lib. Results: The postulated three factorial model (attitude, subjective norm, perceived behaviour control) of our questionnaire based on the TPB was rejected. A two factorial model with a combined factor subjective norm/perceived behaviour control was accepted. The explained variance in the ordinal regression was low (Nagelkerke’s R(2)=.12). Neither attitude nor intention were able to predict the use or non-use of arriba-lib (attitude: p=.68, intention: p=.44). For the combined factor subjective norm/perceived behaviour control a significant, but small effect (p=.03) was shown. Conclusions: The TPB is not an adequate theoretical framework to guide the development of a generic questionnaire in the context of the implementation of complex interventions. To enable the successful implementation of complex medical interventions evaluators have to go through the whole development and evaluation process according to the MRC-model, without short cuts. Further, it has to be discussed if a generic instrument can be valid and useful. Regarding the TPB a publication bias regarding the theory’s applicability might have to be considered.