A Rasch analysis assessing the reliability and validity of the Arizona CoVHORT COVID-19 vaccine questionnaire

BACKGROUND: Despite the widespread availability of COVID-19 vaccines in the United States, many that have chosen not to be vaccinated have done so because of vaccine hesitancy. This highlights the need for tools that accurately capture the knowledge, attitudes, and beliefs towards COVID-19 vaccines, and provide steps toward improving vaccine acceptance. METHODS: Participants of the Arizona CoVHORT (COVID-19 Cohort) received a one-time, electronic based cross-sectional questionnaire intended to capture underlying motivations regarding vaccination, as well as hesitations that may prevent people from getting vaccinated. Rasch analysis was conducted among 4703 CoVHORT participants who had completed the vaccine questionnaire to assess questionnaire reliability and validity. Response categories were grouped to optimize scale functioning and to ensure independent probabilities of participant endorsement. RESULTS: A total of 4703 CoVHORT participants completed the questionnaire, of whom 68% were female, and who had a mean age of 48 years. Participants were primarily White (90%), highly educated (63% with a college degree or above, with most respondents (45%) having an income of more than $75,000 per annum. The results indicated the questionnaire has good reliability and construct validity for assessing attitudes and beliefs about the COVID-19 vaccines. In-fit mean-squares for included items ranged from 0.61 to 1.72 and outfit mean-squares ranged from 0.56 to 1.75, and correlation coefficients ranged from 0.25 to 0.75. The person-item map indicated normal distribution of logit scores measuring perceptions about COVID-19 vaccinations. CONCLUSIONS: The CoVHORT vaccine questionnaire demonstrated satisfactory reliability and construct validity in assessing attitudes and beliefs about COVID-19 vaccines. Overall results provide a starting point for a reliable and valid tool to assess knowledge and perceptions about COVID-19 vaccination, ultimately providing public health professionals with an instrument to assess the factors that are associated with vaccine acceptance or hesitancy.