Implementation Degree Assessment Sheet for Health Program in Japan by Customizing CFIR: Development and Validation

PURPOSE: More than 70% of public health nurses in Japan belong to government agencies, and there is a need for further evidence-based capacity development for program implementation. The purpose of this research was to develop an Implementation Degree Assessment Sheet (IDAS) by customizing the Consolidated Framework for Implementation Research (CFIR) to health programs in Japan. METHODS: The draft IDAS (five domains, 31 constructs) created by customizing the CFIR was refined by the researchers and modified through pre-testing. The survey covered full-time public health nurses (PHNs) affiliated to all prefectures and the cities with health centers of Japan. The survey was conducted as an anonymous, self-administered questionnaire survey by mail. RESULTS: One hundred eighty-five of the 334 institutions (55.4%) agreed to participate in our survey. Of the 966 questionnaire forms distributed, 709 forms (73.4%) were collected, of which 702 responses (72.7%) were valid. No item required consideration of deletion based on the results of item analysis, and our confirmatory factor analysis on model fitness between the five IDAS domains and CFIR showed sufficient fit indices after modification. With regard to reliability, Cronbach’s coefficient alpha, a measure of internal consistency, stayed above 0.8 overall. Our verification of stability with the split-half (odd/even) method resulted in a Spearman–Brown reliability coefficient of 0.95. The correlation coefficient between the IDAS scores and the research utilization competency score, used as an external criterion, was 0.51 (p<0.001), supporting the coexistence validity of the criterion-related validity of the IDAS. The significant differences were observed between known-groups, supporting the known-group validity of the IDAS. CONCLUSION: This study developed the IDAS and confirmed constant reliability and validity. Hereafter, it is necessary to promote the required capacity development based on the actual degree of implementation in order to use the IDAS for the competency development of public health nurses and related professions to deliver health programs. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s43058-022-00270-w.