Development and evaluation of an appraisal form to assess clinical effectiveness of adult invasive mechanical ventilation systems

BACKGROUND: Rapid developments in intensive care medicine have made mechanical ventilation an essential method in the resuscitation and comprehensive treatment of critical care patients. This study aimed to develop and evaluate an appraisal form assessing the clinical effectiveness of adult invasive mechanical ventilation systems. METHODS: An appraisal form was designed according to the effectiveness evaluation theory of the American Weapons Systems Effectiveness Industry Advisory Committee (WSEIAC) along with literature review and expert panel review. Content validity of the preliminary form was analyzed in a cohort of 200 patients. Exploratory and confirmatory factor analysis was used to assess appraisal form validity. Discriminate validity of different ventilation outcomes was analyzed by t test. Test/retest reliability and inter-scorer reliability were evaluated with 30 patients after a 2-week interval by Cronbach's alpha. RESULTS: Exploratory factor analysis showed eigenvalues for 3 dimensions (availability, dependability, capability) to be 7.85, 4.43, and 4.22, respectively. Cronbach’s α for internal consistency of the appraisal form was 0.957, and 0.922, 0.961 and 0.937, respectively, for the 3 dimensions. Test-retest reliability of 3 dimensions was 0.976, and 0.862, 0.857, 0.885, respectively. Intra-class correlation coefficient verified test-retest reliability; ICC 0.976 and 0.862, 0.857, 0.885 for 3 dimensions, respectively. CONCLUSIONS: The appraisal form for clinical effectiveness of adult invasive mechanical ventilation systems has high reliability and validity and may be used in clinical setting.