Inter-rater and test-retest reliability of the “standardized endoscopic swallowing evaluation for tracheostomy decannulation in critically ill neurologic patients”

BACKGROUND: Removal of a tracheostomy tube in critically ill neurologic patients is a difficult issue, particularly due to the high incidence of oropharyngeal dysphagia. For an objective evaluation of decannulation readiness the “Standardized Endoscopic Swallowing Evaluation for Tracheostomy Decannulation in Critically Ill Neurologic Patients” (SESETD) – a stepwise evaluation of ‘secretion management’, ‘spontaneous swallows’ and ‘laryngeal sensibility/cough’ – has been introduced. With the recent study detailed data on inter-rater and test-retest reliability are presented. METHODS: To obtain inter-rater reliability levels both in a group of raters with at least 5 years of experience (‘experts’) and in a group of raters with no or only minor experience using the SESETD (‘non-experts’), for each single item of the protocol and the sum score α-, respectively κ-levels were determined. The ‘experts’ assessed the same videos after a four-week interval to determine test-retest reliability. Ten videos from tracheostomized neurological patients completely weaned from mechanical ventilation were assessed independently by six ‘experts’. 27 ‘non-experts’ applied the SESETD on 5 videos from the same patient population after introduction to the protocol in a one-hour workshop. RESULTS: For the items ‘secretion management’ and ‘spontaneous swallows’ α-levels were identified at > 0.800 both in the group of ‘experts’ and ‘non-experts’. With regard to the item ‘laryngeal sensibility/cough’ in both groups, the α-level was ≥0.667. With κ-levels of 1.0 for ‘secretion management’, 0.93 for ‘spontaneous swallows’ and 0.76 for ‘laryngeal sensibility/cough’ test-retest reliability showed at least substantial agreement for each item. Intraclass correlation coefficient for the sum score was excellent in both groups (α ≥ 0.90). CONCLUSIONS: The SESETD demonstrates good to excellent agreement for each single item included as well as the sum score in experienced and unexperienced raters supporting its usefulness for implementation in daily clinical routine and as an outcome measure for clinical trials.