Standard setting for a novel esophageal conduit questionnaire: CONDUIT Report Card

BACKGROUND: The purpose of this study was to establish the clinical thresholds for five domains (dysphagia, reflux, dumping-hypoglycemia, dumping-GI symptoms, pain) to support the use of the CONDUIT questionnaire as a screening tool to identify patients who might benefit from an educational or clinical intervention. METHODS: A panel of 16 experts met to develop descriptions of “poor,” “moderate,” and “good” conduit performance. They were trained to use the modified and extended Angoff standard-setting method. Each judge provided item ratings that reflected borderline good and borderline moderate patients. The average item ratings were summed and transformed to a 0–100 scale to derive final cut scores. Panelist evaluation of the process and confidence with the rating tasks were collected. RESULTS: Panelists expressed that the training on the method gave them information they needed to complete their assignment. Among other factors, their experience with patients was most influential on their ratings. On the 0–100 score scale, good/moderate cuts ranged from 7.2 to 20.8, and moderate/poor cuts ranged from 37.9 to 64.3, depending on domains and weights. Standard errors of one or both cut scores increased for dysphagia and dumping-GI with weighting. CONCLUSIONS: We described the selection and training of panelists and panelists’ evaluations of the processes they were asked to follow in detail to defend the cut scores. Further prospective validation studies are underway to compare cut scores from this study and clinicians’ judgments and further refine the categorization. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s41687-018-0073-2) contains supplementary material, which is available to authorized users.