Standardized food challenges are subject to variability in interpretation of clinical symptoms

BACKGROUND: Food challenge tests are the gold standard in diagnosing food allergy. Guidelines provide scoring systems to classify symptoms during challenge and typically recommend that challenges are considered positive when objective symptoms occur. However, currently no standard criteria for the definition of a positive challenge outcome exists and interpretation of food challenges mainly depends on clinical judgment. This study aims to assess inter- and intra-observer variability in outcomes of routinely performed peanut challenges in children. METHODS: All complete food challenge score sheets of double blind placebo controlled peanut challenges performed in 2008-2010 in an academic hospital were included. Score sheets were reassessed independently by three clinical experts including double reassessment in a subset of score sheets. Inter- and intra-observer variability was evaluated using kappa statistics. RESULTS: We included 191 food challenge score sheets. Inter-observer agreement on overall challenge outcome was moderate (κ = 0.59-0.65) and was fair (κ = 0.31-0.46) on challenges with symptoms. Intra-observer agreement on overall challenge outcome was good (κ = 0.63-0.77) but was moderate (κ = 0.50-0.60) on challenges with symptoms. Subjective symptoms (oral symptoms, abdominal complaints, food aversion) were significantly associated with disagreement between observers. CONCLUSIONS: We demonstrate that, despite strict adherence to guidelines, there is a considerable amount of variability in reassessment of symptoms recorded on food challenges sheets between and within well trained clinicians, especially when subjective symptoms occur.