Development of a keyword library for capturing PRO-CTCAE-focused “symptom talk” in oncology conversations

OBJECTIVES: As computational methods for detecting symptoms can help us better attend to patient suffering, the objectives of this study were to develop and evaluate the performance of a natural language processing keyword library for detecting symptom talk, and to describe symptom communication within our dataset to generate insights for future model building. MATERIALS AND METHODS: This was a secondary analysis of 121 transcribed outpatient oncology conversations from the Communication in Oncologist-Patient Encounters trial. Through an iterative process of identifying symptom expressions via inductive and deductive techniques, we generated a library of keywords relevant to the Patient-Reported Outcome version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) framework from 90 conversations, and tested the library on 31 additional transcripts. To contextualize symptom expressions and the nature of misclassifications, we qualitatively analyzed 450 mislabeled and properly labeled symptom-positive turns. RESULTS: The final library, comprising 1320 terms, identified symptom talk among conversation turns with an F1 of 0.82 against a PRO-CTCAE-focused gold standard, and an F1 of 0.61 against a broad gold standard. Qualitative observations suggest that physical symptoms are more easily detected than psychological symptoms (eg, anxiety), and ambiguity persists throughout symptom communication. DISCUSSION: This rudimentary keyword library captures most PRO-CTCAE-focused symptom talk, but the ambiguity of symptom speech limits the utility of rule-based methods alone, and limits to generalizability must be considered. CONCLUSION: Our findings highlight opportunities for more advanced computational models to detect symptom expressions from transcribed clinical conversations. Future improvements in speech-to-text could enable real-time detection at scale.