Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS)

The 2013 U.S. Veterans Administration/Department of Defense Clinical Practice Guidelines (VA/DoD CPG) require comprehensive suicide risk assessments for VA/DoD patients with mental disorders but provide minimal guidance on how to carry out these assessments. Given that clinician-based assessments are known not to be strong predictors of suicide, we investigated whether a precision medicine model using administrative data after outpatient mental health specialty visits could be developed to predict suicides among outpatients. We focused on male non-deployed Regular U.S. Army soldiers because they account for the vast majority of such suicides. Four machine learning classifiers (naïve Bayes, random forests, support vector regression, elastic net penalized regression) were explored. 41.5% of Army suicides in 2004-2009 occurred among the 12.0% of soldiers seen as outpatient by mental health specialists, with risk especially high within 26 weeks of visits. An elastic net classifier with 10-14 predictors optimized sensitivity (45.6% of suicide deaths occurring after the 15% of visits with highest predicted risk). Good model stability was found for a model using 2004-2007 data to predict 2008-2009 suicides, although stability decreased in a model using 2008-2009 data to predict 2010-2012 suicides. The 5% of visits with highest risk included only 0.1% of soldiers (1047.1 suicides/100,000 person-years in the 5 weeks after the visit). This is a high enough concentration of risk to have implications for targeting preventive interventions. An even better model might be developed in the future by including the enriched information on clinician-evaluated suicide risk mandated by the VA/DoD CPG to be recorded.