Assessment of the Importance of a New Risk Factor in Prediction Models

BACKGROUND: Discovery of new risk factors poses new challenges on how to quantify their added value and importance in risk prediction improvement. OBJECTIVES: The aim of this study was to apply different statistics and to quantify the importance of some risk factors in acute myocardial infarction (AMI). PATIENTS AND METHODS: In a retrospective cohort study, 607 patients with AMI, aged more than 25 years were studied. They were admitted to the CCU of Imam Reza hospital in Mashhad, Iran from 2007 to 2012. Health information and death registration systems were used to identify patients and to assess their outcome. At first a model containing all variables was fitted (full model). Importance of variables was compared in terms of standardized regression coefficient and inclusion frequency in bootstrap samples. Then, a series of reduced models were fitted, where in each of them only one of the independent variables was excluded. Models were compared in terms of goodness of fit, accuracy (Cindex, R square), separation of patients into risk groups (SEP), and net reclassification improvement (NRI). RESULTS: Age was selected as the important factor based on all 7 statistics. Exclusion of age variable decreased C index from 0.75 to 0.68 and R square from 0.25 to 0.15. Duration of hospitalization was important based on 4 statistics. Exclusion of this variable decreased R square from 0.25 to 0.21. While gender was a useful variable in separation of patients into risk groups, its omission did not reduce model likelihood. The opposite was true in the case of using streptokinase during hospitalization. CONCLUSIONS: Our results revealed that a variable with high separation ability might not necessarily be useful in terms of goodness of fit. Therefore, importance should be defined carefully based on clinical objectives of the study.