Evaluation of the Interrater Reliability of End-of-Life Medical Orders in the Physician Orders for Life-Sustaining Treatment Form

IMPORTANCE: Despite its spread in much of the United States and increased international interest, the Physician Orders for Life-Sustaining Treatment (POLST) paradigm still lacks supporting evidence. The interrater reliability of the POLST form to translate patients’ values and preferences into medical orders for care at the end of life remains to be studied. OBJECTIVE: To assess the interrater reliability of the medical orders documented in POLST forms. DESIGN, SETTING, AND PARTICIPANTS: This cross-sectional study was conducted in a public university hospital in southeastern Brazil. Two independent researchers interviewed the same patients or decision-making surrogates (n = 64) during a single episode of hospitalization within a time frame of 1 to 7 days. Eligible participants were hospitalized adults aged 21 years or older who were expected to remain hospitalized for at least 4 days and whose attending physician responded no to the question, Would I be surprised if this patient died in the next year? Data collection occurred between November 1, 2015, and September 20, 2016, and first data analyses were performed on October 3, 2016. MAIN OUTCOMES AND MEASURES: Interrater reliability as measured by κ statistics. RESULTS: Of the 64 participants interviewed in the study, 53 (83%) were patients and 11 (17%) were surrogates. Patients’ mean (SD) age was 64 (14) years, and 35 patients (55%) and 8 surrogates (73%) were women. Overall, in 5 cases (8%), disagreement in at least 1 medical order for life-sustaining treatment was found in the POLST form, changing from the first interview to the second interview. The κ statistic for cardiopulmonary resuscitation was 0.92 (95% CI, 0.80-1.00); for level of medical intervention, 0.89 (95% CI, 0.76-1.00); and for artificially administered nutrition, 0.92 (95% CI, 0.83-1.00). CONCLUSIONS AND RELEVANCE: The high interrater reliability of the medical orders in POLST forms appears to offer further support for this advance care planning paradigm; in addition, the finding that this interrater reliability was not 100% underscores the need to ensure that patients or their surrogates have decision-making capacity and to confirm that the content of POLST forms accurately reflects patients’ current treatment preferences.