Reliability of Paper-Based Routine Documentation in Psychiatric Inpatient Care and Recommendations for Further Improvement

Background: Health services research is of increasing importance in current psychiatry. Therefore, large datasets and aggregation of data generated by electronic routine documentation due to legal, financial, or administrative purposes play an important role. However, paper-based routine documentation is still of interest. It remains relevant in less developed health care systems, in emergency settings, and in long-term retrospective and historical studies. Whereas studies examining the reliability of electronic routine documentation support the application of routine data for research purposes, our knowledge regarding reliability of paper-based routine documentation is still very sparse. Methods: Basic documentation (BADO) was completed on paper forms and digitalized manually for all inpatients of the Department of Psychiatry and Psychotherapy, University Hospital Hamburg-Eppendorf, Germany, treated within the time period from 1998 to 2006. Four hundred twelve cases of first-episode psychosis patients were chosen for comparison with clinical data from paper-based patient files. The percentage of missing information, the percentage of correct classifications, sensitivity, and positive predictive value were calculated for all applicable variables. Results: In eight cases (1.9%), a BADO form was available, but was not filled in. In 37 cases (7.0%), the patient files were lost and could not be obtained from the centralized archive. Routine data were available for all other cases in 20 (58.8%) of the examined 34 variables, and the percentage of missing data for the remaining variables ranged between 0.3% and 22.9%, with only the variables education and suicidality during treatment having more than 5% missing data. In general, the overall rate of correct classifications was high, with a median percentage of 86.4% to 99.7% for the examined variables. Sensitivity was above 75% for eight and <75% but above 50% for six of the examined 17 variables. Values for the positive predictive value were above 75% for nine and <75% but above 50% for three variables. Conclusion: In summary, paper-based routine documentation reaches acceptable reliability, but this is dependent on the chosen documentation categories and variables. Based on the present findings, paper-based routine documentation can indeed be used for quality management, organizational development, and health services research. Its limitations, however, have to be kept in mind.