All tests are imperfect: Accounting for false positives and false negatives using Bayesian statistics

Tests with binary outcomes (e.g., positive versus negative) to indicate a binary state of nature (e.g., disease agent present versus absent) are common. These tests are rarely perfect: chances of a false positive and a false negative always exist. Imperfect results cannot be directly used to infer the true state of the nature; information about the method's uncertainty (i.e., the two error rates and our knowledge of the subject) must be properly accounted for before an imperfect result can be made informative. We discuss statistical methods for incorporating the uncertain information under two scenarios, based on the purpose of conducting a test: inference about the subject under test and inference about the population represented by test subjects. The results are applicable to almost all tests. The importance of properly interpreting results from imperfect tests is universal, although how to handle the uncertainty is inevitably case-specific. The statistical considerations not only will change the way we interpret test results, but also how we plan and carry out tests that are known to be imperfect. Using a numerical example, we illustrate the post-test steps necessary for making the imperfect test results meaningful.