Advantages of large medical record database for outcomes research: Insights into post‐menopausal hormone therapy

Approximately 25 years ago, our team initiated studies to determine whether outcome results from a large medical record database would yield valid results. We utilized the data in the United Kingdom (UK) General Practice Research Database (GPRD) to replicate the randomized controlled trial (RCT) study result and compared them to confirm the database results. The initial studies compared favorably, but some subsequent studies did not. This prompted development of a new strategy to determine and correct for unrecognized confounding in the database. This strategy divided outcome rates prior to initiation of therapy in the database study (which should include both identified and unidentified confounders) into the outcome rates during the treatment interval. When they differed from Cox‐adjusted results, it reflected unrecognized confounding. We called this strategy Prior Event Rate Ratio (PERR)–adjusted outcome. One of our previously published observational studies replicated the Women's Health Initiative (WHI) RCT study of hormone therapy in post‐menopausal women. Our study results replicated the WHI RCT results except it did not exhibit an increase in heart attack in contrast to the RCT. Furthermore, we could not evaluate death reliably since our analytic approach to overcome unrecognized confounding does not work for this outcome. In Volume 1, Issue 1 of the Learning Health Systems open access journal, we published a new study (titled “A new method to address unmeasured confounding of mortality in observational studies”) that reported a novel death method, based on our prior methodology, that could analyze unrecognized confounding of the death outcome. This new methodology, termed Post Treatment Event Rate Ratio (PTERR), permitted a reliable examination of mortality in post‐menopausal women undergoing hormone therapy. These results are reported in this manuscript. The study used the data from our previous observational study. It demonstrates that estrogen therapy markedly reduced death in post‐menopausal women. This work also illuminates principles of database construction and correspondingly demonstrates the use of novel methodologies for obtaining valid results, which can be applied to enable learning from such databases. Work to advance such methodologies is essential to advancing the scientific integrity Core Value underpinning learning health systems (LHSs). Indeed, in the absence of such efforts to develop and refine methodologies for learning trustworthy lessons from real‐world data, we risk inadvertently creating mis‐learning systems.