Stochastic models of Mendelian and reverse transcriptional inheritance in state-structured cancer populations

Recent evidence suggests that a polyaneuploid cancer cell (PACC) state may play a key role in the adaptation of cancer cells to stressful environments and in promoting therapeutic resistance. The PACC state allows cancer cells to pause cell division and to avoid DNA damage and programmed cell death. Transition to the PACC state may also lead to an increase in the cancer cell’s ability to generate heritable variation (evolvability). One way this can occur is through evolutionary triage. Under this framework, cells gradually gain resistance by scaling hills on a fitness landscape through a process of mutation and selection. Another way this can happen is through self-genetic modification whereby cells in the PACC state find a viable solution to the stressor and then undergo depolyploidization, passing it on to their heritably resistant progeny. Here, we develop a stochastic model to simulate both of these evolutionary frameworks. We examine the impact of treatment dosage and extent of self-genetic modification on eco-evolutionary dynamics of cancer cells with aneuploid and PACC states. We find that under low doses of therapy, evolutionary triage performs better whereas under high doses of therapy, self-genetic modification is favored. This study generates predictions for teasing apart these biological hypotheses, examines the implications of each in the context of cancer, and provides a modeling framework to compare Mendelian and non-traditional forms of inheritance.