Neurogenomic divergence during speciation by reinforcement of mating behaviors in chorus frogs (Pseudacris)

BACKGROUND: Species interactions can promote mating behavior divergence, particularly when these interactions are costly due to maladaptive hybridization. Selection against hybridization can indirectly cause evolution of reproductive isolation within species, a process termed cascade reinforcement. This process can drive incipient speciation by generating divergent selection pressures among populations that interact with different species assemblages. Theoretical and empirical studies indicate that divergent selection on gene expression networks has the potential to increase reproductive isolation among populations. After identifying candidate synaptic transmission genes derived from neurophysiological studies in anurans, we test for divergence of gene expression in a system undergoing cascade reinforcement, the Upland Chorus Frog (Pseudacris feriarum). RESULTS: Our analyses identified seven candidate synaptic transmission genes that have diverged between ancestral and reinforced populations of P. feriarum, including five that encode synaptic vesicle proteins. Our gene correlation network analyses revealed four genetic modules that have diverged between these populations, two possessing a significant concentration of neurotransmission enrichment terms: one for synaptic membrane components and the other for metabolism of the neurotransmitter nitric oxide. We also ascertained that a greater number of genes have diverged in expression by geography than by sex. Moreover, we found that more genes have diverged within females as compared to males between populations. Conversely, we observed no difference in the number of differentially-expressed genes within the ancestral compared to the reinforced population between the sexes. CONCLUSIONS: This work is consistent with the idea that divergent selection on mating behaviors via cascade reinforcement contributed to evolution of gene expression in P. feriarum. Although our study design does not allow us to fully rule out the influence of environment and demography, the fact that more genes diverged in females than males points to a role for cascade reinforcement. Our discoveries of divergent candidate genes and gene networks related to neurotransmission support the idea that neural mechanisms of acoustic mating behaviors have diverged between populations, and agree with previous neurophysiological studies in frogs. Increasing support for this hypothesis, however, will require additional experiments under common garden conditions. Our work points to the importance of future replicated and tissue-specific studies to elucidate the relative contribution of gene expression divergence to the evolution of reproductive isolation during incipient speciation. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12864-021-07995-3.