Varenicline Interactions at the 5-HT(3) Receptor Ligand Binding Site are Revealed by 5-HTBP

[Image: see text] Cys-loop receptors are the site of action of many therapeutic drugs. One of these is the smoking cessation agent varenicline, which has its major therapeutic effects at nicotinic acetylcholine (nACh) receptors but also acts at 5-HT(3) receptors. Here, we report the X-ray crystal structure of the 5-HT binding protein (5-HTBP) in complex with varenicline, and test the predicted interactions by probing the potency of varenicline in a range of mutant 5-HT(3) receptors expressed in HEK293 cells and Xenopus oocytes. The structure reveals a range of interactions between varenicline and 5-HTBP. We identified residues within 5 Å of varenicline and substituted the equivalent residues in the 5-HT(3) receptor with Ala or a residue with similar chemical properties. Functional characterization of these mutant 5-HT(3) receptors, using a fluorescent membrane potential dye in HEK cells and voltage clamp in oocytes, supports interactions between varenicline and the receptor that are similar to those in 5-HTBP. The structure also revealed C-loop closure that was less than in the 5-HT-bound 5-HTBP, and hydrogen bonding between varenicline and the complementary face of the binding pocket via a water molecule, which are characteristics consistent with partial agonist behavior of varenicline in the 5-HT(3) receptor. Together, these data reveal detailed insights into the molecular interaction of varenicline in the 5-HT(3) receptor.