Structural adaptation of vertebrate endonuclease G for 5-hydroxymethylcytosine recognition and function

Modified DNA bases functionally distinguish the taxonomic forms of life—5-methylcytosine separates prokaryotes from eukaryotes and 5-hydroxymethylcytosine ((5hm)C) invertebrates from vertebrates. We demonstrate here that mouse endonuclease G (mEndoG) shows specificity for both (5hm)C and Holliday junctions. The enzyme has higher affinity (>50-fold) for junctions over duplex DNAs. A (5hm)C-modification shifts the position of the cut site and increases the rate of DNA cleavage in modified versus unmodified junctions. The crystal structure of mEndoG shows that a cysteine (Cys69) is positioned to recognize (5hm)C through a thiol-hydroxyl hydrogen bond. Although this Cys is conserved from worms to mammals, a two amino acid deletion in the vertebrate relative to the invertebrate sequence unwinds an α-helix, placing the thiol of Cys69 into the mEndoG active site. Mutations of Cys69 with alanine or serine show (5hm)C-specificity that mirrors the hydrogen bonding potential of the side chain (C–H < S–H < O–H). A second orthogonal DNA binding site identified in the mEndoG structure accommodates a second arm of a junction. Thus, the specificity of mEndoG for (5hm)C and junctions derives from structural adaptations that distinguish the vertebrate from the invertebrate enzyme, thereby thereby supporting a role for (5hm)C in recombination processes.