Large-scale genomic and transcriptomic profiles of rice hybrids reveal a core mechanism underlying heterosis

BACKGROUND: Heterosis is widely used in agriculture. However, its molecular mechanisms are still unclear in plants. Here, we develop, sequence, and record the phenotypes of 418 hybrids from crosses between two testers and 265 rice varieties from a mini-core collection. RESULTS: Phenotypic analysis shows that heterosis is dependent on genetic backgrounds and environments. By genome-wide association study of 418 hybrids and their parents, we find that nonadditive QTLs are the main genetic contributors to heterosis. We show that nonadditive QTLs are more sensitive to the genetic background and environment than additive ones. Further simulations and experimental analysis support a novel mechanism, homo-insufficiency under insufficient background (HoIIB), underlying heterosis. We propose heterosis in most cases is not due to heterozygote advantage but homozygote disadvantage under the insufficient genetic background. CONCLUSION: The HoIIB model elucidates that genetic background insufficiency is the intrinsic mechanism of background dependence, and also the core mechanism of nonadditive effects and heterosis. This model can explain most known hypotheses and phenomena about heterosis, and thus provides a novel theory for hybrid rice breeding in future. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13059-022-02822-8.