A practical guide to estimating the light extinction coefficient with nonlinear models—a case study on maize

BACKGROUND: The fraction of intercepted photosynthetically active radiation (fPARi) is typically described with a non-linear function of leaf area index (LAI) and k, the light extinction coefficient. The parameter k is used to make statistical inference, as an input into crop models, and for phenotyping. It may be estimated using a variety of statistical techniques that differ in assumptions, which ultimately influences the numerical value k and associated uncertainty estimates. A systematic search of peer-reviewed publications for maize (Zea Mays L.) revealed: (i) incompleteness in reported estimation techniques; and (ii) that most studies relied on dated techniques with unrealistic assumptions, such as log-transformed linear models (LogTLM) or normally distributed data. These findings suggest that knowledge of the variety and trade-offs among statistical estimation techniques is lacking, which hinders the use of modern approaches such as Bayesian estimation (BE) and techniques with appropriate assumptions, e.g. assuming beta-distributed data. RESULTS: The parameter k was estimated for seven maize genotypes with five different methods: least squares estimation (LSE), LogTLM, maximum likelihood estimation (MLE) assuming normal distribution, MLE assuming beta distribution, and BE assuming beta distribution. Methods were compared according to the appropriateness for statistical inference, point estimates’ properties, and predictive performance. LogTLM produced the worst predictions for fPARi, whereas both LSE and MLE with normal distribution yielded unrealistic predictions (i.e. fPARi < 0 or > 1) and the greatest coefficients for k. Models with beta-distributed fPARi (either MLE or Bayesian) were recommended to obtain point estimates. CONCLUSION: Each estimation technique has underlying assumptions which may yield different estimates of k and change inference, like the magnitude and rankings among genotypes. Thus, for reproducibility, researchers must fully report the statistical model, assumptions, and estimation technique. LogTLMs are most frequently implemented, but should be avoided to estimate k. Modeling fPARi with a beta distribution was an absent practice in the literature but is recommended, applying either MLE or BE. This workflow and technique comparison can be applied to other plant canopy models, such as the vertical distribution of nitrogen, carbohydrates, photosynthesis, etc. Users should select the method balancing benefits and tradeoffs matching the purpose of the study. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13007-021-00753-2.