Comparing Numerical Comparison Tasks: A Meta-Analysis of the Variability of the Weber Fraction Relative to the Generation Algorithm

Since more than 15 years, researchers have been expressing their interest in evaluating the Approximate Number System (ANS) and its potential influence on cognitive skills involving number processing, such as arithmetic. Although many studies reported significant and predictive relations between ANS and arithmetic abilities, there has recently been an increasing amount of published data that failed to replicate such relationship. Inconsistencies lead many researchers to question the validity of the assessment of the ANS itself. In the current meta-analysis of over 68 experimental studies published between 2004 and 2017, we show that the mean value of the Weber fraction (w), the minimal amount of change in magnitude to detect a difference, is very heterogeneous across the literature. Within young adults, w might range from < 10 to more than 60, which is critical for its validity for research and diagnostic purposes. We illustrate here the concern that different methods controlling for non-numerical dimensions lead to substantially variable performance. Nevertheless, studies that referred to the exact same method (e.g., Panamath) showed high consistency among them, which is reassuring. We are thus encouraging researchers only to compare what is comparable and to avoid considering the Weber fraction as an abstract parameter independent from the context. Eventually, we observed that all reported correlation coefficients between the value of w and general accuracy were very high. Such result calls into question the relevance of computing and reporting at all the Weber fraction. We are thus in disfavor of the systematic use of the Weber fraction, to discourage any temptation to compare given data to some values of w reported from different tasks and generation algorithms.