The ability to evaluate arguments in scientific texts: Measurement, cognitive processes, nomological network, and relevance for academic success at the university

BACKGROUND: The evaluation of informal arguments is a key component of comprehending scientific texts and scientific literacy. AIM: The present study examined the nomological network of university students’ ability to evaluate informal arguments in scientific texts and the relevance of this ability for academic success. SAMPLE: A sample of 225 university students from the social and educational sciences participated in the study. METHODS: Judgements of plausibility and the ability to recognize argumentation fallacies were assessed with a novel computer‐based diagnostic instrument (Argument Judgement Test; AJT). RESULTS: The items of the AJT partly conform to a 1‐PL model and test scores were systematically related to epistemological beliefs and verbal intelligence. Item‐by‐item analyses of responses and response times showed that implausible arguments were more difficult to process and correct responses to these items required increased cognitive effort. Finally, the AJT scores predicted academic success at university even if verbal intelligence and grade point average were controlled for. CONCLUSION: These findings suggest that the ability to evaluate arguments in scientific texts is an aspect of rationality, relies on reflective processes, and is relevant for academic success.