The “Moon Test”: A Step Towards Evaluating Comprehension of Educational Text through Model Mediation

BACKGROUND: This paper addresses the issue of educational text comprehension, which is one of the major problems in secondary schools, especially when such texts are introduced in the natural sciences. Studies on text comprehension often regard reading as a standalone skill: its mechanisms are discussed from leading theoretical approaches (cognitivism, constructivism, etc.), and variables are distinguished and evaluated. Most of the researchers consider text comprehension to be active reconstruction of the meaning which the text delivers, and regard the application of the information retrieved from the text to problem-solving as the indicator for a deep comprehension level. Since we work within the framework of Cultural-Historical Activity Theory (CHAT), we consider educational text comprehension to be mediated through special content-related models which students have to acquire. Unfortunately, there are no studies which have directly linked reading, corresponding problem-solving, and working with content-related models (symbolic means, schemes); hence, with this research, we are seeking to fill in the gap. OBJECTIVE: Our goal is to elaborate the perspective on educational text comprehension as mediated through mastering special modeling (symbolic) means. In this article we illustrate this approach with the “Moon test”— an assessment procedure which we designed to materialize the components of orientation of students’ action as they succeed or fail to solve problems by relying on the educational text provided. DESIGN: We conducted the “Moon test” among the fifth graders (10-12 years old). The text, which told the students how to use the moon’s visual transformations as a calendar, was followed by 12 tasks on the topic. The tasks required using the text to master the model provided, and then solve challenging tasks which only referred to the model implicitly. RESULTS: To analyze the results, we grouped the tasks in four blocks: 1) model acquisition; 2) mastering; 3) application; and 4) experience. The results showed a statistically significant decrease in the students’ performance on tasks of the third and the fourth blocks, which required reasonable application of the models. Further analysis of individual patterns of performance allowed us to distinguish clusters of students with different levels of success in each block. CONCLUSION: Our results attest to the importance of model mediation for reading comprehension and the development of scientific literacy.