An investigation into the relationship between multimedia lecture design and learners’ engagement behaviours using web log analysis

The purpose of this transaction log analysis was to evaluate university students’ engagement behaviours with a catalogue of multimedia lectures. These lectures incorporated selected instructional design principles from the cognitive theory of multimedia learning (CTML). Specifically, thirty-two multimedia lectures which differentially employed the signalling, segmenting and embodiment principles from the CTML were delivered to a cohort of 92 students throughout an academic trimester. Engagement with each multimedia lecture was measured in three domains: affective engagement was measured using a Likert-style survey that accompanied each multimedia lecture; behavioural engagement was measured using the web logs provided by YouTube Studio analytics (average watch time); cognitive engagement was measured using students’ average score on a quiz that accompanied each multimedia lecture. Separate multiple linear regression analyses for measures of affective, behavioural and cognitive engagement revealed that multimedia lectures that ‘stacked’ the instructional design principles of embodiment (whereby the lecture was interspersed with clips of an enthusiastic onscreen instructor), segmenting (where lectures were divided into shorter, user-paced segments) and signalling (where onscreen labels highlighted important material) increased measures of engagement, including overall watch time, number of survey submission and number of quiz attempts (P < 0.05). There was no association between any of the tested principles and students’ quiz scores or their responses on the Likert-style survey. This study adds to the available literature demonstrating the effectiveness of the signalling, segmenting and embodiment principles for increasing learner engagement with multimedia lectures.