Cargando…

Automated feedback and writing: a multi-level meta-analysis of effects on students' performance

INTRODUCTION: Adaptive learning opportunities and individualized, timely feedback are considered to be effective support measures for students' writing in educational contexts. However, the extensive time and expertise required to analyze numerous drafts of student writing pose a barrier to tea...

Descripción completa

Detalles Bibliográficos
Autores principales: Fleckenstein, Johanna, Liebenow, Lucas W., Meyer, Jennifer
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10351274/
https://www.ncbi.nlm.nih.gov/pubmed/37465061
http://dx.doi.org/10.3389/frai.2023.1162454
_version_ 1785074309709955072
author Fleckenstein, Johanna
Liebenow, Lucas W.
Meyer, Jennifer
author_facet Fleckenstein, Johanna
Liebenow, Lucas W.
Meyer, Jennifer
author_sort Fleckenstein, Johanna
collection PubMed
description INTRODUCTION: Adaptive learning opportunities and individualized, timely feedback are considered to be effective support measures for students' writing in educational contexts. However, the extensive time and expertise required to analyze numerous drafts of student writing pose a barrier to teaching. Automated writing evaluation (AWE) tools can be used for individual feedback based on advances in Artificial Intelligence (AI) technology. A number of primary (quasi-)experimental studies have investigated the effect of AWE feedback on students' writing performance. METHODS: This paper provides a meta-analysis of the effectiveness of AWE feedback tools. The literature search yielded 4,462 entries, of which 20 studies (k = 84; N = 2, 828) met the pre-specified inclusion criteria. A moderator analysis investigated the impact of the characteristics of the learner, the intervention, and the outcome measures. RESULTS: Overall, results based on a three-level model with random effects show a medium effect (g = 0.55) of automated feedback on students' writing performance. However, the significant heterogeneity in the data indicates that the use of automated feedback tools cannot be understood as a single consistent form of intervention. Even though for some of the moderators we found substantial differences in effect sizes, none of the subgroup comparisons were statistically significant. DISCUSSION: We discuss these findings in light of automated feedback use in educational practice and give recommendations for future research.
format Online
Article
Text
id pubmed-10351274
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-103512742023-07-18 Automated feedback and writing: a multi-level meta-analysis of effects on students' performance Fleckenstein, Johanna Liebenow, Lucas W. Meyer, Jennifer Front Artif Intell Artificial Intelligence INTRODUCTION: Adaptive learning opportunities and individualized, timely feedback are considered to be effective support measures for students' writing in educational contexts. However, the extensive time and expertise required to analyze numerous drafts of student writing pose a barrier to teaching. Automated writing evaluation (AWE) tools can be used for individual feedback based on advances in Artificial Intelligence (AI) technology. A number of primary (quasi-)experimental studies have investigated the effect of AWE feedback on students' writing performance. METHODS: This paper provides a meta-analysis of the effectiveness of AWE feedback tools. The literature search yielded 4,462 entries, of which 20 studies (k = 84; N = 2, 828) met the pre-specified inclusion criteria. A moderator analysis investigated the impact of the characteristics of the learner, the intervention, and the outcome measures. RESULTS: Overall, results based on a three-level model with random effects show a medium effect (g = 0.55) of automated feedback on students' writing performance. However, the significant heterogeneity in the data indicates that the use of automated feedback tools cannot be understood as a single consistent form of intervention. Even though for some of the moderators we found substantial differences in effect sizes, none of the subgroup comparisons were statistically significant. DISCUSSION: We discuss these findings in light of automated feedback use in educational practice and give recommendations for future research. Frontiers Media S.A. 2023-07-03 /pmc/articles/PMC10351274/ /pubmed/37465061 http://dx.doi.org/10.3389/frai.2023.1162454 Text en Copyright © 2023 Fleckenstein, Liebenow and Meyer. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Artificial Intelligence
Fleckenstein, Johanna
Liebenow, Lucas W.
Meyer, Jennifer
Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
title Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
title_full Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
title_fullStr Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
title_full_unstemmed Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
title_short Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
title_sort automated feedback and writing: a multi-level meta-analysis of effects on students' performance
topic Artificial Intelligence
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10351274/
https://www.ncbi.nlm.nih.gov/pubmed/37465061
http://dx.doi.org/10.3389/frai.2023.1162454
work_keys_str_mv AT fleckensteinjohanna automatedfeedbackandwritingamultilevelmetaanalysisofeffectsonstudentsperformance
AT liebenowlucasw automatedfeedbackandwritingamultilevelmetaanalysisofeffectsonstudentsperformance
AT meyerjennifer automatedfeedbackandwritingamultilevelmetaanalysisofeffectsonstudentsperformance