Cargando…
Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
INTRODUCTION: Adaptive learning opportunities and individualized, timely feedback are considered to be effective support measures for students' writing in educational contexts. However, the extensive time and expertise required to analyze numerous drafts of student writing pose a barrier to tea...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10351274/ https://www.ncbi.nlm.nih.gov/pubmed/37465061 http://dx.doi.org/10.3389/frai.2023.1162454 |
_version_ | 1785074309709955072 |
---|---|
author | Fleckenstein, Johanna Liebenow, Lucas W. Meyer, Jennifer |
author_facet | Fleckenstein, Johanna Liebenow, Lucas W. Meyer, Jennifer |
author_sort | Fleckenstein, Johanna |
collection | PubMed |
description | INTRODUCTION: Adaptive learning opportunities and individualized, timely feedback are considered to be effective support measures for students' writing in educational contexts. However, the extensive time and expertise required to analyze numerous drafts of student writing pose a barrier to teaching. Automated writing evaluation (AWE) tools can be used for individual feedback based on advances in Artificial Intelligence (AI) technology. A number of primary (quasi-)experimental studies have investigated the effect of AWE feedback on students' writing performance. METHODS: This paper provides a meta-analysis of the effectiveness of AWE feedback tools. The literature search yielded 4,462 entries, of which 20 studies (k = 84; N = 2, 828) met the pre-specified inclusion criteria. A moderator analysis investigated the impact of the characteristics of the learner, the intervention, and the outcome measures. RESULTS: Overall, results based on a three-level model with random effects show a medium effect (g = 0.55) of automated feedback on students' writing performance. However, the significant heterogeneity in the data indicates that the use of automated feedback tools cannot be understood as a single consistent form of intervention. Even though for some of the moderators we found substantial differences in effect sizes, none of the subgroup comparisons were statistically significant. DISCUSSION: We discuss these findings in light of automated feedback use in educational practice and give recommendations for future research. |
format | Online Article Text |
id | pubmed-10351274 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-103512742023-07-18 Automated feedback and writing: a multi-level meta-analysis of effects on students' performance Fleckenstein, Johanna Liebenow, Lucas W. Meyer, Jennifer Front Artif Intell Artificial Intelligence INTRODUCTION: Adaptive learning opportunities and individualized, timely feedback are considered to be effective support measures for students' writing in educational contexts. However, the extensive time and expertise required to analyze numerous drafts of student writing pose a barrier to teaching. Automated writing evaluation (AWE) tools can be used for individual feedback based on advances in Artificial Intelligence (AI) technology. A number of primary (quasi-)experimental studies have investigated the effect of AWE feedback on students' writing performance. METHODS: This paper provides a meta-analysis of the effectiveness of AWE feedback tools. The literature search yielded 4,462 entries, of which 20 studies (k = 84; N = 2, 828) met the pre-specified inclusion criteria. A moderator analysis investigated the impact of the characteristics of the learner, the intervention, and the outcome measures. RESULTS: Overall, results based on a three-level model with random effects show a medium effect (g = 0.55) of automated feedback on students' writing performance. However, the significant heterogeneity in the data indicates that the use of automated feedback tools cannot be understood as a single consistent form of intervention. Even though for some of the moderators we found substantial differences in effect sizes, none of the subgroup comparisons were statistically significant. DISCUSSION: We discuss these findings in light of automated feedback use in educational practice and give recommendations for future research. Frontiers Media S.A. 2023-07-03 /pmc/articles/PMC10351274/ /pubmed/37465061 http://dx.doi.org/10.3389/frai.2023.1162454 Text en Copyright © 2023 Fleckenstein, Liebenow and Meyer. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Artificial Intelligence Fleckenstein, Johanna Liebenow, Lucas W. Meyer, Jennifer Automated feedback and writing: a multi-level meta-analysis of effects on students' performance |
title | Automated feedback and writing: a multi-level meta-analysis of effects on students' performance |
title_full | Automated feedback and writing: a multi-level meta-analysis of effects on students' performance |
title_fullStr | Automated feedback and writing: a multi-level meta-analysis of effects on students' performance |
title_full_unstemmed | Automated feedback and writing: a multi-level meta-analysis of effects on students' performance |
title_short | Automated feedback and writing: a multi-level meta-analysis of effects on students' performance |
title_sort | automated feedback and writing: a multi-level meta-analysis of effects on students' performance |
topic | Artificial Intelligence |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10351274/ https://www.ncbi.nlm.nih.gov/pubmed/37465061 http://dx.doi.org/10.3389/frai.2023.1162454 |
work_keys_str_mv | AT fleckensteinjohanna automatedfeedbackandwritingamultilevelmetaanalysisofeffectsonstudentsperformance AT liebenowlucasw automatedfeedbackandwritingamultilevelmetaanalysisofeffectsonstudentsperformance AT meyerjennifer automatedfeedbackandwritingamultilevelmetaanalysisofeffectsonstudentsperformance |