Cargando…

Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements

The COVID-19 pandemic meant that, in 2020, students in England were unable to sit their examinations and instead received predicted grades, or “centre assessment grades” (CAGs), from their teachers to allow them to progress. Using the Grading and Admissions Data for England (GRADE) dataset for stude...

Descripción completa

Detalles Bibliográficos
Autor principal: Magowan, Louis
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer Nature Singapore 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10184100/
https://www.ncbi.nlm.nih.gov/pubmed/37363803
http://dx.doi.org/10.1007/s42001-023-00206-x
_version_ 1785042099046973440
author Magowan, Louis
author_facet Magowan, Louis
author_sort Magowan, Louis
collection PubMed
description The COVID-19 pandemic meant that, in 2020, students in England were unable to sit their examinations and instead received predicted grades, or “centre assessment grades” (CAGs), from their teachers to allow them to progress. Using the Grading and Admissions Data for England (GRADE) dataset for students from 2018 to 2020, this study treats the use of CAGs as a natural experiment for causally understanding how teacher judgements of academic ability may be biased according to the demographic and socio-economic characteristics of their students. A variety of machine learning models were trained on the 2018–19 data and then used to generate predictions for what the 2020 students were likely to have received had their examinations taken place as usual. The differences between these predictions and the CAGs that students received were calculated and then averaged across students’ different characteristics, revealing what the treatment effects of the use of CAGs were likely to have been for different types of students. No evidence of absolute negative bias against students of any demographic or socio-economic characteristic was found, with all groups of students having received higher CAGs than the grades they were likely to have received had they sat their examinations. Some evidence for relative bias was found, with consistent, but insubstantial differences being observed in the treatment effects of certain groups. However, when higher-order interactions of student characteristics were considered, these differences became more substantial. Intersectional perspectives which emphasise interactions and sub-group differences should be used more widely within quantitative educational equalities research.
format Online
Article
Text
id pubmed-10184100
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Springer Nature Singapore
record_format MEDLINE/PubMed
spelling pubmed-101841002023-05-16 Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements Magowan, Louis J Comput Soc Sci Research Article The COVID-19 pandemic meant that, in 2020, students in England were unable to sit their examinations and instead received predicted grades, or “centre assessment grades” (CAGs), from their teachers to allow them to progress. Using the Grading and Admissions Data for England (GRADE) dataset for students from 2018 to 2020, this study treats the use of CAGs as a natural experiment for causally understanding how teacher judgements of academic ability may be biased according to the demographic and socio-economic characteristics of their students. A variety of machine learning models were trained on the 2018–19 data and then used to generate predictions for what the 2020 students were likely to have received had their examinations taken place as usual. The differences between these predictions and the CAGs that students received were calculated and then averaged across students’ different characteristics, revealing what the treatment effects of the use of CAGs were likely to have been for different types of students. No evidence of absolute negative bias against students of any demographic or socio-economic characteristic was found, with all groups of students having received higher CAGs than the grades they were likely to have received had they sat their examinations. Some evidence for relative bias was found, with consistent, but insubstantial differences being observed in the treatment effects of certain groups. However, when higher-order interactions of student characteristics were considered, these differences became more substantial. Intersectional perspectives which emphasise interactions and sub-group differences should be used more widely within quantitative educational equalities research. Springer Nature Singapore 2023-05-15 /pmc/articles/PMC10184100/ /pubmed/37363803 http://dx.doi.org/10.1007/s42001-023-00206-x Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Research Article
Magowan, Louis
Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements
title Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements
title_full Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements
title_fullStr Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements
title_full_unstemmed Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements
title_short Centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements
title_sort centre assessment grades in 2020: a natural experiment for investigating bias in teacher judgements
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10184100/
https://www.ncbi.nlm.nih.gov/pubmed/37363803
http://dx.doi.org/10.1007/s42001-023-00206-x
work_keys_str_mv AT magowanlouis centreassessmentgradesin2020anaturalexperimentforinvestigatingbiasinteacherjudgements