Cargando…
Gestalt assessment of online educational resources may not be sufficiently reliable and consistent
PURPOSE: Online open educational resources are increasingly used in medical education, particularly blogs and podcasts. However, it is unclear whether these resources can be adequately appraised by end-users. Our goal was to determine whether gestalt-based recommendations are sufficient for emergenc...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Bohn Stafleu van Loghum
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5383576/ https://www.ncbi.nlm.nih.gov/pubmed/28243948 http://dx.doi.org/10.1007/s40037-017-0343-3 |
_version_ | 1782520300872663040 |
---|---|
author | Krishnan, Keeth Thoma, Brent Trueger, N. Seth Lin, Michelle Chan, Teresa M. |
author_facet | Krishnan, Keeth Thoma, Brent Trueger, N. Seth Lin, Michelle Chan, Teresa M. |
author_sort | Krishnan, Keeth |
collection | PubMed |
description | PURPOSE: Online open educational resources are increasingly used in medical education, particularly blogs and podcasts. However, it is unclear whether these resources can be adequately appraised by end-users. Our goal was to determine whether gestalt-based recommendations are sufficient for emergency medicine trainees and attending physicians to reliably recommend online educational resources to others. METHODS: Raters (33 trainees and 21 attendings in emergency medicine from North America) were asked to rate 40 blog posts according to whether, based on their gestalt, they would recommend the resource to (1) a trainee or (2) an attending physician. The ratings’ reliability was assessed using intraclass correlation coefficients (ICC). Associations between groups’ mean scores were assessed using Pearson’s r. A repeated measures analysis of variance (RM-ANOVA) was completed to determine the effect of the level of training on gestalt recommendation scale (i. e. trainee vs. attending). RESULTS: Trainees demonstrated poor reliability when recommending resources for other trainees (ICC = 0.21, 95% CI 0.13–0.39) and attendings (ICC = 0.16, 95% CI = 0.09–0.30). Similarly, attendings had poor reliability when recommending resources for trainees (ICC = 0.27, 95% CI 0.18–0.41) and other attendings (ICC = 0.22, 95% CI 0.14–0.35). There were moderate correlations between the mean scores for each blog post when either trainees or attendings considered the same target audience. The RM-ANOVA also corroborated that there is a main effect of the proposed target audience on the ratings by both trainees and attendings. CONCLUSIONS: A gestalt-based rating system is not sufficiently reliable when recommending online educational resources to trainees and attendings. Trainees’ gestalt ratings for recommending resources for both groups were especially unreliable. Our findings suggest the need for structured rating systems to rate online educational resources. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi: 10.1007/s40037-017-0343-3) contains supplementary material, which is available to authorized users. |
format | Online Article Text |
id | pubmed-5383576 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | Bohn Stafleu van Loghum |
record_format | MEDLINE/PubMed |
spelling | pubmed-53835762017-04-20 Gestalt assessment of online educational resources may not be sufficiently reliable and consistent Krishnan, Keeth Thoma, Brent Trueger, N. Seth Lin, Michelle Chan, Teresa M. Perspect Med Educ Original Article PURPOSE: Online open educational resources are increasingly used in medical education, particularly blogs and podcasts. However, it is unclear whether these resources can be adequately appraised by end-users. Our goal was to determine whether gestalt-based recommendations are sufficient for emergency medicine trainees and attending physicians to reliably recommend online educational resources to others. METHODS: Raters (33 trainees and 21 attendings in emergency medicine from North America) were asked to rate 40 blog posts according to whether, based on their gestalt, they would recommend the resource to (1) a trainee or (2) an attending physician. The ratings’ reliability was assessed using intraclass correlation coefficients (ICC). Associations between groups’ mean scores were assessed using Pearson’s r. A repeated measures analysis of variance (RM-ANOVA) was completed to determine the effect of the level of training on gestalt recommendation scale (i. e. trainee vs. attending). RESULTS: Trainees demonstrated poor reliability when recommending resources for other trainees (ICC = 0.21, 95% CI 0.13–0.39) and attendings (ICC = 0.16, 95% CI = 0.09–0.30). Similarly, attendings had poor reliability when recommending resources for trainees (ICC = 0.27, 95% CI 0.18–0.41) and other attendings (ICC = 0.22, 95% CI 0.14–0.35). There were moderate correlations between the mean scores for each blog post when either trainees or attendings considered the same target audience. The RM-ANOVA also corroborated that there is a main effect of the proposed target audience on the ratings by both trainees and attendings. CONCLUSIONS: A gestalt-based rating system is not sufficiently reliable when recommending online educational resources to trainees and attendings. Trainees’ gestalt ratings for recommending resources for both groups were especially unreliable. Our findings suggest the need for structured rating systems to rate online educational resources. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi: 10.1007/s40037-017-0343-3) contains supplementary material, which is available to authorized users. Bohn Stafleu van Loghum 2017-02-27 2017-04 /pmc/articles/PMC5383576/ /pubmed/28243948 http://dx.doi.org/10.1007/s40037-017-0343-3 Text en © The Author(s) 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. |
spellingShingle | Original Article Krishnan, Keeth Thoma, Brent Trueger, N. Seth Lin, Michelle Chan, Teresa M. Gestalt assessment of online educational resources may not be sufficiently reliable and consistent |
title | Gestalt assessment of online educational resources may not be sufficiently reliable and consistent |
title_full | Gestalt assessment of online educational resources may not be sufficiently reliable and consistent |
title_fullStr | Gestalt assessment of online educational resources may not be sufficiently reliable and consistent |
title_full_unstemmed | Gestalt assessment of online educational resources may not be sufficiently reliable and consistent |
title_short | Gestalt assessment of online educational resources may not be sufficiently reliable and consistent |
title_sort | gestalt assessment of online educational resources may not be sufficiently reliable and consistent |
topic | Original Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5383576/ https://www.ncbi.nlm.nih.gov/pubmed/28243948 http://dx.doi.org/10.1007/s40037-017-0343-3 |
work_keys_str_mv | AT krishnankeeth gestaltassessmentofonlineeducationalresourcesmaynotbesufficientlyreliableandconsistent AT thomabrent gestaltassessmentofonlineeducationalresourcesmaynotbesufficientlyreliableandconsistent AT truegernseth gestaltassessmentofonlineeducationalresourcesmaynotbesufficientlyreliableandconsistent AT linmichelle gestaltassessmentofonlineeducationalresourcesmaynotbesufficientlyreliableandconsistent AT chanteresam gestaltassessmentofonlineeducationalresourcesmaynotbesufficientlyreliableandconsistent |