Cargando…

A fairer way to compare researchers at any career stage and in any discipline using open-access citation data

The pursuit of simple, yet fair, unbiased, and objective measures of researcher performance has occupied bibliometricians and the research community as a whole for decades. However, despite the diversity of available metrics, most are either complex to calculate or not readily applied in the most co...

Descripción completa

Detalles Bibliográficos
Autores principales: Bradshaw, Corey J. A., Chalker, Justin M., Crabtree, Stefani A., Eijkelkamp, Bart A., Long, John A., Smith, Justine R., Trinajstic, Kate, Weisbecker, Vera
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8432834/
https://www.ncbi.nlm.nih.gov/pubmed/34506560
http://dx.doi.org/10.1371/journal.pone.0257141
_version_ 1783751248834461696
author Bradshaw, Corey J. A.
Chalker, Justin M.
Crabtree, Stefani A.
Eijkelkamp, Bart A.
Long, John A.
Smith, Justine R.
Trinajstic, Kate
Weisbecker, Vera
author_facet Bradshaw, Corey J. A.
Chalker, Justin M.
Crabtree, Stefani A.
Eijkelkamp, Bart A.
Long, John A.
Smith, Justine R.
Trinajstic, Kate
Weisbecker, Vera
author_sort Bradshaw, Corey J. A.
collection PubMed
description The pursuit of simple, yet fair, unbiased, and objective measures of researcher performance has occupied bibliometricians and the research community as a whole for decades. However, despite the diversity of available metrics, most are either complex to calculate or not readily applied in the most common assessment exercises (e.g., grant assessment, job applications). The ubiquity of metrics like the h-index (h papers with at least h citations) and its time-corrected variant, the m-quotient (h-index ÷ number of years publishing) therefore reflect the ease of use rather than their capacity to differentiate researchers fairly among disciplines, career stage, or gender. We address this problem here by defining an easily calculated index based on publicly available citation data (Google Scholar) that corrects for most biases and allows assessors to compare researchers at any stage of their career and from any discipline on the same scale. Our ε′-index violates fewer statistical assumptions relative to other metrics when comparing groups of researchers, and can be easily modified to remove inherent gender biases in citation data. We demonstrate the utility of the ε′-index using a sample of 480 researchers with Google Scholar profiles, stratified evenly into eight disciplines (archaeology, chemistry, ecology, evolution and development, geology, microbiology, ophthalmology, palaeontology), three career stages (early, mid-, late-career), and two genders. We advocate the use of the ε′-index whenever assessors must compare research performance among researchers of different backgrounds, but emphasize that no single index should be used exclusively to rank researcher capability.
format Online
Article
Text
id pubmed-8432834
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-84328342021-09-11 A fairer way to compare researchers at any career stage and in any discipline using open-access citation data Bradshaw, Corey J. A. Chalker, Justin M. Crabtree, Stefani A. Eijkelkamp, Bart A. Long, John A. Smith, Justine R. Trinajstic, Kate Weisbecker, Vera PLoS One Research Article The pursuit of simple, yet fair, unbiased, and objective measures of researcher performance has occupied bibliometricians and the research community as a whole for decades. However, despite the diversity of available metrics, most are either complex to calculate or not readily applied in the most common assessment exercises (e.g., grant assessment, job applications). The ubiquity of metrics like the h-index (h papers with at least h citations) and its time-corrected variant, the m-quotient (h-index ÷ number of years publishing) therefore reflect the ease of use rather than their capacity to differentiate researchers fairly among disciplines, career stage, or gender. We address this problem here by defining an easily calculated index based on publicly available citation data (Google Scholar) that corrects for most biases and allows assessors to compare researchers at any stage of their career and from any discipline on the same scale. Our ε′-index violates fewer statistical assumptions relative to other metrics when comparing groups of researchers, and can be easily modified to remove inherent gender biases in citation data. We demonstrate the utility of the ε′-index using a sample of 480 researchers with Google Scholar profiles, stratified evenly into eight disciplines (archaeology, chemistry, ecology, evolution and development, geology, microbiology, ophthalmology, palaeontology), three career stages (early, mid-, late-career), and two genders. We advocate the use of the ε′-index whenever assessors must compare research performance among researchers of different backgrounds, but emphasize that no single index should be used exclusively to rank researcher capability. Public Library of Science 2021-09-10 /pmc/articles/PMC8432834/ /pubmed/34506560 http://dx.doi.org/10.1371/journal.pone.0257141 Text en © 2021 Bradshaw et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Bradshaw, Corey J. A.
Chalker, Justin M.
Crabtree, Stefani A.
Eijkelkamp, Bart A.
Long, John A.
Smith, Justine R.
Trinajstic, Kate
Weisbecker, Vera
A fairer way to compare researchers at any career stage and in any discipline using open-access citation data
title A fairer way to compare researchers at any career stage and in any discipline using open-access citation data
title_full A fairer way to compare researchers at any career stage and in any discipline using open-access citation data
title_fullStr A fairer way to compare researchers at any career stage and in any discipline using open-access citation data
title_full_unstemmed A fairer way to compare researchers at any career stage and in any discipline using open-access citation data
title_short A fairer way to compare researchers at any career stage and in any discipline using open-access citation data
title_sort fairer way to compare researchers at any career stage and in any discipline using open-access citation data
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8432834/
https://www.ncbi.nlm.nih.gov/pubmed/34506560
http://dx.doi.org/10.1371/journal.pone.0257141
work_keys_str_mv AT bradshawcoreyja afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT chalkerjustinm afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT crabtreestefania afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT eijkelkampbarta afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT longjohna afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT smithjustiner afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT trinajstickate afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT weisbeckervera afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT bradshawcoreyja fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT chalkerjustinm fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT crabtreestefania fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT eijkelkampbarta fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT longjohna fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT smithjustiner fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT trinajstickate fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata
AT weisbeckervera fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata