Cargando…

Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England

Objective To assess the robustness of patient responses to a new national survey of patient experience as a basis for providing financial incentives to doctors. Design Analysis of the representativeness of the respondents to the GP Patient Survey compared with those who were sampled (5.5 million pat...

Descripción completa

Detalles Bibliográficos
Autores principales: Roland, Martin, Elliott, Marc, Lyratzopoulos, Georgios, Barbiere, Josephine, Parker, Richard A, Smith, Patten, Bower, Peter, Campbell, John
Formato: Texto
Lenguaje:English
Publicado: BMJ Publishing Group Ltd. 2009
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2754504/
https://www.ncbi.nlm.nih.gov/pubmed/19808811
http://dx.doi.org/10.1136/bmj.b3851
_version_ 1782172410099793920
author Roland, Martin
Elliott, Marc
Lyratzopoulos, Georgios
Barbiere, Josephine
Parker, Richard A
Smith, Patten
Bower, Peter
Campbell, John
author_facet Roland, Martin
Elliott, Marc
Lyratzopoulos, Georgios
Barbiere, Josephine
Parker, Richard A
Smith, Patten
Bower, Peter
Campbell, John
author_sort Roland, Martin
collection PubMed
description Objective To assess the robustness of patient responses to a new national survey of patient experience as a basis for providing financial incentives to doctors. Design Analysis of the representativeness of the respondents to the GP Patient Survey compared with those who were sampled (5.5 million patients registered with 8273 general practices in England in January 2009) and with the general population. Analysis of non-response bias looked at the relation between practice response rates and scores on the survey. Analysis of the reliability of the survey estimated the proportion of the variance of practice scores attributable to true differences between practices. Results The overall response rate was 38.2% (2.2 million responses), which is comparable to that in surveys using similar methodology in the UK. Men, young adults, and people living in deprived areas were under-represented among respondents. However, for questions related to pay for performance, there was no systematic association between response rates and questionnaire scores. Two questions which triggered payments to general practitioners were reliable measures of practice performance, with average practice-level reliability coefficients of 93.2% and 95.0%. Less than 3% and 0.5% of practices had fewer than the number of responses required to achieve conventional reliability levels of 90% and 70%. A change to the payment formula in 2009 resulted in an increase in the average impact of random variation in patient scores on payments to general practitioners compared with payments made in 2007 and 2008. Conclusions There is little evidence to support the concern of some general practitioners that low response rates and selective non-response bias have led to systematic unfairness in payments attached to questionnaire scores. The study raises issues relating to the validity and reliability of payments based on patient surveys and provides lessons for the UK and for other countries considering the use of patient experience as part of pay for performance schemes.
format Text
id pubmed-2754504
institution National Center for Biotechnology Information
language English
publishDate 2009
publisher BMJ Publishing Group Ltd.
record_format MEDLINE/PubMed
spelling pubmed-27545042009-12-30 Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England Roland, Martin Elliott, Marc Lyratzopoulos, Georgios Barbiere, Josephine Parker, Richard A Smith, Patten Bower, Peter Campbell, John BMJ Research Objective To assess the robustness of patient responses to a new national survey of patient experience as a basis for providing financial incentives to doctors. Design Analysis of the representativeness of the respondents to the GP Patient Survey compared with those who were sampled (5.5 million patients registered with 8273 general practices in England in January 2009) and with the general population. Analysis of non-response bias looked at the relation between practice response rates and scores on the survey. Analysis of the reliability of the survey estimated the proportion of the variance of practice scores attributable to true differences between practices. Results The overall response rate was 38.2% (2.2 million responses), which is comparable to that in surveys using similar methodology in the UK. Men, young adults, and people living in deprived areas were under-represented among respondents. However, for questions related to pay for performance, there was no systematic association between response rates and questionnaire scores. Two questions which triggered payments to general practitioners were reliable measures of practice performance, with average practice-level reliability coefficients of 93.2% and 95.0%. Less than 3% and 0.5% of practices had fewer than the number of responses required to achieve conventional reliability levels of 90% and 70%. A change to the payment formula in 2009 resulted in an increase in the average impact of random variation in patient scores on payments to general practitioners compared with payments made in 2007 and 2008. Conclusions There is little evidence to support the concern of some general practitioners that low response rates and selective non-response bias have led to systematic unfairness in payments attached to questionnaire scores. The study raises issues relating to the validity and reliability of payments based on patient surveys and provides lessons for the UK and for other countries considering the use of patient experience as part of pay for performance schemes. BMJ Publishing Group Ltd. 2009-09-29 /pmc/articles/PMC2754504/ /pubmed/19808811 http://dx.doi.org/10.1136/bmj.b3851 Text en This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
spellingShingle Research
Roland, Martin
Elliott, Marc
Lyratzopoulos, Georgios
Barbiere, Josephine
Parker, Richard A
Smith, Patten
Bower, Peter
Campbell, John
Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England
title Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England
title_full Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England
title_fullStr Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England
title_full_unstemmed Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England
title_short Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England
title_sort reliability of patient responses in pay for performance schemes: analysis of national general practitioner patient survey data in england
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2754504/
https://www.ncbi.nlm.nih.gov/pubmed/19808811
http://dx.doi.org/10.1136/bmj.b3851
work_keys_str_mv AT rolandmartin reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland
AT elliottmarc reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland
AT lyratzopoulosgeorgios reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland
AT barbierejosephine reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland
AT parkerricharda reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland
AT smithpatten reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland
AT bowerpeter reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland
AT campbelljohn reliabilityofpatientresponsesinpayforperformanceschemesanalysisofnationalgeneralpractitionerpatientsurveydatainengland