Cargando…

Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination

PURPOSE: The purpose of this study was to improve the quality of items on the Korean Nursing Licensing Examination by developing and evaluating case-based items that reflect integrated nursing knowledge. METHODS: We conducted a cross-sectional observational study to develop new case-based items. The...

Descripción completa

Detalles Bibliográficos
Autores principales: Park, In Sook, Suh, Yeon Ok, Park, Hae Sook, Kang, So Young, Kim, Kwang Sung, Kim, Gyung Hee, Choi, Yeon-Hee, Kim, Hyun-Ju
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Korea Health Personnel Licensing Examination Institute 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5729210/
https://www.ncbi.nlm.nih.gov/pubmed/28900070
http://dx.doi.org/10.3352/jeehp.2017.14.20
_version_ 1783286145834024960
author Park, In Sook
Suh, Yeon Ok
Park, Hae Sook
Kang, So Young
Kim, Kwang Sung
Kim, Gyung Hee
Choi, Yeon-Hee
Kim, Hyun-Ju
author_facet Park, In Sook
Suh, Yeon Ok
Park, Hae Sook
Kang, So Young
Kim, Kwang Sung
Kim, Gyung Hee
Choi, Yeon-Hee
Kim, Hyun-Ju
author_sort Park, In Sook
collection PubMed
description PURPOSE: The purpose of this study was to improve the quality of items on the Korean Nursing Licensing Examination by developing and evaluating case-based items that reflect integrated nursing knowledge. METHODS: We conducted a cross-sectional observational study to develop new case-based items. The methods for developing test items included expert workshops, brainstorming, and verification of content validity. After a mock examination of undergraduate nursing students using the newly developed case-based items, we evaluated the appropriateness of the items through classical test theory and item response theory. RESULTS: A total of 50 case-based items were developed for the mock examination, and content validity was evaluated. The question items integrated 34 discrete elements of integrated nursing knowledge. The mock examination was taken by 741 baccalaureate students in their fourth year of study at 13 universities. Their average score on the mock examination was 57.4, and the examination showed a reliability of 0.40. According to classical test theory, the average level of item difficulty of the items was 57.4% (80%–100% for 12 items; 60%–80% for 13 items; and less than 60% for 25 items). The mean discrimination index was 0.19, and was above 0.30 for 11 items and 0.20 to 0.29 for 15 items. According to item response theory, the item discrimination parameter (in the logistic model) was none for 10 items (0.00), very low for 20 items (0.01 to 0.34), low for 12 items (0.35 to 0.64), moderate for 6 items (0.65 to 1.34), high for 1 item (1.35 to 1.69), and very high for 1 item (above 1.70). The item difficulty was very easy for 24 items (below −2.0), easy for 8 items (−2.0 to −0.5), medium for 6 items (−0.5 to 0.5), hard for 3 items (0.5 to 2.0), and very hard for 9 items (2.0 or above). The goodness-of-fit test in terms of the 2-parameter item response model between the range of 2.0 to 0.5 revealed that 12 items had an ideal correct answer rate. CONCLUSION: We surmised that the low reliability of the mock examination was influenced by the timing of the test for the examinees and the inappropriate difficulty of the items. Our study suggested a methodology for the development of future case-based items for the Korean Nursing Licensing Examination.
format Online
Article
Text
id pubmed-5729210
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher Korea Health Personnel Licensing Examination Institute
record_format MEDLINE/PubMed
spelling pubmed-57292102017-12-19 Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination Park, In Sook Suh, Yeon Ok Park, Hae Sook Kang, So Young Kim, Kwang Sung Kim, Gyung Hee Choi, Yeon-Hee Kim, Hyun-Ju J Educ Eval Health Prof Technical Report PURPOSE: The purpose of this study was to improve the quality of items on the Korean Nursing Licensing Examination by developing and evaluating case-based items that reflect integrated nursing knowledge. METHODS: We conducted a cross-sectional observational study to develop new case-based items. The methods for developing test items included expert workshops, brainstorming, and verification of content validity. After a mock examination of undergraduate nursing students using the newly developed case-based items, we evaluated the appropriateness of the items through classical test theory and item response theory. RESULTS: A total of 50 case-based items were developed for the mock examination, and content validity was evaluated. The question items integrated 34 discrete elements of integrated nursing knowledge. The mock examination was taken by 741 baccalaureate students in their fourth year of study at 13 universities. Their average score on the mock examination was 57.4, and the examination showed a reliability of 0.40. According to classical test theory, the average level of item difficulty of the items was 57.4% (80%–100% for 12 items; 60%–80% for 13 items; and less than 60% for 25 items). The mean discrimination index was 0.19, and was above 0.30 for 11 items and 0.20 to 0.29 for 15 items. According to item response theory, the item discrimination parameter (in the logistic model) was none for 10 items (0.00), very low for 20 items (0.01 to 0.34), low for 12 items (0.35 to 0.64), moderate for 6 items (0.65 to 1.34), high for 1 item (1.35 to 1.69), and very high for 1 item (above 1.70). The item difficulty was very easy for 24 items (below −2.0), easy for 8 items (−2.0 to −0.5), medium for 6 items (−0.5 to 0.5), hard for 3 items (0.5 to 2.0), and very hard for 9 items (2.0 or above). The goodness-of-fit test in terms of the 2-parameter item response model between the range of 2.0 to 0.5 revealed that 12 items had an ideal correct answer rate. CONCLUSION: We surmised that the low reliability of the mock examination was influenced by the timing of the test for the examinees and the inappropriate difficulty of the items. Our study suggested a methodology for the development of future case-based items for the Korean Nursing Licensing Examination. Korea Health Personnel Licensing Examination Institute 2017-09-11 /pmc/articles/PMC5729210/ /pubmed/28900070 http://dx.doi.org/10.3352/jeehp.2017.14.20 Text en © 2017, Korea Health Personnel Licensing Examination Institute http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
spellingShingle Technical Report
Park, In Sook
Suh, Yeon Ok
Park, Hae Sook
Kang, So Young
Kim, Kwang Sung
Kim, Gyung Hee
Choi, Yeon-Hee
Kim, Hyun-Ju
Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
title Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
title_full Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
title_fullStr Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
title_full_unstemmed Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
title_short Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
title_sort item development process and analysis of 50 case-based items for implementation on the korean nursing licensing examination
topic Technical Report
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5729210/
https://www.ncbi.nlm.nih.gov/pubmed/28900070
http://dx.doi.org/10.3352/jeehp.2017.14.20
work_keys_str_mv AT parkinsook itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination
AT suhyeonok itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination
AT parkhaesook itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination
AT kangsoyoung itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination
AT kimkwangsung itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination
AT kimgyunghee itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination
AT choiyeonhee itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination
AT kimhyunju itemdevelopmentprocessandanalysisof50casebaseditemsforimplementationonthekoreannursinglicensingexamination