Difficulties in using simulation to assess abdominal palpation skills

OBJECTIVES: Abdominal palpation is an essential examination to diagnose various digestive system diseases. This study aimed to develop an objective and standardized test based on abdominal palpation simulators, and establish a credible pass/fail standard of basic competency. METHODS: Two tests were designed using the newly developed Jucheng abdominal palpation simulator (test 1) and the AbSim simulator (test 2), respectively. Validity evidence for both tests was gathered according to Messick’s contemporary framework by using experts to define test content and then administering the tests in a highly standardized way to participants of different experience. Different simulator setups modified by the built-in software were selected from hepatomegaly, splenomegaly, positive McBurney’s sign plus rebound tenderness, gallbladder tenderness (Murphy’s sign), pancreas tenderness, and a normal setup without pathologies, with six sets used in test 1 and five sets used in test 2. Different novices and experienced were included in the tests, and test 1 was also administered to an intermediate group. Scores and test time were collected and analyzed statistically. RESULTS: The internal consistency reliability of test 1 and test 2 showed low Cronbach’s alphas of 0.35 and -0.41, respectively. Cronbach’s alpha for palpation time across cases were 0.65 for test 1 and 0.76 for test 2. There was no statistical difference in total time spent and total scores among the three groups in test 1 (P-values (ANOVA) were 0.53 and 0.35 respectively), nor between novices and experienced groups in test 2 (P-values (t-test) were 0.13 and 1.0 respectively). It was not relevant to try to establish pass/fail standards due to the low reliability and lack of discriminatory ability of the tests. CONCLUSIONS: It was not possible to measure abdominal palpation skills in a valid way using either of the two standardized, simulation-based tests in our study. Assessment of the patient’s abdomen using palpation is a challenging clinical skill that is difficult to simulate as it highly relies on tactile sensations and adequate responsiveness from the patients.