Reliability in Healthcare Simulation Setting: A Definitional Review

The construct of reliability in health professions education serves as a measure of the congruence of interpretations across assessment tools. When used as an assessment strategy, healthcare simulation serves to elicit specific participant behaviors sought by medical educators. In healthcare simulation, reliability often refers to the ability to consistently reproduce a simulation and that reproducing a simulation setting can consistently expose participants to the same conditions, thus achieving simulation reliability. However, some articles have expressed that simulations are vulnerable to error stemming from design conceptualization to implementation, and the impact of social factors when participants interact and engage with others during participation. The purpose of this definitional review is to examine how reliability has been conceptualized and defined in healthcare simulation, and how the attributes of simulations may present challenges for the traditional concept of reliability in health professions education. Data collection and analysis was approached through a constructivist perspective and grounded theory strategies. Articles between 2009-2019 were filtered applying keywords related to simulation development and design. Data winnowing was structured around a framework viewing simulation as a social practice where participants interact with simulation setting attributes. Healthcare simulation setting reliability is not directly defined but described as errors introduced by the interactions between simulation design attributes and tasks performed by simulated participants. Based on the ontology of simulation’s design attributes believed to introduce setting errors, lexical terms related to reliability suggest how simulated participants are trained to refine or maintain their performance tasks that aim to mitigate errors. To achieve reliability in health professions education (HPE) and healthcare simulation, both domains seek to assess the consistency of a construct being measured. In HPE, reliability refers to the consistency of quality measures across a range of psychometric tests used to assess a participant’s medical aptitude. In healthcare simulation setting, reliability refers to the consistency of a simulated participant (SP) performing a task that is tailored to mitigate errors introduced by simulation design attributes. Consequently, inconsistencies in SP performance subject participants to setting errors exposing them to unequal conditions that influence competency achievement. What is already known on this subject: Performance competency assessment using healthcare simulation is increasingly common. The types of design attributes incorporated into a simulation setting. The use of incorporating simulated participants into a simulation setting. Simulated participants require training prior to simulation setting implementation. What this paper adds: Identifies attributes of a simulation setting that are most commonly thought to interfere with setting reliability. Identifies the relationships among setting attributes and simulated participant performances that influence setting reliability. Identifies terms tied to the achievement of simulation setting reliability. Examines simulated participant training processes aimed to mitigate errors introduced by simulation design attributes.