Smart agent system for insulin infusion protocol management: a simulation-based human factors evaluation study

OBJECTIVE: To compare the insulin infusion management of critically ill patients by nurses using either a common standard (ie, human completion of insulin infusion protocol steps) or smart agent (SA) system that integrates the electronic health record and infusion pump and automates insulin dose selection. DESIGN: A within subjects design where participants completed 12 simulation scenarios, in 4 blocks of 3 scenarios each. Each block was performed with either the manual standard or the SA system. The initial starting condition was randomised to manual standard or SA and alternated thereafter. SETTING: A simulation-based human factors evaluation conducted at a large academic medical centre. SUBJECTS: Twenty critical care nurses. INTERVENTIONS: A systems engineering intervention, the SA, for insulin infusion management. MEASUREMENTS: The primary study outcomes were error rates and task completion times. Secondary study outcomes were perceived workload, trust in automation and system usability, all measured with previously validated scales. MAIN RESULTS: The SA system produced significantly fewer dose errors compared with manual calculation (17% (n=20) vs 0, p<0.001). Participants were significantly faster, completing the protocol using the SA system (p<0.001). Overall ratings of workload for the SA system were significantly lower than with the manual system (p<0.001). For trust ratings, there was a significant interaction between time (first or second exposure) and the system used, such that after their second exposure to the two systems, participants had significantly more trust in the SA system. Participants rated the usability of the SA system significantly higher than the manual system (p<0.001). CONCLUSIONS: A systems engineering approach jointly optimised safety, efficiency and workload considerations.