Hands-Free Authentication for Virtual Assistants with Trusted IoT Device and Machine Learning

Virtual assistants, deployed on smartphone and smart speaker devices, enable hands-free financial transactions by voice commands. Even though these voice transactions are frictionless for end users, they are susceptible to typical attacks to authentication protocols (e.g., replay). Using traditional knowledge-based or possession-based authentication with additional invasive interactions raises users concerns regarding security and usefulness. State-of-the-art schemes for trusted devices with physical unclonable functions (PUF) have complex enrollment processes. We propose a scheme based on a challenge response protocol with a trusted Internet of Things (IoT) autonomous device for hands-free scenarios (i.e., with no additional user interaction), integrated with smart home behavior for continuous authentication. The protocol was validated with automatic formal security analysis. A proof of concept with websockets presented an average response time of 383 ms for mutual authentication using a 6-message protocol with a simple enrollment process. We performed hands-free activity recognition of a specific user, based on smart home testbed data from a 2-month period, obtaining an accuracy of 97% and a recall of 81%. Given the data minimization privacy principle, we could reduce the total number of smart home events time series from 7 to 5. When compared with existing invasive solutions, our non-invasive mechanism contributes to the efforts to enhance the usability of financial institutions’ virtual assistants, while maintaining security and privacy.