Defining TestOps: Collaborative Behaviors and Technology-Driven Workflows Seen as Enablers of Effective Software Testing in DevOps

Context: DevOps is an increasingly popular approach to software development and software operations. Being understood as mutually integrated, both activities have been re-united under one single label. In contrast to traditional software development activities, DevOps promotes numerous fundamental changes, and the area of software testing is not an exception. Yet, the exact appearance of software testing within DevOps is poorly understood, so is the notion of TestOps. Objective: This paper explores TestOps as a concept rooted in industrial practice. Method: To provide a pluralist outline of practitioners’ views on What is TestOps, the YouTube platform was searched for digital content containing either “TestOps” or “DevTestOps” in the content title. Through a qualitative lens, the resulting set was systematically annotated and thematically analyzed in an inductive manner. Results: Referring to DevOps, practitioners use the notion of TestOps when characterizing a conceptual shift that occurs within the area of software testing. As a matter of fact, two dominant categories were found in the data: (i) TestOps as a new organizational philosophy; (ii) TestOps as an innovative software technique (i.e. process supported by technology). A set of high-level themes within each of these categories was identified and described. Conclusion: The study outlines an inconsistency in practitioner perspectives on the nature of TestOps. To decrease the identified conceptual ambiguity, the proposed model posits two complementary meanings of TestOps.