How to design & implement a modern communication middleware based on ZeroMQ

In 2011, CERN's Controls Middleware (CMW) team started a new project aiming to design and implement a new generation equipment access framework using modern, open-source products. After reviewing several communication libraries [1], ZeroMQ [2] was chosen as the transport layer for the new communication framework. The main design principles were: scalability, flexibility, easy to use and maintain. Several core ZeroMQ patterns were employed in order to provide reliable, asynchronous communication and dispatching of messages. The new product was implemented in Java and C++ for client and server side. It is the core middleware framework to control all CERN accelerators and the future GSI FAIR [3] complex. This paper presents the overall framework architecture; choices and lessons learnt while designing a scalable solution; challenges faced when designing a common API for two languages (Java and C++) and operational experience from using the new solution at CERN for 3 years. The lessons learnt and observations made can be applied to any modern software library responsible for fast, reliable, scalable communication and processing of many concurrent requests.