Multiagent Q-Learning-Based Mobility Management for Multi-Connectivity in mmWAVE Cellular Systems

Effective mobility management is crucial for efficient operation of next-generation cellular systems in the millimeter wave (mmWave) band. Massive multiple-input–multiple-output (MIMO) systems are seen as necessary to overcome the significant path losses in this band, but the highly directional beam makes the channels more susceptible to radio link failures due to blockages. To meet stringent capacity and reliability requirements, multi-connectivity has attracted significant attention. This paper proposes a multiagent distributed Q learning-based mobility management scheme for multi-connectivity in mmWave cellular systems. A hierarchical structure is adopted to address the model complexity and speed up the learning process. The performance is assessed using a realistic measurement data set collected from Wireless Insite in an urban area and compared with independent Q learning and a heuristic scheme in terms of handover probability and spectral efficiency.