Joint Beamforming Design for RIS-Assisted Integrated Satellite-HAP-Terrestrial Networks Using Deep Reinforcement Learning

In this paper, we consider reconfigurable intelligent surface (RIS)-assisted integrated satellite high-altitude platform terrestrial networks (IS-HAP-TNs) that can improve network performance by exploiting the HAP stability and RIS reflection. Specifically, the reflector RIS is installed on the side of HAP to reflect signals from the multiple ground user equipment (UE) to the satellite. To aim at maximizing the system sum rate, we jointly optimize the transmit beamforming matrix at the ground UEs and RIS phase shift matrix. Due to the limitation of the unit modulus of the RIS reflective elements constraint, the combinatorial optimization problem is difficult to tackle effectively by traditional solving methods. Based on this, this paper studies the deep reinforcement learning (DRL) algorithm to achieve online decision making for this joint optimization problem. In addition, it is verified through simulation experiments that the proposed DRL algorithm outperforms the standard scheme in terms of system performance, execution time, and computing speed, making real-time decision making truly feasible.