Sensor-Model-Based Trajectory Optimization for UAVs to Enhance Detection Performance: An Optimal Control Approach and Experimental Results

UAVs are widely used for aerial reconnaissance with imaging sensors. For this, a high detection performance (accuracy of object detection) is desired in order to increase mission success. However, different environmental conditions (negatively) affect sensory data acquisition and automated object detection. For this reason, we present an innovative concept that maps the influence of selected environmental conditions on detection performance utilizing sensor performance models. These models are used in sensor-model-based trajectory optimization to generate optimized reference flight trajectories with aligned sensor control for a fixed-wing UAV in order to increase detection performance. These reference trajectories are calculated using nonlinear model predictive control as well as dynamic programming, both in combination with a newly developed sensor performance model, which is described in this work. To the best of our knowledge, this is the first sensor performance model to be used in unmanned aerial reconnaissance that maps the detection performance for a perception chain with a deep learning-based object detector with respect to selected environmental states. The reference trajectory determines the spatial and temporal positioning of the UAV and its imaging sensor with respect to the reconnaissance object on the ground. The trajectory optimization aims to influence sensor data acquisition by adjusting the sensor position, as part of the environmental states, in such a way that the subsequent automated object detection yields enhanced detection performance. Different constraints derived from perceptual, platform-specific, environmental, and mission-relevant requirements are incorporated into the optimization process. We evaluate the capabilities of the sensor performance model and our approach to sensor-model-based trajectory optimization by a series of simulated aerial reconnaissance tasks for ground vehicle detection. Compared to a variety of benchmark trajectories, our approach achieves an increase in detection performance of 4.48% on average for trajectory optimization with nonlinear model predictive control. With dynamic programming, we achieve even higher performance values that are equal to or close to the theoretical maximum detection performance values.