Deep Learning Approaches for Autonomous UAV Control and Mapping in Cluttered Outdoor Environments

Abstract

In complex unstructured environments such as under the canopy of densely forested areas, autonomous and intelligent unmanned aerial vehicle (UAV) flight continues to be an unaddressed challenging problem. The main predicament in the development of a system supporting autonomous and intelligent fly lies primarily due to the lack of an available dataset. During this research, a method was developed to enable rapid data labelling in real-time, which is suitable for data gathered from most mounted cameras (handheld, ground vehicles and/or UAV), regardless of their resolution. Leveraging this dataset creation capability, the proposed trail following approach outperforms the state-of-the-art approaches for trail navigation within a complex forest canopy environment. Additionally, extensive testing demonstrated that the proposed approach does not suffer loss of performance even when replacing the camera between flights.

Another issue with navigating in such an environment (i.e. under the canopy) is the lack of generalisation across varying domains (i.e. farmland, deserts, savanna etc). This is further aggravated by seasonal environmental variations such that a model trained during summer will not be generalised to the same environment during the winter. To solve this issue a multi-task approach presents a deep neural network which eliminates the need to follow a trail, by providing the UAV with the ability to identify the safest area to navigate, regardless of the existing altitude and whether or not a trail is visible in the scene. This new approach increases the navigational control from three to six DoF (Degrees of Freedom). Furthermore, the final results demonstrated superior performance for navigation than the comparators approach.

Besides autonomously navigating in the environment, an intelligent UAV is expected to increase its resourcefulness by mapping while simultaneously exploring the environment. This would facilitate the commencement of search and rescue operations to be carried by either one or a swarm of intelligent UAV. This research entailed the development of a grid-based map that can store the position of obstacles and trails within the search area, while allowing onboard processing. Additionally, this approach adopts a hybrid network that combines Reinforcement Learning and Online Learning to improve navigability, exploration and adaptability to unseen environments, thus allowing search and rescue operations to be extended without requiring offline training.

Overall, this research aims to investigate new and efficient ways for intelligent UAV to perform an end-to-end operation and achieve real-time mapping and autonomous navigation in an unstructured environment. The findings in this research are important in that they culminate into three extensively tested approaches for navigation and exploration that can be deployed either on a ground control station or on-board of a UAV.