Pisanti, Dario (2024) Vision-based geo-localization of future Mars rotorcraft in challenging illumination conditions using deep learning. [Tesi di dottorato]

Documento PDF pisanti_dario_36.pdf Download (199MB)

Abstract

The new aerial mobility dimension enabled by Ingenuity has unlocked unprecedented potential for groundbreaking science investigations in astrobiology, geology and climate on Mars. The next generation of Martian rotorcraft will need advanced navigation capabilities to conduct long-range flights over diverse and challenging terrains. Accurate geo-localization within a global reference frame is essential to mitigate cumulative drift from on-board visual odometry, ensuring precise navigation during extended traverses. Absolute visual localization can be achieved in a map-based approach by matching real-time images from the rotorcraft's navigation camera with pre-referenced orbital maps stored on-board. However, significant illumination differences between query observations and on-board maps can challenge visual geo-localization performance, restricting the mission's operation envelope to certain times of day. This work investigates deep-learning-based methods to perform robust Map-based Localization (MbL) in challenging lighting. We proposed a novel multi-modal deep-learning framework that utilizes cross-attention mechanisms to fuse visual and depth data from ortho-projected maps and Digital Terrain Models (DTMs) obtained from orbital assets, effectively leveraging geometric context to learn illumination and scale invariance. To support training and validation, we developed a custom rendering framework to generate a synthetic Mars dataset with HiRISE ortho-images and DTMs, simulating aerial observations under varying lighting and altitude. Comprehensive evaluations show that our multi-modal method improves localization accuracy and robustness to a wide range of lighting offsets between maps and observations compared to single-modality models, including both deep-learning-based and traditional methods. Additionally, the integration of depth information has been shown to provide a degree of scale invariance and enhanced robustness to varying terrain morphology.

Downloads

Actions (login required)

Modifica documento