Improve the robustness of TSDF-based SLAM

The research was done in a five months project. The main goal of this project is to build a SLAM system which is robust to tracking-loss, drifting and able to perform detail object-level reconstruction. In the period I used the sliding windows and keyframes methods [1] to handle the tracking loss and relocalization respectively. Moreover, an object centroid tracking system was added to allow users to take certain object during reconstruction, so as to reconstruct the whole model (for example, the bottom part).

In this video you can see that:
First objects are incrementally segmented [2],
Second, the tracking is switch from frame-to-model to frame-to-target-object.
Thrid, the display switched between segmented classes, normal distribution and RGB image.

Reference

[1] Engel, Jakob, Vladlen Koltun, and Daniel Cremers. “Direct sparse odometry.” IEEE transactions on pattern analysis and machine intelligence 40.3 (2018): 611-625.

[2] Tateno, Keisuke, Federico Tombari, and Nassir Navab. “Real-time and scalable incremental segmentation on dense slam.” 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2015.