This is a demonstration of ArduPilot using an Intel RealSense T265 camera for non-GPS navigation using the scripts created by Thien Nguyen (see blogs here) as part of GSoC 2019 under the wise mentorship of Patrick Poirier.
A few others have already successfully reproduced the setup but I wanted to use it myself for an upcoming search & rescue competition which requires the vehicle to fly into a building. I also wanted to make sure the setup was easy for others so Jaime Machuca and I created an APSync image for the RPI4 (details are here on the wiki).
Overall I was quite surprised at how well it worked. You can see especially at the beginning of the video that the vehicle’s position hold is very stable, more stable than when using a GPS actually. Also about halfway through the video I tried flying it higher with no objects near the vehicle and it still performed quite well.
On the other hand if you look at the end of the video you’ll see that it completely loses its position estimate after I yaw it around a few times. In the image below the yellow line shows where the EKF thought the vehicle was (over in my neighbour’s yard) but in reality the vehicle was always safely in my backyard.
I also bumped into a known but very annoying hardware problem with the T265 which requires the camera to be powered up after the companion computer. This is resolved in the APSync image using uhubctl to power-down/power-up the USB ports after the RPI4 has started.
There is more work to do on the integration of this camera with ArduPilot but it already works well. If you want to get involved I recommend getting a vehicle working yourself with the T265 (wiki) and report any issues you find. If you’re a developer looking to help us advance the non-GPS features of AP then ping us on the ArduPilot/VisionProjects gitter channel.
By the way, the frame is an Hexsoon EDU450 with a CubeOrange autopilot.
Thanks again to Thien, Patrick and Jaime for their help with this.