Servers by jDrones

Easier setup for Intel RealSense T265


This is a demonstration of ArduPilot using an Intel RealSense T265 camera for non-GPS navigation using the scripts created by Thien Nguyen (see blogs here) as part of GSoC 2019 under the wise mentorship of Patrick Poirier.

A few others have already successfully reproduced the setup but I wanted to use it myself for an upcoming search & rescue competition which requires the vehicle to fly into a building. I also wanted to make sure the setup was easy for others so Jaime Machuca and I created an APSync image for the RPI4 (details are here on the wiki).

Overall I was quite surprised at how well it worked. You can see especially at the beginning of the video that the vehicle’s position hold is very stable, more stable than when using a GPS actually. Also about halfway through the video I tried flying it higher with no objects near the vehicle and it still performed quite well.

On the other hand if you look at the end of the video you’ll see that it completely loses its position estimate after I yaw it around a few times. In the image below the yellow line shows where the EKF thought the vehicle was (over in my neighbour’s yard) but in reality the vehicle was always safely in my backyard.


I also bumped into a known but very annoying hardware problem with the T265 which requires the camera to be powered up after the companion computer. This is resolved in the APSync image using uhubctl to power-down/power-up the USB ports after the RPI4 has started.

There is more work to do on the integration of this camera with ArduPilot but it already works well. If you want to get involved I recommend getting a vehicle working yourself with the T265 (wiki) and report any issues you find. If you’re a developer looking to help us advance the non-GPS features of AP then ping us on the ArduPilot/VisionProjects gitter channel.

By the way, the frame is an Hexsoon EDU450 with a CubeOrange autopilot.

Thanks again to Thien, Patrick and Jaime for their help with this.

8 Likes

Hello @rmackay9 great work as usual. Have a couple of questions:

1 - Does the T265 work together with GPS in cohoperation if GPS active to make precision better?
2 - Does T265 work on EKF2 or it needs EKF3?

Thanks!!

2 Likes

@Corrado_Steri,

  1. Currently the T265 cannot be used together with a GPS but this is high on the to-do list.
  2. The T265 only works with EKF2 but making it work with the EKF3 is also on the to-do list but slightly lower down.
1 Like

Great thanks, looking forward to the cohoperation with GPS. I am glad it works on EKF2 that is what almost everybody uses.

1 Like

Finally had some time to put the Q450 - Mule back to work, flashed the image and started.
I can see it booting but I cannot log using hdmi-usb, it show apync login but cannot enter using keyboard. I connected FTDI to tty1 but, obviously after a while it is spitting MavLink.

Try to find the APSYNC AP but it is radio silence.

@jmachuca77 or @rmackay9 is there something I am missing ?

1 Like

Thanks for giving it a try!

Supporting linux is tough… so many things can go wrong starting with the ability to log in apparently.

I think maybe using a monitor (connected to the hdmi) and keyboard to login is the best way forward.

I’ve also found the wifi access point is spotty. Some times it appears and sometimes it doesn’t. I think it’s related to having the camera plugged in or not but I’m not completely sure.

… So I think maybe it would be best to try disconnecting the autopilot and the camera and see if that helps. Then I’d try a regular keyboard and hdmi monitor.

1 Like

Yep disconnecting Camera allows to use keyboard an shows the ArduPilot SSID
I carry on and will update

2 Likes

@ppoirier, thanks for confirming. It’s quite odd but I suspect we will get to the bottom of it.

By the way Stephen Dade (who produces the PiConnectLite) also confirmed an issue with the Wifi AccessPoint and he said this,

Initially had the same issues are you described. I have fixed the issue via adding in a few extra lines to /boot/usercfg.txt as per the instructions in the Setup -> Software section of the Pi-Connect documentation at https://www.docs.rpanion.com/products/pi-connect-lite#software.
Wifi now works on boot quite nicely. Let me know if this works for you

I have not followed through to test his advice yet.

1 Like
Servers by jDrones