A few others have already successfully reproduced the setup but I wanted to use it myself for an upcoming search & rescue competition which requires the vehicle to fly into a building. I also wanted to make sure the setup was easy for others so Jaime Machuca and I created an APSync image for the RPI4 (details are here on the wiki).
Overall I was quite surprised at how well it worked. You can see especially at the beginning of the video that the vehicle’s position hold is very stable, more stable than when using a GPS actually. Also about halfway through the video I tried flying it higher with no objects near the vehicle and it still performed quite well.
On the other hand if you look at the end of the video you’ll see that it completely loses its position estimate after I yaw it around a few times. In the image below the yellow line shows where the EKF thought the vehicle was (over in my neighbour’s yard) but in reality the vehicle was always safely in my backyard.
There is more work to do on the integration of this camera with ArduPilot but it already works well. If you want to get involved I recommend getting a vehicle working yourself with the T265 (wiki) and report any issues you find. If you’re a developer looking to help us advance the non-GPS features of AP then ping us on the ArduPilot/VisionProjects gitter channel.
By the way, the frame is an Hexsoon EDU450 with a CubeOrange autopilot.
Thanks again to Thien, Patrick and Jaime for their help with this.
Finally had some time to put the Q450 - Mule back to work, flashed the image and started.
I can see it booting but I cannot log using hdmi-usb, it show apync login but cannot enter using keyboard. I connected FTDI to tty1 but, obviously after a while it is spitting MavLink.
Try to find the APSYNC AP but it is radio silence.
Supporting linux is tough… so many things can go wrong starting with the ability to log in apparently.
I think maybe using a monitor (connected to the hdmi) and keyboard to login is the best way forward.
I’ve also found the wifi access point is spotty. Some times it appears and sometimes it doesn’t. I think it’s related to having the camera plugged in or not but I’m not completely sure.
… So I think maybe it would be best to try disconnecting the autopilot and the camera and see if that helps. Then I’d try a regular keyboard and hdmi monitor.
@ppoirier, thanks for confirming. It’s quite odd but I suspect we will get to the bottom of it.
By the way Stephen Dade (who produces the PiConnectLite) also confirmed an issue with the Wifi AccessPoint and he said this,
Initially had the same issues are you described. I have fixed the issue via adding in a few extra lines to /boot/usercfg.txt as per the instructions in the Setup → Software section of the Pi-Connect documentation at Pi-Connect Lite [Rpanion Electronics Documentation].
Wifi now works on boot quite nicely. Let me know if this works for you
I have not followed through to test his advice yet.
I did a test in which the EKF2 was setup to use the T265 and the EKF3 was setup (running in parallel) to use regular GPS. I then walked around in a roughly square pattern (in a clockwise direction) to see how the camera position compared with the GPS position.
The EKF2+T265 position is shown on the map in red. The EKF3+GPS position is shown in blue. We can see at least two problems with the EKF2+T265 position:
The T265 position is rotated by about 20deg for the first walk around the yard.
It goes bonkers at one point and diverges off by about 50m. This happened when walking along the southern border and the camera was quite close to a low fence. I suspect the camera became confused by the regular pattern of the fence and though it was moving far more quickly than it actually was.
Interestingly there is a moment (shown by the yellow arrow) when the camera figures out where it is and resets to a position very close to the GPS position. It also correctly it’s heading although it doesn’t happen instantaneously. It must have remembered it’s location from a previous walk around the yard. I actually walked around the yard twice before remembering that I hadn’t switched on logging.
Anyway, what I take away from this is that we need to handle position resets and also inflight alignment of the AHRS and camera yaw.
Here’s another picture showing the X and Y position from the camera and we can see the moment the position reset happens.
I’m trying to give this system a go but when I start up my Pi4 with the APSYNC image on a 16G SD card inserted, the Pi appears to go through the startup, but then restarts the whole process again… over and over. It looks like it restarts at various spots. I know its me, but what???
Can you connect to a HDMI screen ?
Like I reported initially I had some difficulties to connect as well,and I did some modification in order to get the system starting correctly and to connect easily to the 5 Ghz band.
Here are my mods:
A) For the USB reset, leave Reset USB3 but do reset USB2
Comment out the the USB2 LINE in this script:
apsync@apsync:~/start_uhubctl$ cat start_uhubctl.sh
#./uhubctl -a cycle -l 1 -p 1-4
B) WIFI-AP mods
apsync@apsync:~$ sudo wifi-ap.config get
debug: false
dhcp.lease-time: 12h
dhcp.range-start: 10.0.1.129
dhcp.range-stop: 10.0.1.138
disabled: false
share.disabled: false
share.network-interface: eth0
wifi.address: 10.0.1.128
wifi.channel: 48
wifi.country-code: CA (put your country code here)
wifi.hostapd-driver: nl80211
wifi.interface: wlan0
wifi.interface-mode: direct
wifi.netmask: 255.255.255.0
wifi.operation-mode: a
wifi.security: wpa2
wifi.security-passphrase: ardupilot
wifi.ssid: ArduPilot
Having some LINUX knowledge really helps to get this thing started as you can setup , configure and monitor what’s happening on the RPI
So I’m able to see my rover in Africa. Yippee. Upon further reading of the Intel RealSense T265 blog, the SERIAL2_BAUD and SERIAL2_PROTOCOL should be set to 1. I get that…no sense in overloading the Pixhawk. But I gather I have to make similar changes to the RPi4. I’m unable to start or install the raspi-config. Is there a link describing how to make the necessary changes to the Pi?
Thanks
PS Is this the right place for real beginners to ask very elementary questions regarding the T265/RPi/Pixhawk system on a Rover.?
Actually it is running on UBUNTU - not Raspian so there is no raspi-config.
The interface between the Flight Controller and the RPI is using
mavlink router, that has been mainly developped by @lucasdemarchi.
The config file is /etc/mavlink-router/main.conf
You can look at it by simply typing cat /etc/mavlink-router/main.conf
You’ll see entries for 10.0.1.255 and the ports it has open. You’ll also see entries for 127.0.0.1 which are the ports used internally on the RPi
The T265 is started as a service that call the connection string to the internal network so it can start transmitting the odometry messages. All the startup files are in home directory (the one you log by default).
Hope it helps (Thanks to @jmachuca77 for the info )
@RoboBill, if you’re seeing the vehicle in Africa then it should work.
It shouldn’t matter if the SERIALx_PROTOCOL = 1 vs 2 (either should work).
You’ve used the APSync image for the RPI mentioned on the T265 wiki page?
It’s possible that it won’t work because of Rover’s low main loop rate (only 50hz) but this can be increased by setting SCHED_LOOP_RATE to 100 or 200 but before that it’s probably best to post an onboard log file (the .bin file) so we can see if the camera’s position estimates are arriving in AP.
@ rmackay9 Yes, I’m using the APSync. With the baud at 921600, I have gone into the Mavlink Inspector and seen the data for Vehicle #1.
It appears that with the Baud rate at 1200, the time to load the configs onto the MP gets stalled. But I can see the Pixhawk Configs at 10.0.1.128… NICE!
As to tracking a simple square mission… well that’s not so good, but the “day is young”. I’ve got the latest MP beta. I’ve input the VISO offsets and wheel encoder specs. I’m using EKF2. I’m hoping other “rovers” will join in.
Here is a little more info… I’m able to see my rover in Africa and the parameter list at 10.0.1.128 for baud rates as low as 38400. Any speed below 38400, I can only see the parameters lists.
