Integration of ArduPilot and VIO tracking camera (Part 4): non-ROS bridge to MAVLink in Python

My RPi3 is running Ubuntu MATE 16.04 LTS.

All the variables are updated in the main loop. The scheduled job only read them and send the value to MAVLink messages.

Right, sorry, I inverted write and read in my sentence.

Also I am using Ubuntu 16.04 but from Ubiquity because it comes with ROS preinstalled, so I cannot figure out such different values for CPU load. Can I know the version of librealsense you use?

I am using librealsense version 2.24.0. It has been a while since the last time I updated the system. I will get it up to date and see if it relates to the issue somehow.

1 Like

I am using 2.32.1, a change in librealsense version could imply also a change in T265 firmware, my firmware is 0.2.0.926

So great to see this project pushing forward - it’s the future!!

USB is notoriously fickle timing wise, have you tried a pi4 with usb3?

Thanks @fnoop but i don’t think the cause of the problem is USB, at least for this problem, for two reasons:

  • same setup has no problem with ROS solution
  • same setup and same non-ROS script has no problem when I stopped using the scheduler lib

I am almost sure that with RPi4 and USB3 we have no such problems but I think that if you only want the position estimate from T265 USB2 and less power hungry SBC like RPi3 or Banana Pi Zero (used by @ppoirier) are more then enough.

Where RPi4 and USB3 become really useful is if we want to get also the stereo camera streaming from T265, maybe for testing other VIO solution and confront them with that onboard the sensor.

Thank you for your contributions. Now I follow this article to realize the F450 uav flight based on T265 and Raspberry Pi4, It worked. It was very stable in stabilized mode. But when I switch to LAND or ALTHOLD mode, the uav suddenly falls down. Please help me to analyze whether there is something wrong with my parameter setting.

Hi @beyond21299, I suggest you take a look at the log to see where the problem lies. If the vision data (VISP) is smooth and follows the vehicle’s movement (see the plots above for examples), then the issue might be in the control and settings that you are using.

Now I was unable to export the log,But I will try. again. I want to ask how you set ‘EK2_ALT_SOURCE’ parameter first.

I set EK2_ALT_SOURCE=1 since I am using a range finder.

@LuckyBird I did a test with your script from non_ros_timing branch, I had the same problems as you can see from attached graph but there is no “The job crashed :(” message on the log.

t265_to_mavlink_log.txt (301.1 KB)

Hi @anbello, here’s the elapsed time of the messages from your log:


For comparison, here’s the elapsed time I just tested with RPi3B:

Both scripts are sending the vision messages at 30Hz, but the disparity is quite apparent. As you said, the CPU usage is high when the script uses apscheduler but is not when just running the main loop or in ROS.
I can only say there is some bad interaction between apscheduler and the CPU of your specific companion computer. If the reason for high CPU usage is not clear, I would suggest that you try with another board/computer if possible to see if the issue can be replicated.

2 Likes

@LuckyBird just to be sure that we are using the same HW: I have a Raspberry Pi 3 model B V1.2.
It seems strange to me that it is a HW problem so I will try to reinstall from scratch all the SW starting from Ubuntu MATE 16.04 LTS, librealsense and all needed libraries. Apropos have you tested with latest librealsense?
if, even with the same software stack you use, I will have the same problems then I will try to change the Raspberry Pi too.

1 Like

Hi Thien
I am looking at your post and doing a good test.
I’m testing raspberry pi 4 raspbian and t265.
I have a question in the t265_to_mavlink.py post.

Default global position of home / origin

home_lat = 151269321 # Somewhere in Africa
home_lon = 16624301 # Somewhere in Africa
home_alt = 163000

From here ,
1.Could you know which coordinate system of home latitude and longitude?
2. Is there any reason your home altitude is marked as 163000?

With built test, pos_vertical error sometimes appears during test flight
You lose your posture. At that time, the altitude came out at a strange value …

Finally, when should I click on the Mission Planner’s EKF home in non-ROS? (To somewhere other than the stated African region …)

Thank you

1 Like

Hi @111182,

  • The values of home_lat, home_lon, home_alt are totally random. You can change them to whichever values you like.
  • Error in vertical position indicates that something is not right with the position data. If the position data from the T265 is not good (when it loses track, diverges or relocalises for example), this error or horizontal error would appear. If you have multiple data sources and they don’t agree, this might also happen.
  • You can click set the EKF home origin after the messages “GPS Glitch” and “GPS Glitch cleared” appear on Mission Planner’s message view tab. You can click on anywhere in the map. Note that this can only be done once after the FCU boots, so if you want to reset the home, you need to reboot the FCU.
1 Like

Hi @LuckyBird I think the problem (still unresolved) that causes a high CPU load in my RPi3 highlights a synchronization problem between threads in the script.

I was reading this tutorial to understand thread synchronization in Pyhton:
http://effbot.org/zone/thread-synchronization.htm

Here is an excerpt:
“One important issue when using threads is to avoid conflicts when more than one thread needs to access a single variable or other resource. If you’re not careful, overlapping accesses or modifications from multiple threads may cause all kinds of problems, and what’s worse, those problems have a tendency of appearing only under heavy load, or on your production servers, or on some faster hardware that’s only used by one of your customers.”

So I tried to insert locks as written in the tutorial and the result is that I no longer see errors in the logs.
This means that the errors I saw were the result of attempts by two threads to use the same global variable at the same time (that becomes evident only under high CPU load).

Here the script with lock.

Only three line added:
72, 205, 427

4 Likes

This is great @anbello! Thank you so much for your insights!
To be honest, I did not perform much evaluation on this part. At the time, it seemed to work fine for me and @ppoirier on RPi3, RPi4, Jetson Nano, Up board, so we stick with it and move on.

I will try out your solutions on my setup. If everything works well (as it already worked for you), I will add the changes to the main script on my Github repo.

3 Likes

Hi @LuckyBird would you like that I open an issue related to the thread synchronization problem on your repository https://github.com/thien94/vision_to_mavros?

@anbello that wouldn’t be necessary. I have the code in a branch in my local repo but haven’t pushed to remote yet. This weekend I will push it upstream and if you have time, maybe you can take a look.

1 Like