Integration of ArduPilot and VIO tracking camera (Part 1): Getting started with the Intel Realsense T265 on Rasberry Pi 3B

Thien,

Success!

Finally compiled librealsense, I would note that a couple of other packages needed to be installed as per the Intel realsense installation instructions for linux on their GitHub repo.

Currently going through your instructions for the python mavlink bridge, I’ll let you know how it goes.

Quick question, is their any reason you chose to use a 3B instead of a 3B+? Also, just ordered a couple of pi 4’s in 2 and 4 GB, would they work too?

Thanks mate,

Hugh

Hi @hugh, glad it worked out for you.

One of the goals of our project is to demonstrate that the system can be implemented on even the not so powerful computer board, hence the RPi was selected. I used the 3B instead of 3B+ since it was available for me.

Any board that has USB 2.0 ports should be enough for this project, hence the Pi 4 should work with no issues. Furthermore, the Pi 4 has USB 3.0, so you can also stream images from the T265, which is not applicable for the Pi 3.

Awesome,

I figured i’d start with the python mavlink bridge, as im not too comfortable with ROS; however i can not get pyrealsense2 to install.

The command “sudo pip3 install pyrealsense2” continuously returns " no matching distribution found for pyrealsense2"

any ideas?

Thanks,

Hugh

If the libraries are not available for your system, I can see a few workarounds, you can try one by one:

  • Copy the built libraries librealsense2.so and pyrealsense2.so in ~/librealsense/build/ and place them next to the script that you want to run.

  • Install python2 version: sudo pip install pyrealsense2. In that case, you might need to change the scripts to fit python2 version of APIs.

Hello , i need help for step 3 ,4 ,5 ,I work with arducopter and mavlink but i did not work much on the ros and mavros , i have odroid xu4 and pixhawk ,i installed the library , the camera t265 work ,
what I understood is i must take the camera position ( translation , rotation ) from topic /tf and transmit it with mavros, VISION_POSITION_ESTIMATE [#102 ] message , 20 HZ , it is like that ?

@Tarek-H You are right, but with one missing part. You also need to perform frame transformation to take into account different camera orientation (if not using the default forward facing) and align 0 degree heading (otherwise you will see the vehicle facing east at startup).

We also have a non-ROS implementation in Python in part 4, so you can jump to that first to have a quick up-and-running system, and maybe come back when you feel adventurous with ROS.

hi i’m a super begineer ,will i be able to do this project

Hello

If you already have a flying vehicle with ArduPilot, You need knowledge of these 3 fundamentals systems:

  • ArduPilot flight controller and how to set advanced parameters and read the logs and setup communication using serial devices
  • Companion Computers , like a RaspBerry Pi and how to configure Linux environment and install packages or build from source
  • Python programming language

You need to go step by step and start with having a fully functional flying platform before trying to implement a project like this. I suggest you start by reading the excellent wiki we have on ArduPilot as most of the stuff you need to know is already there. Actually this series of blogs will be part of the wiki.

Thank you , i will see the link , but now i have problem with pyrealsense2 , i know that sudo pip3 install pyrealsense2 doesn’t work , i tried to rebuild librealsense 2.24.0 with DBUILD_PYTHON_BINDINGS=TRUE in cmake , i have a problem when i do make make[1]: *** [wrappers/python/CMakeFiles/pyrealsense2.dir/all] Error 2

Hi @Tarek-H, I read above that you are using Odroid xu4, correct? In that case, you might need to follow this instruction page.
It looks like Intel does not officially support the Odroid line of devices. Furthermore, there are several known known issues with running librealsense on Odroid, so you will have to try some patches and see how it goes.

Hi , yes i use odroid xu4 , it work very well for me but i use C++ not python ,so i rebuild the library to have the pyrealsense2 but i have error for pyrealsense2 with the make , i try to clean and uninstall completely the library and redo install instruction .

Just want to make sure, are you building the main Cmakelists.txt or the one inside the python wrapper folder?

You should navigate to librealsense root directory, run the top level CMake command with the additional flag -DBUILD_PYTHON_BINDINGS, for example:

cd /path/to/librealsense
mkdir build
cd build
cmake ../ -DBUILD_PYTHON_BINDINGS=bool:true

I am building in the build folder inside the librealsense folder , my librealsense folder is in the catkin_ws /src because before i had problem with the link between the ros and librealsense , now i want to use the pyrealsense so when i use cmake …/ -DBUILD_PYTHON_BINDINGS=bool:true ( in the build folder ) and the make i have internal error because pyrealsense ,and some times during the make execution the odroid is frozen!

I believe that means the odroid runs out of memory during the build. Can you try again with make -j1 instead? Note that this can take quite some time for the build to complete.

yes i use make -j1, the error that i have is
make[2]: *** [wrappers/python/CMakeFiles/pyrealsense2.dir/python.cpp.o] Error 4
CMakeFiles/Makefile2:233: recipe for target ‘wrappers/python/CMakeFiles/pyrealsense2.dir/all’ failed
make[1]: *** [wrappers/python/CMakeFiles/pyrealsense2.dir/all] Error 2
Makefile:127: recipe for target ‘all’ failed

maybe is about the librealsense 2.24.0 and must change version ,or some thing missing in python

Unfortunately, I have tested with Odroid XU4 and did not encounter your issue. Maybe you can delete the build folder and try again.

Below are the steps that worked for me.

Install librealsense and pyrealsense2 on Odroid XU4 running Ubuntu 16.04:

  1. Upgrade kernel to 4.14: I followed the instructions here. After which, my Odroid’s kernel is:
$ uname -a
Linux odroid 4.14.111-139 #1 SMP PREEMPT Tue Apr 16 17:31:00 UTC 2019 armv7l armv7l armv7l GNU/Linux
  1. Build librealsense from source:
# Make Ubuntu Up-to-date: update Ubuntu distribution, including getting the latest stable kernel:
sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade 

# Download the complete source tree with git
cd 
git clone https://github.com/IntelRealSense/librealsense.git

# Install the core packages required to build librealsense binaries and the affected kernel modules:
sudo apt-get install git libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev 

# Distribution-specific packages, for Ubuntu 16:
sudo apt-get install libglfw3-dev

Important: Unplug any connected Intel RealSense camera before continue:

# Navigate to librealsense root directory to run the following scripts.
cd librealsense

# Run Intel Realsense permissions script located from librealsense root directory:
./scripts/setup_udev_rules.sh

# Odroid XU4 with Ubuntu 16.04 4.14 image Based on the custom kernel provided by Hardkernel 
./scripts/patch-realsense-ubuntu-odroid.sh

# Install Python and its development files via apt-get (Python 2 and 3 both work)
sudo apt-get install python python-dev && sudo apt-get install python3 python3-dev

# Run the top level CMake command with the following additional flag -DBUILD_PYTHON_BINDINGS=bool:true:
cd ~/librealsense
mkdir build
cd build
cmake ../ -DBUILD_PYTHON_BINDINGS=bool:true
make -j4
sudo make install
sudo ldconfig

sudo reboot
  1. Check installation and test pyrealsense2:
$ ls /usr/local/lib

You should see librealsense2.so.2.24 and pyrealsense2.cpython-35m-arm-linux-gnueabihf.so. Then you can test Python code:

# update your PYTHONPATH environment variable to add the path to the pyrealsense library
export PYTHONPATH=$PYTHONPATH:/usr/local/lib

# Navigate to python example folder
cd ~/librealsense/wrappers/python/examples/

# Run examples:
python3 t265_example.py

Then you will see a stream of data coming from the T265, for example:

Frame #49
Position: x: -3.80651e-05, y: -0.000489848, z: 6.75867e-05
Velocity: x: -0.000446847, y: 0.000521675, z: -0.000202993
Acceleration: x: -0.00395986, y: 0.0395705, z: 

Let me know if this works for you.

thank you so much,I fixed the problem for pyrealsense , before i didn’t use git clone https://github.com/IntelRealSense/librealsense.git ( Master branch ) , I downloaded the zip files for specific version (2.24.0) ,it is for realsense-ros specific version too , now with Master branch I managed to build realsense library with pyrealsense , i tested the t265_example.py it work

Hello I’m following this guide, but in my case I’m giving a shot to the Intel Up board with ubuntu 16.04, the step that is giving me some trouble is the installation of the apsync to connect the board to the ardupilot using MAVROS, can you help me with this?

Can you give us more details of your installation process and what errors are you having?

Yes, I was able to set up everything until the apsync, the images provided are only for TX1 TX2, and raspberry pi3, but I’m using the Intel Up board, I’m using the tx rx from the GPIO to connect to my Pixhawk, but the apsync step is a little bit confussing. Thanks for your response