Optical flow on raspberry pi camera

Hardware
1

Hello, Have anybody already tried to use the GPU of the raspberry pi in order to retreive the h264 motion vector and thus achieving optical flow without using any CPU power ?

We have a python script that does it but don’t know how to merge it in the ardupilot code.
If a native C solution already exists, better use it but if not how could we use our code inside ardupilot for it ?

Thanks,

Philippe Crochat
Anemos Technologies

2

@anemos - very interesting, can you share the script? Would be interesting to see and play around with.
You’re not going to get python merged with the ardupilot code (c++), so it would either have to be translated in c++ or possibly just kept separate in a companion computer.

3

Hello fnoop,

here it is
motion_analysis.txt (9.4 KB)

It was meant to be run in a separate microcomputer, apply a low pass filter to the motion vector result, get the lidar range value and pass the corresponding filtered result to the flight controler for flight stabilisation.

All of that could be greatly simplified if all could be done by ardupilot natively. I don’t know why the python code could not be transformed in C++.
optical flow could be realy more useful if it was able to get podometric value not only on the XY plane but also in the Z direction (vertical axis). This could be achieved if the raspi cam was mounted on a small gimbal.

4

Hi @anemos,

I am designing my own optical flow implementation and stumbled upon your post. Great work!

How did you manage to communicate the flow information with the flight controller? What kind of interface (serial, i2c etc) did you use and which api?

Thanks.

5

Hi anemos, have you succeed merging the pi camera code with ardupilot? I am doing the similar task. With the raspi camera code written in C, I want to merge it with ardupilot. Could you tell me how to make it? Thanks!

6

No, I was not able to do it, and don’t know how to do it but I’m
interested if you succeed

7

I am trying it. Using navio2, the raspberry pi based autopilot, with the raspberry pi camera to take images.

8

Hello zhanghaijason, have you succeded?

9

Hello guys,
If you are still wondering how to merge python code with ardupilot, I would suggest DroneKit


It has python api for conpanion computer and there are some sinilar projects with it.
Best regards
And good luck!