Vision based precision landing

Hello everyone, I am developing a drone with the integrated fmuv5 and I wanted to implement vision-based precision landing and not infrared blocking, since most of the pads have the letter H i want to the drone detect the pad and thus perform the precision landing, but I don’t know what would be the best system to perform such action on a complementary computer, any help ??

1 Like

Hello

You can use precision loiter-landing with a companion computer running OpenCV or a JeVois camera system…

Here is an example of Presision Landing using a JeVois that is targetting an ARUCO tag.

1 Like

thanks for the suggestion, but jeVois seems kind of complex, they don’t even provide the schematic of the card and I would like to implement it on my pcb, so I think it wouldn’t be a good idea, thanks

if I use openmv h7 and execute the text detection algorithm and use the telemetry Mavlink so that when Open MV detects the text, send the message to the controller, causing the drone to land following the letter and not the april tag, would it give right?

Yes it works , I presume the new generation had enough power to process VGA @ 20Hz

1 Like

thanks for the help i will implement the h7 plus and will send the error codes if it goes wrong

1 Like

According to the tests I end up without uart port to connect to the complementary computer, so if I use the h7 plus for optical flow and by using Python I can add an imu with custom code?, would it be highly accurate?

You can use optical flow over MavLink as it is implement now. You need make sure the calibration is optimal and OF delay is adjusted like shown in the wiki.

Adding an additional IMU is not really helping for precision as it is more used for reliability.

Btw I checked for April Tag tracking on a H7 OpenMV and it is still lacking memory to process VGA. Doing tracing on QVGA is too restrictive as it limits the use case to a few feet

1 Like

Yes I know that, but in this case I wanted to deploy the h7 plus with 31mb of usable RAM and for some test it seems that the drone follows the april tag and in daily use people do not use the april tag, but the landing pad, or if it doesn’t work with Open MV, I will create a Flow uav CAN and use GIMBAL and the main camera for precision landing, is it a good idea? correct me if I’m wrong

Precision Landing has been successfully demonstrated with different methods, the choice is yours.

1 Like

I think it’s better to keep Optical Flow,in the next product after improving I will correct, thanks for the assistance

@NiltonMadede, I implemented a few months back a vision based precision landing system for a customer based on https://github.com/goodrobots/vision_landing from @fnoop. I thoroughly tested it and it seemed really robust and reliable. You can check some videos of the testing here:

https://www.youtube.com/channel/UCuGLhZrV1fFCm5osRPh2VGA?view_as=subscriber

The tests above are on a RPi 4.

Best regards!

1 Like

Hi
Thanks for your help sastre,but in this case it works following the april tag, so since I saw that with Open MV it is not possible, I will use the precise landing using gimbal, as in the video👇🏾

hi ppoirier

Would you be so kind to share the libraries used for the solution you propose?

Thanks!

Hello , look on my github

Hey, i am trying to use the open mv7 for precision landing as well without using a separate onboard computer.
with using it as flow control and precision landing, i was wondering if you simply could use both programs in the mv7 and send it a 5v signal to decide which one you wanted to run.

Anyhow, how have you got the mv7 connected?

Openmv connects to ardupilot via mavlink connection on telemetry ports, you can see more instructions on communication here …

https://ardupilot.org/copter/docs/common-openmv-optflow.html

Openmv due to its limited storage and performance, it is not able to perform two actions at the same time (currently), and according to my tests since the landing pad has the letter H, you could use text recognition to for openmv to recognize the letter and land precisely by following it, I am currently improving a part of the code so that it activates text recognition only when the flight controller arrives at the departure point …

And after I finish I can send the code here to you, with updates almost often …
@Bruce_Baxter

Has anyone else had the JeVois camera randomly crash on them? During flight testing today I lost all serial communication to RPi companion computer that I am using to read ArUco tag target values from the JeVois camera. After landing and inspecting the camera, I discovered that no lights were present on the back of the camera (including the green light), but the fan was running. No amount of cable wiggling helped to bring life back to the JeVois. After unplugging / replugging the JeVois, it fired right up. This happened two more times today during testing.

It was warm out today during testing, perhaps around 85 deg F. I’m thinking the camera maybe got too hot and shutdown? Haven’t messed with the CPU performance settings and I believe the default is max performance, so I’ll look at that next.

Anyone else have this issue?

Hello all,

UPDATE for JeVois camera issue: Turned out to be a bad camera. I emailed JeVois and they had a bad batch of black cameras. I have an orange one installed currently and it seems to be working fine.

Does anyone know how to change the gain / PID of the Landing_Target messages through QGroundControl or SITL? I am using a JeVois Camera to calculate angles to an ArUco tag and when the camera first detects the tag the drone reacts quite violently, pitching and rolling rapidly and eventually losing the tag entirely. I am using the RawSensor setting and currently have sensor lag set to 0.

The frequency of the Landing_Target message is usually around 8.5 Hz. I would like to see a higher number but am limited by the speed at which the JeVois camera outputs the serial messages with the tag information.

ArduCopter version: 4.0.3
Python Code to parse JeVois serial & send Mavlink Landing message: Patrick Poirier Github
The camera is hard mounted on the frame.

Thanks for the help!

–UPDATE–

I have recreated the issue in SITL. Even if I have a completely stationary ArUco tag, PL.px and PL.py values are rapidly oscillating. So it is something with the value output from the RPi / JeVois that is really off.

Flight Log (no tag movement)

Referring to this poster, you can see that I am actually tracking the Target angle using the Mavlink Inspector on the GCS. This is actually invoked with ctrl-f on Mission Planner.

https://github.com/patrickpoirier51/JeVois–Python-Tracking/blob/master/PRECISION_LANDING_SETUP.jpg