Maverick - vision_landing

Precision Landing in ArduCopter has been implemented in various ways in the past - IRLock, and the previous vision based efforts from Sebastian Quilter and Daniel Nugent - all brilliant bits of work. However at some point in the past AC_PrecLand was updated with a mini-EKF, tuned for very low latency sensors (IRLock?), and laser rangefinders were added as a dependency, which effectively removed the ability to use a Companion Computer vision-based system.

So a while ago I started what I thought would be a fun little project, to improve the Precision Landing function in ArduCopter, or at least improve the integration/implementation of it. Easy, I thought, a quick, fun little project for an hour or two in the evening to de-stress from work.
Six(+) months on, somewhat like the core Maverick project, this has turned into a bit of an obsession. However, also like the core Maverick project it has been a voyage of discovery and a fantastic learning experience. The boxes of shattered quad pieces tell another story…

However, vision_landing is now getting to be quite usable. It uses Fiducial Markers (patterned tags) like Aruco or April tags as the landing markers, and uses these to calculate the position and distance of the landing site - so no additional laser rangefinder is necessary. It can optionally use multiple markers (as in the examples below) to facilitate really accurate landing - the closer the UAV gets to the landing site, so it locks onto incrementally smaller targets for better accuracy. It has lots of config options available for maximum flexibility, and an optional realsense branch which takes advantage of Intel RealSense cameras for extra funky goodness. The marker boards are available as PDFs from the github page and can be cheaply and easily printed.

Here are a few examples of it in action. Apologies for the terrible camera quality, I’ve had a lot of problems with integration in general, mainly around cameras and USB3. Unfortunately I obliterated my last quad against the side of my house before I could get better footage, so once retooled I will add better examples here:
https://www.youtube.com/watch?v=E5PEEvcGPGc

https://www.youtube.com/watch?v=4ahQUu4kxuk

https://www.youtube.com/watch?v=OnWrUb7D-T4

vision_landing is available as a pre-installed, pre-configured component of Maverick. To start it, simply edit the config file to your liking (~/data/config/vision/vision_landing.conf) and start the service: maverick start vision_landing. It is also available separately to install manually or integrate into other systems:

This is very much a work in progress. It’s not always reliable, so it’s thoroughly recommended that you test it in an open space to start with. The PrecLand library in ArduCopter has a few issues that are still being worked through, in particularly it doesn’t seem to deal well with high latencies, so it’s best to run at the highest framerate possible (all of the above examples were 60fps). If you find any bugs (you don’t need to look too hard) or have any suggestions or questions, please reply here or do raise an issue in github. I really hope this might be useful to you, and welcome any input to improve it. This isn’t a great writeup, I’ll update it when I have a bit more time.

Thanks to the great developers of ArduPilot for the awesome work they do, and in particular @rmackay9 for putting up with an awful lot of stupid questions from me with great patience. Also to @ppoirier for putting his life on the line during testing :slight_smile:

11 Likes

Awesome stuff! Thanks for sharing. Looking forward to test this soon on the QP!

Thanks for sharing @fnoop!!!

That does look very promising indeed. Are you going to make a pull request against ArduPilot master so we can get it into a main release?

I had to google Fiducial Markers as I had no idea what they were. I’m still not quite sure what I’m looking at on the ground in the videos. Is it a patterned large piece of paper with smaller and smaller identical images?

Thanks, Grant.

@gmorph Sorry, the videos aren’t very good! You can see the landing pattern used in the videos here:
https://github.com/fnoop/vision_landing/blob/master/markers/a1-landing.pdf
The github readme has a bit more info about it.
There’s nothing really to push into ArduPilot - there were a few small updates but they’re in the latest (Copter) 3.5 betas. A bit more work would be good in the future to improve autopilot response and make it more forgiving to different latencies. I’ve seen a bit of chat from the developers lately to turn AC_PrecLand into a more generic targeting library available across vehicles, which would be great :slight_smile:

@fnoop Any study material to understand your PrecLand workflow? when i was configuring maverick in RPI4, RPi is taking a lot of time.