Summary of Optical Flow Development in ArduPilot

Hello Greg,

If you want to do garage fly, I would recommend that you take off stabilize and then switch alt-hold to check on the rangefinder (sonar in your case) setting makin shure it does not jumps on the ceiling…and the switch to loiter.so that opticalflow kick-in and that the quad is keeping a fixed position (well depend how much vortex you create in this enclosed space). Make sure that you have a good textured surface, optical flow needs it == This is why I have this nice carpet, and that the focus is set to approx 3-5 feet and finally good lighting. Some would recommend installing a net for protection, but at minimum wear protective glasses.

LOL I just saw rhat the sequence above is the on that you used for your outdoor flight, so you should repeat the same sequence for the first test, and then, once everithing is well settled, you can takeoff loiter, but make sure that you dont move throttle too fast once you get to middle, there is some lag on the dead zone and sometimes it makes the quad jump in the air !!

Ok, thanks for the tips! It’s been so cold lately that I don’t even want to go in my garage. :slight_smile:

We had a nice weather break in NY for February so I got a chance to test my PX4 Optical Flow setup on a hill climb and in circle mode. Although the hill climb seemed to work fine, the circle mode had an odd dip in altitude at the 180 degree and 360 degree positions. The EK2_GPS_TYPE parameter was set to 3 to ignore GPS and use the flow sensor using APM Copter v3.4.3.

Does anyone know what the odd dips were in Circle mode? And is Optical Flow meant to work in circle mode?

Hi @GregCovey,

first of all, thanks very much for sharing and documenting your experience with px4flow! I sure appreciate it a lot!

Just two things I’d like to discuss a little bit here:

  1. The sonar on the px4flow sensor.
    The Maxbotix EZ4 sonar is not entirely unused since its readings is used by px4flow internally to convert the flow vectors of the images into real-world velocities. I tested the sonar using an Arduino and the readigs were actually quite accurate. It was however just a stationary test.
    The test also confirmed that px4flow does output its sonar readings through i2c. It’s just that those distance messages don’t seem to be accepted by APM. However, there is a way for pixhawk to read the sonar outputs using [this] (https://www.lambdrive.com/depot/Robotics/Controller/Firmware/ArduPilot/OpticalFlow.html) tutorial. The reading on APM seem less accurate than those of the Arduino, but I think I just need to adapt some of the parameters.

  2. Optical flow
    Following the hill contour has likely less to do with optical flow, but more with the sonar (extra sonar). As far as I am not mistaken, the way px4flow works is that it takes two consecutive images in a short time, tries to recognize similar features in both of them and then calculates the vectors of those features. These vectors, scaled with the distance readings, would represent your quadcopter’s 2d movement in the horizontal plane, relative to the objects in the frame of the camera (which is the ground in most cases). The x and y velocity is then passed to APM, which makes control adjustments based on unwanted drift of your quad. Therefore, I think that only horizontal movements are compensated by px4flow.

Regarding the dips, maybe it was caused by some leaves flying around, blocking the sonar’s direct view to the ground. Can you reproduce those dips? Be careful with your drone though, those sudden dips were quite dangerous. It is strange however that they happened exactly at 180° and 360°. Could be some software issues.

Cheers, thanks for reading and please keep updating your progress!

Nafnaf

1 Like

Hi Nafnaf,

Thank you for the detailed information. I will try to replicate the circle mode dips in another area (or surface) when the weather permits. I will also look at your tutorial link and see if it deviates from the APM Wiki where I set up my PX4 Flow board and firmware.

Cheers!

1 Like

Hi Greg,

I noticed that you have two sonars installed on your copter (one on PX4Flow and one Maxbotix). You can’t have two sonars powered at the same time because they will interfere with each other. The way they work is to send a pulse and then listen for that pulse to bounce back and measure the latency. If you have two sonars sending pulses, they will hear each other’s pulses and return incorrect readings.

-Ben

Hi Ben,

It is true that there are two sonars but only the one external to the PX4Flow board is activated. The sonar that comes with the PX4Flow board is not accurate enough for good distance measurements.

The installation I used is in the APM Wiki here. The Wiki also contains a list of appropriate external sonar devices with a wide range of pricing.

1 Like

Hello All,

Is it possible to use a sonar or LIDAR as obstacle avoidance system (pointing to the front of the UAV)?

Thank you
Rotem

@Rotemshf Have a look in the documentation http://ardupilot.org/copter/docs/common-optional-hardware.html

Hi Greg

Thanks for sharing your experience. I am also trying to use Optical flow and I am sure your experience will help.

I have a few doubts:

  1. Why do you need to arm in loiter mode? How is it different from Arming in STABILIZE mode and changing it to LOITER when attained certain height?
  2. Did you tried getting position hold using Optical flow in GUIDED mode or GUIDED_NO_GPS mode?
1 Like

Hi AKM,

I do not know why you need to arm in Loiter mode. I typically arm in Stabilize mode and move the quad out onto the field for testing before changing to other modes. Currently, Stabilize mode is the only mode I can count on for accurate flight when Optical Flow is enabled. Note that you still need to pick up the quad before arming to calibrate the range finder.

I just happened to re-test my Tarot 650 quad yesterday in a wild field of long grass and bushes. I did video the two flights but decided not to render them on Vimeo since the performance was so poor. Several times, even though I was at a proper height, the quad was almost destroyed. I am still using the same firmware and parameters as mentioned above in my previous post and video.

When switching from Stabilize mode to Alt Hold mode, the copter would always dip significantly in height forcing me to raise the throttle setting to gain height. When switching to Loiter mode, the copter would usually stay in proper position. In this tall grass field, Circle mode did not work well. Switching back to Loiter mode from Circle mode made the copter move off rapidly in a wrong direction. I was forced to switch back to Stabilize mode and fly manually home.

It seems that BARO works much better for altitude hold and my M8N GPS works best for position hold than using the Optical Flow feature. It may be that my choice of an inexpensive rangefinder (Maxbotic I2C EZ4 $40.) does not perform well. Using a more expensive Lightware SF11/C 120m unit for $249 or Lightware SF10/B laser rangefinder for $400 may provide better results but they are not in my hobby budget.

I will likely abandon my Optical Flow testing for now and simply switch back to BARO and GPS with a parameter change. Perhaps the better sensors will come down in price for DIY users and I can revisit Optical Flow on a new copter firmware release. I am also waiting for prices to drop on RTK since $700 is beyond my hobby budget.

Hi Greg,

Thanks for sharing your experience. I can understand the difficulties you have gone through. Even I am on the same page.

Let me share my story:

My stack:

  1. Firmware installed - APM:Copter V3.5.0-rc6 (efab409b)
  2. PixFlow - Optical Flow
  3. Range Finder - Garmin Lidar Lite V3. Used PWM Connection.

I followed this guide step by step - http://ardupilot.org/copter/docs/common-px4flow-overview.html#pre-arm-checks

Test 1:

I tried the First Flight (Copter Only) as mentioned in the guide - I started flying the copter indoor in STABILIZE mode with EKF_GPS_TYPE = 0 at a height of 1 meter (approx). After a few minutes of flying, I switched to ALT_HOLD mode and my copter suddenly took an extreme hike in altitude and hit the ceiling and crashed, damaging the legs. Though it was not enabled to let ELF use Optical Flow, I am not able to figure out what went wrong.

Test 2:

I went to open space, set the EKF_GPS_TYPE parameter to 3 to make the EKF ignore GPS and use the flow sensor. Started flying in STABILIZE mode, took to copter to a height of 2 meters (approx), then switched to LOITER mode - Just in a nick of time, the poor copter dropped down damaging the legs, props and one motor. It didn’t get disarmed, I guess it just couldn’t respond to the throttle or something I don’t know.

Attached the logs for reference.
test1-stabilize-alt_hold-crash.bin (2.6 MB)
test2-stabilize-loiter-crash.bin.log (3.3 MB)

I am expecting to find the reason for these behavior before I do any further experiment with my copter.

My today’s goal was to test Position Hold using Optical flow and Lidar Lite V3 in LOITER & ALT_HOLD mode. But my ultimate goal is to make this thing work in GUIDED & GUIDED_NO_GPS mode. I want my copter to hold position even if GPS is not available for which I thought I could use Optical flow. So after this, i need to figure out - How to make this work in GUIDED mode.

Thanks.

Hi
Was you able to achieve position hold using PX4Flow in guided or guided_noGPS mode?

Thank you

Very helpful tread, thank you for posting!

What is the most up to date information on optical flow integration?
Best firmware version for success?

1 Like

I’ve had a CUAV PX4 flow 2.1 sitting in the projects drawer for ages, waiting for a companion lidar to be purchased and for me to grow a pair.

I’ve just realised the sonar mounted is the maxbotix EZ4 1242 - so this should be a plug and play PX4 flow for anyone wanting to get started without buying an additional sonar or needing an i2c splitter as it’s done onboard. Correct me if I’m wrong. I’ll try it out. Nice work CUAV

https://store.cuav.net/index.php?id_product=52&id_product_attribute=0&rewrite=cuav-px4flow-21-optical-flow-sensor-smart-camera-for-px4-pixhawk-flight-control-without-sonar-or-with-sonar&controller=product&id_lang=1

Aside, is there any mileage to mounting the PX4flow to the bottom of a gimbal so it always faces straight down or is it ardupilot coded to account for pitch and roll? My Git fu isn’t strong enough to check.

1 Like

hello @GregCovey
according to your tutorial that you have discarded above and that
http://ardupilot.org/copter/docs/common-px4flow-overview.html#common-px4flow-overview
the drivers of PX4flow and separate Maxbotix sonar are already taken by arducopter ??
just change some parameters in Mission Planner
to make a servo and landing with the PX4flow and sonar, is it true or it is necessary to modify the code that I clone since the source git clone https://github.com/ArduPilot/ardupilot.git ?

Hi Alex,

Yes, that’s correct. The standard release of copter now contains the PX4flow support.

Cheers!

Hi @GregCovey
according to the wiki PX4flow

The PX4FLOW (Optical Flow) Sensor is a specialized high resolution downward pointing camera module and a 3-axis gyro that uses the ground texture and visible features to determine aircraft ground velocity. Although the sensor may be supplied with a built-in Maxbotix LZ-EZ4 sonar to measure height, this has not been reliable enough over a range of surfaces in testing, so its readings are not used

just out of curiosity, why the maxbotix of Px4flow is not used?
in what situation are these data false?
thanks for any answer

Hi Anna,

The PX4FLOW Wiki Overview was not my writing. I do not know the author.

Cheers!

1 Like