Pxflow - ardurover support

@rmackay9,
Are these “plug & play” with ArudCopter? I really like the PMW3901 from Tindie, and am interested in possibly fitting it to my Solo.

If weight is an issue, that could be an alternative, but the PX4Flow is a much better option than pwm3901 for a 3D vehicle. The integration with the EKF is excellent and quite optimal, the integrated IMU and the possibility to use different type of lens offers a lot of flexibility

2 Likes

Sajin,

As far as I know nobody has integrated the pmw3901 with ardupilot except on the skyrocket vehicles which are all manufactured in a factory. It should work but I can’t make any promises.

@ppoirier, I’m not sure it matters which sensor is used actually. I don’t think the PX4Flow sensor makes use of the integrated IMU does it? I suspect you know better though…

2 Likes

Thanks for the info, guys.

I may try just to see if it helps at all at lower altitudes.

1 Like

Hi all,

I was away for a couple days working towards a paper deadline. But now I’m back and should have time to work on this. Thanks for the suggestions Randy. I will order a couple PWM3901 to experiment with those. But for the moment, I’ll try getting the PX4Flow sensor working.

The PX4flow only has a gyro (the L3GD20), not an IMU. The PX4/Flow/master and PX4/Flow/klt branches use it for compensating for rotational motion when computing the translation velocities.

I took a look at the recommended firmware source code, and the main differences I found with the current head on PX4/Flow/master are that it is using the KLT algorithm from PX4/Flow/klt_flow2 and that it has opposite signs for the mean flow output.

I didn’t see any changes on the i2c protocol. So PX4/Flow/master should be compatible with ardupilot after flipping the sign of the optical flow output, unless the reported flow from the KLT version has different units or meaning. i’ll do a test tomorrow and let you know how it goes.

– Juan

2 Likes

So, with the latest PX4Flow firmware, the optical flow measurements look a lot better than what I’m getting with the recommended firmware.

This is the image I get with the recommended firmware

This is the optical flow measurement I get when rotating the sensor, first around the x axis then around the y axis

This is the image I get with HEAD on master

And these are the flow measurements I get, again first rotating around the x axis then around the y axis

The axes and signs in both cases appear to be consistent with a right handed coordinate system, and with the axes marked on the back on the sensor. So I don’t think the sign flipping is necessary. Going to test on the vehicle soon.

– Juan

2 Likes

So, we are getting optical flow messages through the pixhawk (using the latest PX4Flow firmware), which is promising. But they’re stuck at a constant value. Time to dig into the ardupilot code.

BTW, the ground distance measurement is changing over time, even though it is supposed to be fixed by setting RNGFND_TYPE to 21, RNGFND_ORIENT to 25 (Down) and RNGFND_GNDCLEAR 25ccm. Is this because the range finder distance is somehow being filtered over time? Is it being integrated with the barometer measurements?

I still don’t fully understand how to make the newer version of the firmware work. I decided to give a try to the recommended firmware again, this time compiling it from source. Now the images I’m getting look like this:

And the signal to noise ratio seems to be much better. I hardcoded the focal length for the lens I’m using:

Again, this is using the recommended firmware, which uses the klt algorithm, compiled from source. I’ve tried it on my vehicle, and if I set AHRS_GPS_USE, now i’m getting reasonable ground speed measurements (i.e not infinity or zero, and pretty close to my eyeballing estimate).

A couple questions I have now are

  1. Is it possible to get an estimate of the position indoors, using the optical flow sensor only?
  2. If the GPS receiver is enabled and available (AHRS_GPS_USE), is the optical flow sensor ignored?
  3. Why am I getting 0 and nan for the flow_rate values? Why are flow_x and flow_y always zero?
  4. With the new px4flow firmware, I’m getting optical flow mavlink messages, but their values seem stuck. Why doesn’t this firmware work as the old one?

Cheers,

– Juan

1 Like

Hi Juan,

I quickly want to check what the status of your optical flow integration is? Are there still some open issues with the pull request (https://github.com/ArduPilot/ardupilot/pull/9334)?

I am trying to use a raspberry pi camera for optical flow on my rover. So this would be a great base to start with.

Cheers,
Daniel

Hi all,

I’ve been doing some experiments with the PX4Flow sensor and I am still puzzled. The optical flow messages look like garbage, but the reported ground velocity actually looks good (messages obtained through mavros).

I thought it would be possible to some odometry from the optical flow sensor + IMU, but it doesn’t look like something that would work out of the box. I’m trying to run some controller learning experiments on the rover, and since running experiments outdoors is more time-consuming, I was hoping I could get something working for indoors. Is there a way to send a fake gps signal at the beginning (to set the starting location) and have the ardupilot state estimator do odometry based on optical flow + IMU?

The firmware I am using for the pixhawk is available here. It was compiled from this PR on the ardupilot repo.

For the PX4flow sensor, the firmware I’m using is available here. I compiled it from this fork: https://github.com/juancamilog/Flow/tree/apm_klt_flow. It is essentially the head on PX4/Flow/master, with the KLT algorithm implementation in the repo by priseborough.

Hope this helps,

– Juan

Hi Juan,

Thanks for the update. I have a working optical flow sensor using the raspberry pi camera now. Good work with the rover integration!
In the mavlink messages, flow_comp_m_XXX only contains the gyroscope data, but not the compensated flow. So be careful with that. For me the mavlink flow_x and flow_y look good.
Have you calibrated the px4flow and verified that it is working correctly? (see http://ardupilot.org/copter/docs/common-px4flow-overview.html)

I still have trouble with the EKF data fusion though. My ground speed seems to constantly increase despite the optical flow being zero.
I generally wonder if it would be better to fuse optical flow more like the visual odometry on a rover. The optical flow measurements should be proportional to horizontal position changes.

Cheers,
Daniel

Take note that the opticalflow is a velocity estimator so if there is no movement, thr signal goes to zero. Thus the only signal being processed by the EKF is the IMU and as you know these units produce a lot of gyro and accelerometer noise that translates into drift.

As soon as it moves, the error gets close to zero and the position estimation start integrating more accurately but unfortunately the vehicle location is wrong because of the IMU noise induced drift

1 Like

Here is a quick update. Slowly things start making sense to me. Some problems I ran into:

  • My RPi optical flow sensor returned quality 0. But Ardupilot ignores samples with quality 0. Oops, my bad.
  • The optical flow fusion will ignore samples if no takeoff was detected and the height above ground is low. This is always the case for rovers, so the data was ignored.
    if (!takeOffDetected && ((terrainState - stateStruct.position.z) < 0.5f)) {
        ofDataDelayed.flowRadXYcomp.zero();
        ofDataDelayed.flowRadXY.zero();
        flowDataValid = true;
    }
  • In EKF2_Measurements, there are some hard-coded flow limits that are too small for a rover (Where the camera is very close to the ground).
if ((rawFlowQuality > 0) && rawFlowRates.length() < 4.2f && rawGyroRates.length() < 4.2f)
  • There is an additional parameter that limit optical flow measurements which was too low (EK2_MAX_FLOW).

  • The default optical flow innovation threshold was too low (EK2_FLOW_I_GATE). When there is an sudden change in speed, the EKF discarded the optical flow measurement because they were too far off. This triggered velocity runaways caused by noisy accelerator measurements.

  • I had to add LEDs to the optical flow camera to get better optical flow data.

I still have problems with with run-away of the z-velocity. It looks like the runaway happens when I see lange optical flow innovations. I wonder if the optical flow fusion influences the z position/velocity. It would make sense for a copter. When the optical flow is larger than expected, the copter could simply be lower (instead of flying faster). But this would make no sense for a rover.

1 Like

Daniel,

I have a bunch of Rpi and cameras and I’m very interested in getting optical flow working with my mower rover. Do you have a link with information about your setup? The work you are doing is very exciting.

vince

Fusing Optical flow data as Visual Odometry seems to work much better. A quick and dirt change is here. EKF3 has to be used and the scheduler loop frequency must be higher than optical flow measurement rate.

@Vincent_Miceli, are you running ardupilot on a RaspberryPi? E.g. Navio2 or Erle-Brain. Otherwise things become more tricky as the Optical Flow data would need to be transfered somehow.

1 Like

Right a now Im using pixhawk cube. I will try any combo that might work for my mower.

@Daniel_Widmann This is a really interesting idea! I’ll try it out on my rover tonight. Thanks!

– Juan

Did this feature get added to a recent version of rover? I didn’t see anything in the ardurover documentation for it.

Hello,

No, we haven’t progress on this

This should be looked at again, with all the new flow sensors avalible it would be a cheap way to do precision offroad without RTK.

1 Like