HereFlow Optical Flow Sensor Testing

[ukmook]

My hereflow works well.

I was flying over grass which I’m not sure is the best texture for it. Also I’m using TFmini Lidar. Followed the wiki and it pretty much worked out of the box.

Malc

[Thi_Van (Thi Văn)]

I’m currently have a drifting problem with the HEREFLOW. There is a small constant drift in whichever direction the Hereflow camera is facing. For example, I have the camera side of the Hereflow pointing at the nose of the Pixhawk Cube and the drone would drift forward. Then when I flip it around 180 degrees and changed the Flow_orientation to 18000. It starts to drift backward. This is when I noticed that the OF.bodyY line is smaller than the OF.flowY. I followed the calibration process and try to change the FlowY_scaler so that OF.bodyY & OF.flowY would line-up but the number of FLowY_scaler doesn’t seem to change anything on the graph. Let me know if you want to see the logs.

[Thi_Van (Thi Văn)]

does your hereflow work indoor (without GPS)? if so, how does it perform? My main goal is to have it fly indoor similar to those DJI Phantom. However, I’m having a drifting and yawing problem. The drift is related to which direction the camera side of the Hereflow is pointing (I explained this issue in more details on the comment above). The yawing problem is related to the position of the Hereflow relative to the center of the drone. It does not yaw relative to the center. I’ve measured and set all of those POSITION parameters but still now luck. I wonder if there has been a manufacture defect on my Hereflow.

[Thi_Van (Thi Văn)]

here is what mean by the OF.bodyY is smaller than OF.flowY. No matter what value of FlowY_scaler I change to, I still get the same result.

[Alberto_Ds]

Hi all, on my miniquad 3" i use this optical flow similar (the same?) heraflow. Works well indoor an outdoor (tested at 26m of altitude), but if i use this with TF Luna lidar (benewake), the behavior is unstable, and often leans in roll or pitch and becomes unmanageable.

This is only optical flow:

Test optical flow hold. Arducopter 3" 125 gr

Here i post the log and pics

New LiDAR sensor Benewake TF02-Pro TFmini-S TF-Luna for drones UAV

Hi all, unfortunately it didn’t perform any better than the VL53L1x. In low light it works well, in direct sunlight the behavior was not good, making the quad unstable and I also had to land in an emergency because I didn’t have good control. On the Yappu telemetry the reading seemed correct. Also, I have mounted TF luna on a piece of copper PCB but the GPS doesn’t work well (didn’t see any satellite in over 10 minutes of flight). I also used the auxiliary function to switch off the lidar, but…

This is my optical flow module

https://it.aliexpress.com/item/33005508643.html?spm=a2g0o.productlist.0.0.43ae73f85kNr5K&algo_pvid=1d838233-7151-4fa3-a44d-844634163803&algo_expid=1d838233-7151-4fa3-a44d-844634163803-8&btsid=0bb0623416102088667053740ebba9&ws_ab_test=searchweb0_0,searchweb201602_,searchweb201603_

[proficnc]

@bugobliterator, @rmackay9, any suggestions for the scaling issue?

[vosair]

I can confirm the same issue regarding scaling. Some of my hereflow have the drift issue as well.

[rmackay9]

All I can say really is that the calibration and setting of the FLOW_POS_X/Y/Z is very important and greatly affects performance. It is also important that the rangefinder provide accurate values with few glitches.

I also don’t think that FlowHold mode works well for most users and I’ve toyed with the idea of removing it. Too many users have issues with FlowHold that it causes a significant support burden and leads to unhappy users.

I’ve got it on my to-do list to add an inflight flow calibration routine that would help users get a more accurate calibration. The code is not simple though and it would also require the users at least temporarily attach a GPS.

[vosair]

I love the idea of an inflight-flow calibration. For our aircraft we use it to augment the GPS, so no problems there.

[Thi_Van (Thi Văn)]

I’ve been working on & off with the Hereflow + LightWare LW20b lidar for about 3 months. Here are all of my experiences with it:

• The rangefinder works great and the drone held its altitude accurately every time.
• The Optical Flow work best in PosHold (indoor/outdoor).
• When flying indoor, make sure to have plenty of lighting, do not fly on a reflective surface. For best performance, lay several strips of colorful tapes on the ground the create a more contrasting surface.
• Pitch/Roll work perfectly fine while Yaw is off-center no matter what Flow_POS_X/Y/Z I set too.
• There is a small constant drift in the direction of the Hereflow heading. This is most likely has to do with the FlowY_Scaler, which I’ve spent many hours trying to calibrate the Hereflow to get the correct graph but never worked. I’ve share some pictures on the top to show you what I mean.
• I’ve been using EK2 to use the optical flow as its primary guiding position but I’ve also read somewhere that EK3 works better with external guiding aid.

All I want is to complete my drone build so it could somewhat perform like a DJI Phantom when flying indoor if you have ever seen one.

[Alberto_Ds]

Randy, please don’t remove optical flow! I don’t think there are other sensors so cheap to hover a mini quad (like DJI mini), especially in indoor flight! I struggled a lot with the settings but now it works quite well outdoors and in direct sunlight as well. It costs about 10 usd and is very small even for a 3 "quad. My problems arise if I try to use it together with a small lidar like the benewake TF luna (small and inexpensive). Both sensors give correct readings, but if used together they are difficult to calibrate IMHO we should implement the wiki with documentation about it, optimize the code and add a calibration system by optimizing the blending even with traditional sensors (gps, mag, baro) Thanks!

US $10.19 15% di SCONTO|Flusso ottico Spot del Sensore di Posizione di Volo...

[rmackay9]

@Alberto_Ds,

ok. To be clear though, I never suggested that I would remove support for optical flow. This is what I wrote:

I also don’t think that FlowHold mode works well for most users and I’ve toyed with the idea of removing it.

FlowHold is a mode that relies only on the optical flow sensor and does not require a range finder. This mode does not work well for most users. Other modes like Loiter and PosHold can work quite well with optical flow.

[Alberto_Ds]

Ok, thanks Randy
This is my FlowHold mode and works well (25 meters) without lidar:

This is my Loiter mode with same optical flow sensor + TF luna lidar:

In loiter the behavior of the quad becomes absolutely unstable, even if the reading in height is correct. In practice I have the opposite problem compared to the majority of users
If it helps, here I put a log with optical flow + lidar:

https://drive.google.com/file/d/1WTaUe8nHfsY8835HbvFl-bqd7JW1axjd/view?usp=sharing

[rmackay9]

Hi Alberto,

Thanks for the log. It looks like the RNGFND1_MAX_CM is set to perhaps 300 meaning that the rangefinder is only trusted to 3m. I’m not sure what the maximum range of the rangefinder is but it might be good to check this is correct.

There’s no parameters in the log file so I can’t really look for configuration issues. In general a .bin file is better than a .log file (and it’s smaller).

In any case, I think the next step towards improving our optical flow support is an inflight calibration routine. I can’t immediately promise when that will be done though.

[Alberto_Ds]

Hi Randy,

sorry, the correct file is this, I hope it is useful:

https://drive.google.com/file/d/15xF_4Zy9IFm3NsWfpEchQQ-paCk9vIWN/view?usp=sharing

That’s right, I put the range at 300cm (outdoor use) as indicated in the wiki for my TF luna lidar, here:

https://ardupilot.org/copter/docs/common-benewake-tf02-lidar.html?highlight=lidar

• [RNGFND1_MAX_CM]For TF-Luna use 800 for indoor, 300 for outdoor. This is the distance in centimeters that the rangefinder can reliably read.

As you can see from the log, I took the quad up to 10 meters high to see if, after the 3 meters limit, the behavior was better

[WangLouis]

Hi Thi_Van

i use lightweight SF20 + hereflow, I’ve been using EK2 no gps and mode is loiter.
Optical flow is installed under the copter, FLOW_POS_X/Y/Z is set zero.
I have not used FlowHold mode.
Hereflow is work correctly(indoor/outdoor), but Fly over 15 meters, hereflow is not well,
This is the result of my test, for your reference~

[rmackay9]

@WangLouis,

I suspect if you set the FLOW_POS_X/Y/Z parameters accurately you’ll find it works better above 15m. That’s my personal experience anyway.

[Thi_Van (Thi Văn)]

Thank you for your reply. I could fly my quad 650 with my current setup with the Hereflow. However, the only problem that I’m having is the quad not Yawing with the center. It made a loop like it’s being yaw off-center and also drift a little bit to which direction it being yawed like cc or ccw. I’ve measured and change those FLOW_POS_X/Y/Z to where the Hereflow was positioned relative to the center of the quad.

• FLOW_POS_X: 0.07 m (Positive X is forward of the origin)
• FLOW_POS_Y: 0.06 m (Positive Y is to the right of the origin)
• FLOW_POS_Z: 0.06 m (Positive Z is down from the origin)

I’m still getting very inconsistent Yawing.

[rmackay9]

@Thi_Van,

I guess this could be caused by one of two things:

1. the hereflow sensor itself cannot handle yawing motion or maybe it simply isn’t reporting the yaw motion to AP
2. the EKF is not handling a rotating vehicle properly. The EKF2 does not estimate horizontal accelerometer biases while the EKF3 does.

It might be interesting to see if the vehicle does better when using EKF3 and/or whether the vehicle also has trouble holding position when using when using GPS.

[Jai.GAY]

looking forward to the inflight calibration in the next release.