Hi @buckker,
Txs for the report. I’ve added it to the 4.1 issues list (see “surface tracking poor stability”) so it won’t be forgotten. There have been changes in the position controller but we’re not expecting that any re-tuning should be required. Of course your test indicates we’re wrong but let’s see. Txs again.
I think I have the same observation with 4.1.0-beta3 and a TFmini Lidar.
First I thought it is a bad Lidar reading when flying over the not solid surface of the cornfield.
As far as I can remember it was fine with 4.1.0-beta1 (cornfield was not so high as now with beta3 
@RainFly I had never success with Benewake lidar sensors for missions over cornfields. If the plants are small the performance is acceptable, but as soon as they grow the lidar sensor signal becomes very unreliable. Maybe your problem is not firmware related, its probably more due to the sensor type.
@rmackay9 Thank’s a lot for adding to the list.
Hi @RainFly,
Txs for the report. There weren’t any changes that I know of in this area of the code between -beta1 and -beta3 so I suspect it is an environmental issue (e.g. taller corn). Surface tracking has always been a bit rough when flying over crops or trees when using lidar because the altitude changes so much.
In any case, if you’d like me to look into this further could you perhaps provide an onboard log? I can’t promise 4.1 will work better than 4.0 but at least it shouldn’t be any worse.
I did today another test flight. I replaced the TeraRanger Evo 60m with a Lightware LW20 to make sure that the issue is not rangefinder related. The behavior is the same. Unfortunately it’s pretty bad 
I hope the performance can be improved for the stable release. For me it’s not usable for safe flight at the moment.
I also checked if the dolphin fly occurs in Loiter. It does not, it’s smooth as always.
Does the new s-curve navigation also affect the altitude controller?
Have a look at the video from the flight this afternoon:
And finally I have another strange behavior. My copter suddenly pitch back during take off and landing in Loiter. I have installed a proximity sensor. This sensor sees obstacles where I can’t see anything 
Never took a note during 4.0.7…
Here is the log: https://1drv.ms/u/s!AunrEY0aDm-h7VLULxRAP0elZwDO?e=HeGDCN
The video looks similar to what I see with my copter. Also the airflield looks quite similar to mine
The frequency looks identical. But I have lower amplitude of the oszillation. But I fly also lower, only 3m relative above ground.
It can be, that my statement in a previous post was wrong.
I’m not sure any more, if 4.1.0-beta1 was OK.
Too many toys with new AP-version to remember - copter, rover, boat, plane
Maybe a side effect from switching from EKF2 to EKF3?
Hi @buckker,
OK, thanks for the video. This is very clearly caused by a change to the filtering we do on the rangefinder between 4.0 and 4.1. I think we will restore the original filter value but also make it configurable using a new RNGFND_FILT parameter (see PR here)
This will likely be included in -beta5 in a couple of weeks but if you would like to test before then I’ve created a developer build here for CUAVv5Nano. This is almost exactly the same as -beta4 but with this new parameter added and the default filtering restored. If you’re OK to test this I’d love to hear if it resolves the problem or not.
Hi Randy
Thanks a lot, I have just downloaded the developer build. I hope I find some time this week to test this version.
Michael
I’m back from the field. I tested RNGFND_FILT values from 0.15 up to 0.7. It does’t help. My quad flies still like a dolphin…
What does the message ERR: TERRAIN-2 mean?
To make sure that the drone is not the problem, I changed all waypoints from Terrain to Relative. No problem, it flies like it should be.
Log:
https://1drv.ms/u/s!AunrEY0aDm-h7gCLYXKQ_2CHMLAY?e=mN6hEd
Thanks very much for testing this. From a look at the log it seems like various values were tested but the autopilot wasn’t rebooted between changes. Sorry, I clearly should have explained that a reboot is required between changes. Once this is released the parameter descriptions will say it is a “RebootRequired” parameter.
I suspect though that maybe the real issue is this innocuous looking change that I did at the request of Lightware to increase the timeout for the Lightware lw20 I2C which is the lidar you’re using (I think).
… so, could you try this new firmware instead and see if it fixes the problem? This one also has the RNGFND_FILT parameter so maybe you could try setting it to various values including 0, 0.25, 0.5 and 10 and see how it does. https://www.dropbox.com/s/nvan137m3ds6nv9/Copter410-dev-rngfnd-filt-and-lwi2cfix-CUAVv5Nano.zip?dl=0
ERR: TERRAIN-2 means that it is missing terrain data. This error message seems to be appearing every time the vehicle enters Auto mode so I’ll have a look into this as well. The lidar seems OK and we don’t see the vehicle executing a terrain failsafe… anyway, i’ll look into it and add it to the “to be investigated list”
@rmackay9 Thanks for having a look at the log file. I will update the firmware and test your suggested values.
@rmackay9 I did another 10 flights with different settings. I was able to reduce the dolphin behavior. It seems that a value of 5 works the best for my quad. The overall altitude control is not that bad. Unfortunately the motors pulses more and getting hotter than normal.
Compared to 4.0.7 it’s still worse, so I think we have to put another effort in this topic.
the new logs: https://1drv.ms/u/s!AunrEY0aDm-h7gNupg2OqZUfX3hL?e=BffIvE
Flight with RNGFND_FILT set to 10:
Flight with RNGFND_FILT set to 5:
Hi @buckker,
OK, thanks for testing and it makes sense that increasing the filter improves things.
I suspect the additional roughness (vs 4.0) is coming from the changes to the altitude controller done as part of the SCurves project. So I’ll have a talk with @Leonardthall about this. Below is a graph of the desired climb rate in Auto (1st half) and Loiter (2nd half).
Hi @buckker Thanks for your help here!
Could you please try this firmware:
https://drive.google.com/file/d/15uJoMdicUWNiKz6rpO5YrXD8IOxT8OAN/view?usp=sharing
This adds some logging so it would be good to turn off your FFT batch logging.
It would be great if you could do an auto flight with:
PSC_TC_Z 0.25 and 0.5
The logs will be quiet large so it is a good idea to do a short mission that would recreate the problem and not spend too much time flying around after.
Thanks again!
@Leonardthall Your modified firmware looks much better I will do some further tests in hilly terrain and look how it performs there.
Logs: https://1drv.ms/u/s!AunrEY0aDm-h7j1IWknjvT6NvXTs?e=iGAHdI
PSC_TC_Z 0.25:
PSC_TC_Z 0.50:
Thanks, I will get that into master!!
We are experiencing a similar problem with our drone dolphing. We do not see a parameter PSC_TC_Z in ArduCopter 4.3.0. Additionally I would like to know what the paramater do, where to access it and how it fits in with the greater control logic. The same goes for RNGFND_FILT. We can can change this one, but I am not aware of any explanation to its function and how it ties in with the control loops?
If you start a new thread with a full discription and logs I can take a look.
I will get the logs and post them in a new thread.
