EKF3 - Uncommanded periodic high throttle out in PosHold

[Thorsten]

Hi all,

I ran some tuning tests today on a 400mm quad. After several attempts, I now have a good tune (Autotune consistently fails to tune Roll + noisy Pitch and Roll) thanks to suggestions from @Leonardthall (https://discuss.ardupilot.org/t/250-quad-with-pixfalcon-help).

The copter is relatively new but was in the air for several hours. After the last autotune I performed some flight tests. During the first test something strange happened, and I cannot determine the cause from the logs.

• I had the copter in PosHold about 2m above ground.
• Then the copter started a series of uncommanded jumps.
• The intensity of the jumps was increasing as was the time between them.
• I switched to stabilize and it was gone.
• Switching back to PosHold it came back again with increased intensity.

You can see a divergence in Roll vs. DesRoll. This was me… I touched the legs to provide some external force to see how it reacts (don’t try this at home…). I first though the jumps were somehow induced by my interference. But it already started before (see second screenshot). It is not related to IMU inconsistency and vibrations are low. Also a 30min flight directly after the event and a reboot as fine. Anyway, this is definitely not a normal behaviour and I don’t think it is a mechanical issue.

I would really appreciate if someone more experienced in log file analysis could have a look. The log is at: http://thorstenbehrens.de/PeriodicThorttle.bin (Uploading on discuss.ardupilot.org always stops at about 34%)

Best regards,
Thorsten

[freezer (fritz barnard)]

I Had a quick look. it seems your vibration levels is quite high.
I am quite new to the hobby, so I might be wrong.

[Thorsten]

Hi @freezer,

thanks for having a look!
Why do you think the vibrations are high? In contrary I think they are low - at least when looking at the IMU.Acc values and the vibes and comparing to the thresholds listed at: http://ardupilot.org/copter/docs/common-measuring-vibration.html

For sure it is higher at full speed, but even then the vibes are below 20.
But maybe I am missing something.

Thanks again!

[freezer (fritz barnard)]

Hi Thorsten

when I looked at first, the z seemed a bit higher. I am used to lower vibraion levels on my my octaquad .

[Thorsten]

Hi Fritz,

ok, so you must have really low vibrations!

[Thorsten]

I had another look at the logs. It seems to be related to the Baro and that (just a guess) EKF tries to compensate something:

[freezer (fritz barnard)]

Hi Thorsten

I use the bulb and earplug method. works for me. however it does seem that vibrations is not the issue here.
It almost looks like you are using the barometer indoors. Might be wrong again!

[freezer (fritz barnard)]

from an very aggressive 3 minute flight. I can unfortunately not find a better one to show.
Hope you find the solution to your problem. I myself had a bad crash a few weeks ago because I had my props the wrong way around . crashed due to uncontrollable yaw…

[Thorsten]

These are really low vibes! Looks like a rigid frame!

[Thorsten]

GPS and Baro alt seem to be consistent. But CTUN.Alt and DAlt are off:

I increased EK3_ALT_M_NSE to 4. But this should be ok I guess.

[Thorsten]

XKF4.SH increases after about 3min 40 sec, while XKF9.SH remains stable (similar to IPD).
So it seems the first EKF core had problems. What could be the source?

[Thorsten]

I switched back to EKF2 and made some more test flights including a longer mission of 36min without any incident. Ayyway, I am still very interested to learn where these jumps came from.

[Thorsten]

It may be offtopic, but why is DAlt changing in an auto mission?

[jfield (Jeffrey Field)]

I had a look a few days back and could not see what is causing the desired climb rate to spike like that.

Also, we do some long multirotor flights and have observed the desired altitude changing for seemingly no reason. I have not figured it out. Terrain features are off, you can check that on your side too. I have no idea how, but it looks as if we are seeing a long term pressure trend correction.

[Thorsten]

@jfield thanks for having a look!
It didn’t happen again. But there must be reason for the spikes…

There is a drift in air pressure over time and thus the estimated height when using a baro. During longer flights the baro and GPS based heights can easily diverge > 10m.
There was a PR recently which adressed this problem with some internal height correction. Maybe this is what I am seeing in the log. But I thought it was off by default. Even not sure if it is in AC3.5.2 at all. I’ll check.

[sergbokh (Sergey)]

Your RC.IN_3 is about 1650 here, so it is a climb command.
And the FC does his job - it climbs and Alt does match desired Alt pretty well.
But FC does that in a strange way… It climbs like in a PWM manner.

May be it is related to a PID values?

[Thorsten]

Thanks Sergey!

Yes, it really does it in a strange way! All other things can be explained I guess.

Baro and GPS alt:
they differ but follow throttle. So they are not the cause.

RCIN.C3:
At 83.5 - 86.5 (line number 10^3) I brought the copter to an agl of about 2.2m, because I wanted to touch the legs.
At 92.5 I started responding to the jumps.
At 95, 97 and 102.5 I tried to reduced throttle/altitude in direct response to the jumps.
You are right that RCIN.C3 is at 1650. But maybe this is due to the deadband in PosHold.
Interestingly the GPS.Alt is closer to what I saw. However, my initial altitude adjustment is not visible in the GPS.Alt graph (but it was only about 30cm or so).

IMU.AccZ:
The first spike, which can clearly be seen in IMU.AccZ is at 85. Previous spikes maybe at 84.6 and/or at 84.3. Hence, it started already while I was adjusting altitude to the 2.2m agl. So, there might be a relation to the process of adjusting altitude.

The copter flies fine with the PIDs so I doubt that it is a PID issue. What is different to my other copters regarding ArduPilot is fast IMU sampling (PixRacer), OneShot and EKF3.

Any suggestions for further tests?

[sergbokh (Sergey)]

Baro and GPS alt:
they differ but follow throttle. So they are not the cause.

Agree, raw Baro and GPS altitude does not match well the altitude estimated by FC. But that I believe it is because of acceleration spikes that are affected on EKF estimation.

It’s not very clear for me why issue was gone after you enabled EKF2. Because as per my understanding EKF is just a attitude estimator and should not affect on throttle behavior.

Did you tried to fly with EKF3 again after that?

[Thorsten]

I am not sure if it is really an EKF3 problem. I was in the air for several hours with EKF3 without any problems. To revert back to EKF2 was just a precaution. This strange behavior only occurred once - at least when I realized it. But it was definitely scary and I really like to know the reason.
I’ll make some tests with EKF3 again this week.

[sergbokh (Sergey)]

By the way what is your props? I also saw a strange vibration in the beginning of your log. I had something similar when used a poor quality flexible 10’’ props. They generally did fly well but on some exact RPM there was a hard bumpiness and vibrations.