EKF3 leans & accelerometer bias but low vibe

allanm84 · June 19, 2020, 8:58pm

I have an issue with APM 4.0.3 running on a MRo Control Zero on a 550mm quad. I’m running the EKF2 and EKF3 simultaneously with the EKF3 as the primary. Reported vibe appears to be quite low and reasonable. The EKF2 has a reasonable roll/pitch estimate, but the EKF3 roll/pitch estimates get dragged off during flight. Additionally, the EKF3 accel biases seems to be drifting and settling at an offset value, which I’d imagine is affecting the roll/pitch estimates.

I’ve attached the log and some screenshots of the issue.

Log here: https://www.dropbox.com/s/nijv6yzbh6rqsqc/2020-06-19%2009-00-48.log.zip?dl=0

amilcarlucas · June 21, 2020, 2:38pm

There are some EKF fixes on 4.0.4-rc2. Have you considered upgrading?

allanm84 · June 21, 2020, 3:13pm

I have not. Looking at the git history, nothing stands out as to what could be contributing to the leans; most changes between 4.0.3 and 4.0.4-rc2 appear to relate to compass or GPS, which I suppose could contribute but I think it unlikely. As a side-note: I’ve seen the issue with and without GPS enabled.

My intention is to check the FFT and see if I need to turn the notch filter on, even though the vibe looks low.

jasclarke308 · June 22, 2020, 2:47pm

This is most likely an error in the yaw estimate feeding through to the pitch and roll estimate. I’d guess that EKF2 is using Mag1 and EKF3 is using Mag2 (which is seeing high interference).

Try disabling Mag2 and flying again, I suspect that will fix your problem. In most circumstances it’s a good idea to disable the internal compasses as they are much more prone to interference.

allanm84 · June 22, 2020, 3:49pm

Ah, interesting, thanks @jasclarke308. I’m guessing the AX/AY/AZ biases in XKF2 are NED-frame, meaning a poor heading estimate would result in a poor bias, which would result in a poor roll/pitch estimate? Or is there a shorter path which would explain why the poor heading estimate would result in a poor roll/pitch?

Regardless, this was a very helpful find; I’m going to test with the internal compasses disabled.

vosair · June 22, 2020, 7:41pm

I had a similar issue with 2 Control Zero units. When I updated to the latest master build, it went away.

jasclarke308 · June 23, 2020, 1:17am

I haven’t looked into this too much, but I believe that the accel biases are in the body frame (generally, X, Y, Z are used to denote body frame, N, E, D are used for earth frame). My theory is that the mag measurements, which are also in the body frame, are conflicting with the accel measurements and causing the accel biases to wander. But that’s pretty much pure speculation by myself.

allanm84 · June 25, 2020, 2:03am

I’ve tried the following with no success:

Disable the 2nd (and 3rd, non existent) compasses
Calibrate and enable a notch filter (this helped quite a bit with gyro and accel noise)

I’m still seeing “the leans” and the yaw estimate seems to drift (+/- 10deg?) even while sitting on the pavement. It drifts enough that I see “DCM: yaw inconsistent by … deg” errors. The logs from these flights are here: https://www.dropbox.com/sh/h45scvffcldj5g2/AAAwgGpjcOpitd1kI-IDYKiga?dl=0

Next up: I’m going to try APM 4.0.4-rc2.

jasclarke308 · June 25, 2020, 3:35am

Good choice changing firmware, I had some problems with compasses in 4.0.3 as so jumped straight back to 3.6.12.

Some other options that may help with the leans and/or yaw discrepancy:

calibrate accelerometers again
set COMPASS_TYPEMASK to 256 which will disable detection of the internal ICM-20948 compass on the MRO Zero on startup and then calibrate compasses again (after this is done, you should notice just the MAG message and not MAG2 in the logs)
calibrate compass again with “strict” enabled
sometimes a poor gyro calibration can cause a yaw measurement error. This can be a real problem in cold climates, where the calibration of gyros done on startup (ie. cold) is no good once the board warms up. So rebooting the controller after it’s had time to warm up but before flight can be a good idea.

I must admit I haven’t used EKF3 ever really, but I do believe there are some differences in the heading calculation so EKF2 so that may also explain some of the behaviour you’re seeing as well.

I think you’re on the right track with the firmware change though. Good luck!

allanm84 · June 26, 2020, 8:30pm

Another update: I tried 4.0.4-rc2 with no success (same leans issue, and the heading wandered). I also tried the COMPSS_TYPEMASK=256, rebooting, and recalibrating compass with the same results. At this point I’m going to go back to the EKF2, which appears to work fine. At some point I’ll dig into the code to backtrack the offsets, but for now I’m happy with the EKF2 performance. Thanks for the help!