Intermittent EKF flags

My large Hexacopter likes to keep me guessing. Some days it flies great, no EKF warnings, and some days it still flies great but EKF fluctuates during flight. the compass column fills up/ goes red. Yesterday I started receiving the following message on a flight “EKF2 IMU1 switching to compass 1” and then more switching. The aircraft was about 1 mile away at the time but still appeared to be flying normally. Error compass variance appeared on the HUD but wasn’t recorded anywhere else. After seeing the compasses switching in flight I decided to RTL and then EKF messages cleared. back to low values in the EKF tab.

A second flight after a reboot had similar results.

Bad EKF values usually started on the turn northbound
EKF cleared during first flight after changing heading during RTL.

Aircraft has a Pixhawk 2.1 running V3.6.10 (I know I should update). Here2 GPS Using compass 1 (external)and 2(Internal) 3 is disabled.

Logs for both flights

Yeah, the EKF is rather unhappy with the compass, although I can’t find any apparent cause. The “incorrectness” of a misbehaving compass will usually vary with heading, so the apparent intermittency is not surprising. This is a visualization of the compass variance:

The green line is EKF “innovation,” which is how far off the EKF thinks the sensor reading is, measured in the relevant units (Gauss for magnetometers). The red line is the EKF consistency check. This is the “meter” that Mission Planner displays on the HUD. It’s the innovation divided by the max acceptable error - it should be close to 0, and when it exceeds 1, the EKF begins rejecting the sensor reading. This is why the EKF switched compasses just as you hit RTL, although it switched right back to compass 1 for some reason. Of course, that didn’t solve the problem, so the innovation climbs right back up until you land.

I couldn’t find anything that is clearly related to mag readings or consistency except heading, which is not terribly informative. This is what you get when your compass is poorly calibrated, though, so it would be worth trying a recalibration. Another thing you could try is set COMPASS_LEARN = 2. This will tell the EKF to use the GPS ground course to learn new compass offsets.

Not sure if this is related, but before you started flying, there is something causing your compasses to get really noisy. It happens at the same time as the battery current draw changing, so maybe it was you plugging/unplugging something. Any idea what it is? There’s no motor spinup or anything.

1 Like

The aircraft is a gas/electric hybrid so the mag noise is likely caused by the generator starting. I agree that the cause is likely a bad compass calibration as it is very difficult to calibrate due to the size of the aircraft and the interference of having a generator strapped to it. I’m looking forward to auto learning offsets that is coming in 4.0 I believe, in the meantime I’ll give compass learn = 2 a try. Thanks for the insight!