Reading logs... Warning: EKF primary changed: 1

What specifically should I be looking for in the logs to try to troubleshoot this warning? Occurs in 3.5 and 3.6, multiple full calibrations and get the warning message intermittently in Mission Planner and QGroundcontrol after the tilt alignment message and before GPS lock message on start up. No issues showing in auto analysis of log. Thanks.

1 Like

Patrick,

AP runs at least 2 EKFs by default with each one running off a separate IMU (accelerometer, gyro). One of the EKFs is used as the primary source of attitude and position estimates while the other runs as a backup. This message means that for a moment at least the EKF system wasn’t happy with the main EKF and switched to the backup. It does this when it thinks the sensor data isn’t adding up well… i.e. there’s an inconsistency in the data.

Normally it’s the compass that causes problems but it can also be the GPS. If you have a dataflash log we can have a look.

3 Likes

Thanks for offering to look. This log is from the last flight. I have subsequently replaced the GPS and compass hardware but not wiring and still get the warning. The warning occurs before arming and then on flying the flight drifts with abnormal behavior like no yaw on RTH. The warning is intermittent after a reboot.

Edit: I see both IMU1 and IMU2 raw accelerometer values don’t overlay. Is this the issue and if so how does one fix it? And also in params EK2_ALT_M_NSE in my latest param file using 3.6.2 is not the default ‘1’ per the documentation (http://ardupilot.org/copter/docs/common-apm-navigation-extended-kalman-filter-overview.html) and is ‘3’. I haven’t changed it, though. Is the documentation still correct?

1 Like

Hi mackay9, what I really need is some documentation.

I could use a full list of params with an explanation of what they do–more than what’s in MP. What the default values are, what the mins and max are and why you would change them.

Full and accurate list of log items in particular what each EKF item is in the log. What the acronyms mean. How the values are chosen.

The website docs aren’t complete with every item. Or am I missing something? Thanks so much for your help.

3 Likes

First, the documentation:
Full parameter list for ArduCopter. In most cases, the listed min and max numbers aren’t actual hard limits; they can be exceeded safely if your configuration calls for it.
Description of most EKF-related parameters and log fields. It’s somewhat outdated, so some fields aren’t described.

The error doesn’t happen during your flight, so it’s not visible in the log. I can’t say exactly what causes it without the error present. You can set LOG_DISARMED = 1 to capture the error if it only happens while the drone is disarmed.

I did notice that the flight controller’s position estimate is a bit off from the GPS reading, but I can’t say for sure if that’s the cause without seeing the error in the log.

Hi Rick, thanks very much for looking. I’m linking the new log captured pre-arm. I notice it has a NaNs fail now. I don’t know what that is and can’t find anything on it in the docs, but see from a forum post it is not relevant. Is that the case?

Edit: I can see that NKF4.SH is ranging between 1 and 5 contrary to the docs EKF4 SH which reads shouldn’t go over 1. Am I reading that right that NKF4 in the log is actually EKF4 in the Dev docs? If the issue is indeed the barometer. How would one correct the issue?

1 Like

Hm I think I see the problem. Yes, it’s the altitude consistency check that is failing, but the barometer is fine. To see this, compare CTUN.BAlt (barometer altitude) and CTUN.Alt (EKF altitude estimate). You can see that the EKF altitude is going crazy for some reason.

The other main source used for altitude estimation is the accelerometers. They are used to determine climb rate (CTUN.CRt), which is then integrated to contribute to altitude estimation. For some reason, IMU 1 thinks the drone is ascending. Compare IMU.AccZ and IMU2.AccZ. IMU2 is noisier, but you’ll see that its Z acceleration is more accurate, being centered around -9.8 m/s, which is acceleration due to gravity.

IMU1, on the other hand is inaccurately measuring gravity to be between -10 and -10.2, which the EKF would interpret as a constant upward acceleration, which does indeed appear in the climb rate measurement. I also plotted IMU1 temperature, as it seems like the Z measurement trends downwards as temperature increases. Temp only changes by a few degrees, though, which shouldn’t bias the accelerometer that much, but it’s a possible culprit if your IMU is defectively susceptible to temperature bias. You may be able to test this by performing your accelerometer calibration after you let the IMU warm up for a while. Or you could use the realtime tuning graph on Mission Planner to watch az and az2 (and pressTemp?) as the board warms up.

1 Like

Rick, thanks very much. I see what you mean. Interestingly the EKF Primary change warning occurs mainly when starting cold rather than warm–leave it a couple minutes and get the warning, or reboot immediately and don’t get it. That would tie in with it being a temperature change being the instigator of the issue.

1 Like

I am facing the same issue. Warning autopilot EKF primary change 1 or 0. when i use maxbotix rangefinder.

EKF switching can be caused by many things, we need a log to be able to determine what is causing it for you.

My Tarot 680 Pro crashed with same error message, can expert help me to understand cause of crash