EKF failsafe leads to crash, please help with the log

Hello, a few days ago I had a very strange crash, the link to the bin file is here.

The vehicle is a big quad, commanded via python script to ascend to altitude and then fly to a certain point in GUIDED. The vehicle / script combo had been flight tested multiple times and works fine.

Expected flight path was a takeoff to 40m, then straight line to the point. Instead what happened was an ascent significantly shorter than 40m, followed by an ascending horizontal flight towards trees, hitting the top branches and crashing.

Looking at the .bin the mode changes from GUIDED (take off) to LAND about 8 seconds after takeoff. It also seems that a GPS glitch caused the EKF to give up and trigger a failsafe.

However, I am not terribly convinced that the GPS was at fault. To begin with, the first thing I checked was if the cable had disconnected. In the screenshot below, you can see the last GPS message (top) just before the error message (just below the middle):

You can see that a seemingly legit message has been received, with good HDop and NSats so the cable seems to have been in place.

Now, plotting the trajectories using the online plotter shows some odd things. Here are the trajectories according to AHR2, GPS and POS:



The AHR2 and GPS trajectories seem to be correct, given my own impression of the actual trajectory. The POS seems to be wildly inaccurate, there was absolutely no sharp turn to the side, the vehicle went pretty much straight, as shown in the AHR2 and GPS graphs.

Digging a bit further, it seems the position innovations started getting increasingly desperate as shown below:

I have underlined the GPS Glitch message and the NKF4 message immediately following it. In this NKF4 message, SV is not looking good and SP is on its way to not looking good.

So, at the time of the failsafe, velocity seemed to be off and position also suspect? Looking again at the last GPS message before the error, the GPS reports a speed of 7.927 (presumably m/s) and ground course of 323.89 (presumably degrees). The heading seems spot on, the speed is most likely somewhat exaggerated but nothing extravagant, it was indeed heading rather purposefully towards the trees.

So, this log seems to offer me more questions than answers.

Why did the vehicle start heading towards the trees in the first place? Apparently this movement started before the EKF freaked out so what caused it when it should have been heading straight up?

Why did the EKF point the finger at the GPS? Although the GPS-reported velocity is possibly a bit on the high side, there doesn’t seem to be anything suspicious in the GPS-reported trajectory.

Anyone have any thoughts on this?