Solo flyaway and crash during auto mission

Hi Folks,

I was flying my solo using tower and running an autonomous mission. It worked great at least once earlier in the afternoon. I armed solo then switch into auto. It took off straight up and started heading towards the first waypoint. Then it went nuts and started to fly away.

I tried to bring it back with the sticks but got no response. Then I hit the pause button (ie brake mode) a few times which had no response. I then tried the fly button (loiter mode) which didn’t do anything either. It got to a point where it was about to reach some buildings so I disarmed it mid flight using the (a+b+pause) button combo.

Unfortunately it was too dark by the time I got to the crash site. I waited until morning and with the help of a friend found solo about a metre under water in the creek. Remarkably no damage but it was water logged with salt water and the battery firmly in place :’( . I’m cleaning all the pcbs now.

I’ve flown ardupilot for years and know these flight controllers don’t just fly away. I’m very happy to admit fault if it’s something I did but I couldn’t see anything from the time I spent going over the logs. The behaviour looked like a GPS glitch but the log and the fact it was exactly where the last GPS co ord was makes me think otherwise. I would really appreciate if someone with a more experienced set of eyes could take a look at my logs. Thanks!

Dataflash Log: 150.BIN (1.0 MB)
Telemetry Log: solo.tlog (2.3 MB)

3DR uses their own version of ArduPilot, you should be talking to them so that they can determine what happened. If you aren’t satisfied with their answer I’m sure someone in the community can look at your logs to confirm it, but your first course of action should be to report this to 3DR and wait their analysis.

Yeah I’ve reported it to them but from what I’ve seen around they’ll take their time and may not be that helpful. Regardless of what platform I’m using, it’s still running arducopter and I just wanted some help analysing the logs. I guess I’ll have to wait at the mercy of 3DR.

Actually, these flight controllers can and do fly away if provoked. If the software receives garbage sensor data, it may go insane.

Which is what happened here; the EKF had no idea what was going on for a lot of this flight.

Here’s what the GPS said:

… but the EKF didn’t believe that. Here’s what the EKF thought was going on:

As you can see, the EKF had no idea. You can see it resetting its position a few times in there to the GPS position.

Now the EKF did work out that it was in trouble. The vehicle should have decided to land itself, frankly, and perhaps on a recent ArduPilot it would have.

Why the EKF went insane I haven’t worked out. I can see that it is deeply unhappy with the compass (and the field lengths are significant, and the magnetometers aren’t agreeing particularly well) - but it would be nice to know how that suddenly became a problem for it given you’d already had a flight.

Hey Peter thanks for having a look. I could see that the EKF gets unhappy very quickly but I couldn’t work out why. It seemed to lose the horizontal position within metres of taking off with the velocity and others following suit once further into the flight.

That’s what I don’t understand. Not even 30 mins prior I was successfully running that same survey mission without a problem, and I’d been flying all afternoon. Also I just realised it may not be obvious from my username but it’s Jaimyn, and unfortunately this was my new Solo.

Is there anything particular you recommend I have a look at to see what could have made the EKF go nuts? Also what tool did you use for generating that map?

Thanks :slight_smile:

The only thing that really struck me in the sensor data was the discrepancy in the magnetometers:

Note what happens to the Z components (and total field strength) between the two magnetometers.

You might like to check earlier, more successful, flights to see if the same signature is present.

It might be that in the earlier flights EKF was relying on one magnetometer and in the flight chose poorly. Digging into the EKF messages may tell you that - few people probably remember how magnetometer data was merged into the EKF in the version of ArduPilot running on Solo ATM.

The tool used to create those maps was mavflightview.py. I had to use screen-grab this time around; my changes to include the flight mode legend don’t make it into the optional output file you can specify.

1 Like

Thanks Peter, I’ll have a look at the earlier flights for comparison.