Copter flew away in GUIDED mode

We’re testing an autonomous copter, that is controlled by a Raspberry PI. The mission is, basically, to take off, fly to the destination point, fly back, and land.

We had a lot of successful attempts, but there was a failed attempt which finished with a crash.

The copter was flying back, but then (as I saw in the logs), error “Subsystem: 16, Errorcode: 2” happened. The copter started to fly in wrong direction. Than there was “Subsystem: 17, Errorcode: 1” error. Then the copter said “EKF variance” and “EKF primary changed:0\r\n” to the Raspberry, and went to LAND mode.

After that it crashed somewhere, so we could find it later hardly.

Here is the link to the log file: https://drive.google.com/file/d/0Bw9Vkl56vQBLcXpLalJOaTBPZ1M5VXI0Rkh2R3pfdV9kZzYw/view?usp=sharing

Can anybody help with this?

Please post the logfile so people can better help. Thanks

I added it to the post.

For the first error
I reckon that the subsystem is ERROR_SUBSYSTEM_EKFCHECK
https://github.com/ArduPilot/ardupilot/blob/master/ArduCopter/defines.h#L422

and the error is ERROR_CODE_EKFCHECK_BAD_VARIANCE
https://github.com/ArduPilot/ardupilot/blob/master/ArduCopter/defines.h#L461

which occurs here

https://github.com/ArduPilot/ardupilot/blob/master/ArduCopter/ekf_check.cpp#L58

and here

https://github.com/ArduPilot/ardupilot/blob/master/ArduCopter/ekf_check.cpp#L76

If that is correct,What that means in detail though, I don’t know!

I would guess it should be possible to find the meaning of the other error messages by looking them up in the first link also

Thank you for response.

I understand, that this is something, connected with the EKF, EKF compass variance actually.

But what has been this error caused with, that’s the question… Maybe a bad GPS/compass module?

Yes it could be, However my guess is that something went wrong and the frame starts to vibrate, which made EKF2 loose its reference.

if you see after line 380K your vibration levels skyrocket. That is my 2 cents.

Vibrations increasing

EKF2 mag innovations increases

That leads to an unhappy EKF2 engine.

Could you please verify your setup to see if any propeller got loose?

Thanks for the report. It looks like you’re doing interesting stuff because your log covers an incredible 16hrs. I’ve never seen a log covering that length of time so well done!

I’ve had a look at the logs, and it’s certainly an EKF failsafe that triggers the land but I’m slightly unsure what has caused the EKF to become unhappy. I’ve asked Paul Riseborough if he can have a look.
The GPS seems fine and vibration levels are not too bad (around 20m/s/s which is acceptable) making me think it’s the compass.

The ARMING_CHECK is set to zero meaning arming checks are disabled. It’s generally a good idea to find and fix the cause of problems rather than removing the check. Worst case remove the check for the one item causing problems.
My guess is that it was a “compass inconsistency” pre-arm failure? The offsets on the internal compass (compass 2) are quite high so perhaps there’s a lot of wires near the flight controller. It’s possible to disable the internal compass by unchecking the compass-use on the compass calibration page. It’s mentioned in the picture on the wiki here: http://ardupilot.org/copter/docs/common-compass-calibration-in-mission-planner.html#onboard-calibration

Sorry I don’t immediately have a better answer but hopefully Paul will.

Randy, and Oleg,

I see that the last flight was done with the PI solution index at “1”, I take that as the 2nd ekf instance. When the problem develops, the Mag and GPS velocity innovations exceed 1 causing the “lane switch” to index “0”, and the trigger of LAND mode.

I believe the landing would have been successful if it were over a suitable lading site. I think it only appeared to fly away during the descent because the RCIN channels 1 and 2 appear to be trimmed 60 to 65 points from there trim value. This should have been obvious in the previous loiter flights. I think channel 1 and 2 are pitch and roll, but did not confirm. LAND_REPOSITION is set to 1, so these trims had the vehicle lean 3+ degrees(nose up, left arm down?) during the ~40m decent. Hit something and bounced off it half way down!

Looks like it was “re-positioned” into a building, then a tree, with what looked to be a fine solution “0”. Sorry for your troubles. Have a look at the safety radio trims! Not sure what sent the solution 1 into confusion.

The only thing I can’t understand is why the vibration levels have raised that much. Do you have any clue?

Fernando,
Looks to me that the vibes are a bit higher at altitude. Maybe more breeze up there, more asymmetric lift vibrations, etc. It seems fairly common.

it makes sense. Great analysis!

How did you plot like that? I am interested in it.

Thank you very much for all the answers!

rmackey9, yes we (Copter Express limited) have a lot of very long logs, because actually we’re developing an autonomous drone and the charging station for it, so the drone can do some job, then automatically land to the charging station, wait until the battery is charged, and then fly again. For precise landing we’re using our own system, that lands drone using camera and markers.

You can check out the video of our system work if you interested: https://youtu.be/jVYJgF9WNKA.

With this autonomous system we need the copter to work really robustly, that’s why we’re making a lot of tests. First, I think we’ll enable the prearm checks, and then may maybe we’ll try to disable the internal compass, as it is not necessary.

So I’ll post our results later.

Hi, Any updates on this topic?