Flyaway in land mode after GPS glitch

Log Analysis -- deprecated Copter Logs -- deprecated
1

We’re quite new to ArduPilot, so we may be missing something obvious here…

The other day we were doing a fairly simple test with our tricopter. The test area was right next to a river — probably not the best choice in hindsight! The mission was to fly to a few waypoints in a fairly small area (all on the near side of the river), and then RTL.

The first flight went fine. However, during the second flight, we got a “GPS Glitch” and the drone suddenly started flying across the river. At the time, we observed that it was flying on its side, and appeared to be gradually descending. We tried to gain control using the RC, but were unable to. We then lost sight of the drone, and were unsure whether it had landed in the river or on the far side of it.

A few days later, we managed to get access to the property on the other side of the river and found our drone — remarkably almost completely unharmed! We were able to access the dataflash logs, but we’re having trouble figuring out exactly what happened.

Here’s our interpretation of the log file.

  • The first flight (from about timestamp 16:54:00 to 16:56:30) was fairly uneventful.
  • After that, we tried the same mission again (starting at about timestamp 16:59:00), and fairly shortly after takeoff we got a GPS Glitch which caused the drone to go into Land mode.
  • The GPS Glitch seemed to be caused by a variance between the “GPS” and “POS” positions. (As seen plotted on the map while viewing the log in Mission Planner.)
  • That variance disappeared very suddenly at the same time as the “EKF primary changed” message appeared, which we assume means that switching to a different IMU fixed the incorrect position reading. Our guess is that the EKF position was incorrect (as a result of an IMU problem) while the GPS position was correct.
  • At this point, the drone was still on the near side of the river, although not where it should have been. However, it suddenly started going across the river while still apparently in Land mode.

The altitude readings in the log show that the drone was indeed descending (as was appropriate for Land mode) while crossing the river. What we don’t understand is why it wasn’t staying in the same spot while doing so. Could this be related to the fact that the drone was flying on its side?

The log analysis shows the following. Could the compass issues raised by the analysis be the problem here?

Size (kb) 6104.32421875
No of lines 70675
Duration 0:05:49
Vehicletype ArduCopter
Firmware Version V3.5.4
Firmware Hash 284349c3
Hardware Type 
Free Mem 0
Skipped Lines 0
Test: Autotune = UNKNOWN - No ATUN log data
Test: Brownout = GOOD - 
Test: Compass = FAIL - WARN: Large compass offset params (X:37.00, Y:-316.00, Z:27.00)
WARN: Large compass offset in MAG data (X:37.00, Y:-316.00, Z:27.00)
Large change in mag_field (294.88%)
Max mag field length (1792.30) > recommended (550.00)

Test: Dupe Log Data = GOOD - 
Test: Empty = GOOD - 
Test: Event/Failsafe = FAIL - ERRs found: CRASH GPS_GLITCH 
Test: GPS = UNKNOWN - join() takes exactly one argument (2 given)
Test: IMU Mismatch = GOOD - (Mismatch: 0.51, WARN: 0.75, FAIL: 1.50)
Test: Motor Balance = GOOD - Motor channel averages = [1544, 1538, 1591]
Average motor output = 1557
Difference between min and max motor averages = 53
Test: NaNs = GOOD - 
Test: OpticalFlow = FAIL - FAIL: no optical flow data

Test: Parameters = FAIL - 'THR_MIN' not found
Test: PM = WARN - 2 slow loop lines found, max 8.38% on line 56193
Test: Pitch/Roll = UNKNOWN - 'BarAlt'
Test: Thrust = GOOD - 
Test: VCC = UNKNOWN - No CURR log data
2

Without GPS, it will not hold position. It will drift with the wind and inertia.

What do you mean it was “on its side”?

3

It was an ekf failsafe caused by a extensive gps glitch. When land is initiated by ekf failsafe it will NOT use gps for holding position. The code assumes that position information are compromised.
The point is, that once a failsafe is initiated a land without gps, there will be no mode changes or start using gps, even if the glitch is cleared and ekf position solution become valid again. Only an RC or Mavlink command can change back to loiter or any other mode.

4

I’d be very concerned about the compass error. Without a reliable compass you can end up with some really erratic behavior.

Is your compass/gps unit mounted on a mast or directly on the frame? Any big electrical currents near it?

5

19 sec after the GPS Glitch, you switched on the RC radio, but also there, failed to change mode out of LAND.
RC on does not fix things alone, you should have switched mode.
You could have switched mode on GCS as well.

6

Without GPS, it will not hold position. It will drift with the wind and inertia.

We thought it would start holding position once it had regained a good GPS signal, but Eosbandi’s reply explains why this was not the case.

7

The point is, that once a failsafe is initiated a land without gps, there will be no mode changes or start using gps, even if the glitch is cleared and ekf position solution become valid again.

That makes sense.

I assume this means there’s no sure-fire way of ensuring that the drone will land at (or close to) the position where the glitch happened in a case like this. I thought the IMU would have at least been able to maintain a roughly level attitude during the landing, but I guess if the glitch affected the IMU too then this may not have been possible.

We were considering putting the GPS/compass unit on a mast (as suggested by FlyingPotatoes), so I suppose we should go ahead with that to reduce the chances of glitches like this in the future.

Thanks for your insights everyone.

8

That may or may not make any difference. Multi-pathing of GPS signals is the most common reason for “glitches”, and that can happen at any time where terrain or objects can reflect GPS signals, the receiver picks up the signal, but the time stamp is off. It is for this reason that GPS glitches will normally happen closer to the ground, or around objects that can reflect the GPS signal and cause it to multi-path. They rarely happen in flight where the receiver has a good clear view of the GPS satellite constellation and it is more difficult to pick up reflected signals from ground-based objects.

My recommendation has always been that the ArduPilot system is definitely capable of fully autonomous takeoffs and landings, but the GPS reliability can cause a crash. And there’s nothing that can be done about that. If you are flying in an area where the GPS reliability is good, it will likely work time after time. But if you don’t want to risk it you always have the option of taking off and landing manually, and starting and ending your auto flights with the aircraft airborne. If you, for instance, use a RTL command to bring it home, as soon as it reaches the hover point and does it’s slight delay before beginning descent to landing, switch to a non-GPS mode (I prefer to use Stabilize) and land the aircraft manually from that point.

Long ago I experienced the same thing you just had happen. And have since used the manual method of takeoff and landing to make sure the aircraft is “in the clear” before letting it fly auto. And have never had an issue in hundreds of auto flights with a GPS glitch causing a runaway helicopter. But I remember at least a dozen times in all those flights when the ground station announced a GPS glitch during landing, which did not affect it because I was in a non-GPS flight mode. Especially near HV powerlines and bodies of water like lakes or rivers, which is an excellent reflector of GPS signals.

1 Like
9

Agreed on the takeoff/landing in stabilize. Flying manually gives you a feel for how the craft is behaving and will give you a heads up if anything is wrong (low power, sluggish controls, poor handling), and also gives you a chance to fly out of any potential crashes. It’s saved my bacon a few times.

I always like to take off in stabilize, bring the craft up to about 10 feet AGL and switch to loiter, then hang there for 5-10 seconds to make sure it’s actually stable before starting a mission. I’d rather have the GPS derp out in a controlled environment than have it take off sideways on launch. :slight_smile:

EDIT: Putting the GPS on the mast won’t help with multipath but it’d definitely help with his crazy compass errors. :slight_smile:

10

Yep. I didn’t have anybody to hold the phone camera when I was doing this, so people could see what I was doing, and had to just set it down. But this is an outline of of my preflight checks for a helicopter before flying an auto mission. Basically can’t afford to be wadding up $3,500 helicopters because of a simple GPS glitch.