Crash on auto mode - unexplained

I had crash which I can not explain. Please review this and if you have any clue what was wrong - make sugestion…

I have 10 years of drone experience. Drone is Skyhero, 7,5 kg - 8 motors - pixhawk - version 3.2.1.

Drone was flying two auto missions for aerial mapping. In mission two - the crash happened.
I have bin log, telemetry data.

Drone was flying at 70 meters at windy conditions. Wind was aprox 8 m/s in bursts.
At 19 waypoint - the anomaly started. Drone sudenly drop about 15 meters and pitch increased to 22 degrees. After that - it just fell down. After it fells to 40 meters - I give 100% throttle and change to rth mode. After no effect - I tried full left turn to adjust fall. No effect.
It was over in few seconds. It crashed vertical aprox 12 m/s.

After that - we made analysis of logs.
We find in logs that:

  • no failsafes triggered
  • battery voltage is ok
  • power supply is ok
  • since we have 8 motors - it is very unlikely that something went wrong with multiple motors at same time

Some things are strange:

  • look at CTUN - desired altitude and altitude. It appears that autopilot crashed deliberatly. Both altitudes matched. But if you look target altitude and alt error - it is expected range. DALT want to drop copter - target altitude want to rise copter. Why?

  • look at GPS data in BIN and CMD . Some strange spikes. Example: longitude changed from 14 to -155. Longitude changed from 46 to 199. During flight… This was 1 min before crash.

  • If you look GPS in BIN (time) - it has coordinates at north pole at autopilot inicialization.

  • if looking at telemetry log: when it switched to RTL - very strange - distance to waypoint resets to zero. It should be distance to home (play log). Copter did not even try to return home.

  • I am not sure about motors outputs - current dropped to 25 A. Hover is at 44 AMP current. However - it seems delibarate if you look at CTUN altitudes.

  • look at CMD ALT - check values. Why are values switching? Why did it stopped in flight? It is aprox at the time of crash!

Bin log is compressed in win RAR - it is too large to post it here.

I’m brand new to the hobby of automated flight, but I have some questions and observations. Take them or leave them though given my newbie status.

  1. You switched the copter to RTL mode from Auto mode vs. Loiter, AltHold, etc. so your stick commands were ignored immediately prior to the crash. I believe that is normal behavior (ignoring stick commands, not crashing!). The logs show your inputs, but given the mode the outputs aren’t in your control at all.

  2. Looking at your Dalt, GPS Alt, Location, etc. your drone appears to have crashed itself intentionally. There was no increase in vibrations prior to the crash that would account for such a massive deviation in the DAlt. So it looks to me like your quad did exactly what it thought you wanted.

  3. I notice that the CMD_Num goes smoothly up to 18 and then hits an ERR condition thereafter. I have no experience there but it did jump out at me. Do you have other logs of successful flights you can compare that to? Seems odd.

  4. Given that everything looks normal (to my admittedly untrained eye) my gut would be that your board locked / crashed during navigation or got some corrupted instructions. The ERR condition in ground station comms contributes to my opinion. My limited experience (have I said that enough times???) suggests that these flight controllers are fantastic, but experience stability problems like any other microcontroller / computing device. That’s my guess.

If you had thrown it into Loiter or Stabilize mode to try to regain control it would be interesting to see if that would have changed the outcome. But you went from one autopilot mode to another - so you never really had any control if something got corrupt / went haywire with the autoflight function(s).

-Jeff

Thank you.

Here is log from flying 10 min before crash - first mission. I did not find anything wrong here…
dropbox.com/s/ju9vzuf5xxshkpc/36.BIN?dl=0

Yes, it would be better in stabilize mode - but it was split second decision. Drone was already in free fall - like 12m/s when I changed mode.

good point:
CMD NUM just stopped working. This is important.

ANGSTBST in CTUN has dropped to -2500.
Normal value is 0-40.

Any idea why can this happen?

Look at your RCOU.Chan4 and 5
One motor is commanded to max, another goes low, most likely 5 lost thrust. , so 4(opposite) reduced thrust to maintain attitude. Then you have 4 other clockwise motors that must reduce thrust in order not to yaw , given the lost CCW thrust…

You were pretty underpowered in the time before failure, flying already at 70% throttle - once the failure happened Throut were 100% , but remaining in an navigation mode did not help, as maintaining yaw has priority over throttle. (ans so you see reduced altitude target)

RTL did not navigate home, as there were no room for navigation as it had insufficient thrust even to hover in one spot. (increasing angle would increase descend rate even more) To be honest, once pilot sees that AP fails to do the job, it’s not a good idea to think RTL will save the day.

Pilot should have switched to Stabilize mode, could even climb at the cost of yaw-rate but to a (skilled) pilot it’s not a big problem to get home even with a constant yaw. It’s also possible to gain altitude with higher yaw rate, then navigate at slower yaw rate while slowly losing altitude - then repeat.

“unexplained” crash explained :slight_smile:

Andre K - you were correct. We examined motor number five and determined it had lost thrust.

This feature - priority to yaw before thrust - can be really dangerous in motor failure in auto mode. I did not know that. Hard to believe but true - one motor failure in octocopter can bring down drone.
I had once before one motor failure in hexacopter - but it was not problem to fly in any mode. This one was very different.

This is solved -since it is not related to APM code.

yes, it’s ironic, AP is (technically speaking) capable of flying much better with constant yaw than any pilot. (especially at a distance when it’s hard to evaluate directional corrections when flying manually)

The feature is just not there, it would need proper problem-detection and kick in only then, otherwise , if we just gave yaw priority over altitude, the RPAS could ignore yaw error at any altitude error, giving worse flight experience.

BTW: click “solved” to help moderator.