Wing crash after failsafe - what does the log tell me?

ArduPlane Plane 3.5
1

I am hoping this crash will prove easy to diagnose. I am not the best at reading log files, and could use some help.

The plane is a 60"+ wing running an AUAV-X2 and ArduPlane 3.5.3. I was flying LOS after a decent launch, and had done a few successful race track passes. Keeping one eye on the plane, I went to take my seat, and tried putting on my goggles. As I was doing so, I had a failsafe. As I was getting the goggles in place I found myself seeming to lose power, in failsafe, and not able to pull up, with no meaningful control. These were very brief impressions, however, so I am very unsure as to what exactly occurred.

There are a number of real and potential problems happening during this flight and I’ll enumerate them. But first, a crash video. Doesn’t everyone love a good crash video?

Definite problem #1:

I’m running an explictly beta version of the TBS Crossfire firmware, one with a known propensity to failsafe at close distances when running telemetry, which I was. I also could have blocked the antenna with my body at an inopportune moment. So the failsafe itself is explained and does not indicate a loose connection to the flight Controller, for example.

While I believe I understand the failsafe, I do not understand the subsequent crash.

Possible problem #2:

I was testing out an “Auto Launch” mode done through OpenTX config on my Taranis. I’ll attach the .eepe file for the interested, but the screenshot shows the important part.

AutoLaunchGladScreenshotOpenTX.png1026×608 53.1 KB

There’s a voice announcement, which plays a warning and countdown, then with the plane in FBWA applies 100% up elevator and ramps up the throttle. This allows me to focus on throwing the plane without rushing to get to the sticks before the plane gets stable. Here is where I stole the idea:


www.thedoble.com/uncategorized/lazy-auto-take-off-with-apm/

It does work - the launch is very good if not flawless (I think it needs just a bit more throttle set). But I am still working on better programming for exiting the mode, so it is possible/likely in the heat of the moment switching from LOS to goggles that I hit something and re-set the mode, which could have killed the throttle, or that I just killed throttle entirely myself by pushing the stick down. I have tried to diagnose this possible failure myself in the logs, but so far am just confused by what I find.

Possible problem #3:

I have had a problem in the past where one of the servo connections was loose and it would cause left elevon failure. Immediately before this flight I believed I had tracked this problem to a bad servo extension cable, and had replaced it. Maybe the problem was still present.

Possible problem #4:

When I found my crashed wing, I discovered one of the motor phases was disconnected, and there were some marks and nicks on the wires which lead me to believe the prop may have fouled and caused the disconnection in flight, and not the crash itself. I’ve now secured the phases more securely, and don’t expect this to ever be a problem again, but if this was the issue I would like to be confident about it.

Here’s a zip with the various files:

https://github.com/StewLG/catfood/blob/master/06-03-2016%20-%20Glad%20Failsafe%20Crash.zip

2

This shows the dramatic drop in altitude during failsafe pretty clearly:

http://s287.photobucket.com/user/StewLG/media/Glad%20Crash/FailsafeCrashTrack_zpsggg8wejl.png.html

The part of the flight I find most confusing is around the short failsafe, where the plane enters CIRCLE mode for 2-3 seconds.

It’s the narrow blue section here:

http://s287.photobucket.com/user/StewLG/media/Glad%20Crash/Throttle-mode-rssi-overview_zps4vopusyq.png.html

Zoomed in, with notes:

http://s287.photobucket.com/user/StewLG/media/Glad%20Crash/Throttle-mode-rssi-detail-annotated_zpsigqgeyes.png.html

Why does the throttle appear to get cut during the CIRCLE mode?

While it is certainly possible I cut the throttle accidentally while moving into goggles, the throttle being cut lines up exactly with the CIRCLE mode.Although the data makes it seem as though the throttle is being cut before entering CIRCLE mode, the fact that we see the same gap between throttle change and mode change when exiting the mode and throttle resumes makes me assume this is an artifact of logging or something else, and in fact they are happening at the “same” time, at least for the purposes of this discussion. Please someone correct me if this is wrong.

So I’m concerned that my impression is that while ArduPlane did the right thing, and detected failsafe just when anticipated, it doesn’t appear to have applied any throttle during the failsafe, arguably causing (or at least contributing) to my crash. RSSI is zero so ArduPlane should have complete responsibility for the throttle during this time, correct?

The throttle value before and after circle is a precise 70%, indicating I did not touch the throttle during that time.

I actually would prefer to find that I had accidentally bumped something, since I can be more careful in the field. Is this impression correct, or am I mis-reading things?

3

@StewLG,
I get this message when I follow your link to your files: (Sorry about that, but we can’t show files that are this big right now.)
Regards,
TCIII AVD

4

TC III:

Do you get the “View Raw” link next to that message? Just click that, and you’ll be able to download it. GitHub is just refusing to preview the file - you should be able to download it.

ClickViewRaw.png999×482 25.2 KB

5

@stewlg,
Yes, I did. I will try again. Thanks.
Regards,
TCIII AVD

6

This morning I found myself leaning towards the cause of the crash being the disconnected motor phase I found on the crashed wing:

I’ve fixed this for good (see inset photo above), but I’d wanted to know for sure this was the problem, if it was. So I’ve been looking again at the throttle values. I think my original take on this was wrong; CURR.Thr is defined as “Pilot input throttle from 0 ~ 1000” (Downloading and Analyzing Data Logs in Mission Planner — Plane documentation), so it seems to make sense that it went to 0 in throttle controlled mode like CIRCLE. The throttle variable I think I was looking for is CTUN.ThrOut, defined as “Final throttle output sent to the motors (from 0 ~ 1000). Normally equal to ThrIn+AngBst while in stabilize mode.”

So here’s a graph showing that, with my latest guesses as to what happened:

I’ve finally put the Altitude (BARO.Alt) on there, as well as RSSI (RSSI.RXRSSI), so you can see where the problems start, where it starts to drop, and once the crash is complete.

I’m enough of an optimist to be doubtful there is some terrible bug in ArduPilot that caused CIRCLE to crash, so I’ve been trying to find parameters to blame. Here’s my lineup so far:

THR_MAX

Currently set to 75, I intend to put this to 100%. It’s a big heavy wing, and it may need all the power it can muster in an extreme circumstance.

TRIM_THROTTLE

When no airspeed sensor is available some parameters are not used for
some purposes:

the TRIM_ARSPD_CM parameter is not used as an airspeed target in auto flight. Instead the TRIM_THROTTLE parameter is used as base throttle, with extra throttle added/removed to retain the target altitude.
[ Fixed Wing FAQ — Plane documentation ]

Right now I have this set to 45%, which is definitely too low, possibly even a stall during level flight, and a recipe for disaster during a steep turn.

I will probably put this at 75%, maybe higher.

ARSPD_FBW_MIN

STALL_PREVENTION is already set to 1, so ARSPD_FBW_MIN should be active (even without an airspeed sensor, which this plane lacks). It’s currently set to 9 m/s, which comes out to 32.4 km/h, which judging from the flight footage is way too low. I will set this at 60 km/h, I figure. ARSPD_FBW_MAX I’ll put at 120 km/h.

If there are other throttle/stall/speed parameters I should look at, please let me know.

7

I had a better flight with the above parameters. I’m thinking the very low TRIM_THROTTLE is what did me in on the original crash.

TBS has corrected the failsafe-prone Crossfire firmware, so I was able to control when I exited FBWA. I tried CRUISE mode, since it would continue in a straight line and I would hopefully only be trying to regain altitude/pitch if something went wrong. Cruise worked fine - although I (re)discovered that 100% throttle kills the batteries I have set up this wing to use, and now I was pushing the batteries below 10 volts, cranking 60 amps.

I’ve reviewed the flight footage, trying to find the sweet spot for performance. Here’s the highly suspect spreadsheet I came up with; treat this as a series of rows of anecdotes.

40% throttle functions, but is a bit wobbly on FBWA, since it is somewhere uncomfortably close to the stall speed. 100% throttle destroys batteries at 59 amps (and it’s also a rough ride). Best I think is about 60-65% throttle. It flies fine and only puts moderate pressure on the batteries.

Trying to hit this optimal band of speed, I’m changing the following parameter values:

TRIM_THROTTLE = 75 => 60
THR_MAX = 100 => 70
ARSPD_FBW_MAX 30 => 20
ARSPD_FBW_MIn 22 => 18

I also found turning a wee bit too slow, and I want to see if slightly faster ascent/descent is comfortable, so I also adjusted the following:

LIM_ROLL_CD 5000 => 6000
NAVL1_PERIDO 20 => 19
WP_RADIUS = 40
LIM_PITCH_MAX = 2000 => 3000
LIM_PITCH_MIN = -2500 => -3500

I haven’t flown any of these changes yet.

(See notes below for the underpinnings of these numbers:)

(ARSPD_FBW_MIN = 65 kmh (40% throttle = 9.94 amps. Wobbly but OK.) = 18 m/s)
(ARSPD_FBW_MAX = 75 kmh (65% throttle = 15.57 amps. Quick but not insane battery pressure.) = 20 m/s)
TRIM_THROTTLE = 75 => 60 (%). 13.5 amps.
WP_RADIUS. We cruise at 60% throttle, which is approx 75 kmh. Which is 20 m/s. So we go 40 m/s in 2 secs

8

I’ve had a number of flights on this new configuration now, and things are finally flying nicely.

Again, I believe the original problem was with the speed and throttle settings; they were too low for this wing, and once adjusted everything started to click.