Nimbus 1800 VTOL Crash Analysis

[Chad_Frazer]

I have eliminated a fair amount of mechanical problems. I could use some help trying to figure out why this bird took a nose dive after more than 24 successful missions.

Log File C:\Users\thear\AppData\Local\Temp\tmp4AD4.tmp.log
Size (kb) 52167.8056640625
No of lines 527443
Duration 0:05:30
Vehicletype ArduPlane
Firmware Version V4.0.5
Firmware Hash 0bfa2638
Hardware Type
Free Mem 0
Skipped Lines 0
Test: Autotune = NA -
Test: Brownout = GOOD -
Test: Compass = FAIL - WARN: Large compass offset params (X:-43.85, Y:-135.90, Z:312.92)
WARN: Large compass offset in MAG data (X:-43.00, Y:-135.00, Z:312.00)
Large change in mag_field (251.04%)
Max mag field length (561.65) > recommended (550.00)
Test: Dupe Log Data = GOOD -
Test: Empty = GOOD -
Test: Event/Failsafe = GOOD -
Test: GPS = GOOD -
Test: IMU Mismatch = GOOD - (Mismatch: 0.53, WARN: 0.75, FAIL: 1.50)
Test: Motor Balance = NA -
Test: NaNs = FAIL - Found NaN in POS.RelOriginAlt
Test: OpticalFlow = FAIL - FAIL: no optical flow data
Test: Parameters = GOOD -
Test: PM = NA -
Test: Pitch/Roll = NA -
Test: Thrust = NA -
Test: VCC = UNKNOWN - No CURR log data

Mission starts at the 79% mark.

FOUND: One motor on the right wing had a motor that was binding. Unknown if that was a problem before or after the crash.

She had a funky takeoff to 150ft, I chalked it up to crosswind and a change in magnetic declination. She had her accelerometers calibrated before takeoff but not her magnetometers. I do not recall doing a reboot but I might have. No errors were displayed at takeoff.

She went to 150ft, transitioned and then went 20degs down until she hit the ground at higher than her max allowed speed when in auto. She did not appear to respond to max up elevator I was giving her during the downward decent.

I’m pretty sure its not a mechanical failure, it might have been an ESC failure as the left wing caught fire. The battery was deformed but not punctured and it did not ignite (fire was from ESC). I can’t see anything in the logs that would prove this though.

I am open to all suggestions!!

I have uploaded params, .bin, .tlog and a few photos.

Crash Log 8-30-2020

Firmware Version V4.0.5 on the Cube Black
Mission Planner version 1.3.73 build 1.3.7530.26934

[Kywon.M_Vlogs]

…

At this point, i can’t even tell if this is a software issue or a critical ESC issue so bad your FC cranked out on and crashed the entire plane… What software and firmwares are you running and what ESCs did you use?

[GregCovey]

Hi Chad,

It is indeed a peculiar situation. Just after the “Transition Done” message at 332, the rear motor stops as expected, but the front two motors also start to reduce and then go off at 334.7 well before the crash at 337.7. Everything seemed to be functioning but the flight controller appeared not to know what mode it was in.

If you watch the flight in UAV Log Viewer, it looks like the plane just floated without throttle until it stalled. The control surfaces did what they could to keep it level. You can see the current drop to 0 just after the transition done. After the crash, there was tons of current which liked burned out the ESC. The advantage of using the UAV Log Viewer is that you don’t need a tlog from the GCS and you can plot parameters while you view the flight replay.

If I had to guess, I suspect an issue with the mapping plan. Typically, I use commands like:

• VTOL_TAKEOFF
• DO_VTOL_TRANSITION
• WAYPOINT

In your plan, it reads:

• TAKEOFF
• DO_CHANGE_SPEED
• WAYPOINT

[Kywon.M_Vlogs]

Well… I guess it was either a issue with either the ESCs or the FC itself. Looks like after it did its little helicopter to plane thing, it’s back motor wasn’t the only thing that stopped. Looks like all the motors stopped. Your plane took it’s last flight without power before falling to it’s demise…

I don’t know how the fire started.

I think what the problem is is that you have it set so that in plane mode no motors worked. Like if you attach a glider to a drone or something and let it go only but it’s the glider that puts it’self un the air and drops itself.

[Chad_Frazer]

I updated the main post to have the answer to you question Kywon:

Firmware Version V4.0.5 on a Cube black

I was using Mission Planner version 1.3.73 build 1.3.7530.26934

This was a self build but I used the components that Foxtech uses, for the ESC it is the Hobbywing Xrotor Pro 50A for the front two motors (one each) and a Hobbywing Xrotor 40A for the rear motor.

I can say this too, the front motors are rated for 70A continuous draw but the ESC will only go 50A. This has worked because the nominal current drawn in flight is about 10amps plus/minus 5amps. Hover with a full battery is 38amps, with a nearly depleted battery its 48amps give or take 5 amps. That’s total current draw. That does not stress the ESC’s at all, not even during a takeoff (vertical takeoff).

[Chad_Frazer]

First off, that UAV Log Viewer is wicked cool! I’m sold! Bookmarked!

Yes, after watching it I can say that’s exactly what we watched from the ground. It has NEVER struggled like that. I’ve launched from wet/dry ground, on stones, from rebar’ed concrete and sand and gravel. In all test flights (tests flights after tuning was done) and on all of its two dozen or so actual missions it never launched like that.

Winds aloft in the log reported 8-14mph which the bird likes, even more than zero wind. I did not abort because I thought that the hangar (which is a good 50 feet high and made of metal) was perhaps causing a mini vortex that I was just too unfortunate to takeoff next to, coupled with being close to a large metalic structure which might of been affecting the magnetometer.

As for the mission plan that is what I have always done and this is why:

If the quad plane parameters are active, I am not aware that there is any difference between VTOL Takeoff vs regular Takeoff (I could be wrong). I have used that command extensively in the past.

I have to immediately do a DO CHANGE SPEED command because during the transition the bird like to build up an excess of speed, like in the 40-50mph range, cruise is 32mph. While I could tinker and find the correct value to put in the PID control section (specifically the %throttle for cruise under the Basic Tuning Tab) I find that telling the bird what to cruise at while in auto mode is much much safer and reliable. The %throttle I did set in the Basic Tuning page is 13% which is the median for throttle range for the majority of flights she has done.

Typically she burns .5 volts from a fresh battery in order to complete a transition programmed to be completed at 150ft.

If I ever forget to have the do change speed command, I’ve had birds overspeed and pile drive into the ground with parts in tow. I’m very careful about that still

I will research the VTOL vs regular takeoff command.

[Chad_Frazer]

I have reinspected the power distribution board and can find no physical defects or smells of an electrical fire/overheat condition (she smells good).

I have put some more pics in the folder for you guys. This was the “autopsy” photos I took while trying to figure it all out.

The power distribution board is rated to handle 160A and when you hold it, it FEELS like a trough mother. I doubt the Power Distribution Board failed but then again I have not tested it post mortem either so that’s still a possibility.

[Kywon.M_Vlogs]

What about the esc? From what i see now, it looks like both of the escs failed.

[Chad_Frazer]

Both ESC’s are in the newly uploaded pics…one is a charged remnant and the other “appears” to be completely fine! I can’t tell if the ESC caught fire in flight or after the crash. One witness said he saw smoke on impact whereas I saw smoke only after the impact.

A short in the ESC would potentially cause a system wide failure of motors like what is showing in the logs and would most definatley overheat the ESC to the point of fire.

I did find that one motor had a suspected dislodged magnet causing a rubbing between the windings and that magnet but I can’t tell you if that occurred before or after the crash.

I watched the current draw (which should be excessive if the magnet was rubbing the windings) and I don’t know if 80A for a vtol takeoff is excessive or not…I’ve never noticed the current draw on a vtol takeoff because I’m always watching the bird instead of the GCS but I will dig into my past missions and find out ASAP.

[Kywon.M_Vlogs]

Where do you think the fire was from?

[Chad_Frazer]

OK! I think I got it!

There is an excessive current draw on takeoff! 97amp pull where a momentary 60amp pull is normal with an average of a 50-55amp draw on accent.

This means that either a motor or an ESC was on its way out. For this theory I am betting that the motor was damaged/faulty before takeoff.

The damaged motor (binding motor) was on the right side of the aircraft and the ESC that caught fire was on the left side. This suggests that the left side motor was being forced to compensate for an under performing right side motor, overheated, shorted the current going to both motors, caught fire in flight and the rest is a glide all the way to the crash site.

Throttle commands reached 100% during its downward glide further indicating that there was a problem with propulsion. Typical “full throttle” is around 70% and never higher.

EDIT: I misread this graph. The 97amp power draw happened AFTER impact. Throttle at peak altitude was about 55% (pretty normal) and then, as Greg was saying, the throttle commanded by the autopilot fell to zero at impact all the while decreasing as the aircraft pitched down. Throttle command decreasing while airspeed is increasing is normal IF the airspeed is near or below stall speed. As stated further below airspeed was adequate at the end of transition but was around 50+mph at the point of impact.

I have no idea why except for maybe a partial blockage of the airspeed sensor??

[Kywon.M_Vlogs]

Do you have video of the whole situation throwing down?

[Chad_Frazer]

The proof:

Abnormal wear pattern on windings:

FOD (Foreign Object or Debris) found in motor. It is about 3mm long, has threads and is magnetic, maybe a screw tip or the tip of a small drill tap:

So that piece created friction, the friction caused and excessive current draw, the current draw overheated the ESC, the ESC shorted/caught fire thereby killing power to both motors and down she went.

I have no idea where that FOD came from.

[Chad_Frazer]

I know there were at least 3 cameras recording but I don’t have access to them.

[Kywon.M_Vlogs]

Oh. Big oof! Having a rig is like having a aircraft. aircraft retire. If they don’t they crash. Kinda like what happend here only exept it’s a drone. With aircraft they referbish them so nothing wrong. But idk if you did or not

[Chad_Frazer]

I’ve been messing with drones both military and civilian since 2009. I’ve seen this kind of thing before. It’s kinda hard to find FOD in an electric motor as symptoms don’t present until its too late (usually).

As for refurbishing stuff…I tend to not reuse electrical components to save a buck anymore…you never know when that “cheap” part is going to go out on you.

So for gen 2 of this bird I am upgrading the ESC’s and will add a motor spin freely to all checklists. That still won’t guarantee that it won’t happen again.

I can say this, had I been paying attention to the current draw and throttle commands that the GCS was showing during takeoff, that coupled with the abnormal liftoff I MIGHT of been able to save the bird.

[Kywon.M_Vlogs]

My gosh… You gotta treat models like aircraft these days… If i was going to crash, there was nothing i cold do to save my model, i would steer it into to some concrete or a place where nothing can catch fire, and put it down peacefully. That way, nobody gets hurt, and no fire.

[GregCovey]

Chad,

I still suspect a possible firmware issue. As stated above, the transition was done at 332 and the crash happened at 337.7. Below are graphs for Altitude, Current, and ESC control.

The Altitude shows the “Transition Done” at 332 and the crash at 337.7, almost 338. The logs continue which shows the flight controller was still working after the crash but bad things happened like full power to the ESCs.

The current dropped to zero right before the transition was done, yet the throttle stick was about mid. This seems like a software issue. There was no further current draw until AFTER the crash.

The ESC control signals look fine up to the “Transition Done” phase at 332. The flight controller turned off the motors at 334.8, long before the crash at 337.7. It’s interesting that both C5 and C6 motor controls from the flight controller are doing the same thing. This would seem more like software control than a dual motor or ESC failure.

[GregCovey]

As further discovery, here is an interesting graph of Altitude, Airspeed, and Current. You can see the original “Transition airspeed reached” of 11ms at 325, but the “Transition done” does not happen until the plane reaches 19ms and the current has already dropped to about 4amps at 332. I’m not sure if this has to do with your speed change command.

From the graph above, at 332, the C5 and C6 outputs are going down, but do not drop until almost 335. What bothers me is the current drop right at the transition done period. It would seem too much a coincidence to be hardware.

[Chad_Frazer]

I see what you are saying Greg…that is a big coincidence. To help some I uploaded a very typical flight BIN file to the folder you should already have access to. It’s called Typical Profile.BIN. I think that might help for comparisons.

I can say that the bird builds up a lot of speed during transition, I’ve seen 45mph before. So when it hits the next command line in the mission profile (do change speed) the current does drop a lot, 4amps would be right if it was flying into a headwind. But falling out of the sky is not normal

Maybe I should submit this to the code guys? How would I do that?