Pixhawk 2 RCOUT C3 went to zero and flat-lined causing my quad to crash. Anybody else see anything like this? Newbie here. Log file too large to be uploaded, but can be downloaded from my Google drive here: https://drive.google.com/open?id=0B0OMLBwWNbKjal9NNjJPU1l5eW81
Trying to confirm this is what actually happened through testing - hardware? Software? Any inputs would be appreciated.

unable to download log.

link to log on Google drive https://drive.google.com/open?id=0B0OMLBwWNbKjal9NNjJPU1l5eW8

I just tried the link to the log file and it seems to be working. Again, I’m new to posting on this forum.
$5,000 quad carrying a $12,000 agriculture sensor and it simply dropped out of the sky about 15mins while running an auto mission. 5th mission of the day, flew dozens before this one on tis aircraft.
The log shows loss of receiver, but that happens when quad is long way out. Parameters are set to keep flying mission and regains Tx signal after a few seconds. Slight motor imbalance, and compass error, but it threw the EKF/compass error as it was tumbling to earth.
Only solid failure I see if flatline of C3 (motor #3) all three other motors were going crazy trying to correct.
Really need to nail down the point of failure before launching another aircraft again.
My thanks in advance for any replies or feedback.

Sorry to hear about your crash. Was any of the quad or the sensor recoverable?

It looks like a mechanical failure of Motor #4 - you can see it go to command 100% thrust shortly before #3 flatlines - the #3 flatline is a consequence of trying to balance for the lack of thrust in #4.

It looks like it recovers a couple of times, but that’s probably very brief periods where it doesn’t need thrust from that motor as it was tumbling. The continued 100% command for #4 and 0% for #3 points to a proper mechanical/electrical problem for that motor rather than just an ESC desync.

ps - if you’re running really expensive sensors like that, please do consider running a Hex or better yet an Octo or X8, that have a much better chance of not crashing from this kind of issue.

1 Like

Thanks FNOOP. Very interesting analysis. Motor#4 got destroyed in the crash as the aircraft basically landed with that motor down. This could support your theory that #4 wasn’t generating any thrust. Also, the prop on #4 is not damaged, which made me think it wasn’t turning when it impacted with the ground. I’ll never know if that motor took a crap, because it got toasted on impact, but I can check the ESC performance.
Thanks also for asking about the damage. The sensor, thankfully wasn’t damaged. The aircraft is our own design and it’s a tank. Motor#4 and the motor mount took the brunt of the crash and got destroyed, but the rest of the aircraft is pretty well intact.
I understand about using a hex or even better an octo for redundancy, but the efficiency is much lower and hence, much less acreage/flight…Everything’s a trade-off…
Thanks again!

Very glad to hear the equipment mostly survived :slight_smile:
Understand your wish to run a quad for redundancy (and simplicity) - I do too although I only run very small quads and it makes me nervous when I send them out on missions past my garden! How about a parachute system?

Also might be worth looking at your motors/props and FC mount, you’ve got odd periods of sporadic Z vibes and consistent clipping on one of the accelerometers. I don’t think this had anything to do with your crash, but it can certainly cause undesired behaviour and crashes. (The blue line in the background is the clipping increasing throughout the flight):

One of the things that can cause Z-vibes is play in the motor bells - as the bell turns it moves vertically along the motor shaft causing Z-vibes. Can also be caused by too hard or too soft mounting of the FC.

We’ve been wrestling with vibration. Had the Pixhawk 2 hard mounted as recommended, then went to the thin foam pads supplied. That needs som more looking into.
Regarding motor #3 going to zero, wouldn’t you expect some more activity from that channel as the aircraft tumbled? It went to zero and stayed there - flatline as I called it. Seems odd.

Your vibes look fine to me. Less than 30 in the z is where you want to be.

I think motor three or the ESC failed. I am postulating, motor 4 went to 100% to compensate for the loss of thrust from motor 3 failing.

It doesn’t make sense aerodynamically that if 4 fails, 3 would be commanded to zero output.

On a related note :
You might be interested in looking at these guys http://www.quaternium.com

Hybrid petrol/battery quad with 3-4 hour endurance and on a forum post
on DIY drones the founder claims a power density of 5 times of LIPO batteries.
In addition they claim mission turnaround is refuel and relaunch with a maintenance
cycle every 50 hours. Startup spanish company and I believe they use a Pixhawk 2.1 controller.
Can’t vouch for them, but they look interesting.

Wow, another interesting theory…similar to FNOOP’s thoughts, if motor3 failed, wouldn’t motor #4 go to zero and not max? Also the log shows the Pixhawk output and not the actual ESC or motor thrust performance. For some reason, even if motor/ESC #3 failed, the Pixhawh output went to zero - flatline.
More questions…
Thank you for your inputs.

I’ll qualify this by saying I’m certainly no authority or expert on the matter!

But from what I understand, a motor/rcout trace going to 100% means that motor has failed. There’s no feedback from the motor so all the flight controller knows is that corner suddenly starts to drop (due to lack of thrust) so it tries to compensate by increasing the output on that motor. As the corner continues to drop due to lack of thrust so the FC tries to compensate even more and eventually goes to 100%, which is what you see on #4. #3, which is the opposite corner, therefore rises because the opposite side of the axis has dropped, so the FC tries to compensate by turning off thrust/dropping rcout on that motor. Key is that you can see #4 rise first, and #3 drops afterwards - clearly a reaction to #4.

WRT the vibrations, clipping that increases throughout the flight is generally not a good thing. I’ve had several crashes or uncontrollable throttle caused by high Z vibes and clipping. The firmware is much more resilient these days and in addition you’re only clipping on a single accel which I assume the EKF will reject, but still worth looking at. The firmware docs say:

Graph the Clip0, Clip1 and Clip2 values which increase each time one of the accelerometers reaches it’s maximum limit (16G). Ideally these should be zero for the whole flight but low numbers (<100) are likely ok especially if they occur during hard landings. A regularly increasing number through the log indicates a serious vibration problem that should be fixed.

It would be great if one of the devs/experts could chip in with their opinion :slight_smile:

I noticed that way before any motors start to mess about the Z axis compasses 2 and 3 go a bit teenager on the plot.So something else was goiing on beforehand or a failing motor/ESC was interfering with the magnetics.I’m not good eough to decide.

another good observation. MAG1 is my primary, but we’ve never been able to figure out the interaction between compass 1 and compass 2. Even with the internal compass 2 disabled, the PH still uses it for heading info.
Could definitely be an indication of a motor or ESC going south.

Can I be so bold as to suggest you rename this post and mark as resolved? I think it’s pretty clear that a motor/esc failure caused the crash, not the autopilot.

Good news that the camera survived, and nice log analysis from @fnoop

Hi JP, I didn’t realize the topic name presupposed an autopilot failure, but I guess it does. I’d be happy to rename it, but I’d like to leave it open for any other inputs. FNOOP’s analysis is very good and most likely identifies the problem, But the additional input from JAGGER’s showing the compasses were seeing some extraordinary interference is also very helpful. This is why I’d prefer to leave it open - for any additional inputs that might help avoid future crashes of very expensive gear.
Not sure how to rename topic or what the advantage is of marking it solved - new to forum posting…

The interesting point being that those two compasses are inside the Pixhawk 2.1 and not out on a stalk so something was ramping up the EMI in that area.

I don’t know why people are so defensive of the Pixhawk 2.1.The one I had was crap.

Right, so your analysis may point to an ESC going south. We have surely had our problems with the Pixhawk 2. Even to a point where went back to the Pixhawk (1) for some of our aircraft. Are you using the Pixhawk 2 or another FC?

Mine has gone back and I’ve moved back to CUAV Pixhack V3s.I’ve got 4 Pixhacks and rate them.The Pixhawk 2.1 was a first edition and nothing but hassle.Scary on occasions.Couldn’t get any support about it’s misbehaviour so gave up on it.The Pixhacks just work and the V3s have the newer sensors fitted (same spec as the 2.1) but don’t claim to have central heating.

Something I’ll add as this may be an ESC or motor thing.I no longer use bullet connectors if I can help it.All solid solder connections.I’ve had three bullets fail so far.Two on the ground and one not.Not had a problem with the motors or ESCs just the connections.This doesn’t count if you’ve got an overlong screw in a motor.