RTL rocketed copter away from launch to powerlines, why?

ArduCopter Copter 3.1
1

Hi all,

Would you please help me to dig reason of these series of problems which caused hard crash and help to avoid this in the future.

It should have been normal heavy camera copter gimbal pitch axis tuning test flight, gimbal roll axis was locked. Flight ended to unpowered power lines and crashed. I got footage from copter and helmet camera. Log file is attached. Firmware is V3.1.2 (ddd4d881).

What happened: copter was on ALT HOLD when I noticed that left stick was mechanically stuck so that YAW worked only to right side. Throttle was fine. Radio has done this earlier but I assumed reason to be a stone I found from stick bottom. Perhaps it wasn’t. This flight was certainly last for this 9XR radio.

After realizing stick problem I was trying to switch STABILIZE and land but logs tells opposite, APM was switched to RTL. Most likely this was human error. Copter was just above the launch.

When RTL was activated, copter took of high speed away from launch, spinned one full round to left (stick was stuck to center-right) and then boosted to power lines.

This leaves one very good question:
Why and where copter boosted when RTL was activated.

I have done earlier vibration test with unbalanced propels and it has been very smooth. And after that propels are balanced. Also compassmot is done and it gave result 13%. Current is high 100A-140A on normal flight but compass and GPS are about 100mm from power cables and more from ESC:s. ESC:s are calibrated and compass declinations entered manually. Every setting in APM is like it has been on earlier flights. AUTO flight settings I haven’t touched.

One thing is what might be related to problem but I can’t be sure. Compass wasn’t in its place after crash, like pretty much any component on top of the copter because it dropped upside down. Why I’m wondering this is that compass was only glued in its place and were waiting plastic screws to arrive. I have checked its firmly in place prior to every flight but this is in theory possibility that compass is teared off. One full round to left was a thing why I’m wondering this because I couldn’t have done it. But I can’t find any proof for this from logs. I wish it was a problem and therefore is easy to solve and avoid in the future.

Video: dropbox.com/s/tn4weeot43grt … launch.mp4

In the end, no one get hurt and crash offered good opportunity to rebuild few things better which I have found during building :slight_smile:. Price of the crash after full immediate disassembly was: full reassembly, all carbon booms, test bench result 7 motors, 6 propels, one main plate, one battery and something I haven’t found yet.

2

You did not log MAG, but generally speaking, you should test loiter, then test some more - with speed - before expecting RTL to work.

RTL flyaways are “always” caused by bad GPS, or bad magnetormeter.
your HDOP seems to go to 99.99 (no fix) before crash.
test, test , test.

3

[quote=“Andre-K”]You did not log MAG, but generally speaking, you should test loiter, then test some more - with speed - before expecting RTL to work.

RTL flyaways are “always” caused by bad GPS, or bad magnetormeter.
your HDOP seems to go to 99.99 (no fix) before crash.
test, test , test.[/quote]

Thanks for your input Andre!
You are right. I should have tried Loiter first but RTL wasn’t test plan today and neither Loiter.
RTL was accidental and I didn’t plan to do it before testing flight modes in order how documentation tells to test. That was planned to be future testing and RTL was there just because upcoming law in Finland seems to require so. Just a backup even not tested yet.

Back in to topic; if magnetometer and GPS fails on RTL, what exactly APM is supposed to do? There are still information available if copter has acceleration to some direction and without deeper knowledge of APM code I guess it should try to hover still, not accelerate at full speed to random direction. I believe there is something not yet discovered in this crash.

RTL “Float away” or force throttle reduction would be understandable.

4

Well, in your configuration, FS_GPS_ENABLE says that it will switch to LAND 5 sec after GPS fix loss. (you can also see it actually did enter LAND in log)

In your case it did not flew to a badly set home , or based on wrong GPS position, because the actual distance to target increased as it flew. /(see NTUN/WPDst)

  • This indicate bad compass calibration, or bad compass orientation… or anything related :slight_smile:
5

[quote=“Andre-K”]Well, in your configuration, FS_GPS_ENABLE says that it will switch to LAND 5 sec after GPS fix loss. (you can also see it actually did enter LAND in log)

In your case it did not flew to a badly set home , or based on wrong GPS position, because the actual distance to target increased as it flew. /(see NTUN/WPDst)

  • This indicate bad compass calibration, or bad compass orientation… or anything related :slight_smile:[/quote]

Thanks again for your help!
So, without GPS and Compass information copter could fly where ever and speed is SET in some parameter, which one?
Is there parameter which makes copter trying to hover and waiting that 5 seconds rather than speeding randomly?

This double problem seems to produce uncontrolled high risk behavior. GPS value is currently monitored, should Compass be as well? On EU we’ll have strickt regulations soon for RPAS and requirement for risk assesment wouldn’t surprise. Lowering any risk created by autopilot in any abnormal situation might be smart to have high priority on development now. If Finnish (partially based on some earlier regulations somewhere) regulations this year goes through as proposed, it means that any single failure can’t cause accident. We could think what this means to our hobby.

By the way, has anyone done risk assesment for autopilot? I mean a chart where all sensors and critical components are listed and if any of those are in malfunction, result is written to particular line. At least for all known single failure situations.

6

You are set FLTMODE6= RTL, so check the PWM on CH5 of your 6 switches, if some value are too close to the border. Maybe it’s reason you are go to RTL mode.

What happened in RTL is obvious, GPS give the right position but compass give the wrong direction, your copter goes the wrong way.

After crash, your GPS broken and the copter go to LAND mode, I think your copter is already on land at that moment.

You should not mount the GPS without a good compass. If your GPS is not there at that moment, your copter will land when RTL is active.

7

It does not fly randomly at set speed. What happened, was that it build up speed in (wrong) direction, and lost GPS while at speed, there’s no “break” feature without GPS. So it just kept travelling at what would finally be the speed of wind, unless getting GPS back.

“No single fault can cause accident”, is a nice goal, not always achivable, even commercial airlines have some systems, that can cause a disaster in the right circumstances, like MD80’s jackscrew for horizontal stabilizer. It’s all about minimizing the risk thru maintenance and proper routines.

-assuming your CAA rules are not very different from Norwegian ones, you should minimize the risk, and a risk assesment does not require 0 risk, but acceptable risk. So you may say main battery connector fault would be a disaster, but the chance of it to fail is small, because you have soledred it at correct temperatures therefore risk is acceptable.
CAA does not ask you to do risk assessment for APM/Pixhawk, but the whole UAS.
And yes - I do use Ardu* professionally, with permissions from CAA and Norwegian National Security Authority

8

[quote=“Andre-K”]It does not fly randomly at set speed. What happened, was that it build up speed in (wrong) direction, and lost GPS while at speed, there’s no “break” feature without GPS. So it just kept travelling at what would finally be the speed of wind, unless getting GPS back.

“No single fault can cause accident”, is a nice goal, not always achivable, even commercial airlines have …[/quote]

Yes those sounds overshoot regulations. There are two copter operators on panel, so they should have some influence for reality. Duplicating every possible failing part is one option (at least sensors and connectors), but is not possible currently.

If I have understood correctly for avoiding same accident I need to: switch stabilize immediately when something abnormal is happening and from settings side divide that traveling speed at least for 10.

Any other ideas for avoiding very fast uncontrolled speedings like that are very welcome.

9

Thanks for your advices!
I checked radio PWM values and they were 988, 1500 and above 2000. So this leaves only two options, program failure or my own mistake which is most likely.

This is assumption what happened based on what I’ve read so far and found from logs:

  1. Left stick didn’t work properly and made me confused, copter was at alt hold
  2. Confusion caused irrational action to switch RTL instead of Stabilize
  3. Magnetometer perhaps was teared off and caused copter to speed to wrong direction, away from spot where it was supposed to land. It was already on top of landing spot
  4. On the way to who knows where copter lost GPS and spinned one round without knowing its heading
  5. Copter flew to power lines

So quite a series of problems. I have slight flashback that I switched to stabilize before crash on time when copter spinned around. But another memory flashback is that I wondered why copter don’t fall even throttle is zero. I don’t know which was first. These needs to be checked.

Still I don’t get it why copter speed to some direction if it already was on top of the landing spot and why so fast. It didn’t need to go anywhere, just land even spinning

If this is not logical conclusion, is wrong or do not contain all details, please guys let me know.

10

It’s on the top of the landing spot, but not precisely, in 2 meters . Still RTL will work first.

Even it start land, but your GPS is working, it will use the GPS data to keep in the landing spot, with the wrong compass, it still will go to wrong direction. Fast. Because more distance to home, more speed will have. Your GPS give the right information about the distance.

It should work if your disconnect the GPS. If GPS is not there, it will go to LAND mode without horizontal position keeping. It will drift in wind, but it will land maybe just few meters away if the wind is not strong.

11

Is there exact time or some timer saved in to logs?

I’m comparing events with video/logs at frame by frame and timing would help a lot for finding out what happened and in which order.

What I have found so far is that GPS were lost long time after RTL command. When exactly, it is unknown.