Looks like AP3.5 GUIDED mode crashed my plane (solved by AP 3.5.2)

First, Tridge and WickedShell, thanks for your time looking at this, I do appreciate it.
Sorry for my delayed reply, I forgot that this forum does not send Email to warn you you got an answer to your thread (by default). Kind of a misfeature…

Random replies:

  • I had TRIM_ARSPD_CM at 15 and ARSPD_FBW_MIN at 12 last time we talked Tridge, and I seem to remember that you recommended I lower TRIM_ARSPD_CM to the same value than ARSPD_FBW_MIN, so I did before this flight and crash.

  • You did mention I got as low as 7.5m/s, as per my reply in the bug, that’s not what I found in the logs, nor do I believe it’s possible. This plane stalls below 11m/s. But as agreed, the airplane did get too slow and did at least for a while need pitch down to maintain flying airspeed. The AP made sure it never stalled.

  • You’re correct about full throttle being only 80% per my request in the config, I had half forgotten about that, but at the same time on that motor 80% was mostly maximum thrust anyway, and I didn’t want the motor to be overstressed and burnt during normal flight. You could make the case that allowing takeoff power might be allowable for GUIDED with a minimum altitude deck, would be a good thing, but that’s half a detail.

  • Sorry, I forgot to give my channel mapping
    o CH01 : Right Aileron
    o CH02 : Elevator
    o CH03 : Throttle
    o CH04 : Rudder
    o CH05 : left aileron
    o CH06 : R flap
    o CH07 : L flap
    o CH08 : OSDCmg: up 12V+cams off, mid: 12V on, down: GUIDED. 2 mode flips: OSD rotate
    o CH09 : FPV Camera L/R (S2)
    o CH10 : free
    o CH11: RSSIQual from RX
    o CH12 : Ardupilot Mode Switch (6 position switch mod)
    I’m pretty sure that both flaps were full down due to manual passthrough

  • Having override set in manual, yes, it’s a reasonable idea. I kind of wanted GUIDED to save me in case I did something stupid, and lost control, but right now it’s more likely to crash my plane, than me, so manual removing the geofence is not a bad idea

  • Putting crow flaps actually causes my plane ot pitch up, I have a crow flaps to pitch down mix. However I never fly with flaps only (my controller is not setup for it), so I cannot swear to you that flaps cannot cause pitch down, but I’m pretty certain they won’t because every single full size plane I’ve flown requires trim down as you add flaps, in other words flaps cause nose up.
    However full flaps down on that plane will definitely add a fair amount of drag (that’s the point), so trying to fly back up with full flaps should be possible, but harder. Given that the airspeed went as fast as 26m/s with flaps down, I assume some of that energy could easily have been used to fly at least level, or back up.

I read your detailled analysis Tridge (thanks), and you also found that keeping pitch down was harder with the flaps, which agrees with flap causes pitch up. On that plane full throttle also causes pitch up, but not horribly so. When I fly in FBWA or CRUISE, the AP seems to handle that condition transparently and keeps things in check ok.
So, I do think the flaps caused loss of pitch control much more than throttle because I used full allowed throttle in FBWA/CRUISE many times without problems

Tridge, to see if throttle was more an issue than flaps, can you compare with
marc.merlins.org/tmp/6_GUIDED_st … _crash.BIN
from
github.com/diydrones/ardupilot/issues/3357
where I went from FBWA to GUIDED and a similar looking issue happened except no flaps should have been involved then.
This will help validate (on the same airframe) if your analysis and fixes would have helped with the issue there too

Actually another data point that could be useful for analysis:
github.com/diydrones/ardupilot/issues/3032
AP 3.4: RTL and CRUISE both give occasional oscillations and stalls of death

marc.merlins.org/tmp/MANU_to_GUIDED_4.BIN
youtube.com/watch?v=UnuNdcZ … be&t=18m5s

Looking at it again, I don’t think it’s a pitch up due to throttle because not much throttle was applied, and I don’t think RTL motor power up would have caused what was a perfectly flying plane above RTL target altitude to start flying so badly, but since we’re talking about control loop issues, this may still be worth looking into, to see if it’s related.

Airframe is a BFG 2600 which a T tail aircraft. Elevator has always had plenty of authority in all flight conditions as far as I can tell

Hi Tridge,
My apologies if this is totally unrelated, but I’m bringing it up in case it can help validate your recent fix.
On my last post about github.com/diydrones/ardupilot/issues/3032 you may want to scroll to
’marcmerlin commented on Oct 18, 2015
Actually I was just able to retrieve some screenshots from the OSD (recording is incomplete, but those are enough to help).’

Lower down in that bug:
Example 1: CRUISE failure:
youtu.be/y-TKmxJ5Tyg?t=10s
I’m not sure why CRUISE was commanding such a pitch up attitude causing repeated stalls, and with only 22% throttle. Usually it puts a lot of power when it goes up, and here virtually none (I see 1.93A draw, that means the motor is barely turning).
So why

  1. so much pitch up
  2. no motor power
  3. no detection of the stalls and attempt to regain
    marc.merlins.org/tmp/Cruise_Stall_4.BIN

Example 2: RTL failure
youtu.be/uD5Ou2tTKjM?t=1m41s
Pausing on a non snowy frame shows pitch up of 23 degrees, no idea why, and RTL missing the target heading by more than 90 degrees.
I can see how eventually the airspeed drops enough to do this, but there is no reason for RTL to fly so erratically with such a nose up altitude and not full power. And if it gets into that mode, shouldn’t it detect the stalls, and add power?
Switch to CRUISE at 2:19 fixes the problem
marc.merlins.org/tmp/RTL_Stall_3.BIN

There was the issue that ARSPD_FBW_MIN had been raised to 14 to see if it helped flight behaviour while TRIM_ARSPD_CM was still 12, but there still seems to be a pitch control issue here.

For anyone reading this thread, AP 3.5.2 has a fix that is supposed to prevent the crash that happened here.
Given spare time (always in short supply obviously), @tridge is going to have a look at the other logs with weird pitch issues to see if they were related and most likely addressed by the fix he committed.
Thanks again @tridge for the code fix and reviewing the logs.

@tridge , I just did some test flights with 3.5.2 today, with full flaps down, breached the low altitude fence, and let GUIDED take over a few times.
While it struggled to stop the descent the bit and regain altitude (and that’s expected, the flaps are big), it did the right thing and went back up.
So at this point, I have some confidence that your fix did indeed fix something :slight_smile:
The other logs I gave in this bug may still point to some separate issues since the last near crash happened without the flaps down if I recall correctly, but at least the GUIDED takes over with flaps down case seems to work ok now.

Also, I have crow fflaps with differential throws and going to 130%, so I’m not sure how possible or easy it is to set them up as flaps controlled by AP, but if I do, should I assume that GUIDED wiill do the right thing and retract the flaps when it needs to fly back up?

@tridge, you asked for logs showing GUIDED fly back up with the flaps down.
I just did this today,
http://marc.merlins.org/tmp/12_GUIDED_FLAPS_CLIMB.BIN

At 3:22, I disabled geofence, used MANUAL and put full flaps to drop below 100m,
https://youtu.be/NgUKE-YJfHg?t=3m45s
guided takes over and climbs back to 100m

So that’s good :slight_smile:
What’s not so good was that my fence return altitude was 150m, and it only went back to 100m
PARM, 651332702, FENCE_ACTION, 1
PARM, 651332718, FENCE_TOTAL, 6
PARM, 651332734, FENCE_CHANNEL, 8
PARM, 651332839, FENCE_MINALT, 100
PARM, 651332855, FENCE_MAXALT, 1000
PARM, 651332870, FENCE_RETALT, 150
PARM, 651332886, FENCE_AUTOENABLE, 0
PARM, 651332902, FENCE_RET_RALLY, 1

I did not have a rally point, so it went back to the takeoff point, which is good, but ignored the 150m and went back to 100m.
Any idea why?
Then, there are clear issues with airspeed, but I put that in another message: Why does airspeed not reset to 0 before takeoff and why did autocal make it worse?

Mmmh, actually in my next flight where I crashed RTL failed to recover from inverted flight and spun backwards into the ground check
https://youtu.be/wFwTneYeNY4?t=06:52s (offset 06:52)
CRUZ is hoping up and down, which looks a biti weird

@tridge you can load the 57s CRUIISE segment between GUIDED 1s and FBWA, it’s the part where the plane keeps nudging up and down.
I had a look with Mavproxy, but I’m not sure if I’m figuring out what’s wrong. Is it a problem with PID? I had flown that airframe a fair bit since autotune, and never experienced this up/down problem (actually I think it didn’t happen for the rest of the flight)
http://marc.merlins.org/tmp/12_GUIDED_FLAPS_CLIMB.BIN

Looking at these graphs, all I seem to see is AP commanding that pitch to go up and down



@tridge more reading seems to say that PTCH2SRV_P could cause porpoise.
Looks like autotune gave me 0.92 for it, not sure if it’s too high.
Still, weird that I never had porpoise behaviour until this flight.

PTCH2SRV_D 0.074393
PTCH2SRV_FF 0.000000
PTCH2SRV_I 0.062053
PTCH2SRV_IMAX 2000.000000
PTCH2SRV_P 0.929917
PTCH2SRV_RLL 1.000000
PTCH2SRV_RMAX_DN 90.000000
PTCH2SRV_RMAX_UP 90.000000
PTCH2SRV_TCONST 0.400000

I guess when I get my plane repaired after its last crash, I can try the steps from http://ardupilot.org/plane/docs/roll-pitch-controller-tuning.html if autotune isn’t doing quite the right thing for my plane anymore.

Is RTL supposed to auto-land? Or does the plane circle overhead? Thanks.

@ripfree: you are totally asking this in the wrong place. It circles overhead. Please use a new thread for questions, don’t hijack another one :slight_smile:

absolutely brilliant your analisis above, thanks a lot for contribute to this knowledge.

I have experienced many different behaviors thinking that something is wrong in multicopters and plane but almost always the solution turns out to be a summation of conditions and parameters set correctly . This discipline takes time and patience.

373 01-01-1970 05-30-00.bin (2.0 MB)
Hi, Crashed Skywalker Arduplane 3.40 when switched to RTL while ythe plane was in guided mode “Go to Here”

I can see that there is no Airplane firmware upgrade for APM beyond 3.40 and I can see that the bug was resolved in Arduplane 3.52. Can someone help for a patch for APM Arduplane 3.40 ?

@Rana As you may have seen in more than one announcement, the orginal APM isn’t supported anymore, so unfortunately you shouldn’t really expect firmware updates for your flight controller anymore.
The original APM has been obsolete for more than 2 years now, my recommendation if you care about your airframes is to stick to a pixhawk clone, or better from now on.

Hi,
I have many many APM’s and want to use them, so if either Tridge or someone else can share the patch to take care the issue of crash in guided then I could make use of these APM’s

I have many pixhawk too but want to use the APM also.

@Rana, I had a look at your log (“373 01-01-1970 05-30-00.bin”) and it doesn’t look anything like the issue that Marc previously saw.
The first thing I notice in your log is that no roll or pitch tuning has been done - it is just using the defaults. It looks like the gains are way too low. Because of the very low gains it takes a long time to build up enough elevator to start to pull up when it is descending. The main thing you need to do is run an autotune, or even just manually increase the P and I gains or both roll and pitch. It takes nearly six seconds to build up enough pitch integrator to start to pull up, and that is much too long. You probably need about 5x the integrator gain, and 3x the P gain at a rough guess.
I am happy to do a new bug fix only release for the APM2 if a really serious bug is found and I have a log that shows it is needed, but this is not an example of that sort of problem.
Cheers, Tridge

Trige, many many thanks for the reply bro !

If gains are suspected culprit then the same skywalker plane with which I have done more than 200 successful auto mission flights with APM2 only should have not been.

I mean I never initiated RTL in guided mode in any of my flights with APM prior to 3.4.

Prior to this crash, the this skywalker plane with same APM2 has done few successful flights in auto mode without any issue and that is too with default parameters. Below is link to some of very successful long missions with Skywalker with default parameters in APM2.



Below is the telemetry log file’s video clip of small portion of this flight in which crash happened.


Telemetry Log.tlog (523.9 KB)

I mean I have seen that Skywalker always does well with APM2 default parameters. But this does not mean that I should keep on using default parameter.

I will do as you suggested. I will change the parameters approximately to what you suggested or autotune it.

When the APM2 bug fix you are planning to release ?

Best Regards
Rana

That is one of the problems with the ArduPilot controllers - it does fly OK in many situations even with terrible tuning. That is nice for people who don’t want to bother tuning, but it means they get complacent and don’t tune. Then when the vehicle is in a bit different situation and has to recover attitude quickly it will not recover.
I flew a skywalker with an APM2 as my main test plane for a couple of years. My parameters were:

  • PTCH2SRV_D 0.07
  • PTCH2SRV_I 0.3
  • PTCH2SRV_IMAX 3000
  • PTCH2SRV_P 1.8
  • PTCH2SRV_RLL 1.1
  • RLL2SRV_D 0.1
  • RLL2SRV_I 0.3
  • RLL2SRV_IMAX 3000
  • RLL2SRV_P 2.2

note that they are a very long way from your parameters! The control surfaces on a skywalker are really quite small, so it needs a lot of gain to fly well.
Don’t just use my parameters btw, better to run an autotune. The right parameters depend on several factors that are dependent on the specific airframe (eg. which hole in the control horn you use).
Cheers, Tridge

I will only do a new APM2 release if a serious bug is reported that warrants the work to do that release. So far I haven’t seen any bug reports that are sufficiently serious to do a new release. I looked into your log as I thought perhaps that would be one, but it turn out it is just tuning.
Cheers, Tridge

Hi Tridge, once again many many thanks for you time and support.
Also thanks for sharing your parameters.

I would try to use your parameters and then check in in stabilized mode at ground it all control surfaces are behaving properly.

Only then would do a small test flight at around 200mt alt then try to see if the model is doing well in stabilized and auto.

If it does well then at around 200mt altitude, I will put the plane in guided mode preferably not much away from me and then will initiate RTL and see.

Thanks for confirming that no serious bug reported so far in AP3.40 for APM2

Once again thanks a lot for your wonderful support bro.

Best Regards
Rana