I’ve seen this behavior since the 2.x code and on three different planes. It can be triggered easily by setting FBW-min slightly above the plane’s stall speed, or by setting the max-climb value too high.
Once APM:Plane flies into a stall under any auto mode, it will go full aileron deflection and spiral down until manual mode is entered to recover.
I’ve seen this happen in flight twice due to an air-speed sensor failure. The first one was on an APM flying through fog which resulted in the pitot getting clogged with moisture resulting in falsely high airspeed readings. That plane went into the ground as I wasn’t able to recover it in time. Luckily the damage was minor. The second time was with a pixhawk where the i2C airspeed sensor actually failed in flight. I was fortunate enough to be able to recover it in flight and fly all the way home on manual with the airspeed reading about 30mph higher than it actually was.
I really don’t understand why the code drives the ailerons so hard in this case. I thought it was that the APM was too slow to respond to the rapid attitude changes, but since I have now seen the exact same behavior with a Pixhawk, I know that’s not the cause.
I’m surprised that I haven’t seen this issue reported by others since it’s so bad and very easy to hit.
Post logs of that happening. - be sure to log RCIN and RCOUT too.
I have never seen airspeed fail on APM or Pixhawk (analog/digital) , of course, here north pitot tube heating is needed during winter - but other than that - I’ve not seen a sensor-failure. If that was common, we’d use redundant sensors (which is most likely to happend on Pixhawk - but airspeed is not there yet.)
The incident occurs in loiter mode between lines 36000 and 39000 of this log when it’s in a steep dive spiraling to the ground. I was able to easily recover it after switching back to manual.
You can see the aileron output (RCOUT CH1) go hard to the stops and stay there for the duration.
ARSPD_FBW_MIN is clearly too low if you stall at <18m/s (it’s not a reason to the odd behavior of roll)
Anyway - your minimum airspeed is never reached, so there is no real reason to see actual stall handling.
When I plot RCOUT.Ch1 va ATT.Pitch and BARO.Alt , I see that Ch1 output is hard to read, with little consistent influence on pitch.
Is the plane well tuned, when it comes to PID’s ? (it does not look like it is)
This is a little bit too hard to figure out, for once, is high PWM = nose down ? it seems so during the dive, but not around line 14k-15k, where PWM increase increases pitch.
Actual stall speed for this plane is around 9-10 meters per second. The airspeed sensor was failing during this flight. (I still have that bad sensor but have swapped it out).
I have around 5000 miles and hundreds of hours flying on those PID values and that airframe with very good results. It’s pretty crisp, much more so than what auto-tune would provide.
The actual stall/spiral dive is only about 2 seconds right before I switch it back to manual mode at around line 39000. I noticed it happening and since I’d seen it before knew exactly what to do to recover.
On another similar plane, that spiral dive happened for 10s of seconds before it hit the ground. It was unfortunately an APM and it took me about an hour to find so I have no log data for that one.
I do have both ground station and on-board video of that one though.
I don’t see how it was failing, the airspeed is proportional to GPS speed regardless of heading, so there was no strong wind. It’s a bit low all the time, but that’s just a matter of tuning.
It increases just fine during the dive.
Still, unless a proper dev chips in here, I say what you see is not stall handling, but something else going wrong.
I agree that “Don’t stall” is good advice to avoid the problem, but that’s not the point.
The point I’m making is that if you do stall in auto mode, giving full aileron deflection seems like a very bad thing.
This issue is very easy to simulate by just turning FBW_MIN down to slightly above stall speed. APM will fly into a stall then give full aileron until it hits the ground.
I’m guessing that some I term builds up and overflows causing the problem.
In my log, you can clearly see it’s stalling by looking at the elevator output before the incident. It goes up and up until maxed, then in response, the elevator is driven full minimun while ailerons go full right.
That failed airspeed sensor is really screwy, just putting your hand near it will cause the reading to go way up. The new one works fine so I don’t hit the issue for now, but the real point is that when a stall happens, the handling of it is very bad.
Do other people fly APM:Plane with I-terms set to zero?
PS: I never wrote (pointless) “do not stall” as a suggestion.
I just pointed out that the AP most likely did not try to recover from stall, as I don’t see airspeed be at a level that would make it do stall-recoverey/prevention.
I do not have any “*_I=0” in my configs. - up till 2.78b I had PTCH2SRV_I,0 , but that’s it
I looked at the GPS speed and from it you can see that the airspeed sensor had an offset of around 10 M/S for the entire flight. It was showing 16 M/S when the plane was at rest on the ground at the beginning and end of the flight.
For the rest of the flight, the airspeed reading tracks the GPS speed pretty well but is 8-10 M/S high which is why the plane flew into a stall, since my cruise speed is only set for around 18 M/S.
Looking at the current reading is also pretty obvious. As soon as I switched it to loiter, the throttle was cut trying to get down to 18m/s IAS.
You’re right that there was very little wind that day.
The main point here is to look at what aileron, elevator and rudder do once the stall condition is reached. I don’t understand why aileron/rudder stay over even though the plane did at least one complete roll after stalling.
As I mentioned before, I had another plane do this at altitude and the state will be maintained all the way to the ground and dozens of spins.
I have also experienced unrecoverable stalls in auto modes, and as you know a quick switch to Manual is your only hope to recover. The code does not know how to recognize a stall, and simply does what a poorly trained pilot would instinctively do. I believe coding the autopilot to correctly identify a stall is much harder than it would seem. Even many high end proprietary autopilot systems suffer from this vulnerability. Kestrel uses a panic button that the operator must push if a stall/spin occurs. This allows the plane to recover while remaining in auto flight by changing the logic to prevent up elevator and prioritizes rudder over aileron for directional control. I think this could be implemented in APM much easier than stall detection which would be prone to false positives. This would be a good request for GitHub or DronesDiscuss.
[quote=“Noircogi”] …
… I’ve seen this happen in flight twice due to an air-speed sensor failure. The first one was on an APM flying through fog which resulted in the pitot getting clogged with moisture resulting in falsely high airspeed readings. That plane went into the ground as I wasn’t able to recover it in time. . .[/quote]
@Noircogi
Could that have been freezing fog that caused the pitot clogging? Do you remember the approximate ground temp that day? If close to zero it might be sub zero at 100m height. If it then clogged from freezing fog then the ice would turn into water after the crash, thereby “altering the evidence”.
The thought of just some moisture ruining the airspeed sensing is highly disturbing. I was hoping to come up with a rain tolerant plane (with airspeed sensing). Now I have to reconsider…
@Andre K
Have you implemented pitot heating? If so, your solution would be of great interest, probably not just to me.
Having reliable airspeed indication (and failure management) is becoming increasingly important as the flight controller becomes increasingly relying on it (TECS) while the same flight controllers are being used more frequently in professional applications. Having our UAS fall out of the sky at arbitrary locations due sensor / probe failure is one of our nightmares.
This plane had something like 150 hours of flawless flying on it before that incident btw. It’s got another 20 or so since that event. It now has a pixhawk instead of the APM, but I’ve found that the pixhawk has the same issue following a stall.
Noircogi : this is scary,
-please post your .log so I can see if the behaviour is the same.
-What was the actual stallspeed of that plane ?
-What did you do during the fall, did you switch to manual ?
I pretty much depend on AS for complex missions in northern regions, with strong winds - so if this is a bug, it’ll be important one to squash.
Unfortunately, that was an APM and it took me over an hour to find so the log had rolled over.
Flight mode selection is in the upper left of the OSD. I did switch to manual as well as RTL and circle, but I may not have given it enough time in manual. I only had a few seconds from realizing it was spiraling down until I list video due to the mountain range between me and the plane.
That plane has a pretty low stall speed of around 20 MPH. It was flying into about an 8-12 MPH headwind at the time. The plane is an X-UAV talon modified to a standard tail config with the wing extensions.
ok, I see not even a change in roll-rate s you switch to manual.
I cannot imagine it to be anything else than a mechanical failure, or a servo power loss, servo failure.
AP is working fine, based on the telemetry - and there is no option for AP to interfere with your input when in manual mode. If control surfaces were responding, the roll rate would change(/stop) when you switched to manual, unless you held hard aileron while doing so - not very likely. Also it’s very likely did some stickbanging the last moments, trying elevator - no sign of that moving eighter.
There was no mechanical failure. The plane was fine after being reset. The only damage was to the pan servo and flight motor. The servos etc. are doing great with many more hours of flight since that incident.
Watch the current readings and airspeed readings on the OSD and things become pretty clear.
Airspeed reading goes artificially high. Motor power is cut. Plane stalls and begins spiral dive. Power comes back up and it spirals all the way down. The pitot tube appears to clear out partway through the dive, but the death spiral characteristic of the APM code means it’s too late for that to matter. This is definitely a software issue in the APM plane code. It’s easy to induce other ways as I said earlier.
let me sum up:
so you say you can trigger it at will. (how)
you say switching to manual does not work ?
I actually tried a stall few months ago as part of usual testing after adding another payload option. Stalled manually, then switched to cruise. - needless to say, nothing unusual happened.
Switching to manual does work to recover normally.
I think in the video above I just didn’t give it enough time in manual to recover from the extreme state it was in. I knew I was going to lose video any second.
You need to have the onset of the stall occur in an auto mode to get into this state. I know for certain auto and RTL will do it. I believe that it should also happen in cruise but I haven’t seen it myself.
Simply setting FBW_MIN/cruise close to stall speed, then hitting RTL with the plane heading away from launch will probably induce the situation in most cases. Once the spiral dive happens, you can switch to manual and pull it right back out.
[quote=“Noircogi”]Switching to manual does work to recover normally.
I think in the video above I just didn’t give it enough time in manual to recover from the extreme state it was in. I knew I was going to lose video any second.
[/quote]
maintaining such roll rate takes force, if the ailerons went to mid-position even for half a sec, you would see a clear drop in roll_rate - so I do not completely agre there.
unfortunatly, I have no cheap airframes to test this on, and the “cheapest” plane is scheduled for a paying mission on friday.
Do you have an opinon if actual stall is necessary ?
if, flying auto, actual stall speed is 12m/s , will configuring arspd_min=20 , while flying 15m/s trigger it ?
or - does the problem require the actual stall, (where AP does not achieve des.roll) because it takes time to pick up AS ?
If this is a real issue, I may be testing it as a safety precaution soon.
