AP 3.7 Near Crash: RTL takes too long to recover a positive attitude and regain altitude

@tridge, I wanted to thank you for improving AP so that RTL could be more reliable as a “oh crap” switch to recover from unexpected situations/pilot loss of control. Unfortunately it almost failed me a couple of days ago by losing a lot of altitude before finally recovering.
Video to follow along: https://youtu.be/P5TLpbGoiiE

I used auto takeoff and that worked ok (45sec)
When I switched to FBWA I had an issue with my controller commanding too much pitch down and while I was trying to figure out what was wrong, I realized I was too low and losing altitude fast, so I switched to MANU, managed to over correct, stall, and then switch to RTL.
0:52:, altitude is 31M airspeed is 10kph (too low).
0:54: Airplane pitches down to -24 degrees likely due to the stall, airspeed is back to 55kph, way above flying speed, altitude is 34M
Now is where I’m not very happy with RTL.
0:55: plane is still pitched down, 10 degrees, still losing altitude and on top of that it decides to turn back down to the RTL point when recovering altitude should be #1 priority (RTL point is 100m, altitude is 32m)
0:57: plane still pitched down 10 degrees, airspeed is now 77kph going towards a crash, altitude is now 14m
0:58 : attitude is finally brought back to 0 towards nose up, altitude is a mere 12m, it’s only by luck that the plane didn’t crash
After that RTL does the right thing.

My problem is that RTL lost over 20m of altitude when it could have recovered a positive rate of climb almost instantly from offset 0:54. Sure, only 4 seconds went by, but in those 4 seconds, 22m of altitude were lost and the plane could have crashed into a tree, or the ground if it had started just a bit lower

Plane is a bixler 2, with flaps (not controlled by AP).
BIN from the flight: http://marc.merlins.org/tmp/bix_near_takeoffcrash.bin

I ended up opening a bug for this: https://github.com/ArduPilot/ardupilot/issues/6212