VTOL Plane: QuadPlane with forward motor
Firmware: ArduPlane 4.0.9
FC: Pixhawk2 CubeOrange

I followed the wiki to set Q_RTL_MODE=2 to make my plane RTL as a normal fixed wing and loiter to Q_RTL_ALT, and then trans to VTOL to land. But Q_RTL_MODE=2 did not have any effect. The plane RTL as a normal fixed wing and loiter around HOME with RTL_RADIUS. It never doing a transformation to VTOL. I have to change flight mode using MissionPlanner or RC to land it.
I also tyied Q_OPTION=4 as wiki said and use VTOL_LAND mission command, but I got the same result.
I compiled the 4.0.9 source code to use the quadplane simulator. Sadly, the same test got same result.

I just want to make my plane land as this photo.

Please help me. Thanks.

VTOL approach is 4.1 and later

Thank you but the wiki said:

That is for VTOL approach as part of a mission, not VTOL approach from RTL.

Thanks you and I undersdand this is a part of a mission.
I tried use Q_OPTION=4, and upload a mission with VTOL_LAND command. But it’s not working. The plane will change to VTOL after the waypoint before VTOL_LAND. And move to landing positon as a copter.

bit 4 is not a value 4, it would be a value of 16, use the tick boxes in mission planner.

Thank you @iampete, I think I just found that.
I tried this option on simulator several times, and I noticed that the transition began at the loiter circle and move as a copter, then make its head to wind (or home), not as the wiki picture shown us.

It can do that if your loiter circle is too small for the given transition deceleration.

Ok @iampete
Thank you very much. I will do more test and make it happen.