Plane climbs and stalls when switching to RTL or loiter

I am flying a Skyhunter fixed wing with a Pix32 V5. Whenever I switch from manual to either RTL or loiter it goes into a vertical climb, stalls and spins and I have to revert to manual to catch it. The auto analyzer on the log in Mission Planner gives the following error: Test: Compass = FAIL - Large change in mag_field (93.09%)
Could this be the problem?