Altitude varies with Here+ RTK

I have recently set up a Pixhawk 2.1 with the Here+ RTK. I did the survey to 0.5m and in loiter mode if I move the base the drone also moves the same direction so I know the corrections are happening. The problem I am having is repeatability with altitude. Sometimes the drone will sail way above expected, other times it will practically land and there never seems to be a rhyme or reason behind why. I have the EKF2_ALT_SOURCE set to 2 which I would have thought would help but it hasn’t at this point.

On an older system when I was using the Piksi on a first gen Pixhawk I know there was a option to use the GPS for altitude and that seemed to help tremendously but it seems like that option has went by the wayside once PH2.1 came along.

Thanks for any help.