Rangefinder in GUIDED mode

Hi guys
I need precise control of near surface altitude in GUIDED mode and not performing a landing, GPS and barometer can’t hold its alt so I turned for rangefinder. After I correctly configured the rangefinder, it holds its alt perfectly when using GUIDED mode with no control inputs. I tried this by placing a box under my copter when its flying. However, when I try using simple_goto() function, it can’t hold its alt and I assume its using barometer data because when there’s no wind, it holds its alt, but when there’s strong wind, it just changes altitude.
I’m using a PixHawk 2.4.8 with arducopter 4.3.7 stable, I don’t want to use RTK.
Is there any way to configure rangefinder as primary altitude source in GUIDED mode? My copter operates at around 5m above ground so rangefinder is reliable.