Hi!
I have no experience with ardurover, but I faced a similar issue with arducopter. I don’t know if yours is the same issue, but I had to set the global position explicitly. Ardupilot requires a global position, even if it will only use relative movment commands.
You can explicitly set the global position in different ways. To test if this is the issue, you can right click in the mavproxy map and click “Set origin”. If this is the issue and you want to automate it, you can use the mavros topic /mavros/global_position/set_gp_origin.
It is not sufficient to set it to (0, 0, 0), it has to be a plausible point. I read somewhere that the controller uses the position to correct the magnetometer for known disturbances.
The post I asked when I had the same issue: Guided mode with optical flow, without GPS, in simulation