I am not sure I can help much, but I will point out a few important things that come to mind.
I assume you are using skid steering (SERVO1_FUNCTION=73, SERVO3_FUNCTION=74) as described here http://ardupilot.org/rover/docs/rover-motor-and-servo-configuration.html. If you are using skid steering, then you can enable pivot turns as given here: http://ardupilot.org/rover/docs/rover-tuning-pivot-turns.html. That will give your rover the ability to turn on a dime at the corners. Of course, there is a lot of tuning required to make things work well. Which brings me to what I feel may be your main problem and that is just your speed and tuning needs serious tuning work. This can be very frustrating but is the most important thing to having good control.
Also, the Navigation loop should be tuned once steering and throttle are tuned. It can be tricky in my experience.
Also, having a working and calibrated compass is important.
If you upload your flash logs from your SD card, someone here may be able to help you more. I will take a look but am not as versed in how interpreting logs as others.