I am having a similar problem, I hope someone has any advice. As seen in the attached video when I turn the controller 360 the readings in compass test follows for 180 degrees, then it jumps back 180 and starts over.
[youtube]https://www.youtube.com/watch?v=15XbKxK5oN0[/youtube]
@okolloen,
If the HMC5883L is mounted on the top of the GPS/Compass module, then the compass rotation would be None in relation to an APM2.5/2.6 flight controller assuming that the X and Y coordinates are oriented correctly in relation to the front of the vehicle.
Regards,
TCIII GM