So now there is a great new MagFit tool in WebTools on the firmware server (see @iampete’s MagFit WebTool YouTube Video), I thought I would compare it with the other MagFit tools and it was a bit confusing.
I have a brand new plane, never flown, with a brand new flight controller (MFE PixPilot V3) never flown and I needed to do a large vehicle compass calibration because it’s quite a big plane. (My personal biggest so far 2.1m wingspan).
So I did a flight where I flew around a bit, maybe not the best flight for this, but I was kind of doing an autotune so it gives some variability for the MagFit. In any case, I chose this one log and ran with through all these tools:
So the results are a bit disconcerting. None of them are the same, most somewhat similar, but it leaves me wondering. Which is right? Which one should I use to set the parameters for my plane?
Now that I’ve re-watched Pete’s YouTube video and thought about it some more, I’m thinking I’m asking the wrong question. With no changes to the offsets, I’m thinking the results should match a bit better with the other tools, but the real power of the WebTool Magfit is the dynamic nature of the tuning.
I tuned the same log by choosing different values for offsets.
Its a interesting comparison, MAVExplorer is the real standard here, plot.ardupilot and the web tool are both quite new and no one (Including me) knew about the mission planner one.
Firstly I think your compass 0 Z offset for WebTools is a typo, I get -250.919. I’m not really sure what you did for the “tuned” values so I cannot replicate.
Ultimately there different because they use different methods, although I can only really speak to MAVExplorer and the WebTool.
WebTool minimizes the mean of the X Y and Z errors, MAVExplorer minimizes the length of the error sqrt(X^2+Y^2+Z^2). This is what allows the Webtool to be fast. In your log the tool does 6 fits for each of the three compass in less time than MAVExplorer takes to do one fit on one compass. The speed allows us to do more fits which allows the user to better compare them all. No-one has time to do such a comparison using MAVExplorer. Without getting into the weeds too much this seperating of the axes allows the WebTool to do a direct matrix solve rather than having to use a iterative method. This also means that the WebTool will get the same answer every time. MAVExplorer will not, although its more obvious when your fitting more than just offsets where it cuts off the decimal places (this is because the values are stored as integer parameters in AP).
The WebTool weights the error for each log by the number of times it has see that attitude. If you imagine a sphere representing all possible orientations of the magnetic field vector the WebTool splits this into 80 sections and then weights those sections equally. In MAVExplorer each data point is weighted equally. Say you spend 90% of the log going north and 10% going east MAVExplorer’s solution concentrate 90% of its effort on the north section. This means you get very good results if your flying north but if you turn east the results are less good. In the same case the WebTool will spend 50% of its effort on north and 50% on east. This method is a attempt to ignore sections of the log where nothing useful happens as there is not new data. If you carefully crop in a log to the flying bit and fly lots of figure of eights they should both some out much the same.
MAVExplorer does a slightly odd yaw realignment thing that I don’t agree with so have not implemented.
Ultimately all the tools are just solving a optimization problem. If we gave them all the same problem they would get the same answer but because they all formulate the problem slightly differently they all get different results.
Really the only way to tell which is right is to try them on a vehicle and see which gets on best. The EKF does such a good job dealing with small differences that I doubt you would be able to tell.
Obviously I think the WebTool is best. That is not necessarily because it gets the best solution but because it tries to present the data in such a way that a user can pick the calibration that will work best for them.
@timtuxworth@iampete - Thanks for this comparison and high-level discussion of what makes the results different! This is very timely for some testing I’m currently conducting. From the description of the MAVExplorer and Web Tool, it sounds like MAVExplorer is solving the optimization problem using an iterative approach, while the Web Tool is using a closed-form solution, like least-squares or weight least-squares? I have no idea if that’s accurate as I have not looked at the implementation yet.
It could be interesting to use the tools together. Optimization problems, especially iterative solvers, generally require a good initial guess and being that the Web Tool runs “fast” it could produce a good first optimized calibration that could then be refined using MAVExplorer. This is done in system identification, where something like equation-error (ie least squares) is used to initialize an iterative optimizer. But we have to remember that @iampete said MAXExplorerer applies a “slightly odd yaw realignment thing” that the Web Tool does not use.
I suppose I’m rambling at this point, but it’s late for me. Thanks again for the development and discussion!
