Where do all log values come from? (Raw sensor and control-considered data)

Hey guys

[General question]
I’m having some trouble trying to figure out where do some specific variables come from, what do they mean in terms of in-flight decision making, etc
More like “how is this calculated and what does it feed? what value (among the many variables that means the same thing) is the plane really taking into account when in auto?”

Is there anywhere I can find a list of it?

[My specific problem]
I’m using a piksi rtk module as gps2 and (as expected) getting some discrepancies between GPS1, 2, all the BAROs, POS, etc
How do I know what value is feeding the controller on auto mode (aka the “actual” altitude as far the navigation is concerned)?
Altitude is just one example. There are many physical quantities with different values on many different variables logged.

Quick update: maybe a better way to put this is “what are raw sensor data and what are the calculated quantities”

My log and param files, if anyone is curious

Ardupilot is a complicated system, and there is not a single document where all the log values are explained. (There are good reasons no such document exists)

Depending on your needs, the “hard but best” approach is to learn to read the Ardupilot source code. Once you are familiar with how logging works, you can trace which value in the C++ source code is being logged to DataFlash. There is a well-designed pattern.

An “easier but limited” approach is to just ask specific questions, as you’ve done… so I’ll also try to help with your specific question.

The “best estimate of truth” available to the Ardupilot system comes from the NavEKF system. This is called a “Filter” (specifically EKF=Extended Kalman Filter) which is the technical name for “the math which combines (noisy) sensor data with mathematical models to figure out the truth as best as possible.” So a best-bet for finding what the plane is “really taking into account” is to look at the results of this filter. In the log, the POS and ATT groups stand for Position and Attitude, and these are populated with the outputs of the NavEKF system.

So the short answer: I think you want to look at POS.RelAlt. I think it is the calculated quantity which is used for most (but not all!) altitude-based decisions.

P.S. both your links appear to provide the same .param file, neither seems to provide a log.

2 Likes

Thanks @hunt0r!

Sorry, here is the log.

So, getting the hook, if I’m looking for the “actual” physical quantities (rather than raw sensor data), I should always look for the NKF outputs? POS for position, ATT for altitude and I’m guessing TECS or CTUN for velocities? (Ground speed should be the same on either GPS and airspeed should be directly from ARSP)

Learning how the source code works and tracing the quantities back from the log is my next step. Pretty hard, not impossible!
But then I won’t be bothering here. Actually, I could also help others.

In the version that you’re using, NKF is the abbreviation given to the NavEKF2 filter data, this seems to be your active filter. There are XKF fields, the abbreviation given to the NavEKF3 filter data, but no data in your log, indicating you probably have this filter disabled.

I believe the POS is your best position estimate. It is likely selecting the “best” output from the filters, and converting to Lat, Lng coordinates. (I see that NKF1.PE and NKF1.PN look very similar to POS.Lng and POS.Lat, they may be identical when converted to like units)

ATT is your best attitude estimate, and I see that ATT.Roll is the same as NKF1.Roll.

NKF1 has what seems to be ground-referenced velocities: VN, VE, and VD.

1 Like

You probably know this, but just in case: The best description for all parameters/log variables is here (plane): ardupilot.org/plane/docs/parameters.html

But as Hunter says, if you need additional information in terms of calculations, where they come from and how they are used looking at source code is the way to go.

1 Like

I have been struggling to read logs as well. Once I think I got something, I try to put it into the wiki.

It would be really nice, if log variables had to be commented in the source code. One could periodically and automatically generate a complete log list - as it is already done for the parameters. Maybe that’s already somewhere of the bottom of the devs TODO list in any case. :wink:

1 Like

Thank you guys very much!

Any additional thoughts are well appreciated.
Hints on how to start digging into the code are welcome as well!!!

@moobsen such a request exists, indeed, low down on the priority list: https://github.com/ArduPilot/ardupilot/issues/5864#issuecomment-286475888

@arthursn138 Here are some hints: ArduPlane/Log.cpp, libraries/DataFlash/LogStructure.h and libraries/DataFlash/LogFile.cpp

Good luck!