Dataflash vs tlog protocols

Hello everyone,

I am doing some log diagnostics and am going through .bin and .tlog files. Currently I am relying on Mission Planner to generate cleartext but this is not the point.

The first part of a .bin file is the message format definitions, eg:

FMT, 128, 89, FMT, BBnNZ, Type,Length,Name,Format,Columns 
FMT, 129, 31, PARM, QNf, TimeUS,Name,Value 
FMT, 130, 50, GPS, QBIHBcLLeeEefB, TimeUS,Status,GMS,GWk,NSats,HDop,Lat,Lng,RAlt,Alt,Spd,GCrs,VZ,U 
FMT, 131, 50, GPS2, QBIHBcLLeeEefB, TimeUS,Status,GMS,GWk,NSats,HDop,Lat,Lng,RAlt,Alt,Spd,GCrs,VZ,U 
FMT, 195, 19, GPA, QCCCC, TimeUS,VDop,HAcc,VAcc,SAcc 
FMT, 196, 19, GPA2, QCCCC, TimeUS,VDop,HAcc,VAcc,SAcc 
FMT, 132, 49, IMU, QffffffIIfBB, TimeUS,GyrX,GyrY,GyrZ,AccX,AccY,AccZ,ErrG,ErrA,Temp,GyHlt,AcHlt 
FMT, 133, 75, MSG, QZ, TimeUS,Message 
FMT, 134, 39, RCIN, Qhhhhhhhhhhhhhh, TimeUS,C1,C2,C3,C4,C5,C6,C7,C8,C9,C10,C11,C12,C13,C14 
FMT, 135, 35, RCOU, Qhhhhhhhhhhhh, TimeUS,Ch1,Ch2,Ch3,Ch4,Ch5,Ch6,Ch7,Ch8,Ch9,Ch10,Ch11,Ch12 
FMT, 136, 15, RSSI, Qf, TimeUS,RXRSSI

and other similar lines.

This is not MAVLink format, so far I can tell. What I am trying to understand is what this is and why it is what it is.

Is there documentation for this specification, or is there just the code?
Is it quasi-stable? How often does it change?
Is there a reasoning behind this specification? Why not just write the same telemetry MAVLink messages onto it?

Thanks in advance!

Hi,

The bin format is self described. Those FMT messages are format messages that specify what format the other messages have. The first FMT message describes itself.
The format field is inspired on the Python formatting and the exact meaning can be found at https://github.com/ArduPilot/ardupilot/blob/master/libraries/DataFlash/LogStructure.h#L719-L738

There isn’t any documentation except code, but like I said the bin file is self described. Now if you want to know exactly the meaning of each of those columns in all the messages you’ll need to look throughout the code. It isn’t easy to document as the calls to write to the dataflash are spread out in the code and there are messages that exist in one vehicle type that don’t exist in the other.
I don’t want to say it is stable as it regularly has new messages or existing ones may change. But it isn’t a completely unstable format.

MAVLink messages have less information than the dataflash log. That is the reason behind always asking for it when looking at issues posted in the forum.

If you have more questions, go ahead and I’ll try to help.

Thanks for the reply, Francisco.
It confirms some of my findings.

I have a few more questions, if that’s alright, regarding the related parameters.

First of all, there is LOG_BITMASK, which seems to control the logging frequency of various “families” of log messages.
However, after a cursory browse of the code, I cannot find which messages are included in which “family”. Can you point me to the corresponding source file (I know it’s a bit of a mundane question, sorry).

Additionally, there is the LOG_BACKEND_TYPE parameter which I hadn’t noticed before.
Is it correct to say that if I set it to

  • 2: The logged file will be in the MAVLink format
  • 3: Two files will be recorded, one in Dataflash format and one in MAVLink.

Thanks

Regarding LOG_BITMASK it controls if messages should be logged or not, not the frequency. This can be different in each vehicle although I think they are all the same. Hopefully we can change this to the DataFlash library in the future.
Once again, there isn’t a single place with the info. For Copter, at https://github.com/ArduPilot/ardupilot/blob/master/ArduCopter/defines.h#L311-L330, you can find the list of masks for logging and you can search for them in code.

Regarding the LOG_BACKEND_TYPE:

  • the DF log is never recorded in a MAVLink format, what that option does is send the DF log through MAVLink instead of recording it in a file
  • if you set it to 3, it will save to a file and send it through MAVLink

Don’t hold back if you have more questions, I’ll try to answer.

Right, I got it.

Thanks for the answers, Francisco!

It is a bit discomforting that there are two disjoint “telemetry” protocols where ardupilot reports its health and status, when it comes to diagnostics.

But I completely understand this choice, since MAVLink serves a communication purpose primarily, while the Dataflash is for debugging mostly. And programmers need debugging messages.

I may ask about the content of some Dataflash fields in the future, but I think that’s it for now. :slight_smile:

In case you haven’t found this already, libraries/DataFlash/LogStructure.h
defines these formats, look around line 700 or so for the definitions of
the odd-looking QfffffIIfBB decoding.

Unfortunately I can’t speak to your other questions, someone else chime in?

1 Like