Ardupilog - A DataFlash log to MATLAB converter

Blog
, ,
1

Hello everyone!
Almost two years ago, @hunt0r and I needed to parse some ArduPlane logs. We are both on our track to get a PhD in UAV control systems and Matlab is a very common tool in our trade.

Sadly, at the time the only tool able to convert from DataFlash logs (.bin) to a format readable by Matlab was Mission Planner and it didn’t serve our needs well enough. It would work only on Windows, take a very long time to convert and would ignore all text fields.

So, we decided to create our own converter, Ardupilog. It is Matlab-based, meaning that you need Matlab to run it.
It can read all DataFlash logs, since they are self-defining, and create a versatile log object.
You can even “flatten” the log into a classic struct, if you want the information to be independent of our library.

Example Usage

Reading a log

Add the Ardupilog source code to your path.
Then,

log = Ardupilog()

will open a file browser, where you can select the log file you want to decode.
The variable struct log will be generated with the included message types as fields.
Its contents can be normally processed with Matlab commands:
image
image

Filtering messages

You can optionally filter the log file for specific message types:

log_filtered = log.filterMsgs(<msgFilter>)

msgFilter can be:

  • Either a vector of integers, representing the message IDs you want to convert.
  • Or a cell array of strings. Each string is the literal name of the message type.
    image

Slicing

Typically, only a small portion of the flight log is of interest. Ardupilog supports slicing logs to a specific start-end interval. For example, to keep log messages from boot time 1000sec to 2500sec:

sliced_log = log.getSlice([1000, 2500], 'TimeS')

Exporting to plain struct

Creating a more basic struct file, free of the Ardupilog dependency, is possible with:

log_struct = log.getStruct()

log_struct does not need the Ardupilog source code accompanying it to be shared.
image

More information can be found on the repo’s README.

Next Steps

While presently Ardupilog does not receive regular updates, features that we’d like to see supported at some point are:

  1. Support for MAVLink .tlog logs
  2. Incorporation of DataFlash units for messages fields
  3. Interpolation of message field timeseries, where applicable

We hope this little project proves useful to those who need it. :grinning:

11 Likes
2

Nice work !
would love give it a try …

3

I think in Ardupilog.m the GPS leap second row (405.) has a missing data: ‘Jan 1 2017’
There is +1 second introduced over Dec 31 2016 (https://en.wikipedia.org/wiki/Leap_second).

Otherwise excellent work!

4

Indeed, our tables were out of date.

Thanks for noticing it and for your comments!

5

I tried this script with some of my logs. Very good work!

1 Like
6

New Dataflash logs include additional messages encapsulating field units and multipliers.
With https://github.com/Georacer/ardupilog/pull/77 now Ardupilog can store this information within each logMsgGroup, on the fieldUnits.<fieldName> and fieldMultipliers.<fieldName> attributes.

7

I getting the following error messages

log = Ardupilog()
Undefined function or variable ‘LogMsgGroup’.

Error in Ardupilog/createLogMsgGroups (line 315)
new_msg_group = LogMsgGroup(newType, newName, newLen, newFmt, newLabels);

Error in Ardupilog/readLog (line 134)
obj.createLogMsgGroups(data’);

Error in Ardupilog (line 68)
readLog(obj);

Any help would be appreciated

8

Hi,

Can you provide me with the log you are trying to read?
I’m not getting any errors with the current master on my logs.

9

Thanks George. File attached.

10

Err, are you sure? I don’t see a link or attachment anywhere…

11

Hi, II have noticed that the data flash logs in newer 3.9 7 above base ardupliot are not being read by Matlab. This is the error:
Error using Ardupilog/addprop
‘FMT’ is already defined as a property.

Error in Ardupilog/createLogMsgGroups (line 320)
addprop(obj, newName);

Error in Ardupilog/readLog (line 134)
obj.createLogMsgGroups(data’);

Error in Ardupilog (line 68)
readLog(obj);

12

Thanks for bringing this up. Can you provide me with a log, so that I can test and fix?

13

Hi George,

Thanks for the response. I have uploaded a sample log: https://www.dropbox.com/t/YUf3uycJNNVZCYnA

14

@Georacer : Were you able to download the file?

15

Yes, 1.7MB. I’ll look into it.

16

Hm… I’m getting a different error:

In an assignment  A(:) = B, the number of elements in A and B must be the same.

Error in Ardupilog/buildMsgUnitFormats (line 374)
                multValues(unitIdx) = multValue;

Error in Ardupilog/readLog (line 184)
            obj.buildMsgUnitFormats();

Error in Ardupilog (line 68)
        readLog(obj);

I’ll check it nonetheless, but can you make sure what error you get with the log you sent me?

17

FYI, started an issue here, because it looks a bit perplexing and the discussion might take long: https://github.com/Georacer/ardupilog/issues/79

18

This is super awesome. I’m using it for a GUI application for flight analysis.

EDIT: Is there a way to create empty Ardupilog structs to manipulate multiple sets of logs? I’d like to create a drop down menu with different logs. I want to define it as a public property.

Example:

plogs = [{ardupilog_1}, {ardupilog_2}, {ardupilog_3}, … {ardupilog_n}]

19

Hm… I’m not sure I understand your end-goal and I foresee a lot of back and forth. Care you open an issue with a use-case example where we can talk more?

20

Hi. I realize this is made for MATLAB, but have anyone made it work in Octave? I am attempting to do so but it seem to give up before I get prompted to input a file. As a former Matlab user I believe it is superior, but there are many reasons to use open source software, and Octave is not a bad alternative when it comes to that.

1 Like