How to export BIN and log files to CSV or JSON?

ArduCopter
1

I am using Mission Planner to fly my Foxtech Hover 1, and from the flights I perform I get log files in .BIN format.

I have tried to convert the bin file to csv for some post analysis, but haven’t been successful. In the end I would like to be able to access the data easily from e.g. Python. What I have tried:

  • Mission Planner allows you to convert .BIN to .log but that is not what I want ultimately
  • Mission planner allows you to review a log > display table > right click on table > export as csv but this only exports the current sub selection of data. I would like to be able to export multiple information sections at once (e.g. ATT, BAR0 and CTRL). Also, this method doesn’t allow you to include headers into the CSV.
  • Tried some python scripts with regular expressions, but they tend to not work for the whole file.

So how can I effectively export all of the sets of data which I want from the BIN file to a sensible CSV file?

Thank you very much!

Erik

2

CSV is not a good format, you sill need a lot of parsing to use it.

Do you have access to matlab? try ardupilog.

3

Thanks for the reply. Ardupilog seems to work but only partially (e.g. it imports IMU1 but not IMU2 and IMU3 data). Do you know why this could be?

Or can anyone recommend an alternative?

4

It`s an array of vectors, so 2 and 3 are also there

5

From a technical perspective what’s the issue with CSV? I’ve noticed even if I convert .BIN to .LOG and then try to import that (or a CSV for that matter) in Excel, they never fit in one spreadsheet.

On the other hand, any of the log viewers are able to open the same file (whether .BIN or .LOG) much more efficiently.

Thanks!

6

The .BIN files are complex, more complex than you expect.
The very latest version of ardupilog handles them fine, with vectors and all. We use it daily at our office.

And it is extremely easy to extract the data you want in matlab, process it and visualize it. Heck if you still want .csv files, you can export some of the matlab results into a simple .csv file, that is also easy to do.

But converting the entire unprocessed .bin file into .csv will produce a huge file extremely hard to parse. You have been warned. But of course you can use your time as you please.

2 Likes
7

Hahaha, got it!

I was curious about the reason why .BIN is so much more efficient. In my very simple understanding I was thinking if the logs are represented as a data table, what difficulties might be there in loading them into spreadsheets.

Thanks for the insight, though!

8

It is basically data in 180 different tables interleaved with each other. there are also multipliers and other stuff.

ardupilog works fine, but of course feel free to reinvent the wheel :wink:

9

Is there any python based tool similar to the matlab one?
or anything else other than matlab?

10

mavexplorer is a python tool. Have you tryed that?

1 Like
11

I have, and its not as comfortable as some online tools. If thats the one then I’ll use it… Thanks!