How to export all log parameters from .bin file?

General
1

Hi all,
First time posting to this community, so apologies if this is not the right place to do so.

I am interested in exporting all the parameters from the .bin file into separate files, as I would like to upload and store this data in a central database (MS Access).
Currently, I was thinking about just looping through all params shown in the docs and using the pymavlink tools to export data. Currently, my code loops through all .bin files in a folder.

Log parameters page
https://ardupilot.org/copter/docs/logmessages.html

pymavlink command line tool:

param="BAT"
for fn in *.bin
do
    # echo "$fn"
    python "C:\path\to\script\mavlogdump.py" --format csv --types $param "C:\path\to\bin\files\\${fn}" > "C:\output\folder\path\${param}_${fn::-4}.csv"
done

Is there a better/more succinct way to accomplish this? Should I create an array of all desired params, and loop through those as well?

Thanks in advance!

2

If you’re just doing one or two log files, try this: Review the log file in Mission Planner. With the log file viewer open select the check box to “Show Params”. Then in the pop up that has the parameters “save to file”

3

image
image1016Ă—271 48.4 KB

Another possibility is to go to the DATAFLASH LOGS tab and select CREATE KML+gps. You will then be prompted to select a BIN file. In the same directory as the BIN file, a series of files will be created, including a parameter file.

4

Hi, @Lano @Allister Thanks for the replies. However, it does not seem like these are solutions for my particular application.
We have a fleet of drone pilots. Our idea is that, when they return to office, they would download the .bin files from the drone (to a usb). Then, go to our main server, plug in the usb, and use our script that would:

  • loop through each .bin file and export all parameter data (to json or csv files)
  • then upload those exported data files into MS Access database
  • more stuff down afterwards in script
1 Like
5

You do have the more scriptable/automatable solution. There is no better than that.

I do have a python script that is probably faster, but it needs adaptation for you because it checks out the default parameter values, not the parameter values. If you are interested the script is called extract_param_defaults.py and all you need to change is line 97 and 98 to extract Value instead of Default.

1 Like
6

You can simply use pymavlink to do this.

Here is a function that takes in the file path of the .bin file and returns a dictionary of the parameters:

def get_params(file_path):
    
    parameters = {"PARAM_NAME": [],
                 "Value": [],
                 "Default_Value":[]}
    
    connection = mavutil.mavlink_connection(file_path)
    m = connection.recv_match(type='PARM')
    while m:
        parameters["PARAM_NAME"].append(m.Name)
        parameters["Value"].append(m.Value)
        parameters["Default_Value"].append(m.Default)

        m = connection.recv_match(type='PARM')
    
    return parameters

you will have to import the mavutil.py from pymalink as shown below:

from pymavlink import mavutil

Further you can save these parameters into an file format as per your requirement.

7

@amilcarlucas @Suyash_Mali Thanks for the responses and sharing some code! I think both of your codes will work for my use case.

@Suyash_Mali Can you expand on what the Value and Default_Value params are? What gets populated there?
Does this function return what parameters there are in the .bin file, and then I would use another function to export the data from that dictionary?

8

It is the same code I use, but mine has export functions included

1 Like
9

The Value key contains the present value of the parameters, while the Default_Value contains the default values of the parameters that are set by ardupilot.

Yes, this function returns just a dictionary containing the Param Names, Their present values, and the default values of those parameters. You can use it according to your requirement.
And as said by @amilcarlucas, this function is just a striped-down version of his example.

10

@amilcarlucas thanks for your code, it is very helpful for learning to use pymavlink software. Currently, I am only getting the default value outputs, but I would like the actual measured data (i.e. the GPS time and location as a csv file). I changed the code to have:

if pname not in defaults and hasattr(m, 'Value'):
    defaults[pname] = m.Value

But this only returns a single value for each param.

In your code, how would you output each PARAM as a file, such as the output file generated from the code below:

python "C:\path\to\script\mavlogdump.py" --format csv --types GPS "2024-01-12 10-07-30.bin" > "GPS_OUTPUT.csv"

Edit:
I was able to output the entire file as a text using the following command:

python  "C:\path\to\script\mavlogdump.py" "2024-01-12 10-07-30.bin" > binoutput.txt

It seems like this doc describes the .bin file params well:
https://ardupilot.org/copter/docs/logmessages.html#logmessages

Would you recommend finding a way to loop through this output? Or can your code condense all the params together and output a file for each?

11

I have changed the tool to do what you want, but you need to get the new version from the GitHub - ArduPilot/MethodicConfigurator: A clear ArduPilot configuration sequence repository and then play with the command line parameters

12

Thanks for the quick response. For some reason, your code seems to only print the param_name and value, and not all the data. I was able to work with the pymavlink api, and it seems like I could pull each param using the:

m = mlog.recv_match(type=['GPS'])

Is there a list or function that returns all the types (which are the params in link below)?
https://ardupilot.org/copter/docs/logmessages.html#logmessages

13

Yes, for GPS messages you need to change the code in the way you described. You can add other message types to the list

14

Is this something I should just loop through a list of message types that I create manually?

15

No, you put them on on the list and loop only once.
The body of the loop is a switch-case similar statement that selects what to do depending on the message type