Now that I have system identification data, what do I do?

Greetings. I am thinking that the system identification mode will be really useful as it collects much information which, it seems, could then be post-processed and used to help make things “tight” and “right”.

But, uh, when I look at https://ardupilot.org/copter/docs/systemid-mode.html and at https://ardupilot.org/copter/docs/systemid-mode-operation.html, I see much write-up about how to create the data, but I can’t find anything obviously useful about how to use, or post-process the data.

What am I overlooking? Thanks for the assist.

bob

1 Like

Hello,

You should look at tuning page then ! You have plenties of data in the log that are waiting to be analysed. As the mode description said, you have recorded the system output to a stimulus. From the log you should be able to calculate if the response is the one you want and if not adjust the different controller to reach the response you want.
If you aren’t used to manipulate those data, you could start be something more basic like a PID tuning. Look on the net the theory behind PID tuning. And I speak about the real math behind PID, not the approximation you could read on drone forum…

1 Like

I’m at the same point. I used the latest PR from @Leonardthall (https://github.com/ArduPilot/ardupilot/pull/17345) on top of master and did flights with all possible SID_AXIS combinations.

But towards the end of each System Identification run, the copter climbed a lot, so we manually reduced throttle. I have a gut felling that we should not have done that. Can you comment on this?

I would also like to add information about system input variables and system output variables for each of the runs to the wiki.

My next action will be to create a script that extracts the relevant SID sections of the flights from the dataflash logs to convert them to matlab files.

1 Like

That is not a problem. You need to make adjustments to keep the copter in position.

There is no log output. All of them record all the data. So the logging does not change. The rates are defined as presented in the Wiki. You get the ATT, RATE and PID logs with the additional SID log for the excitation reference data. This is all done lock step to ensure the data all has the same time stamp for each sample.

Hello @Leonardthall.

This is what I figured out so far:

SID_AXIS description SID_MAGNITUDE IN OUT Why
1 Input Roll Angle 15 SIDD.Targ ATT.Roll Complete angle and rate controller and system roll response
2 Input Pitch Angle 15 SIDD.Targ ATT.Pitch Complete angle and rate controller and system pitch response
3 Input Yaw Angle 15 SIDD.Targ ATT.Yaw Complete angle and rate controller and system yaw response
4 Recovery Roll Angle 15 SIDD.Targ ATT.Roll (FF forced to 0) Angle and rate controller and system roll response without the command model (input shaping)
5 Recovery Pitch Angle 15 SIDD.Targ ATT.Pitch (FF forced to 0) Angle and rate controller and system pitch response without the command model (input shaping)
6 Recovery Yaw Angle 15 SIDD.Targ ATT.Yaw (FF forced to 0) Angle and rate controller and system yaw response without the command model (input shaping)
7 Rate Roll 40 SIDD.Targ RATE.R Roll rate controller and system response
8 Rate Pitch 40 SIDD.Targ RATE.P Pitch rate controller and system response
9 Rate Yaw 40 SIDD.Targ RATE.Y Yaw rate controller and system response
10 Mixer Roll 0.6 SIDD.Targ PIDR.Act Roll system response (no controllers *)
11 Mixer Pitch 0.6 SIDD.Targ PIDP.Act Pitch system response (no controllers *)
12 Mixer Yaw 0.6 SIDD.Targ PIDY.Act Yaw system response (no controllers *)
13 Mixer Thrust 0.6 SIDD.Targ PIDA.Act Thrust system response (no controllers *)

* the controllers are active and they will fight against the SID excitation signal. But they do not appear in this response.

Did I got it right? Are the recovery Axis just like the Input Angles with FF parameter forced to 0?
https://ardupilot.org/dev/docs/apmcopter-programming-attitude-control-2.html does not explain where the signals get injected.

I now have a matlab script that prepares .bin files to do processing on all 13 Axis, but I do not know if my above assumptions are correct.

My ultimate goal is to be able to determine all FF parameters and the phase margin of each PID controller.

@Leonardthall can you provide me with the source file of the latest control diagram (the one you used in the Conference 2021)? I will annotate it with the Dataflash log variable names. And give it back to you, so that https://ardupilot.org/dev/docs/apmcopter-programming-attitude-control-2.html can be updated.

I posted my work on https://github.com/amilcarlucas/ardupilog/tree/pr-ardupilot-sid-support

The flight log files are available here https://we.tl/t-KXhvCNx6LL

But the tool is easily customizable to support your own flights

Hello @Leonardthall. this is the place I posted the results.

The script automaticaly creates iddata objects out of flight log files

That is correct.

Here is my latest attitude control diagram.

ArduCopter V5 Attitude PIDs.pdf (49.7 KB)

Rate Roll, Pitch and Yaw axis are injected into the rate request after the angle controller in the diagram.

Mixer Roll, Pitch, Yaw and Thrust are injected into the motor mixer with the rate loop output.

Does that answer your question?

1 Like

Thanks Leonard,

I need the source of that diagram so that I can edit it and send it back to you.
I plan to use diagrams.net to recreate it in editable .png file format so that anyone can edit it.

Can you take a look at the table above? It tries to explain why the axis are there and what they are used for.

I plan to make that table part of the matlab script documentation, together with the annotated diagram where one can see where the signals get injected and which signals get logged together with their name.

The matlab script posted above is a good starting script for anyone, just feed one or more .bin log files to it and it generates fully annotated matlab iddata objects. And it is documented :slight_smile:

So your question marks are actually:
Angle and rate controller and system yaw response without the command model (input shaping)

And the points with:
(no controllers) still have the controllers active so they will fight against your excitation still.

Any of these can be used to create various models the difference is the frequency response due to the filters the excitation pass through and how the controllers fight against them.

I will send the diagram by email as I can’t attach them here. I used Dia.

Where in the diagram do PSC_VELZ_FF, PSC_ACCZ_FF, PSC_VELXY_FF fit?

Inside the “Target to measured error calculation” ?

ArduCopter 4.1 Attitude PIDs - 20210602.dia.txt (19.3 KB)

@Leonardthall here is my latest diagram, just rename from .dia.txt to .dia.
I added notch filters, point where signals get written to the .bin log files, and the injection point for the system identification.

Can you add the remaining missing information?

They aren’t in the stabilize controller. They are an outer loop (wrapped around the stabilize controller for position control) and do not affect the response that you are measuring with the system ID mode

It is important to know how the data is filtered before it is stored. This will impact bare airframe identification if it is not fully understood and properly accounted for in the model identification.

ArduCopter 4.1 Attitude PIDs - 20210603.dia.txt (21.7 KB)

I worked a bit more on it and did some corrections.

@bnsgeyer thanks for the input, I now understand why PSC_VELZ_FF, PSC_ACCZ_FF, PSC_VELXY_FF are not depicted in the figure. The figure describes the stabilize flight more, and there is no position controller on that one.

Yes, it is very important to know the source of the logged signals. That is not documented, this is my attempt at documenting it. I hope this helps other people struggling with the SID mode.

So now it is clear where the SID_AXIS 7…12 are injected and where the RATE* and PID* signals get logged.

Can I get some help in determining where SID_AXIS 1 …6 and 13 get injected? And where ATT* signals get logged?

@Leonardthall In the diagram, shouldn’t you show that the stick position is converted to a pilot attitude request before it goes into the Input Translation block?

@amilcarlucas SID_AXIS 1, 2 and 3 would be the pilot inputs (stick position) as an angle request which would be before the input shaping (which I believe is occurring in the Input Translation block).

SID_AXIS 4, 5, and 6 would be in the same place as the first 3 however the feedforward is off. So in the Angle Controller, the block that says “target to body frame conversion” would be removed from the diagram.

I am pretty confident that what I said is correct but I would like @Leonardthall to verify that.

1 Like

Yes it should.

Correct.

Bill is correct here.

We made some progress on this. Now we have two diagrams:

  • one for RATE_FF_ENAB = true

  • and one for RATE_FF_ENAB = false.

We improved the variable names, and added many details. I will post updated matlab scripts soon.

4 Likes

great job @amilcarlucas I am looking into this more and more,

unfortunately I am unable to open the flight log files, could you upload it to another cloud service such as DropBox.

The log files are not important. The PR is, I plan to work more on the PR once I get some time. but feel free to post pull-requests against my branch.