Notch filters turn on with a delay?

I am messing a lot with my freshly-built Coax based on a Matek H743 WLITE board. I am using the master branch (this commit to be precise) with some changes unrelated to filtering.

The coax it is not yet well tuned and such, but I spotted a really weird behavior of filters which I think should not happen. Here is the post-filter plot obtained from the Filter Review Tool:

Here it is clearly seen that before roughly 30-second mark - which is few seconds into the flight - the notch filters don’t cancel out the noise. This happens both for the propeller noise (including harmonics) and for the ambient 100Hz noise (likely from servos working).

Where is this turn-on delay coming from? In highly tuned vehicles this could be dangerous…

The complete log of the flight is attached.

And yet another flight with a clear transition in terms of filters at 30 seconds, this time based on PID logging. Interesting…

The 30 seconds seem to be related to the moment of powering up the flight controller rather than to the moment of starting up: the two flights above started at roughly 19 and roughly 16 seconds since power-up. Third flight started at roughly 38 seconds after the power-up moment, and there was no indication of delayed filtering.

Is this on the master branch or 4.5? There have been some filtering changes made in master that could be related

It’s on the master (I have indicated the exact commit in the post).

The filtering changes came in after this commit 533eb73622b11bc0d4841ae5268c05079bcb33dc so its worth seeing if you still see the same behaviour at that commit

I had to turn off the “Gyros inconsistent” 10s delay manually in order to be able to check that - otherwise, the lift-off could only happen right at 30s, resulting in no useful data.

There was no 30s event in a flight based on that commit. However, the filters themselves worked worse: out of two motor notches, one seemed to be suppressed a lot less. I don’t know whether it is expected, but probably it is useful.

Note that the filter review tool has not been updated for the latest notch functionality in 4.6.dev.

This does look like a bug to me. We see the desired notch frequencies from FTN move as expected. Then after some time the logged center frequency in FCN jump up to match.

It then tracks correctly for the rest of the flight.

1 Like

can you upload the log? I’d like to reproduce in SITL

1 Like

The very first log is already uploaded (the link is in the body of the post).

The second flight (of which there was a screenshot in my first comment) is this:

The flight at @andyp1per’s request, forked off at the point before the overhaul of filters, is this:

Sorry for being slow. From the filter review tool the issue seems to only be there in the newer logs.


Not in the older log:

In the second log on 4.6 as in the original log we see the FCN only jumps up after some time. What I did not notice before is that we also see the same thing in FCNS and FTNS. So its it seems to be all notches rather than just the those driven from RPM.

Yes, this is all consistent with what I think I observed.

I have done a PR that I hope will help with debugging this issue.

Some other things to try would be to remove the option to update at loop rate. See if it comes good at the same time after the vehicle has armed. The other would be interesting is to try is throttling down and up again in the same log. It would be interesting to see if they have the same issue on the second flight or if its a once per boot thing.

I rebased onto the PR above and did two tests with INS_HNTCH_OPTS set to 6 (originally) and 2 (as advised). Each test contains two flights, one started as early as possible, and a second one. I also performed some throttle excursions to the degree possible with this vehicle and in the test environment available. The logs are below.

The quick summary is:

  • NaN logging seems to work as expected.
  • “Update at loop rate” seems to have no effect on the 30s notch startup delay.
  • Subsequent take-offs have no filter problems, so it seems to be a startup issue.

I seem to have found where these 30 seconds come from.

This is HAL_INS_CONVERGANCE_MS from AP_InertialSensor/AP_InertialSensor_config.h:
#define HAL_INS_CONVERGANCE_MS 30000

Within these first 30 seconds, according to void AP_InertialSensor::HarmonicNotch::update_params(uint8_t instance, bool converging, float gyro_rate), init is called on filters instead of update.

As far as I can see, no center frequency update is actually happening in this case. If I am correct, in this case the center frequency is never communicated to the underlying bank of notch filters, so they are never initialized, and we get NaNs in the log and no filtering. (For fixed notches I am not sure this is the right explanation, but probably there is something I don’t know).

And in any case the problem seems to be somewhere along this code path.

Ah, that makes sense! I remember querying this in the review - need to go and have another look.

Well found!

The easy fix would be to just pre-arm check that the convergence is done (it might be a good idea independent of this notch issue).

The more complex fix would be to allow the update function to take sample rate so it could still run the filters.

Convergence is currently hardcoded to “30 seconds past the FC is turned on”, so this kind of “check” would be disappointing. In some of my craft, 30 seconds can be enough to melt mounting points of a video transmitter and drain some noticeable amount of battery juice - unless one starts with USB power and detaches it later, which is quite a disadvantage in usability.

We always used to filter while converging - I don’t see why we can’t re-instate that code.