Uncommanded periodic spikes in DesPitch and DesRoll

[attachment=0]Spikes_DesPitch.PNG[/attachment]Hi All,

I made some tests with different setups today and one log shows strange (to me) uncommanded periodic spikes in DesPitch and to some degree in DesRoll.
What is even more strange is that i) they vanish out of a sudden and ii) they only occurred in that Loiter (Poshold) period - not in Stabilize and not in other Poshold periods. However, they occurred immediately after switching from Stabilize to Poshold.
I think I checked all other variables but no ones seems to be the cause.
RCIN shows no correlation - although CH3 seems quite noisy (Btw, much noisier compared to another system where I use the same TX but with the DSM satellite connected directly to the Pixhawk and not the main RX via a 3DR PPM encoder as in this case. Well, I just searched the forum and this seems to be an unanswered problem: viewtopic.php?f=68&t=5486 but most probably not related to the spikes).
The copter is a hexa with an original Pixhawk and a ublox M8 running AC3.2RC13 with EKF enabled. Vibrations are quite low.
Please find the log attached.

Any idea, help or suggestion is greatly appreciated!
Thanks a lot in advance!
Thorsten

This is almost certainly position noise getting into the attitude targets because we don’t have an appropriate filter.

It looks like it is timing jitter on the GPS that correlates with the spikes. See attachment.

Jonathan,
thanks for the reply!
I’ll make some further tests the next days. I will also check older logs. I have not seen this before. But maybe the spikes were covered due to higher AccZ vibrations.
Over at drones-discuss you mentioned a 3Hz lpf. Would it help in this case? If so, it would be great if you could post a compiled firmware including the filter for testing.
Thanks again,
Thorsten

It would help in this case but it would not address the root cause which is that the GPS or GPS driver is doing weird things.

Here’s the branch with the filter: github.com/jschall/ardupilot/tr … cel-filter

Ok! Thanks!

Following a recommendation from Paul I reduced EKF_POS_DELAY and EKF_VEL_DELAY from 200 to 120. All other EKF parameters are default values and the M8 is configured properly (at least I hope so). I’ll make a test with a regular 3DR GPS for comparison. But first I want to check if this was an exception of the rule.

Jonathan, I tried to replicate your diff(GPS.t,1). But it is unclear to me what it actually is.
Can you provide some details?
I had some really bad interference with the bluetooth telemetry module. Now I am using an xBee S1. Maybe this is the cause since they are both operating at 2.4 GHz. However, the bluetooth module caused changes in HDOP and the stat count. Which is not the case with the xBee.

Jonathan,
I successfully build a firmware based on your branch. Curious to test it later today. Even more because it is the first time I compiled the ardupilot firmware :slight_smile:
Since I am a GitHub newbie two questions: The resulting firmware is 3.3-dev. So it is based on master instead of the 3.2 branch? What are the differences I have to expect in relation to the latest 3.2RC?
Thanks a lot in advance!
Thorsten

[attachment=1]3HzPatch_01_spikes.PNG[/attachment]

Jonathan, I successfully finished a first test flight with your patch!
Two things: first, this hexa was never that stable in the air - especially with such high PIDs. Perfect! :slight_smile:
Second, in the beginning, after switching to PosHold for the first time, there were some relatively heavy uncommanded spikes in DesPitch again. But, the pattern is completely different this time. Apart from these initial spikes Loiter was perfectly smooth for the rest of the flight - almost scary… I’ll make some more intensive tests later today.
It would be great if you can have a look at the log. I hope you find the cause for these spikes.
Thanks again!
Thorsten

I made some further (short and unsystematic) tests. The filter seems to work very well! There are some twitches now and then but very limited.
What I found is that the EKF gyro biases (EKF1; GX, GY, GZ) are very noisy now. Especially GY is pretty high and noisy. This seems to be introduced with the new firmware but cannot be related to your patch. Hence I assume another cause. Any idea?

Default gyro bias process noise may have increased since whatever version you are used to.

I doubt that it is anything to worry about.

GPS.T is the system time in microseconds when the GPS driver logged that it got a new fix. It is not the actual time that the GPS got a fix. diff(GPS.T,1) is the difference between the current GPS.T and the last GPS.T. That would be GPS.TimeMS

Thanks for the explanation!
I’ll make some tests under different conditions and report.

Oh right, I just remembered that I fixed (several) bugs in the gyro bias logging - the units are different now.

So, the magnitude of that GY noise is about 0.1 deg/sec. You can reduce the EKF_GBIAS_PNOISE (I think) parameter to make it quieter if you really want, but I would caution against messing with it. The value was chosen by Paul Riseborough to be robust.

Jonathan,

Thanks for getting back on this!
I just switched back to 3.2 with your patches #1466 and #1457 included.
Flies fine. Although I am still having a little too high AccZ vibrations for a perfectly smooth Loiter.
The gyro biases look much less noisy now. But, if it is different units etc. I may switch back to master.
Is there any summary of the differences between 3.2 and Master apart from https://github.com/diydrones/ardupilot/compare/ArduCopter-3.2 ? I guess not :slight_smile: But since it is quite a few changes I am wondering if there are any really important changes to be aware of?
I will investigate the GPS timing problem within the next days.

Thanks again,
Thorsten

Could ask Paul about gyro bias noise.

If you want to compare 3.2 and master, you could do:
cd ardupilot
git checkout origin/master
git diff origin/ArduCopter-3.2 – libraries/AP_NavEKF

You can see:

  • No change to GYRO_PNOISE_DEFAULT in AP_NavEKF.h
  • No change that is likely to affect the gyro bias noise in AP_NavEKF.cpp

There was, however, this commit: github.com/diydrones/ardupilot/ … eb137d8f80 though I do not fully understand why it would cause such a drastic change in the way EKF1.G[X|Y|Z] look. Perhaps you could compare with and without that commit and see if the noise goes away?

Jonathan,
maybe it the difference results from the change from int8_t to int16_t.
The differences are obvious:

  • AC3.3-dev
    [attachment=1]Master.PNG[/attachment]

  • AC3.2RC14
    [attachment=0]32.PNG[/attachment]

In both cases they do not stabilize. According to the wiki they should: http://copter.ardupilot.com/wiki/common-apm-navigation-extended-kalman-filter-overview/#EKF1
Do you see any problem here?

Jonathan,

what software are you using to calculate the timing jitter (and to draw the diagram)? Is it part of the log analyzer?

Thanks!
Thorsten