Vibration, EKF variance and associated GPS Glitch results in cascading failsafe

I have several questions about this log of an EKF variance and GPS Glitch.
https://drive.google.com/file/d/1saTjRkaS0cboV-Zhy1M_ri2aMCVCc1QN/view?usp=sharing
The build was about 210g with 4 inch props and a Omnibus nano F4 V6. This particular build had no ESC telemetry, but a similar slightly lighter build (see below) does, and it hovers at about 10500 RPM, or about 175 Hz. I’d guess this frame is closer to 200Hz. The frame is one piece 4mmx4mm CF and I believe it contributes very little vibration. The props resonate fairly strongly at about 382 Hz. The flight controller is hard mounted with Nylon standoffs. I think the firmware as around V4.0.3.

I’m not an expert at analyzing logs, but this is what I see:
It looks like vibration levels are fairly low until the trouble begins, generally below 5 for all 3 axis. No clipping is reported, even after vibration levels soar to 20-30. Just before the EKF variance is reported, all 4 motor outputs increased from their normal hover range of about 30% to almost 100%. I suspect this was due to the EKF thinking it was loosing altitude, though it wasn’t. So the question is, what caused it to think it was falling. The IMU.accZ reading did jump in this area, indicating acceleration downward, but why? Was this caused by sensor aliasing? A little later, the AHR2 and NKF1 and NKF2 attitude estimates diverge and the EKF variance is declared. Am I interpreting the log correctly?

My questions:

Is the resonance at 382 Hz close enough to the MCU6000 sample rate to cause aliasing? I wouldn’t have thought so.
I’ve never understood how aliasing can be such a serious problem without clipping in the first place. Don’t the sensors inherently do some integration or averaging, or are the accel and gyro readings actually instantaneous readings somehow?
I’ve seen that the development branch now includes support for higher gyro sample rates for the MCU6000. Is this likely to help? It is not clear to me if sample rates higher than 1kHz are possible for the MCU6000 accelerometers. When fast sampling of the MCU6000 gyro, say 4kHz is configured in the development branch, what will the effective sample rate be for accel? Is faster sample rate for gyro alone likely to help?

A GPS glitch is reported at about the same time as the EKF variance, but I’m guessing the position difference was caused by the EKF issues. Wouldn’t it be much better if the position information didn’t get called into question? Is there a way to avoid this cascading failsafe?

The vibration levels in the log dramatically increased when the RCout increased, but which came first? The log seems to show both increased at about the same time which makes it hard to establish cause and effect.

The flight ended in a bad crash because I panicked after the ground station reported the problems. I switched modes in an attempt to prevent it from landing where I didn’t want it to land and when I tried to switch back to loiter, it refused and flew away at very high speed and I ended up triggering emergency motor stop hoping it would crash into something soft. It was a very light build, but at the speed it was traveling when I cut the motors, I’m lucky it crashed on a neighbors driveway and not through their window. I’ve rebuilt with a new flight controller and nearly identical frame at the same total weight and I’ve had one more flight with EKF variance and GPS glitch which also caused a forced landing. A nearly identical but 20g lighter build described here has never exhibited these issues with or without the hard mount:

The log for the second instance seems to show bogus GPS vz reading before the vibration increase. So I suspect this might have triggered the increase in RCout in this case. But the result was the same increased vibration and EKF variance and GPS glitch report. Here is the second log:
https://drive.google.com/file/d/1c5jXscqNnUXMsJ1t7VjjAYslU6AVxV57/view?usp=sharing

I would appreciate any suggestions and help with these questions.

Thanks!
–Brad

1 Like

For first log:

Yes, that is true, but after that vibration levels suddenly seem to increase and CTUN.ThO suddenly incrases from normal values, reaching 1 (green trace):

CTUN.ThO, Ctun.Alt and Ctun.DAlt show that you were at 30m high and it climbed, but probably you didn’t want to climb, so CTUN.Dalt is strange. You also have “Vibration compensation on”.

This capture (2142x4673) shows CTUN messages, showing exagerated values for CTUN.ThO:

Now have a look at
Autotune: hexa sudden climb 40m – CTUN.Dalt negative – DANGER

Very similar situation: 11s normal (autotune was on progress with a fully charged battery, but that may be a coincidence), and suddenly it went up.

During those 11s vibration levels were moderate, then seem to increase and copter is suddeny accelerated going up (CTUN.ThO=1 during one second).

On all replies the cause is high vibrations. Perhaps your case shows this is not clear.