ArduPilot 4.1-beta2 going upwards like crazy (EKF2, poshold)

Hey,

I just had a very strange behavior with ArduPilot 4.1-beta2 (self compiled) using EKF2 on my 5" Quadcopter. I flew the whole day and everything looked fine, but during the last two flights, in Poshold, quickly after arming and starting, it went all over the place and I already thought I lost it in a tree.

In the logs you can see, shortly after starting, it suddently went from a stable flight in Poshold to full power and flew up into the air. I tried to figure out what’s wrong from the logs, unfortunately, I don’t understand it.

Purple line: throttle, other lines: Servo 1-4

Can anybody tell me what’s wrong? Is there a sensor broken?I didn’t change the configuration between the good flights and the ones where it went crazy. I use a Matek H743 for FC and Matek M9N-CAN for GPS.

Logs:
[removed]

Video:

I’m thankful for any help as I have no idea :frowning:

Thanks!
Sebastian

Link doesn’t open - access denied

Sorry, should work now. Can you please try again?

Sebastian

Hi @Sebb,

Very high vibrations I’m afraid. The “grey zone” is between 30m/s and 60m/s but the vertical axis vibration levels are very commonly above 40m/s/s and spike to 70 m/s/s or even higher on two occasions. It’s hard to say what’s causing the vibrations but perhaps check the motors and propellers and also how the autopilot is vibration isolated. We have some wiki pages here which may help.

1 Like

Hi @rmackay9,

thanks for the feedback. There was a very strong wind (I’d guess around 30-40km/h), but the same wind was there already before where everything was completely normal.

Same quadcopter, the day before, it was a quick test hover, you can see that I have absolutely no general vibration issue…

As written, that was the 2nd time in a row that this happened, in fact, I just wanted to verify with this flight that everything was okay… When it happened before, it was also in poshold, as soon as I switched it to stabilize mode, it stopped shooting upwards.

For me it’s not clear why vibration/wind causes it to shoot into the air like that?

This was the flight before: (Vibrations on top, servos on bottom):


[removed]

And you can see the same, in poshold, it started shooting up (maybe also in althold but took a bit?)

Here is the almost 9 minute long flight before, where you can see the vibrations changing between flying against and with the wind, where it flew a long mission perfectly…


[removed]

I fully understand vibrations are the first thing you check when somebody provides a log… But should vibration from wind cause it to suddenly raise like that?

As I checked my props, motors and bells before, put tape on it to minimize the vibration and was well below 30 without wind, I don’t think I have a general vibration issue and just strong wind should not lead to almost loosig my drone… (?)

Thx,
Sebastian

@Sebb,

The problem with high vibrations is that the accelerometers reach their limits and the EKF essentially loses information. It clips off the top or bottom of the accelerometer values which means the EKF doesn’t get the full acceleration value. Without the full acceleration value it cannot accurately determine whether the vehicle is climbing or descending and the vehicle can’t hold altitude probably.

In 4.0 (and higher) AP has a vibration failsafe to catch “climb aways” and it looks like this triggered in both logs. This likely kept the vehicle’s altitude under control but it can still oscillate quite a lot. During testing we saw oscillations of 40m at times but it didn’t keep climbing forever which is what happened in 3.6 (and earlier).

If all this sounds scary, please understand that this is the flip side of user expecting very steady position hold (both vertically and horizontally). This is only possible with reallyi good sensor data (and sensor fusion) which is not what the EKF is getting if the IMU is clipping due to high vibration.

The good news is that AP has this vibration failsafe which should stop the vehicle from climbing away completely and new IMUs have higher ranges than older IMUs which means clipping is less likely and so the overall problem is reduced.

Hello @rmackay9,

thanks for the explanation… I hope it’s okay to ask some more questions to fully understand the reason.

So, as far as I understand, that’s a common problem with vibrations and I’m not the first one having these issues :slight_smile:

I understand that the quadcopter loses data due to clipping and doesn’t know it’s height accurately anymore. But for me it’s still not clear why it goes to full throttle then. From what I understand, going full throttle would mean the quadcopter thinks, it’s losing height/not gaining height when raising throttle above hover throttle and trying to compensate?

I found a few possible fields for an estimated height:
AHR2.Alt, CTUN.Alt, POS.Alt

I graphed them, but I don’t see a sudden drop which would explain why it suddenly starts going to full throttle. Also IMU[0].AccZ / IMU[1].AccZ don’t show a lot of changes before…

But still I can see CTUN.ThO suddenly raising to 1 and also DCRt going up.

Also, when the trottle raised, this wasn’t reflected in the CTUN.Alt value. Is it raising ThO more and more because it doesn’t see a reaction on it’s throttle change in the CTUN.Alt value…?

Thx,
Sebastian

1 Like

@Sebb

Yes, you are not the first to hit this problem and it was perhaps my biggest safety concern before the vibration failsafe was added in 4.0. This issue is also why we added the real-time vibration reporting (which appears as the little “Vibe” field on MP’s HUD) and have multiple wiki pages explaining how to diagnose and potentially fix high vibration frames.

The altitude estimate is not the issue, instead it is the climb rate estimate that goes bad. In the logs you will likely see that the estimated climb rate becomes negative (e.g. the vehicle thinks it’s falling) while the altitude is going up. This contradiction is one of the criteria that triggers the vibration failsafe.

So in these situations the climb rate estimate is incorrect. The altitude hold controller sees the vehicle is apparently falling and increases throttle in response.

2 Likes

Hi @rmackay9,

The climb rate is CTUN.CRt, right? Indeed, I can see that value going negative, although the quadcopter raised quickly. But it seems the throttle already raised shortly before CRt went down and the falling CRt might then be caused by the vibration generated from the 100% throttle (?)

Is there a way to limit the max. throttle, especially in automatic modes, to e.g. 80%? The only limit possibility I found is MOT_BAT_CURR_MAX, which is probably not what I want, as I don’t want this in Acro mode…

Thanks,
Sebastian

@Sebb,

You can set the throttle range directly using the MOT_PWM_MIN and MOT_PWM_MAX parameters. Raising the throttle very often increases vibration levels because the major source of vibes is often the propellers passing over the arms. I don’t think this is a good long term solution though. I think it would be better to improve the vibration isolation for the autopilot (see wiki here)

1 Like

Hello @rmackay9,

thanks for your patience and answers :slight_smile:

I will check how to mount the FC more isolated. Currently I use these things
image
as space is limited…

However, at least I don’t have a general issue with my motors, I just tried it quickly inside, I’m usually well below 15 for vibration and clip is at 0.

But as I now know that these throttle issues can happen, especially during heavy wind, and there is a failsafe mechanism preventing it from flying away completely this that won’t decrease my confidence in my drone too much :slight_smile:

1 Like

ok, sounds good. I suspect the rubber dampers shown are too hard. I’m certainly not a mechanical engineer but I think harder substances only get rid of higher frequency vibrations. If it is too soft however then it will get rid of low frequency movements in the control band. mRobotics 3M foam or an autopilot with built in vibration isolation is generally my answer to these problems but I don’t fly really small frames.

1 Like