LAND mode climbing uncontrollably

Hello,

Two separate quads had remarkably similar problems running version to version 4.1 at the exact location.

Short version: Quad goes into LAND mode and goes up uncontrollably. (log attached)

Long version:

  1. First quad: This one had been flying for two or three months. A low-quality board (omnibus f4 pro clone). It was a quad racer and flew very well. After bouncing on the ground due to a lousy maneuver, it started to climb and climb and never came back… We lost the camera feed at an altitude of over 1000m++ above the ground.

As the quad was not mine, I have no previous logs; I just witnessed that the OSD screen informed that the quad was in LAND mode while climbing out of control.

  1. Second quad: This is mine, and the log is attached.

This quad uses a Matek h743 slim and was flying very well. After I removed the anti-vibration plate, I went to test it, and the vibration definitely became a problem.

From the log, I understand that the vibration was responsible for the EKF switching and the subsequent switches to LAND mode.

However, I do not understand why the quad shot upwards at both LAND mode times.

Please, can someone help me with the interpretation? Could both quads have the same problem?

Obs 1: the flying field is approximately 1km above the sea.

Obs 2: the RTL and althold modes in the log were tentatives to get the quad back to home.

Obs 3: The quad is fine! :wink:

log: https://drive.google.com/file/d/15UXOW8gw3NTDmK5ZJMAigVIYIGAZb-3V/view?usp=sharing

@brunoolivieri,

Really sorry about the loss of one of the copters. The issue is very likely that the vibration failsafe is broken in Copter-4.1.0-beta4 (since -beta1). We discovered this problem a week ago and have a fix that will be released with -beta5 later this week. Sorry, we don’t normally issue alerts for beta software but I probably should have put one out for this issue.

… back to the logs, the vibration levels are well above the acceptable range of 30m/s/s at times and there is are huge numbers of clipping events which leads to the EKF not getting the complete acceleration info which causes it to lose its climb rate estimate.

The problem is that this can cause a feedback loop where the altitude controllers increase the throttle to counter the apparent (but incorrect) fast descent which leads to even higher vibration.

Land mode relies on the altitude controller so without the vibration failsafe the best solution is for the pilot to retake control in stabilize mode.

I’m sure you’ve seen the wiki pages on Measuring Vibration and Vibration Dampening but just in case.

P.S. I’ve put a warning at the top of the -beta4 release message and on facebook.

2 Likes

This causing on stable arducopter 4.0.7 also?

2 Likes

@kalai1219,

No, stable (Copter-4.0.7 and earlier) is fine. The vibration failsafe was only broke in -beta1 to -beta4. Sorry if my comment was unclear so I will fix that, txs.

2 Likes

@rmackay9
Thank you so much for your clear clarification.

1 Like

Thanks Randy!

Regarding the vibration, yes, I am aware of that. It was a test to check it… :frowning:

Regarding the beta-5, I am anxious to install it in some quads. Actually, I need it (lol). As I am relying on the use of S-Curve, I cannot

I am assembling some quads for a small project and doing several autotunes over the next couple of weeks on about 4 different frames. If I can do a field test of something specific, please let me know, and I’ll insert it into the process.

Thanks @brunoolivieri,

-beta5 has just gone out now so feel free to give it a try.

If you’ve got that vehicle with high vibrations around still you could try it out on -beta5 to see if it does any better. In general, with high vibration levels, we expect bad altitude hold performance and the vehicle could even climb 40 to 50m but it shouldn’t climb forever. Just be ready to takeover in stabilize mode of course. No pressure though, thanks!

I have the quad in hand and was going to reassemble it to insert anti-vibration pads. I need it soon, but I can do a test this Friday.

@rmackay9: Please confirm that I understand the test correctly:

  1. Updated to beta-5;
  2. I fly in some altitude-assisted flight mode (example: althold or poshold);
  3. I do a flight that shakes the vehicle to force vibration problems;
  4. The failsafe goes in;

If all goes well:

  1. The vehicle climbs quite a bit but still at a comfortable height (less than 100m for sure);
  2. The vehicle lands on its own;

If it doesn’t work:

Putting the vehicle in Stabilize should overwrite the failsafe and give me command over it.

@brunoolivieri,

Yes, that test sounds good. The lack of vibration foam on the vehicle is important and you may also need to do a rapid climb to increase the throttle to push the vibration levels over the limits.

Of course please be careful… if you can’t do the test safely then please don’t worry, we will undoubtedly have other incidents of high vibration.

So… I did the test.

I flew, shaking the quad: https://youtu.be/qAlvvVsTRQc

Visually my impression was that everything happened in the same way as the previous flight. I do not identify that the quad went into Land mode in the logs, but it climbed a lot in Althold.

I confess that visually the impression we had was that it went much higher than the 70 meters of the log. I and some friends usually fly there, and we share such an impression.

I made four markings of ascents below. From left to right:

1st) Small unwanted climb.

2nd) Uncontrolled climb as in the accident in the first post/log. At this point, the log indicates 70 meters, but it looked much more.

3rd) I really think that I understand what was happening at the time, and I confess I had understood that the quad climbed alone. Still, reviewing the log, I fear it may have been me in Stabilize.

4th) This fourth climb I really don’t think was me because when it happened, I put the quad in stab to try to recover it for good, and then I knocked it down.

image

image

In my defense, the climbs were after erros:

image

The logs: https://drive.google.com/file/d/1nyT3uPiD3SEa17Dm1KIcANg0YnuN886I/view?usp=sharing

Unfortunately, this quad is out of service… It was pretty tough in the crash and will fly again with a few repairs, but I will need to remove the vibration to use it.

1 Like

@rmackay9 I guess the problem still present. Am I wrong?

Hi @brunoolivieri,

Thanks for the testing and sorry about the crash.

There are three climbs in the log and as you thought, two of them were in stabilize mode so we can ignore these two (pilot directly controls the throttle in these modes).

The first climb shows the vibration compensation is working although the vehicle climbs 60m before its altitude is brought under control and it starts to descend. After that I think the pilot took over in Stabilize mode (I’m sure it was a bit of a scary event) so we can’t know what would have happened but it didn’t climb endlessly which is good. Both GPS and baro agree the vehicle climbs about 60m so I think it is accurate.

The climb of 60m is a bit more than I was expecting but the vehicle is also very powerful so when the motors go to full power it climbs at about 8m/s…

I’ve asked @priseborough to have a look at the log too and in particular to look at the EKF3’s climb rate estimate which is very jagged.

Thanks again for the test. I’m sure it was a bit scary and the results aren’t perfect but without the vibration failsafe the vehicle would have just kept on climbing until it ran out of batteries.

3 Likes

@brunoolivieri,

@priseborough had a look at the logs and confirmed that it seems to be working correctly. We also discussed the very jagged climb rate shown in green in the screen shot above and determined that this is OK and is caused by high vibrations affecting the the “forward propagation” step.

To provide some more detail, the EKF has two time horizons, one is about 0.2 seconds in the past (required so that we can fuse data from sensors with different time lags) and the other is now. All the hardwork happens at the older time horizon but finally the acceleromter value are used to essentially extrapolate from the older time to now. The high vibrations levels and clipping make this forward extrapolation result very noisy.

Paul and I are really thankful that you produced this log. txs!

1 Like

That’s really nice of you! I’m the grateful one to both of you.

Thanks for the details, as well as this, helps in future diagnoses.

I have six quads that need to use v4.1beta because of receivers (f.port) and S-Curve. I would be pleased to test something from the beta backlog as long as it is valuable or a priority for you.

ps.: I am having some issues with telemetry and f.port. I still need to read the problems with my boards in that thread, but I am afraid that Matek H743 does not send telemetry as expected. I am using several of them.

1 Like

@rmackay9, @priseborough and @brunoolivieri. Not sure if this is related with what we experienced months ago and reported here:


If yes, I daresay that we were able to solve it modifying the portion of the code that ignores the barometer if the innovation is too high. Thus, we forced the barometer to always being fused and that solved the issue. It tried to happen a couple times after that; we tested a lot. However, forcing the barometer to be fused allowed the drone to recover a healthy altitude estimation. When the innovation reaches the threshold in which the EKF stops fusing the barometer, the drone trusts entirely in the accelerometer for the altitude estimation. The accelerometer by itself is not reliable for estimating the altitude.

The innovation was already growing uncontrollably as in our case even before the pilot hit landing mode.

This may be because it has not been fusing the barometer.

1 Like