Loss Of Altitude in Auto with Copter 3.5

ArduCopter Traditional Helicopter
7

It’s EK2_ALT_SOURCE which I have set to zero and it’s supposed to use the barometer. But obviously it’s not, because if it was it wouldn’t do what it does. The barometer readings in the logs are spot-on, agreeing with what’s actually happening. And the flight plan is not using the altitude information from the barometer, because if it was the helicopter wouldn’t be tending to run into trees that it’s supposed to be 60 feet over the top of.

There’s some other adjustments for there for the filter that reduces or increases weighting of the baro measurement, etc… Whatever is going on there does not work. Or the filter doesn’t work properly. Or it’s applying some other filter, or fusing of GPS, regardless of the EK2_ALT_SOURCE setting. Or something. Whatever it is, it is not safe to fly on an serious autonomous flight.

8

I could have sworn that there was a different parameter with an actual weighing value, but I cannot find it.

Changes in barometric pressure (and temperature variations influencing the onboard sensors on the pixhawks) is a fairly common issue on fixed wings, which can make auto-landings after longer missions an issue, but then we’re talking 1-2 meters, not 17!!.

9

Correct. I don’t get too excited about 1-2 meters because pressure buildup under the main rotor can cause that. But 7 meters off on a short flight, and 17 meters off on a long one - there’s something wrong with the altitude estimator. And the setting of zero for the alt source is obviously not limiting the altitude estimation to just the barometer. It’s being fused with some other sensor readings, and I suspect it’s GPS.

10

According the documentation:
EK2_ALT_SOURCE

This parameter controls which height sensor is used by the EKF. If the selected optionn cannot be used, it will default to Baro as the primary height source. Setting 0 will use the baro altitude at all times. Setting 1 uses the range finder and is only available in combination with optical flow navigation (EK2_GPS_TYPE = 3). Setting 2 uses GPS. When height sources other than Baro are in use, the offset between the Baro height and EKF height estimate is continually updated. If a switch to Baro height needs to be made when the filter is operating, then the Baro height is corrected for the learned offset to prevent a sudden step in height estimate.

If setting EK2_ALT_SOURCE to zero is using the Baro altitude, why is the helicopter not at the correct altitude when the Baro obviously shows it’s not at the correct altitude? Taking a snapshot of this one waypoint the command altitude value (flight plan) was 7.62 meters, which is 25 feet. The helicopter was actually at 1.5 meters, or 5 feet (my visual estimation was 1 meter, or about 3 feet).

Further, since the command altitude was 7.62 meters, why is the CTUN.DAlt only 1.58 meters? How can you have a desired relative altitude of 1.58 meters at that point and space in time when the flight plan is plainly 7.62 meters? Doesn’t make sense. Isn’t the flight plan command altitude the desired altitude?

Screenshot from 2017-10-10 22-45-55.jpeg991×428 77.2 KB

Tracing the GPS altitude estimate from the beginning of the flight to the end, the GPS estimate (height above the reference ellipsoid) agrees with the flight plan. None of the rest does except right at the beginning of the flight, and it went downhill from there (pardun the pun). So my conclusion is that it used GPS altitude for the entire flight and totally ignored the barometer. In other words, what the documentation says it’s supposed to do vs what it actually does in the real world is two different things. All the EKF2 settings are default:

EK2_ABIAS_P_NSE , 0.005
EK2_ACC_P_NSE , 0.6
EK2_ALT_M_NSE , 3
EK2_ALT_SOURCE , 0
EK2_BCN_DELAY , 50
EK2_BCN_I_GTE , 500
EK2_BCN_M_NSE , 1
EK2_CHECK_SCALE , 100
EK2_EAS_I_GATE , 400
EK2_EAS_M_NSE , 1.4
EK2_ENABLE , 1
EK2_FLOW_DELAY , 10
EK2_FLOW_I_GATE , 300
EK2_FLOW_M_NSE , 0.25
EK2_GBIAS_P_NSE , 0.0001
EK2_GLITCH_RAD , 25
EK2_GPS_CHECK , 31
EK2_GPS_DELAY , 220
EK2_GPS_TYPE , 0
EK2_GSCL_P_NSE , 0.0005
EK2_GYRO_P_NSE , 0.03
EK2_HGT_DELAY , 60
EK2_HGT_I_GATE , 500
EK2_IMU_MASK , 3
EK2_LOG_MASK , 1
EK2_MAGB_P_NSE , 0.0001
EK2_MAGE_P_NSE , 0.001
EK2_MAG_CAL , 3
EK2_MAG_I_GATE , 300
EK2_MAG_MASK , 0
EK2_MAG_M_NSE , 0.05
EK2_MAX_FLOW , 2.5
EK2_NOAID_M_NSE , 10
EK2_POSNE_M_NSE , 1
EK2_POS_I_GATE , 500
EK2_RNG_I_GATE , 500
EK2_RNG_M_NSE , 0.5
EK2_RNG_USE_HGT , -1
EK2_RNG_USE_SPD , 2
EK2_TAU_OUTPUT , 25
EK2_TERR_GRAD , 0.1
EK2_VELD_M_NSE , 0.7
EK2_VELNE_M_NSE , 0.5
EK2_VEL_I_GATE , 500
EK2_WIND_PSCALE , 0.5
EK2_WIND_P_NSE , 0.1
EK2_YAW_I_GATE , 300
EK2_YAW_M_NSE , 0.5

The maths that make this system tick are way over my head. But at the minimum I’m going to say whatever it’s trying to do has a serious anomaly.

11

Chris,
are you sure that when the mission was created that the mission was set in relative and not absolute heights?
That was my problem.

Sorry, I am editing this now.
I was away and did not follow this from the beginning. It is a small flat area. Forget what I wrote.

12

Yes. All the altitudes in the flight plan are relative to home. This particular flight is the best one to look at for analysis because it is on flat terrain. And it flies the same waypoints over and over again with a DO_JUMP with the altitude gradually bleeding off as it flies the same waypoints repeatedly. There is no barometer drift, no change in barometric pressure, no change in temperature, There is no problem with the hardware because I reverted to AC3.3.3 and flew the same flight without the issue happening. And the issue is repeatable. It happens at high flight speeds in missions with all spline waypoints making turns. The issue has been noted by multi pilots, as well, back to 3.4:

I’m pretty sure it is a flaw in the EKF2 altitude estimator with an error that grows over time flying under these conditions. Since most people don’t fly that way it’s not reported often enough to worry about. And plane has a different implementation than copter has. And the documentation is inaccurate. The altitude in auto obviously does not use the barometer in 3.5.

13

Found out that Terrain Spline Waypoints work.
Maintained altitude perfectly.
AC 3.5.2

2017-10-11.jpg1056×594 200 KB

14

Same here. If I fly a mission where there is limited turning there is no issue. For me it only happens if the speed gets high enough to where the heli “cuts off” waypoints (due to speed and acceleration limits) and continuous turning on the mission. I think I have enough data on it to be able to submit a proper issue on GitHub so the EKF experts can take a look at it. I’m thinking some kind of error or “leak” in the code that causes a gradual buildup of the error with a resulting decay of the altitude estimate. I think it’s showing the desired altitude at the final waypoint as being way low because the error gradually “leaked” to where it thought the sensors drifted that far, resulting in the altitude estimate also drifting.

I eliminated hardware issues by reverting to 3.3.3 and flying the same mission with no issues. And it is repeatable on the mission illustrated above, although the drift in altitude is not always the same amount.

When I looked at your issue before @pmshop I was convinced it had to be something else. Because the logs don’t really show what’s going on or indicate any error. Everything looks rosy - right up to the point where the aircraft flies into the ground. The ground station indicates proper altitude, and everything, during the flight. If I went back and looked over your logs now, with what I have learned by analyzing mine, I’m betting I’d see the same anomaly. It was much harder to pick out on your flights because it was over changing terrain elevation. My documented “problem flight” is on perfectly flat terrain where the decay in the actual altitude can be easier seen thru the baro readings, visual reference, and onboard camera reference…

1 Like
15

@ChrisOlson Please provide a log and I will have a look at it. From the description and the fact the Baro alt tracks the GPS alt in flight, it could be that the function that slowly adjusts the EKF origin to maintain consistency with GPS is responsible. This behaviour was modified by PR https://github.com/ArduPilot/ardupilot/pull/6288 which made it into 3.5.3.

This PR enabled the origin height adjustment to be disabled via the EK2_OGN_HGT_MASK parameter. It should be disabled if EK2_OGN_HGT_MASK = 0.

16

@ChrisOlson given that your parameter list doesn’t contain the EK2_OGN_HGT_MASK parameter, you must be using 3.5.2 or earlier. If so, then updating to 3.5.3 is recommended.

17

@priseborough
Thanks for taking a look. I wanted to make sure it’s not a simple settings problem.

I’m definitely flying 3.5.3. But you are correct in that there is no EK2_OGN_HGT_MASK param.

This is the BIN log from the described flight:
https://drive.google.com/open?id=0B5oLpNzxih4tT1dlOVBhcXNGVkE

This is the tlog from QGroundControl:
https://drive.google.com/open?id=0B5oLpNzxih4tNlRMQlJmZGl0Sk0

This is the flight plan from QGroundControl:
https://drive.google.com/open?id=0B5oLpNzxih4tLWg0YldZY2tzUEk

Note the flight plan does not use a speed command. WPNAV_SPEED is adjusted on-the-fly with Channel 6. The flight plan is also in QGroundControl’s new JSON file format. If you would like that exported in the more common .txt format I probably have it in another computer with APM Planner2.

The current params are in the BIN log. But I got those in another computer too. I’ll upload them as soon as I get it out of the other PC.

18

@priseborough I just connected with the flight controller to export the params. QGroundControl identifies the firmware version as 3.5.3

Screenshot_2017-10-12-17-49-05.png1368×768 125 KB

But unfortunately I do not see that EK2_OGN_HGT_MASK param in the list

Screenshot_2017-10-12-17-49-54.png1368×768 164 KB

This is the params exported from the Flight Controller…
Scratch that one. Tower has an issue exporting params sometimes. I’ll export from QGroundControl and upload it.

QGroundControl has some extra fields in the params file, but it still should be readable by any text editor. And interestingly it identifies the vehicle type as a multi-rotor instead of a heli frame in the comments header:
https://drive.google.com/open?id=0B5oLpNzxih4tbWhBbEY3VmxxTzA

19

Paul, I read thru the review on that PR. And it was merged into master (3.6-dev). But did it actually get into 3.5 backports? I checked out the 3.5 branch and I don’t see in there for 3.5.3.

20

OK, my mistake. I saw the Copter 3.5.3 release commit in master and assumed that was the same as AC3.5.3 release. It isn’t. The AC3.5.3 release is in a separate branch and does not contain https://github.com/ArduPilot/ardupilot/pull/6288.

The log data is consistent with an EKF origin that is slowly changing through flight to be consistent with the GPS. The EKF height is tracking the baro correctly and the local vertical position is tracking the control system demand. Both are trending down in height, but the average GPS height remains constant through the mission.

Screen Shot 2017-10-13 at 10.22.44 am.png844×452 44.9 KB

@rmackay9 We need to get that PR into AC3.5.4 ASAP

1 Like
21

Thanks for taking a look, Paul. At least it’s been addressed and just a matter of getting the fix into the mainstream release.

At first I thought it wasn’t using the baro. But I did notice the baro tracking the vertical position (DAlt) and that’s what raised an alarm that it was an EKF issue related to the height at arming. And that height remaining consistent with the GPS while the control system went down. That’s why the heli was actually at ~1.5 meters at the final waypoint - the control system (DAlt) said it should be there. What I didn’t know was why. Now, after reading the PR thru I have a better understanding of it.

22

Yes, the EKF uses local NED coordinates internally and publishes a local position and an WGS-84 origin taken from GPS. The navigation code converts that information back to global coordinates. The EKF2 in 3.5.3 is slowly adjusting the published height of it’s origin to be consistent with the GPS and while that may give the most accurate WGS-84 estimate output, it is incompatible with how the data is being used by the control loops.

If a relative height to takeoff location is required, then the origin height at takeoff should be used throughout flight when converting from EKF to nav coordinates.

This issue was debated at length and in the end https://github.com/ArduPilot/ardupilot/pull/6288 was added to give flexibility in dealing with Baro vs GPs drift in a way that met the requirements of the different applications.

For example for long duration missions , Baro drift can be a bigger problem than GPS drift.

23

Cool. And yeah, that’s true. I’ve seen a baro drift 10 meters in 10 minutes in cold weather!

24

@priseborough I’m going to build your changes on my fork and test this on 3.5.3 if that’s ok.

25

This is a crazy parallel, but I mentioned this issue to an avid aviation buff and he said it sounds like the same symptoms the Airbus 380s had on auto pilot. Gradually lost altitude over 200 miles or something like that.
I found that totally bizarre.

26

Full size aircraft are a little different in that respect in that they all use barometric pressure with the altimeter set to 29.92 above FL180. And it is an encoding altimeter connected to the transponder to report altitude to ATC. Altitude hunting is usually the result of not having the gain set properly on the autopilot’s altitude hold function. Some variation is normal as long as IFR separation minimums are maintained. If they aren’t ATC is on the radio telling the pilots they’re a thousand low or whatever, and can they please break off their card game long enough to get the airplane back to cruising altitude.