Unhealthy GPS signal when using "GPS for Yaw"?

ArduCopter Copter 4.1
,
1

Yesterday, I had a first test-flight using the “GPS for Yaw” feature of ArduCopter 4.1.0-beta1. Hardware setup uses two ublox ZED F9P chips with helical antennas mounted with 70 cm distance.

Before going into the air, I noticed that a single board computer (with a CPU working in the range of 1.5 Ghz to 2.5 GHz) mounted also on the frame seems to interfere with GPS. It takes longer to get “RTK fixed” status (status = 6) on GPS2 (as required for a stable GPS based heading) and quite often it went down to “rtk float” (status = 5) or even 3dgps (status = 4). So I decided to shutdown this computer and now the GPS fix status was most of the time stable on “RTK fixed” level (status = 6).

grafik
grafik1617×641 63.6 KB

I was also using RTK correction data via NTRIP from a local service, so most of the time also first GPS was on “RTK fixed” status (status = 6):

grafik
grafik1601×637 62.9 KB

The flight was good, I noticed no issues. Position was held very stable and when I moved the sticks, all reactions were as expected. There was no yaw drift during that flight.

Number of satellites was between 24 and 27 and hdop was between 1.2 and 0.8 on GPS1 and between 0.6 and 0.4 on GPS2. Nevertheless, I got quite often “Unhealthy GPS” warnings spoken on Mission Planner and I did not see a reason for that. There are over 6000 GPS messages in the log, for about 10 minutes flight, this means 5 Hz for each GPS, so there should not be a communication problem.
Also, when you look into the log file, there are a lot of events of type “GPS_PRIMARY_CHANGED” (82 events in 10 minutes flight).

grafik
grafik1615×643 97.3 KB

When you look at the GPS track on the map, it also looks quite astounding:

grafik
grafik809×808 153 KB

You can see a zigzag track but it was not the way, like the drone was flying. It was flying very smooth, so I assume, that this zigzag is just an issue with the way, how Mission Planners log analyzer interprets a track when you have two GPS mounted in a significant distance.

I guess, this is not intended?

EDIT: You can download the onboard log here: https://kopterkraft.com/downloads-static/2021-05-15%2012-10-29.bin

1 Like
2

@Hacky,

Thanks for the report.

Re the zigzag in the MP log browser’s map, I also found this bug a few months ago and Michael Oborne has since fixed it. I’m pretty sure this will be resolved if you update the MP on your machine by clicking on the “Help” menu option and then press either “Check for Updates” or “Check for BETA Updates”. Then wait a few minutes and restart MP.

3

@Hacky,

I also had a peek at the GPS messages in the logs and it looks like the 2nd GPS isn’t keeping up with the 5hz requirement. So below is a screenshot of the GPS messages (for just the 2nd GPS) extracted and we see a 6 second period where it was only providing data at 1hz.

image
image777×530 36.9 KB

If you want to check for yourself you can also see the gap using MP after filtering for only GPS messages (click on the column headers and select, “GPS”). In the screen shot below you can see the “I” column is mostly “0” with only a few "1"s. This shows that the 1st GPS is updating at 5hz but the 2nd is not.

image
image1672×879 64.7 KB

4

The issue with the 2nd GPS (aka the “rover”) not providing updates at 5hz is very likely caused by the autopilot not being able to send the RTCM data from the 1st GPS (the base) through to the 2nd GPS quickly enough.

Getting the data flowing through quickly enough seems to be slightly difficult to setup because the autopilot needs to use “DMA” for the serial ports connected to the GPSs and which serial ports have DMA is difficult to know although there is apparently a way to figure it out by downloading a file from the autopilot using MAVFTP. How to do this is not documented! :slight_smile:

An alternative may be to directly connect the two GPSs with a serial cable but we don’t yet have information on the wiki about how to do this. We will add this soon-ish … hopefully within the next week or so.

5

@rmackay9
Thanks for your analysis, really appreciated.
Regarding the zigzag, I will check behaviour with Mission Planner update 1.3.74.1.

Regarding the 2nd GPS occasionally not keeping up with 5Hz update rate, it sounds likely that it is caused by the data flow going through the autopilot. Surely it would be a better option to unload this task from the autopilot. If you could give me a tentative description (by PN) how to set up the direct communication (meanwhile I am quite familiar with F9P modules, u-center etc.), I could give it a try and report the results.

Nevertheless, I think an update rate of 1 Hz from the second GPS is not really critical, as long as this GPS is only used for heading calculation. To my understanding the main source for heading still is the gyro information and the double GPS is only used to set an initial heading and to compensate gyro drift. So in this case the “Unhealthy GPS” warnings suggest a more critical situation than it actually is.

A bit more concerning for me is the large number of “GPS_PRIMARY_CHANGED" events (82 times within 10 minutes flight). Perhaps this is an aftereffect of the reduced update rate?

6

Hi @Hacky,

I’ve discussed with @tridge about GPS-for-yaw and the issues you’re seeing are all related I think

  • The change in primary GPS happens when we lose the heading estimate from the GPS. This is caused by a limitation in our EKF which requires the position, velocity and yaw all come from the same GPS. When we’re using GPS-for-yaw we want to get the yaw from the 2nd GPS (the rover) but this forces the EKF to use the 2nd GPS for position and velocity as well. When we lose the GPS-for-yaw we switch back to the 1st GPS (the base) because it actually provides a better position and velocity than the 2nd GPS. We have it on the to-do list to allow the position+velocity and yaw to come from different GPSs and this will avoid the switching
  • the 1HZ update from the 2nd GPS will stop GPS-for-yaw from working temporarily. This won’t immediately cause problems for the EKF’s heading estimate because the gyros drift very slowly but if it continues for too long the the vehicle’s heading will drift.
  • You are aboslutely right that directly connecting the two GPSs together is a good idea because it avoids sending the data through the autopilot which can suffer from serial bandwidth problems. We have it on our wiki to-do list to add the setup instruction. Personally I also don’t know how to do this.

By the way, there is a relevant discussion going on in the Rover-4.1 category here.

7

Probably this is the best way for UART F9Ps.

image
image774×1007 225 KB

1 Like
8

Thanks @rmackay9,
I read through the discussion at ArduRover 4.1 but there were some statements that confuse me:

They discuss the direct connection between the base and rover modules in combination with GPS_DRV_OPTIONS=0 but they still report “Unhealthy GPS” warnings. As I understood you, these warnings are caused by the high amount of RTCM traffic going through the flight controller between base and rover modules - which should not happen anymore when using GPS_DRV_OPTIONS=0 and direct wiring.

Additionally, I can report a first successfull “real” flight (not just simple test hovering) with beta3. Before that, I had to do some work on the antenna configuration in order to improve the portion of “RTK fixed” state. This is important because otherwise, if there is too much time on “3dgps” or “rtk float” status, you risk yaw drift and currently there seems to be no fallback to magnetometer in these cases. The flight was about 15 minutes with only a few (may be two or three times) “Unhealty GPS warnings”.

In my configuration there is a lot of UART traffic (I use all 5 UARTs and had to desolder the ADS-B module lines of the carrier board in order to get the fifth UART available), but Pixhawk Orange seems to be able to catch up with the traffic. There are 2 UARTs used for GPS, 1 for “herelink” MAVLink, 1 for companion system and 1 for the gimbal (also contrllerd over serial).

9

Hacky,

Unrelated to your other report, -beta4 includes a fix to the DMA setup on CubeOrange (and possibly other autopilots) that may improve throughput and might possibly help the 2nd GPS maintain its 5hz update rate. I’m really not sure but maybe… so perhaps you could check and report back whether you notice any improvement or not?

10

I will switch to beta4 and report…

1 Like
11

@Hacky,

If you’re still working with GPS-for-Yaw we’ve got an improvement to the serial performance on some boards (including CubeOrange). We are not planning to include this in Copter-4.1.0 (but maybe 4.1.1) but you could test with “latest” and/or I could provide you a modified version of Copter-4.1.0 if you’d like to test it.

12

@rmackay9
Thank you for the offer. I would prefer a modified Copter-4.1.0 version (Cube Orange).

13

I think we have the same problem, on a cube orange with a couple of F9P chips: the GPS Health for GPS2 (the rover in the setup and the source of yaw) is flickering and GPS2 is switching between 4 and 6 (DGPS and RTK fixed). We have the same setup with the RTCM stream going through ardupilot.

I see that rmackay9 said that a fix is in the plans. Did it happen?
Thanks,
Mihai

14

This has not been problematic in quite some time. Are you outside with a clear sky view? And are you using (EDIT) Copter 4.4.0 or newer?

15

You should be using ArduCopter 4.4.0

16

You should update the firmware to the latest version, I was also testing GPS for yaw recently.

17

I was hoping that it has been taken care of - I’m glad to hear that it did. I do have a rather new firmware version (namely 4.3.6), but no, I don’t have 4.4.0. About the sky, yes, very clear (at a field with nothing nearby): the base GPS reports 20 satellites and the rover 25 - both use F9P with dual band antennas.

We can surely try 4.4.0.
Thanks for the suggestion,
Mihai

18

Hi Mihai,

I faced a similar problem and I tried extending the serial stream rate by using the SRX_EXT_STAT

After using this I don’t face the problem during flight or when arming.

19

As far as I know, it’s easy to lose satellites when the F9P doesn’t have adequate power.

20

CUAV - we don’t lose satellites - while throwing the error, the satellite count is solid (sometimes even increasing). Thank you Dhanesh - that’s certainly something that could be the problem (i.e., possibly information from too many satellites to forward, and the serial becoming a bottleneck).

I’d like to try it - what exactly did you change (and where)? Did you changed it “down”, or “up”? And to what value if you could share?

Looking in a bit more detail at SRx_EXT_STAT, it seems that’s a mavlink telemetry parameter. Is that how the RTCM updates make it from the GPS base to the GPS rover?! As MAVLink_RTK messages?

Thanks a lot!
Mihai