One of the great pleasures of being part of the ArduPilot project is the opportunity to test some interesting hardware. That opportunity came up recently when Holybro offered to send me a selection of their new GPS range. In mid-2019 I did initial tests of GPS (really GNSS) receivers from Drotek based on the (at the time) new u-blox F9P module, and I was delighted by the results. Since then I’ve been even more delighted to see more vendors coming out with modules based on the F9P.
After receiving the Holybro GPS modules and antennas I looked around and found that I had 12 different GPS types that I could test, so I thought it would be a good opportunity to do some direct comparisons between all the modules I had to see how much difference there was between them.
The 12 modules I had were:
- Holybro F9P with helical antenna (tri-band CH7604A)
- Holybro F9P with Harxon GPS500 tri-band survey antenna
- Holybro M8P with helical antenna (CH3602A)
- Holybro F9P Rover Lite (with builtin tri-band antenna)
- Drotek F9P with a single band external antenna
- mRo M9N with builtin antenna
- Hex Here+ M8P with builtin antenna
- Matek M8 CAN with builtin antenna
- Allystar HD8040D L5 capable NMEA GPS with builtin antenna
- CUAV M8 CAN GPS with builtin antenna
- Hitec Mosaic Septentrio CAN GPS with Tallysman tri-band antenna
- 3DRobotics M6 GPS with builtin antenna
After starting on the testing I also received one of the new mRobotics F9P modules with builtin antenna. I wasn’t able to include this in the full testing but did do some testing of this against 3 other modules after the main tests.
The older GPS modules (especially the 3DR M6 and the Here+) are included because many people in the ArduPilot community may be familiar with them. They have been replaced by newer modules and are probably hard to buy now, but I thought for people looking to update their GPS they may find the comparison useful.
All but two of the above receivers are based on u-blox modules. The two non-ublox modules are both experimental, and I included them as I was curious how they would do. The Allystar is an extremely low cost (around $20) GPS which is capable of receiving L5 signals.
The L5 band is a very promising band and should allow for much better GPS performance in the future, but is not fully deployed yet. Having L5 in such a low cost module was intriguing, but it should be considered experimental, both because L5 is not in full service yet and because I needed a test firmware from Allystar to get it working with ArduPilot (to support vertical velocity which ArduPilot needs).
The Hitec Mosaic module is also very promising, but at the time I did the testing in this posting the firmware was still being developed. Results will likely improve a lot before the module is released. This module is based on AP_Periph, offering UAVCAN support as well as direct uart access and uses the Septentrio Mosaic chipset, which has a very good reputation.
Holybro GPS Modules
The reason I started this testing was the arrival of a really good selection of GPS modules from Holybro. They sent me both M8P and F9P based modules, with a range of antennas.
These are very well made GPS modules with nice features. For my use case my favourite is definitely the F9P with helical antenna. The antenna is detachable (it uses a SMA connector), allowing it to either be used with a different antenna or you can use a SMA extension cable to place the antenna where you need it in the vehicle. The USB port makes updating the u-blox firmware easy (this is very important with the relatively new F9P module). The helical antenna is a good choice for many vehicles as it can get good performance without having to add an extra ground plane. This is particularly nice for dual-GPS setups where you using moving baseline RTK support to provide yaw without the need for a magnetometer (this is sometimes called a “GPS Compass”).
The F9P and M8P modules that I received came with uart cables suitable for the primary GPS connector on the Holybro range of flight controllers. I gave Holybro some feedback suggesting that an additional 6 pin JST-GH adapter cable would be a big benefit as it would allow two of the modules to be put in a single flight controller, and would also allow them to be used with flight controllers from other vendors. Holybro agreed and told me that a 6 pin JST-GH cable would be included in the future. It is great having a responsive vendor!
The “Rover Lite F9P” GPS was larger than I expected (75mm diameter). It has a more traditional form factor with builtin patch antenna. It has a single cable with a connector suitable for the primary GPS connector on the Holybro range of flight controllers. This makes it less flexible than the modules that come with the helical antennas both because of the fixed antenna, plus the inability to use it as a secondary GPS without cutting the cable and soldering on your own cable. Updating the firmware on the u-blox F9P will also be more difficult without the USB port.
Holybro also included a Harxon GPS500 survey antenna with SMA cable. I was interested to compare it’s performance with the much smaller helical antennas. More on that later.
Testing Methodology
One of the difficulties with GPS testing is trying to make the comparison fair. Ideally I would test all modules with all antenna types (where the antenna is swappable). Ideally I would also have each GPS module under test in exactly the same position at exactly the same time (so as to get the same atmospheric conditions, satellite positions and multi-path).
The tests also should be conducted over a long period of time, preferably 24 hours for each test so that the full range of satellite geometries is tested. Finally the modules under test should all have identical power setups and be connected to the same flight controller.
I couldn’t hit all of these points in the time available and with the hardware I had, but I tried to come as close as I could. I setup 7 flight controllers, all running the same version of ArduPilot plane firmware. The flight controllers were not all the same (I used boards from 4 different vendors), but they were all powered from a common power supply which fed 7 seperate power modules via chained XT-60 connectors.
All the modules were placed on one medium sized plastic table, with the spacing between the modules enough to minimise ground plane interactions.
I chose two locations for testing. One was in my backyard which has a moderate number of trees to simulate a “poor” location for GPS reception. The other was at ANUs Spring Valley farm to the west of Canberra, with full sky view on a beautiful spring day.
For each of the two locations I did two long test runs:
- one run without any source of correction information, so the modules did not get any RTK lock
- one run with RTCMv3 from a nearby high quality NTRIP station provided by Geoscience Australia on the Australian CORS network. This allowed modules that were RTK capable to get RTK float or fixed locks.
The whole setup was definitely not perfect, but I believe it is good enough to provide some useful results.
Comparing results with CEP and HEP
In my previous postings on GPS performance I mostly used graphs showing error and GPS reported accuracy numbers to compare the two GPS modules I was testing. With 12 modules that sort of comparison becomes more difficult and less useful. For these results I will show “CEP” and “HEP” results for each of the four test runs with each module. I will drill down into graphs of if requested in the comments.
The CEP of a GPS module is “Circular Error Probable”. It is a statistical measure that works like this:
- find the median position reported by the GPS module over the test run
- find the radius around that median position which encloses 50% of the points. This is the “CEP:50”
- find the radius around the median position which encloses 99% of the points. This is the “CEP:99”.
If you get a value of 1.2 meters for CEP:50 then that means that 50% of all the points in the test run fell within 1.2 meters of the median point. In other words, the CEP:50 value is the accuracy you should do better than “most of the time”. The CEP:99 value is the accuracy you should do better than “almost all the time”.
As I tested each module in 4 different situations, that would produce 8 CEP values per GPS module. In practice there are more CEP values than that, as for modules which are RTK capable it is not really useful to compare the CEP when the module has a RTK-fixed lock to the times when it has a RTK-float lock. To address this I showed separate CEP values for RTK-float and RTK-fixed time periods for these tests. I also show the percentage of time that the module had RTK-fixed solution compared to a RTK-float solution. How much of the time the module is able to get a fixed RTK solution is really the key to good RTK performance.
Ok, so that is CEP in a nutshell, so what is HEP? HEP is just the same as CEP but for height, so it is “Height Error Probable”. The median height over the data is found, then the difference from that median height that includes 50% and 99% of the points is calculated. As GPS modules typically produce less accurate height than horizontal position it is common that the HEP is larger than the CEP. Which is more important depends on your application.
In either case the golden rule with CEP and HEP results is “smaller is better”.
Backyard Test - no RTK
This test was conducted in my backyard, without any RTK correction data. The location in my backyard has a moderate amount of tree cover, with approximately clear 50% sky view. This test ran over 14 hours. The results below are sorted by CEP50, with better results first.
| Module | CEP50(m) | CEP99(m) | HEP50(m) | HEP99(m) |
|---|---|---|---|---|
| Holybro F9P Rover Lite | 0.73 | 2.60 | 1.39 | 5.01 |
| Holybro F9P Helical | 0.84 | 2.97 | 1.64 | 4.97 |
| Holybro F9P Harxon | 0.94 | 2.93 | 1.52 | 5.57 |
| Hitec Mosaic Tallysman | 0.98 | 2.51 | 1.25 | 8.03 |
| mRobotics M9N | 1.56 | 4.43 | 2.21 | 8.16 |
| Drotek F9P Single | 1.61 | 3.99 | 2.16 | 8.16 |
| Holybro M8P Helical | 1.64 | 4.81 | 2.57 | 10.57 |
| AllyStar L5 | 1.69 | 4.17 | 2.30 | 10.30 |
| Matek M8 CAN | 2.26 | 6.92 | 2.85 | 10.95 |
| 3DR M6 | 2.61 | 10.06 | 3.68 | 22.42 |
| Hex Here+ | 2.70 | 9.76 | 3.54 | 14.43 |
| CUAV M8 CAN | 3.02 | 9.98 | 3.76 | 14.83 |
These results shows a few things very clearly:
- the multi-band GPS modules (F9P and Mosaic) perform a lot better than single band modules, even without RTK corrections
- within the multi-band GPS modules, using a multi-band antenna really helps (eg. Drotek F9P does relatively poorly and it had a single band antenna attached)
Neither of these results are surprising at all.
Also note the wide range of HEP99 results. The newer GPS modules do greatly reduce vertical error.
A note of caution is also worthwhile with the results. You may think that if you see a CEP99 of 9 meters that you are sure to get a GPS position within 9 meters when it actually had a few points out to well over 20 meters of error. That last 1% of points can include some really large outliers.
Backyard Test with RTK Corrections
The next set of tests was in the same location, but with RTCMv3 corrections provided by a local (distance under 3km) high quality NTRIP station. This test was conducted over 11 hours.
For these tests the results below show two rows for GPS modules that are RTK capable. The first row shows the CEP/HEP for the periods of the test where the module had a RTK-Float position. This happens when the GPS is unable to obtain the much more accurate RTK-Fixed lock. The 2nd row is for the time when the GPS had a RTK-Fixed position. On that row I also show the percentage of the test that it had a RTK-Fixed position. The best GPS modules had RTK-Fixed positions for the whole test, and for those there is only one line.
The results are sorted first by the amount of time they have a RTK-Fixed position, followed by the CEP50 result.
| Module | RTK | CEP50(m) | CEP99(m) | HEP50(m) | HEP99(m) | Percent Fixed |
|---|---|---|---|---|---|---|
| Holybro F9P Helical | Fixed | 0.01 | 0.03 | 0.01 | 0.06 | 100% |
| Holybro F9P Harxon | Fixed | 0.01 | 0.04 | 0.02 | 0.09 | 100% |
| Hitec Mosaic Tallysman | Float | 0.19 | 2.45 | 0.14 | 4.76 | |
| Hitec Mosaic Tallysman | Fixed | 0.01 | 0.03 | 0.02 | 0.13 | 98% |
| Holybro F9P Rover Lite | Float | 0.04 | 1.46 | 0.03 | 1.58 | |
| Holybro F9P Rover Lite | Fixed | 0.01 | 0.04 | 0.02 | 0.08 | 98% |
| Drotek F9P Single | Float | 0.08 | 1.83 | 0.08 | 2.63 | |
| Drotek F9P Single | Fixed | 0.01 | 0.07 | 0.02 | 0.26 | 91% |
| Holybro M8P Helical | Float | 0.07 | 1.61 | 0.14 | 2.14 | |
| Holybro M8P Helical | Fixed | 0.02 | 0.06 | 0.02 | 0.14 | 73% |
| Hex Here+ | Float | 0.08 | 0.50 | 0.07 | 1.65 | |
| Hex Here+ | Fixed | 0.02 | 0.06 | 0.03 | 0.34 | 41% |
| AllyStar L5 | - | 1.32 | 3.69 | 1.90 | 8.50 | |
| mRobotics M9N | - | 1.85 | 5.24 | 3.43 | 12.66 | |
| Matek M8 CAN | - | 2.18 | 6.51 | 3.17 | 13.43 | |
| CUAV M8 CAN | - | 2.61 | 7.46 | 2.94 | 14.43 | |
| 3DR M6 | - | 2.63 | 8.99 | 3.39 | 19.22 |
The results really show the huge difference that providing RTK correction data gives. CEP values are nearly 80x better than without corrections, down close to 1cm is achieved by the best performing modules. The two Holybro F9P modules really shine in this test, with 100% RTK-Fixed lock, despite the trees and 50% sky view. The Drotek F9P also does well given it only had a single band antenna (I really wish I had another tri-band antenna to give it, and my apologies to Drotek for making this module look bad when it is really quite good). The Hitec Mosaic does quite well, but doesn’t get 100% RTK-Fixed lock. Hopefully by the time the final module comes out it will really show what a Septentrio can do.
The Holybro M8P does better than the Here+, possibly due to antenna differences in this environment. They are both single band modules with the older u-blox M8P modules, and if you have a choice between a M8P and a F9P (ie. you can afford the newer modules) then you should definitely use the F9P.
Within the non-RTK GPS modules in this test the standout was the AllyStar, taking advantage of the L5 signal. I would caution though that it is considered very experimental, and I did notice the AllyStar L5 sometimes gave really silly results, like claiming to have a 3D fix with 15+ satellites but getting the height wrong by over 500 meters. It didn’t do that in this test run, but did for some other testing I did on another day.
Spring Valley - no RTK
The next set of tests were out at the Australian National Universities Spring Valley robotics facility. It was truly wonderful weather, and sitting out in the paddock among the wildflowers was a pleasure. This was a great location to test GPS modules in ideal conditions, a fairly flat paddock with full sky view.
The first test at Spring Valley was without RTK corrections. The results were not surprising.
| Module | CEP50(m) | CEP99(m) | HEP50(m) | HEP99(m) |
|---|---|---|---|---|
| Holybro F9P Helical | 0.30 | 0.77 | 0.30 | 1.37 |
| Holybro F9P Harxon | 0.37 | 1.16 | 0.45 | 1.57 |
| Drotek F9P Single | 0.51 | 1.25 | 0.99 | 3.07 |
| Holybro F9P Rover Lite | 0.59 | 1.17 | 0.70 | 4.39 |
| mRobotics M9N | 0.61 | 1.87 | 1.15 | 6.63 |
| Hitec Mosaic Tallysman | 0.68 | 2.46 | 0.61 | 3.76 |
| AllyStar L5 | 0.74 | 3.18 | 0.90 | 3.20 |
| Holybro M8P Helical | 0.74 | 2.11 | 0.87 | 3.32 |
| 3DR M6 | 1.09 | 3.25 | 1.85 | 5.71 |
| Hex Here+ | 1.36 | 4.09 | 1.38 | 5.44 |
| CUAV M8 CAN | 1.52 | 11.00 | 2.23 | 19.21 |
| Matek M8 CAN | 1.98 | 5.85 | 1.68 | 11.46 |
All the GPS modules did significantly better than when they were under some trees, with the CEP about 2x better. The improvement in HEP was even greater, around 4x better for some GPS modules. Vertical position really is more of a problem than horizontal when you don’t have full sky view.
Spring Valley with RTK Corrections
The final test was at Spring Valley with RTK corrections from a nearby high quality NTRIP station.
| Module | RTK | CEP50(m) | CEP99(m) | HEP50(m) | HEP99(m) | Percent Fixed |
|---|---|---|---|---|---|---|
| Holybro F9P Rover Lite | Fixed | 0.01 | 0.02 | 0.01 | 0.04 | 100% |
| Holybro F9P Helical | Fixed | 0.01 | 0.02 | 0.01 | 0.05 | 100% |
| Holybro F9P Harxon | Fixed | 0.01 | 0.02 | 0.01 | 0.05 | 100% |
| Hitec Mosaic Tallysman | Fixed | 0.05 | 0.13 | 0.06 | 0.20 | 100% |
| Drotek F9P Single | Float | 0.12 | 0.31 | 0.24 | 0.36 | |
| Drotek F9P Single | Fixed | 0.01 | 0.03 | 0.01 | 0.05 | 99% |
| Holybro M8P Helical | Float | 0.10 | 0.18 | 0.02 | 0.18 | |
| Holybro M8P Helical | Fixed | 0.01 | 0.03 | 0.01 | 0.04 | 98% |
| Hex Here+ | Float | 0.07 | 0.40 | 0.06 | 0.40 | |
| Hex Here+ | Fixed | 0.01 | 0.04 | 0.02 | 0.07 | 82% |
| AllyStar L5 | - | 0.75 | 2.11 | 1.20 | 4.20 | |
| mRobotics M9N | - | 0.76 | 2.68 | 2.13 | 6.49 | |
| 3DR M6 | - | 1.39 | 3.91 | 1.62 | 8.15 | |
| CUAV M8 CAN | - | 1.86 | 5.15 | 2.63 | 8.73 | |
| Matek M8 CAN | - | 1.94 | 6.61 | 3.39 | 10.38 |
This is the test with the best conditions for GPS modules and it really does allow them to shine. Getting 100% RTK-Fixed and 99% of positions within 2cm of the median really is a testament to the incredible power of the GNSS system that we use every day. A few years back getting this type of result would have cost a small fortune, and now it is within reach of a lot of users.
As expected the triple-band capable GPS modules do better, but even the M8P based modules do pretty well in this test, the main difference being that they lose the RTK-Fixed solution and fall back to RTK-Float a lot more often.
mRobotics F9P
A late arrival to my set of test GPS modules was the new mRobotics F9P.
This is a large module with a big groundplane (about 75mm aquare). I would have liked to redo my testing with the full complement of GPS modules and with this GPS included as I think this module will be of interest to quite a few people. Unfortunately I didn’t have time, so I just did a smaller test including 4 GPS modules to see how it stacks up against a few others.
| Module | CEP50(m) | CEP99(m) | HEP50(m) | HEP99(m) |
|---|---|---|---|---|
| Holybro F9P Helical | 0.52 | 1.36 | 0.87 | 7.38 |
| mRobotics F9P | 0.61 | 2.14 | 1.26 | 4.47 |
| AllyStar L5 | 1.26 | 3.61 | 1.90 | 7.60 |
| Holybro F9P Rover Lite | 1.27 | 2.77 | 2.09 | 6.47 |
This test was performed in my backyard, under the same conditions as the original non-RTK backyard test (ie. with about 50% sky view). The results show it performs almost as well as the Holybro F9P Helical. I’d also note that the Holybro F9P Rover Lite was a bit disappointing in this test, not doing as well as it did in previous tests. I don’t have a clear explanation for this change. If I find out more I’ll add to this post.
Final Thoughts
Apart from the raw results, there were a few other things I thought worth mentioning that came out of these tests.
The first is the importance of good power for GPS modules, especially for the F9P based modules. My initial power setup was not good, with some flight controllers running at around 4.8V. I found the F9P based modules did quite badly and after looking into possible causes I found that by ensuring all flight controllers had at least 5V the performance was a lot better. If you have a F9P GPS then be sure to check your power. You can see the board voltage in most boards by looking at the POWR.Vcc messages in onboard logs or in the POWER_STATUS mavlink message in your ground station. If you are dipping below 5V then it would be worth fixing your power setup.
The second is the importance of being able to update the firmware on your GPS, especially for F9P based modules. Some of the F9P based modules arrived with an older firmware and there have been a lot of improvements in the F9P firmware since the module was released. If you are not using the 1.13 u-blox firmware then you should definitely update. Having a USB port on the GPS really helps with this. Otherwise contact your vendor with instructions on updating.
I hope you have found this post useful and happy flying!
