Hi everyone,
We had this very weird behavior: the drone triggered an EKF Failsafe followed by a GPS Glitch message right after the takeoff. This happened with this specific drone, out of 8 other similar units. In discussion with the Ardupilot team, they asked about the compass calibration and suggested some values from a magfit calculation (thank you so much @peterbarker !).
After adjusting the compass parameters with the suggested values, we had a second event in a row, identical to the first one. With the log from the second event, Peter noted the follwing:
your DroneCAN GPS units - what sort of GPS are they? They seem to be lying terribly about their accuracies, and not agreeing with each other on where your vehicle is! Your config has them 60cm apart, but the position they’re feeding back into the vehicle is ~5m apart! They are claiming to have 1.3m/1.6m accuracies. The EKF doesn’t mind getting noisy data - but if you systematically lie to it as these GPS units are doing it does tend to get annoyed. Could you supply some photos of your GPS install, please? Do you have any transmitters co-located with the GPSs?
We do have 4G antennas nearby, but this is alse pretty much the standard for the rest of the fleet and also very similar to other models we have. The GPSs are mounted on the lateral arms of an hexa.
Then I asked if mechanical vibrations could be the cause, and Peter replied:
When the vehicle starts to move it does rapidly start to get errors. This could be vibration, but I discussed that with tridge earlier and we thought that unlikely (you need very high vibration frequecies to annoy the clock crystals on the GPS units for it to happen AFAIK).
One of the GPS units has it’s cable running too close to the main power cable (where all the drone’s electrical current is flowing), and indeed the GPS cable extension could have the shielding compromise. In fact, you can see the offset before takeoff and the latitude value diverging exactly when the current raises:
So basically I see three hypothesis:
- Very high frequency vibration on this specific drone, caused by some difference on the airframe;
- Nasty interference from the main power cables;
- Faulty GPS hardware (one of them).
Our next step will be either re-routing the GPS cable or using a soft mount to compare. If anybody can see any other clue in the attached logs, please let me know.
Setup:
- Cube Orange
- Dual Here3 using blend configuration
- 25kg MTOW Hexa - T-MOTOR MN705 propulsion
Important notes
- The drone did some successful flights before, but ErSc values were always higher than usual;
- The GPS receivers are hardmounted, but just like the rest of the fleet;
- The satellite number was normal for both receivers;
- The position the GPS receivers are feeding back into the vehicle is ~5m apart! They are claiming to have 1.3m/1.6m accuracies;
- 00022.BIN is the log from the previous normal takeoff;
- 00023.BIN is the first event;
- 00024.BIN is the second event.
LOGS Link
https://drive.google.com/drive/folders/15FbpIOYZ08wimz85zc-zOyiR1OlXT71l?usp=sharing
