This is a project that I would like to initiate, and I am in search of a few talented developers to assist me with exporting INS/GNSS data for import to Novatel Inertial Explorer for tightly-coupled post-processing. In my mind, this serves two primary purposes; first, to use the trajectory to directly georeference images or LiDAR points on the earth for mapping, and second, to help improve or cross-compare ardupilot’s navigation algorithms with existing methods of post-processed state estimation. I believe the hardware is 95% there, but the process to get what we need isn’t. Ultimately, I’d like to see raw IMU and GNSS data extracted from the .BIN files in a format that works with Inertial Explorer. While we could use a standalone hardware device for this job, I think it would be wonderful to leverage the existing hardware on vehicles to do amazing stuff!
Novatel’s Inertial Explorer manual states that “In theory, virtually any IMU sensor can be used with Inertial Explorer. The only requirement is that the data be logged in the format provided in this section.” In addition, Inertial Explorer supports uBlox data. The format for these exports is here. https://docs.novatel.com/Waypoint/Content/PDFs/Waypoint_Software_User_Manual_OM-20000166.pdf
Inertial explorer should work just fine with the .ubx or RINEX files that can be generated from the logged ZED-F9P data, but we must make sure to get the correct messages in the GNSS log. I’m not sure exactly what those are, but I know ardupilot can configure these uBlox receivers accordingly. I plan to use a uBlox Zed-F9P through serial logging at 10Hz with RAW GNSS data recording turned on. My camera will send a pulse to the EXT_INT pin on the GNSS receiver that should generate a TM2 message. That message must certainly be recorded, and it currently isn’t. https://github.com/ArduPilot/ardupilot/issues/14400#issuecomment-694110125
Next, I’m concerned about clock drift on the autopilot boards. I’ve tested it and found it to be significant (tenths of seconds over a few hours, and my missions will be long). My understanding is that the way to correct clock drift is to use a PPS signal from the GNSS. I believe we may need some new architecture here to enable the proper timestamping of the 200Hz IMU data. That feature request is here: https://github.com/ArduPilot/ardupilot/issues/10369
This may be a paid project, and I am interested in having these features submitted as PR’s to ardupilot’s master branch so that this community can leverage this revolutionary technology for mapping or any other reason to extract the most accurate trajectory information available. If we do this right, we may be able to make accurate maps with very inexpensive hardware without having to conduct aerial triangulation, especially where pixel matching struggles. I think the relatively low-cost hardware on the market today can do some awesome things. I have Inertial Explorer and the hardware to test now. Please let me know how you could help and what it would take for me to work with you to get it done!
What we want to have and what we currently have access to are currently two separate things. I’m going to postulate that post-flight adjustment of the timestamps is possible, and we’ll try that first because we have the hardware and mechanisms to do that. Let’s assume we collect at 200Hz and the time drift is 0.1s per hour. We would be off by 20 samples in an hour. In otherwords, that’s one sample every 3 minutes. I don’t think it’s going to be an issue to squeeze or stretch the current sample timestamp a little bit to make it more accurate. We’re talking about 0.005s vs 0.0050277s, and I think as long as we provide inertial explorer with the correct timestamp, we should be good.
Now what you bring up is a good point with what may be necessary for live navigation and EKF consumption, but that’s separate from my needs here now.
The problem what is not easy to solve without physical (sync) connection between Pixhawk and F9P is the fact that IMU data is synced to processor time (say 200 Hz), while the F9P data inside the Rinex file is GPST ~+/-0.01 s, but when sending data to the Pixhawk on a serial bus it arrives after reception with ~0.2 sec delay. The F9P internally is able to determine GPST but it does not have clock steering function, hence sometimes it slips 0.01 sec, but more or less running parallel with GPST. The Pixhawk on the other hand has a cheap internal oscillator, what is sensitive to temperature, so after boot you could see a slow thermal drift & if you fly with your a/c you will see also change in temperature and an additional factor for diverging clocks. They can be aligned more or less sufficiently, but tight coupling tends to be rather sensitive for clock sync.
We already have physical accurate time sync through the Camera shutter feedback feature in ardupilot. I can either use just shutter feedback events as sync points, or I can use a PPS output from the GNSS as a consistent pulse to keep the sync regulated. 0.01s isn’t going to make or break this synchronization, and it all can happen in post-processing with adequate filters.
We did not have too much luck with connecting the flash hotshoe to both Pixhawk and F9P at the same time. We worked on it a bit but I guess we have not put too much effort in it. If you make it work and share logs (plus level arms, etc.) I would be happy to run some tightly integrated solutions (my PhD student has some in-house code, as well as running GiNAV).
Other solution what we have not explored is to route flash–>f9p (extint) and f9p PPS → Pixhawk. With this you could have a regularly spaced sync between GNSS and IMU, while the camera events are logged down to 20 ns accuracy.
Emlid Reach RS2 was the base station, Emlid M2 and a Sony RX1RII on the Elanus Duo aircraft. Each flight was about 35 minutes and took about 1250 images. You should find everything you need at the link. For some reason on each flight, the M2 didn’t pick up a few of the trigger releases, but the mRobotics Pixracer pro did. The hotshoe feedback wire was split to include a wire to the servos header. It was obvious what images were missing geotags based on my excel analysis. I’ll probably raise the issue on the Emlid forum.
How do things look? Based on the conversation in the last dev meeting, I think we’re going to have uBlox TM2 support built into ardupilot soon, so that’s exciting to get these events synchronized!
I did exact the same thing to have a shutter feedback on my Durandal FC as well but it doesn‘t pick up all triggers. Mostly when it comes to longer flights. I‘ve also raised the issue on emlid forums but the answer was that it works when I use only the original connection (M2 shutter count wrong - #21 by k.harbort - Hardware integration - Emlid Community Forum).
Also, the clock drift is a major problem for my Flarm device which sends and receives flight data from smaller airplanes which share the same airspace like me here in Germany. So for me it would have a double effect to have a spot on synched time.
I am not a programmer but I am glad to help with testing or delivering test data.
