I was writing a script to string multiple waypoint missions together by loading them sequentially from the SD card when I noticed a peculiarity in the example script method for loading waypoints.
The example scripts read the text files line by line, using a table (called
data) to store each set of waypoint parameters. The latitude/longitude integer values are set like this (where data fields 9 and 10 are in floating point decimal degrees):
Unfortunately, that’s only good to about 5 decimal places (~1m accuracy), and waypoints for my use case require near centimeter accuracy - the full 7 decimal places.
Fortunately, it appears that you can overcome the issue by simply manipulating the string values, like this:
-- the AP Lua implementation appears very susceptible to floating point math errors -- this function takes a decimal number in string format and returns an integer -- representation of its value * 10^7 (as in ArduPilot lat/lng arguments) -- without using any floating point math (takes *much* longer) function strf_to_stri(x) local whole, fraction = x:match("(.+)%.(.+)") if (fraction == nil) then whole = x fraction = '00000000' end if (string.len(fraction) > 8) then fraction = fraction:sub(1, 6 - string.len(fraction)) end fraction = fraction .. string.rep('0', 8 - string.len(fraction)) return whole .. fraction:sub(1, -2) end
If you’re about to say something like, “But Yuri, just use
string.format()!,” I tried that. Same precision error.
I should probably lop off the 8th fractional character a little sooner and improve the overall efficiency of the function. I was accounting for the fact that the typical input to the function is expected to have 8 decimal places (as in Mission Planner’s saved waypoint files).