We are trying to use the new create_terrain.py python script to pre-load the ‘NxxExxx.DAT’ terrain files onto a Pixhawk so that we do not need to upload terrain height information during a flight. However, we found that once we had copied the files to APM/TERRAIN folder on theSD card, ArduPlane was overwriting them with incorrect data. As a further test, we deleted the terrain files and let ArduPlane generate the .DAT files by requesting terrain information via Mavlink. It appears that these files also contain errors.
I wrote a python script to visualize the terrain files that are being generated by the create_terrain.py as well as by ArduPilot to demonstrate the issue:
Using our location as an example S35E018, we obtained two S35E018.DAT files, one generated by the create_terrain.py script and the other from ArduPlane (with an initially empty APM/TERRAIN folder). Going through the lat/lon data contained in the S35E018.DAT file generated by ArduPlane, it seems like most of the blocks being loaded in are from 1 longitudinal degree East of the correct tile (S35E019), and some of the smaller groups of blocks are being loaded from 1 latitudinal degree north of the correct location (S34E018).
Here is the plot of the S35E018.DAT file generated by the script (which is correct):
For anyone else wanting to better understand the terrain file structure, here is a diagram of what I roughly understand the structure to be (there may be mistakes in here so please let me know if there are changes needed):
There is a bug with terrain file handling in that version (as your pictures show). It is fixed in ArduPlane 4.0.6 and later.
There is a terrain file server over at https://terrain.ardupilot.org/ which has the correct tiles. You can use this to pre-load tile on to the SD card.
EDIT:
I don’t think 4.0.6 has been released quite yet.
It looks like ArduPlane 4.0.6 is no longer overwriting the files that are copied to the SD card. However, when the SD card is cleared and ArduPlane generates the .DAT files, it seems to be generating some unreadable blocks.
My understanding is that the .DAT files consist of enough 2048-byte blocks to cover the 1 lat/lon degree for the file, where each 2048-byte block consists of the following data:
TYPE NAME BYTES
---------------------------------------
uint64_t bitmap [0:8]
int32_t lat [8:12]
int32_t lon [12:16]
uint16_t crc [16:18]
uint16_t version [18:20]
uint16_t spacing [20:22]
int16_t[28x32] height [22:1814]
uint16_t grid_idx_x [1814:1816]
uint16_t grid_idx_y [1816:1818]
int16_t lon_degrees [1818:1820]
int8_t lat_degrees [1820:1821]
trailer of 227 bytes [1821:2048]
When the .DAT files are generated by ArduPlane, are they simply meant to contain 2048-byte blocks or is there other information that is included in these files? They don’t seem to be generated in the same way as the create_terrain.py script as, for instance, the blocks generated by ArduPlane are not ordered. I was finding that the first 48 2048-byte blocks in the file that was generated by ArduPlane 4.0.6 were invalid; however, the remaining blocks in the file were correct. Is this perhaps some header information?
@LiamClarkZA Do you by chance still have the plot_terrain.py file somewhere?
Also your graphic is really useful and deserves to be on the wiki somewhere!
