Hey @snktshrma ,
I am curious to know what led you to choose to create your own map instead of using satellite images to directly match incoming image frames.
Hi @snktshrma , i found this thesis http://d-scholarship.pitt.edu/41874/1/LiSanchez_etdPitt2021.pdf while looking for papers related to TRN(terrain relative navigation) .
This thesis could be quite beneficial in providing a deeper understanding of the finer details.
The thesis discusses dividing the TRN(the first approach, where we match the incoming query image to a prior map/ database of images) into two steps: coarse-level localization using a vector of localized descriptors and fine-level localization using SIFT features. Further, it also uses an approximate nearest neighbour search to find the closest match of the query image frame in the database at both the coarse and the fine level instead of the brute force way
Further, instead of storing a stitched map, the database stores visual words and descriptors, leading to low memory requirements.
2 Likes
Hi @radiant_bee! I went through the same idea of matching through satellite images but that was really intense on image processing part due to differences in scale, perspective, resolution, lighting conditions, and different times of acquisition. I was earlier trying with dsmac but right now relied on base level SIFT feature matching.
I will surely build upon that and get satellite image matching done as well as it will be much efficient and easier than forming maps or searching a database of images
Thanks a lot @radiant_bee for sharing this! I highly appreciate your suggestions and support!
This thesis seems to have some good approach and Iāll surely give it a thorough read and try implementing some of its techniques in my algorithm, specifically database optimisation.
Hi @snktshrma , how is your work on that topic going?
Meanwhile I have implemented a solution that works pretty well. It allows the ArduPilot position controller to Loiter stable near ground as well as on 100m above ground and has very low drift. This part does not require aerial images taken / prepared in advance.
Second part is a position detection in order to compensate drift of the optical flow. I also took the approach, to compare my live camera view with orthophotos using SIFT feature matching (only CPU based but with massive parallel processing). That part works well, when the georeferenced images for comparison are also taken and prepared in advance with my camera but not well, when I use images from other sources (e.g. orthophotos provided by Geo-Portals, Google sattelite images etc.). On that part I am still optimizing by using alternative approaches that should do a better job than SIFT:
1 Like
Hi @Hacky ! Thatās great know you got it working at 100m with multiple approaches. Will be interested in knowing your approach.
Yes, optical flow algorithm is also working for me good at high altitudes with position correction.
Currently I am trying with some satellite images positioning for high high altitudes and thatās working good. I am able to dead reckon and get position from satellite image. Drift is also pretty low and a low RMS error. Iāll be releasing that part soon and also how I implemented that.
If youād like to contribute along with this approach, let me know.
You might want to take a look at this also: GSoC 2024 wrapping up: High Altitude Non-GPS Navigation
Thanks, I also took a look at your āwrapping upā thread. As I would like to discuss some things further, where would you prefer to continue the discussion? Here in this thread or in the āwrapping upā thread?
Regarding the optical flow: I implemented a different algorithm and use it at about 4-5 Hz (it could also run at 30 Hz on a 1080p stream for vehicles operating at higher speeds) and after some calibration, I tested in a real flight at 90 m altitude on a windy day (gusts up to 12m/s) that after about 1.3 km waypoint flight (8 WP with changing directions), the drift was about 80m. Do you have some results from real flights with your system for comparison?
Would be really interesting to compare the implementations regarding drift and position stabilization in real life side-by-side!
Regarding the drift compensation, it is interesting that according to this PR, meanwhile there are 3 different message types allowing to reset the position without confusing EKF. Why did you decide for the MAV_CMD_EXTERNAL_POSITION_ESTIMATE message? Did you compare results with the other 2 alternatives mentioned by rmackey9?
Regarding fusion with IMU data:
Do you see improvements when doing this? In my opinion, it may make things worse if you do not have a good wind estimation because the wind based drift not recognized by the IMU is dominating.
Ya we continue here only. Really great to know that itās working for you. If you can share a log and the video, it would be really interesting to analyze it.
-
I donāt have results from real life, but if you are using a powerful companion computer, I am using position correction based on loop closure to correct the drift. Itās still in development as sometimes there are false positives and it can worsen the results.
-
They all serve the similar purpose. Other methods havenāt been merged and tested intensively yet. So I resorted to this one. Also position estimate is a bit delayed in my case, and hence I used MAV_CMD_EXTERNAL_POSITION_ESTIMATE.
-
Yes IMU improved the system. IMU data is back tracked to correct drift and dead reckoned for a very small fraction of time to compensate for processing time.
I am currently trying two different methods right now:
- Which doesnāt require pre-processed dataset of images
- Which needs satellite image or orthophoto of the area.
Also, I have posted an update here: GSoC 2024 wrapping up: High Altitude Non-GPS Navigation - #10 by snktshrma
Hi there!
Thanks @snktshrma for sharing this interesting thread and also other people for contribution!
As Iām working on similar solution(basically the aim-loiter in hight altitude) wanted also to share some results in this thread.
I successfully tested optical flow algorithm(locas-kanade from open cv) which was based on simple mavlink optical flow message(#100) on law altitube ~10meters,mode is loiter. Lidar was also connected,outdor range acorrding to the specification~10, so I didnāt go higher.
Basically everything was configured according to this documentation and log-based calibration, thanks @rmackay9 for detailed instructions.
Sharing some logs:
before the flight
IMU Gur.X and OF.X looks more or less precise, however there are occasional shifts at the
peaks
Then on the flight
the difference seems more significant and script apparently need lots of impovements.
However like i said testing was succesfull) the drone visually was motionless.
The couple of questions I wanted to bring up, eventhough they may seem very basic, but Iām still learing this topicā¦
Based on the logs If discrepancy in flight is very high does it mean that drone can stabilize just using imu + lidar data?
Can we use optical flow for navigation or its loiter only?
What should be taken into consideration when go on higer altitudes except lidar?
What are the benefits of other feature tracking algorithms?
1 Like
Hi @Ketchyn,
Comparing OF.flowX and OF.bodyX is important while doing the Log based Calibration and in this situation we expect the two to be close. While the vehicle is flying though we expect them to be different because this is how the EKF calculates how much the vehicle has moved horizontally.
thanks for sharing!
In this project, can the distance to the target point be calculated using the fov data of a camera and the height data of the drone?
1 Like
Hi @stewen ! Yes, you can calculate the distance if you have cameraās intrinsic parameters and the altitude of the drone. You can easily calculate the focal length using FOV and can get the distance using the following relation:
Hey Guys!
I desided to share one more test flight log here on a distance of 105 meters from the groud rather then creating new topic
Drone coud stabilize and hover perfectly but for short period of time ~2 min and then started drifting slowly on a side.
What I can see from the logs is body.x(gyro.x) larger flow.x which points to the thought of increasing scaler parameters a bit more.
One more obesrvation is gyro/flow spikes which might caused dron drift. But it Is hard to say if flow algorithm produced these spikes or its a gyro glith.
Would appriciate your thoughts @rmackay9 @snktshrma
1 Like
This gives the correct result if my camera is looking down at 90 degrees. So how do I calculate it when the angle of my camera is not 90? Is there a formula for this?