How to Properly Simulate Vision-Based Navigation in SITL Without a Real Camera?

ArduCopter
,
1

Hi everyone,

We’re currently working on a GPS-denied visual navigation system for a quadcopter and are trying to validate our pipeline using SITL before moving to hardware testing. We’ve made decent progress, but we’re running into EKF stability issues and would really appreciate some guidance from people who have worked with ExternalNav or vision pipelines.

We are developing a vision-based localization system that works like this:

  • We generate a pre-stitched orthomosaic map of the operational area.

  • The drone uses a downward-facing camera.

  • We run SIFT feature matching between live camera frames and the orthomosaic.

  • The estimated pose is sent to ArduPilot via:

    :backhand_index_pointing_right: VISION_POSITION_ESTIMATE MAVLink message

  • EKF3 then uses this as the primary position source in GPS-denied mode.

Final Goal

  1. Takeoff using GPS

  2. Switch to vision-only navigation

  3. Perform hover / waypoint navigation

  4. Switch back to GPS and land

. attaching a pdf file so understand about architecture and how the pipeline works

ArduPilot_Vision_Navigation_SITL_Post.pdf (5.2 KB)