Has anyone flown a fixed wing just with vision/VIO?

ArduPlane Plane 4.6
1

Hey everyone, I was wondering if anyone has successfully attempted to fly a fixed wing purely based on vision navigation/VIO and if so how did you go about it and what lessons did you learn?

1 Like
2

I haven’t personally tried it practically, but I did do a lot of academic research on SLAM for aerial platforms and explored some datasets of visual data from fixed wing UAS.

A lot of this information will depend on your flight envelope - velocity, altitude, roll/pitch angles.

  1. Maybe obvious but whatever camera you use should have global shutter. Rolling shutter will result in blurring or other artefacts since the camera may be moving at significant speed.
  2. You will get different results depending on the altitude and speed that you’re flying at. Since the algorithms work by extracting features from the frame, the features will persist and be easier to track if you’re flying slower or higher.
  3. The consequence of flying higher, however, is that the feature extraction may become more difficult… Trade offs.
  4. Make sure your camera system also has an appropriate frame rate, resolution, field of view for your envelope. Higher resolution for higher altitude, higher frame rate for lower altitude or higher velocity, and so on.
  5. I would recommend using a stereo camera system (especially since you suggest ‘pure’ visual or visual-inertial). This allows for depth (altitude) estimation. With a fixed wing platform you can get a bit creative and increase the baseline between your camera sensors to improve the range of your depth estimates (e.g. by moving the sensors out toward the wingtips (maybe complicated if you’re using a COTS stereo camera package))

These are the things that come to mind for me when trying to gather visual navigation information on a fixed wing platform. The biggest thing is (as often is the case) that it is highly dependent on your specific use case.
Do you have any estimates for your flight velocity? altitude? any restrictions on the hardware? cost constraints on the camera system?

3

I would be very careful putting cameras on wingtips as wings depending on loads and structure design can flex enough to make distance estimation unreliable.

4

yes fair point. i’ve seen it achieved in papers, but of course it’s something that needs to be implemented with careful thought