In this project we use a ZED Stereo camera for 3D distance measurements and object detection. The following Arducopter methods are implemented:
- OBSTACLE_DISTANCE(Arducopter 4.0.x and AltHold mode)
- DISTANCE_SENSOR(Arducopter 4.0.x)
- OBSTACLE_DISTANCE_3D (Arducopter 4.1.x in Loiter, Guided, Auto and RTL modes)
The 3D point cloud generated by the ZED camera is processed on a Jetson TX2 and the resulting distance information and data on detected obstacles is transferred to a Pixhawk flightcontroller via mavlink. The Behaviour of the drone in case of detected obstacles ahead is handled in Arducopter’s object avoidance functionality.
In addition, a YOLOv4 based object detection is implemented.
All image processing is done by the Jetson TX2, exposed through different ROS image topics and available as RTSP streams to supported GCS like Solex or Mission Planner.
All further details, code and installation steps are explained in the wiki: https://github.com/mtbsteve/pegasus/wiki
and can be found on my github:
https://github.com/mtbsteve/pegasus
The project is based on the great work of Rishab Singh @rishabsingh3003 of the Arducopter Dev team https://github.com/rishabsingh3003/Vision-Obstacle-Avoidance
and the Stereolabs ZED-YOLO implementation (https://github.com/stereolabs/zed-yolo)
