Project: 3D Obstacle avoidance with a Stereolabs ZED cam

In this project we use a ZED Stereo camera for 3D distance measurements and object detection. The following Arducopter methods are implemented:

  • OBSTACLE_DISTANCE(Arducopter 4.0.x and AltHold mode)
  • DISTANCE_SENSOR(Arducopter 4.0.x)
  • OBSTACLE_DISTANCE_3D (Arducopter 4.1.x in Loiter, Guided, Auto and RTL modes)

The 3D point cloud generated by the ZED camera is processed on a Jetson TX2 and the resulting distance information and data on detected obstacles is transferred to a Pixhawk flightcontroller via mavlink. The Behaviour of the drone in case of detected obstacles ahead is handled in Arducopter’s object avoidance functionality.

In addition, a YOLOv4 based object detection is implemented.

All image processing is done by the Jetson TX2, exposed through different ROS image topics and available as RTSP streams to supported GCS like Solex or Mission Planner.

All further details, code and installation steps are explained in the wiki:
and can be found on my github:

The project is based on the great work of Rishab Singh @rishabsingh3003 of the Arducopter Dev team
and the Stereolabs ZED-YOLO implementation (