Hi @Jazx_Jazx - a full answer to your question will probably take a few posts, but the first part asking to integrate a depth camera into your simulated drone should not be too difficult.
Gazebo Garden and Harmonic support depth cameras directly, so you may not need to used the one from the plugin (you may need to tune the camera intrinsics to match, which would be for a follow up).
I’ve added a branch to ardupilot_gazebo here: GitHub - srmainwaring/ardupilot_gazebo-1 at prs/pr-all-camera-sensors that adds all the available camera sensors to the gimbal mount.
The example is run using
gz sim -v4 -r gimbal.sdf
You can inspect the behaviour of each sensor by selecting the appropriate topcis in the Image Display.
Figure: depth camera
Figure: rgbd camera
Figure: thermal camera
Figure: wide angle camera
Figure: panoptic segmentation camera
The next step is to integrate the data from the camera topics with ArduPilot. For that we’ll probably want to use the tools available in ROS to carry out any image manipulation required, or at least convert the image data into a format recognised by ROS which will be more accessible to other tools. For this we’ll use the ros_gz bridge - more on that in the next post.