Accessing Camera Images via UDP or similar

ArduCopter Copter Simulation
1

I am currently using the camera sensor in gazebo to drop images to a directory at a specified frame rate.

I would rather access these images over UDP. I’m open to the format.

I have noticed that the PX4 sitl has a gstreamer camera plugin
libgazebo_gst_camera_plugin.so

Does something similar exist in Ardupilot sitl?

Note it doesn’t have to be gstreamer - I’m just trying to remove the whole drop to filesystem and read from filesystem inefficiency.

I’m asking what would be the best approach:

  • try and port the PX4_sitl plugin

  • edit the current camera sensor plugin and augment it to send images over UDP (instead of to the file system)

  • some other existing plugin that I am not aware of

  • some other way that I am not yet aware of.

Note: I want to run without ROS in the picture.

Hmmm maybe this is more of a question for the Gazebo forum? Perhaps I’ve confused myself because the gst_streamer plugin comes with PX4_sitl compilation.

2

Ok. I have solved my problem.

I’m using Ardupilot_sitl and I could not find anything in ardupilot sitl that referenced video streaming.
There is a gazebo plugin that I found available in PX4_sitl
So this is how I accessed it for use in gazebo/ardupilot_sitl

git clone https://github.com/PX4/PX4-Autopilot.git --recursive

Edit the build.sh file to get gazebo9 it changes depending on ubuntu version.
Later ubuntu versions will automatically want to build for gazebo11 etc
But we need a build for gazebo9 (or whatever you are running).
So edit to fix, then run

bash ./PX4-Autopilot/Tools/setup/ubuntu.sh —no-nuttx

Once build we need some plugins compiled which will only be compiled
for certain models - in this case the typhoon_h480 model

cd ./PX4-Autopilot
make px4_sitl gazebo_typhoon_h480

This will start gazebo with that model, just exit.
After that

cd build/px4_sitl_default/build_gazebo

You’ll find a bunch of plugins. You can either change your env variable to add this directory or
copy the ones you want to use to the place where gazebo looks for its plugins.

You’ll be interested in:

libgazebo_gst_camera_plugin.so
libgazebo_video_stream_widget.so
and possibly 
libgazebo_camera_manager_plugin.so  -- if you want to control from QGroundControl

In the your model.sdf where you have the vehicle camera you should add"

        <plugin name="GstCameraPlugin" filename="libgazebo_gst_camera_plugin.so">
            <robotNamespace></robotNamespace>
            <udpHost>ip-where-you-want-to-send-the-stream</udpHost>
            <udpPort>port-where-you-want-to-send-the-stream</udpPort>
        </plugin>

In the .world that you are invoking add:

<gui>
        <plugin name="video_widget" filename="libgazebo_video_stream_widget.so"/>
</gui>

This will make a gui button appear that allows you to stop start the video stream. Default is on.

If you are looking to add the camera manager plugin so that you can control the video feed with QGroundControl see the typhoon_h480.sdf for an example. I didn’t need this feature so I have not tested it.

Once you start gazebo with the above changes check that you have no errors and that the plugins have been found.

Then you can start a gstreamer pipeline to display the video,e.g.

gst-launch-1.0  -v udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' \
! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink fps-update-interval=1000 sync=false

Where port is the same as what you specified in the plugin .sdf. Its sending RTP with the video encoded as H264. You can do anything you want with it that a gstreamer pipeline allows.

Note: none of this is directly related to Ardupilot sitl since the camera and the plugin that streams are independent of ardupilot_sitl. However it appears to be a common need - based on previous unanswered questions. So I have posted here.

I even managed to confuse myself on this topic because the plugin was found in the sitl for PX4

You can find more background details at Redirecting to latest version of document (main)

1 Like
3

@geoff thank you so much for this “article”.
I’ve managed to run video via gstreamer. I used ROS prior to this and it’s very slow.
GStreamer is more robust and portable. ROS need loads of packages installed on system.

This is great. Thank you.
Zdanek