Thanks,
The current gazebo simulation supports udp video streaming at port 5600 (if i am not wrong). for this specific project i have a script that converts udp stream to rtsp stream and then the tracking script consumes it.
Thanks for your response.
Did you follow ardupilot-gazebo instructions for adding a camera? Or using any other document?
Could you receive udp video before using your conversion?
In my situation I see test below after using this command gst-launch-1.0 -v udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false:
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Got context from element ‘autovideosink0’: gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)“(GstGLDisplayWayland)\ gldisplaywayland0”;
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
But nothing happen after this.
Try following these steps:
gz sim -v4 -r iris_runway.sdf
gz topic -t /world/iris_runway/model/iris_with_gimbal/model/gimbal/link/pitch_link/sensor/camera/image/enable_streaming -m gz.msgs.Boolean -p "data: 1"
gst-launch-1.0 -v udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false
I used these commands but no difference happened 
A gear icon(something like a setting icon) appears, but I can’t see the video stream.
Edit: When I use second command this error appears in Gazebo console:
(gz sim server:4042): GLib-GObject-CRITICAL **: 10:55:05.139: g_object_set: assertion 'G_IS_OBJECT (object)' failed
[Err] [GstCameraPlugin.cc:421] GstCameraPlugin: failed to create GStreamer elements
Maybe gstreamer plugin is not working. Firstly check you are on the master branch and you have the latest commits. (Master works for me well). Second you can try to update gstreamer libraries.
The version i am using is here
gst-launch-1.0 version 1.20.3
GStreamer 1.20.3
also you can have a look at gst library installations here. Just ignore the opencv library build commands in there.
Thanks a lot for your help.
My gstreamer version is also 1.20.3. I followed the installations you linked above(except opencv), but unfortunately, nothing changed in the result.
1 Like
Hi @khanasif786 ,
I successfully compiled and ran all the source files, but I’m having an issue with the send_camera_information.py script.
I manually set up outputs 14560 and 14570 in MAVProxy, and the send_camera_information script connects successfully.
However, in the MAVProxy console, I can’t see any indication of the camera being connected.
Any ideas? 
Actually, I am having trouble adding a new device with sysid 245.
Update:
I can’t see the ap_camera message in the mavproxy console at all.
I even wrote a script to manually retest.
There will not be any indication right now for camera being connected or disconnected. With the standalone scripts.
Can you tell me exactly which message are you talking about? BTW you can use QGC or MissionPlanner also to watch messages and their publish rates
dear @khanasif786
The list of things I did is as follows:
First, I launched the Gazebo simulator with the following command:
gz sim -v4 -r iris_runway.sdf
Starting SITL (Software In The Loop):
Then, I ran SITL using the command below:
sim_vehicle.py -v ArduCopter -f gazebo-iris --add-param-file=$HOME/ardupilot_gazebo/config/gazebo-iris-gimbal.parm --model JSON --map --console
Enabling Image Streaming:
After confirming that everything was running correctly, I enabled image streaming with this command:
gz topic -t /world/iris_runway/model/iris_with_gimbal/model/gimbal/link/pitch_link/sensor/camera/image/enable_streaming -m gz.msgs.Boolean -p "data: 1"
Converting UDP Stream to RTSP:
I used the gst_udp_to_rtsp script to convert the UDP stream into an RTSP stream.
Running the Tracking Script:
After that, I executed the tracking.py script to start the tracking process.
Fixing MAVProxy Communication Issue:
I noticed that MAVProxy wasn’t sending data on port 14570, so I added the output manually using the following command in the MAVProxy console:
output add 127.0.0.1:14570
After this step, the tracking program started working correctly, and I could see the output in the frame window.
Sending Camera Information:
To send camera information, I ran the following command:
python3 send_camera_information.py --sysid 245 --compid 0 --resh 640 --resv 480 &
This script required a MAVLink connection. To establish it, I added another output in MAVProxy:
output add 127.0.0.1:14560
After this step, the script connected successfully, and I could see the message “Camera information message sent”.
Observing MAVLink Messages:
I noticed that the MAVLink messages didn’t show the camera information as expected. In the test video, there was a proper output like this:
AP_Camera: camera vendor 1232.32 camera model 123.32 fw 1.0.0.1
However, I didn’t see anything like that in my setup, even though it was shown in the video.
Oh okay, maybe you are missing something, Can you try running run_all.bash script there. Also you must be having the same parameters set as i have put, Either you can download my parameter file manually and then upload it, it has the networking setup also, so you don’t need to add that “output add ” command.
The thing is camera information message is surely sent but not received by the SITL or the SITL discarded it. If the SITL has the correct parameter set then it should work, Also don’t use mavlink forwarding instead use the NET parameters to communicate. You can see those into this file
Thank you, everything works fine. Have you considered adding Zoom?
Not yet. Hopefully will do it soon.
1 Like
For future comers, if anyone gets problem in udp stream make sure to get the output of
gz topic -l
and then publish 1 on the enable_steaming topic like this
gz topic -t <topic_name> -m gz.msgs.Boolean -p "data: 1"
So apparently it was
gz topic -t /world/iris_runway/model/iris_with_gimbal/model/gimbal/link/pitch_link/sensor/camera/image/enable_streaming -m gz.msgs.Boolean -p "data: 1"
1 Like
This is my work with the capabilities to select any object in the see field of the drone. The selection is done from the ground station and tracking it by using the SOT algorithm.
Have you tested the csrt and kcf tracking algorithms here on Jetson? I am trying the same thing, but there is a lot of delay in Jetson and A8 mini. I would be glad if you can help me.
I really want to use this code. For example, if I have a catamaran with twin propellers on the water surface, after automatically identifying objects such as floating debris on the water surface, it can move towards the identified object. There is a net bag installed at the bottom of the boat, so that it can intelligently identify and clean up the garbage on the water surface. Is this feasible?
1 Like
Hello @stewen
Yeah, i was also having the same problem what you can do instead is using a low latency pipeline for gstreamer. See this is what i used in my case (you can make use of the parameters which is used) gst-launch-1.0 rtspsrc location=rtsp://192.168.144.25:8554/main.264 latency=50 ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false
Also one more thing for the QGC part the HDMi stream works with good latency you can make use of that on ground station. But after a fresh build of QGC on herelink master i was not having much latency on the rtsp stream (ethernet one) so right now i am using only the Ethernet port stream both on GCS and the jetson.
hello @admired
Its totally feasible!! but not from the approach i am using, the approach i am using is for Air vehicles that is totally dependent on the terrain data and you know the terrain data is not perfect. What you have to do is some kind of navigation technique for the sea or ground vehicles. But its interesting to do this. let me know if you are having any plan.