UDP video stream in MP HUD?

Is there an established way for Mission Planner to use gstreamer UDP video in the HUD? For example, the video stream from a Solo or other companion computer. It seems now that MP can only see an actual video device on the PC. Perhaps there is a third party application that can receive the gstreamer UDP video stream and emulate a hardware webcam that MP could see. Maybe ManyCam can do it?

MP already does this.
just install gstreamer to a default location
I suggest 1.9.2 gstreamer.
and if the remote video source sends to port 5600 MP will automatically display it on the hud.

Well that was simpler than I thought it would be. Lol. Thank you, I will try this when I get home.

I’m having trouble getting thus working, I’ve tried several different versions of gstreamer, including 1.9.3. If I launch gstreamer from the command line I get a feed in a separate window from mp. I only get one or two lines in the cmd window that comes up from mission planner. The only thing I can think of is the mp gsteamer is using different switches than I am. Is there a way to change those settings?

I think the issue is with how picky the Solo video stream can be. Only one device can access that video pipeline. And the video will not even start until it detects an App connected. So when I start MP in the right sequence, the video works.

I’m using an rpi to do essentially the same thing with video streaming as the solo does, my pi is forwarding mavlink along with mavproxy then when someone connects to the mavlink it starts a stream to that device using get-launch on the pi and raspivid. Like I said I get the gstreamer windowto pop open when the steam starts, and the pipeline opens, but it stops getting video almost immediately. If I close the pipeline that mission planner launches and open one manually I get the stream in a separate gstreamer window without fail but would really like to have the hud overlayed on this video.

I’m not using or seeing any other windows or prompts. When I open MP, the video is just there. No gstreamer windows or actions necessary.

I’m using a slightly older version of Mission planner, 1.43, the 1.45 version never opened a window nor did it do anything when I clicked the gstream button in the F menu so I thought that it was broken. Perhaps opening the seperate window has been disabled for cleanliness. I can’t seem to get it to work in any version of MP.

Ok, I’ve kicked the tires on this and used several versions of MP and several versions of Gstreamer over the last week or so and it appears that the magic combination is that mission planner is actively listening for a UDP connection when I power on the airframe. Once the airframe boots up completely and starts the stream I get video in MP. If I do this in any different order I never get the stream in MP, even if I just restart the stream MP still never grabs it. This was all working with MP 1.3.45 and Gstreamer 1.9.2 as was suggested earlier in this thread.

That sounds like what I did the other day. I had done several reboots in the course of changing and calibrating things working through my AC Master upgrade.

the issue is around the app port. MP only trys the app port once. on opening. as it is a very non standard connect sequence

That explains everything right there. Thank you. Is this documented somewhere already?

Nope this doesn’t work for me. I have 1.9 installed and Mission Planner does to anything. However, using Qgroundcontrol shows the video feed on 5600

Same issue here, video works perfect in QGC, nothing in MP. Have tried several different known working streams with no luck. Was getting a pop up video detected, install gstreamer 1.9.2 which I have done (complete install) still nothing.

Also tried the stock stream from APSync http://ardupilot.org/dev/docs/apsync-intro.html#apsync-intro same issue. The stream seems to start automatically when mavproxy.py is started. Also tried a Logitech C920, again works on QGC but not MP.

My slightly tweaked stream from the Pi camera

raspivid -n -fl -w 1280 -h 720 -b 10000000 -fps 30 -t 0 -rot 270 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=10.0.1.25 port=5600

Dan what does the -fl command do ?

is missionplanner and gstreamer allowed though your firewall? everything I’m seeing suggests it should work

i just double checked, and worked as I expected
start the pi


open MP
and it starts

in the MP console windows I see

INFO MissionPlanner.Utilities.GStreamer - .cctor
INFO MissionPlanner.Utilities.GStreamer - Found gstreamer F:\gstreamer\1.0\x86\bin\gst-launch-1.0.exe
INFO MissionPlanner.Utilities.GStreamer - Starting F:\gstreamer\1.0\x86\bin\gst-launch-1.0.exe -v udpsrc port=5600 buffer-size=300000 ! application/x-rtp ! rtph264depay ! avdec_h264 ! queue leaky=2 ! avenc_mjpeg ! queue leaky=2 ! tcpserversink host=127.0.0.1 port=1235 sync=false
INFO MissionPlanner.Utilities.GStreamer - _Start
INFO MissionPlanner.Utilities.GStreamer - Setting pipeline to PAUSED …
INFO MissionPlanner.MainV2 - Loaded 4825 airports
INFO MissionPlanner.Utilities.GStreamer - Pipeline is live and does not need PREROLL …
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstTCPServerSink:tcpserversink0: current-port = 1235
INFO MissionPlanner.Utilities.GStreamer - Setting pipeline to PLAYING …
INFO MissionPlanner.Utilities.GStreamer - New clock: GstSystemClock
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)01640028ffe1000e27640028ac2b402802dd00f1226a01000528ee025cb0, level=(string)4, profile=(string)high
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)01640028ffe1000e27640028ac2b402802dd00f1226a01000528ee025cb0, level=(string)4, profile=(string)high
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/avenc_mjpeg:avenc_mjpeg0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
INFO MissionPlanner.Utilities.GStreamer - Redistribute latency…
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/avenc_mjpeg:avenc_mjpeg0.GstPad:src: caps = image/jpeg, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = image/jpeg, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = image/jpeg, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1
INFO MissionPlanner.Utilities.GStreamer - /GstPipeline:pipeline0/GstTCPServerSink:tcpserversink0.GstPad:sink: caps = image/jpeg, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1

and all is good

I have a similar problem. On some pc everything works properly, on others in the HUD hangs on the first frame.
Do you have any suggestions?

Same problem here on latest beta. On some PC it works, on others it hangs on first frame.

Corrado

the hud hanging is normally an opengl issue with the pc. update your graphics card drivers from the vendor