Ardupilot_gazebo and camera not showing in Gazebo nor UDP stream

phuesler · January 6, 2025, 10:31am

Hi, I need help getting the camera feed to work inside Gazebo Harmonic.

I followed the instructions on GitHub - ArduPilot/ardupilot_gazebo: Plugins and models for vehicle simulation in Gazebo Sim with ArduPilot SITL controllers on a fresh install of Ubuntu 22.04 inside a VM. I got Gazebo to run with iris_runway. The quad copter shows up, I can fly it with Mavproxy etc.

However, I can’t get Gazebo to display the camera image, which I have seen in screenshots and video. It doesn’t show anything there.

Steps:

gz sim -v4 -r iris_runway.sdf

In the logs it says, “No scene or camera sensors available”

Any ideas? Thank you for your time.

Regards

Patrick

phuesler · January 6, 2025, 10:49am

Update: I figured out how to display the image. On the upper right corner, I searched for image and enabled that. Now I can actually see an image.

Have a terrific day.

phuesler · January 6, 2025, 11:04am

Still no video, neither in QGroundControl nor with gst-launch. Any pointers how I can debug this? I have enabled streaming as described in the readme and the console output of Gazebo says, that it started streaming.

And I can see UDP packets being sent to port 5600 on 127.0.0.1

$ sudo tcpdump -i lo -n udp port 5600 -X
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on lo, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:12:39.006369 IP 127.0.0.1.36405 > 127.0.0.1.5600: UDP, length 14
	0x0000:  4500 002a 5d12 4000 4011 dfae 7f00 0001  E..*].@.@.......
	0x0010:  7f00 0001 8e35 15e0 0016 fe29 8060 378f  .....5.....).`7.
	0x0020:  9047 f925 4761 826a 0930                 .G.%Ga.j.0
11:12:39.006397 IP 127.0.0.1.36405 > 127.0.0.1.5600: UDP, length 984
	0x0000:  4500 03f4 5d13 4000 4011 dbe3 7f00 0001  E...].@.@.......
	0x0010:  7f00 0001 8e35 15e0 03e0 01f4 80e0 3790  .....5........7.
	0x0020:  9047 f925 4761 826a 61e5 820a 7ffb b6ca  .G.%Ga.ja.......
	0x0030:  288b 6708 9f47 4de4 dacf cb40 55f6 9748  (.g..GM....@U..H
	0x0040:  3ed2 14c8 3365 c4dc 12e6 a986 fb00 38d9  >...3e........8.
	0x0050:  9c84 043f b885 a431 3182 9133 4776 a6ea  ...?...11..3Gv..
	0x0060:  b71c b71a 6745 1e73 09f0 6766 c3f6 3905  ....gE.s..gf..9.
	0x0070:  96f8 a41c 08f4 b004 d77e 9dc6 80dc 4d4b  .........~....MK
	0x0080:  7da8 c0ce 8242 3438 72d1 2949 d78b e3e0  }....B48r.)I....
	0x0090:  e3d5 17d2 61a7 6c0f 6d90 b970 a73f 6fdc  ....a.l.m..p.?o.
	0x00a0:  dec2 81e3 c54b 2e5b e8a0 a974 93d5 19fd  .....K.[...t....
	0x00b0:  37d0 a25d 614c c73b 887d a0a7 1f77 bf34  7..]aL.;.}...w.4
	0x00c0:  5061 f2ed 64e4 b33f 927e 5690 b261 aea3  Pa..d..?.~V..a..
	0x00d0:  292c 2961 fdb9 11df 46f5 2924 d87b f202  ),)a....F.)$.{..
	0x00e0:  b7d0 1b5e 10e9 27ec 9fa1 c60c 8abf 70f8  ...^..'.......p.
	0x00f0:  ee3d 0e39 be70 25ee 19cf b730 6084 98ae  .=.9.p%....0`...
	0x0100:  68cc 76e7 5581 fa94 50fc ce4e 0f61 0b50  h.v.U...P..N.a.P
	0x0110:  f7a9 0c8c 6919 5e20 60bc 6536 03b8 b98a  ....i.^.`.e6....
	0x0120:  2cb5 169d 9849 38ce f690 e45c 30de 0385  ,....I8....\0...
	0x0130:  25b1 18b7 ee16 a1ef ac53 fcba 525e 4478  %........S..R^Dx
	0x0140:  a0c5 63a1 d21c ab30 c6a2 d45a 962c b9c0  ..c....0...Z.,..
	0x0150:  87c8 1de5 e750 5c77 7e3c d759 ee3f 4415  .....P\w~<.Y.?D.
	0x0160:  abd3 87c0 d8de 452f d8fc 0d83 31f4 8706  ......E/....1...
	0x0170:  35b7 9a08 eaee 6fff 2595 d85e 71d1 5d32  5.....o.%..^q.]2
	0x0180:  f863 3e09 01d3 3815 a4e6 4c32 236f bd1b  .c>...8...L2#o..
	0x0190:  de83 cb7a 4a99 2730 b450 eb03 e26d 2ca2  ...zJ.'0.P...m,.
	0x01a0:  4a27 826a 2484 3cfe 1d83 0f5c 295b 0f06  J'.j$.<....\)[..
	0x01b0:  f64a 6da8 5193 43c9 71e9 d606 35ac 7fef  .Jm.Q.C.q...5...
	0x01c0:  3f18 fe62 20bd e403 491e 433c e6d8 3624  ?..b....I.C<..6$
	0x01d0:  f6d0 6f53 f178 08a4 0305 b09e b33f abea  ..oS.x.......?..
	0x01e0:  53a0 5914 4d1f e647 e159 756b 1456 716e  S.Y.M..G.Yuk.Vqn
	0x01f0:  4ee9 fc16 7e03 dc3e bd44 25a0 339f 42c2  N...~..>.D%.3.B.
	0x0200:  e4f9 313c ac29 e6c6 7d89 6738 fdb8 9e18  ..1<.)..}.g8....
	0x0210:  e2da 6aad 2e3a 5cc0 ac43 787c d3b4 654f  ..j..:\..Cx|..eO
	0x0220:  4d15 966d adbc 83e9 ca1f 5edf fa3c a7e3  M..m......^..<..
	0x0230:  3ad8 9ce2 0a9a cc91 1a5c ec69 2d43 a29a  :........\.i-C..
	0x0240:  3ddf f710 866c fc78 466a cbc3 0f14 03f2  =....l.xFj......
	0x0250:  aa64 4a56 4b8e 8a34 9747 5468 efc4 aba8  .dJVK..4.GTh....
	0x0260:  707a e04f baa8 c1a5 f16f f159 8212 e245  pz.O.....o.Y...E
	0x0270:  37dc db64 15e1 570a b0c4 1ef8 3a2f 9131  7..d..W.....:/.1
	0x0280:  6a6d 0b6a ad60 e6d8 0c97 775f 279a e958  jm.j.`....w_'..X
	0x0290:  314c 993b 8e52 90e1 8d93 8e3a 2114 c61d  1L.;.R.....:!...
	0x02a0:  7952 8d30 e95a bc88 4121 8784 2b14 4730  yR.0.Z..A!..+.G0
	0x02b0:  3bb9 d3b4 f8f1 3a97 39b7 a504 8f03 7f2c  ;.....:.9......,
	0x02c0:  e5e1 fed9 0b47 ab97 cd7f 2c9f e08f 2247  .....G....,..."G
	0x02d0:  fdb2 15d5 6598 f7ce c54d 3aa3 786e 5c97  ....e....M:.xn\.
	0x02e0:  7874 a867 c73b 083b 27f8 6b12 3cde 6b4f  xt.g.;.;'.k.<.kO
	0x02f0:  1315 37d3 290b abd1 5a95 a1f3 a83e 2c9c  ..7.)...Z....>,.
	0x0300:  8cee c9f8 065a 1da1 37a2 d9a8 d691 e3f4  .....Z..7.......
	0x0310:  cd80 b7c1 a28a e7bb e101 9162 3809 a7d6  ...........b8...
	0x0320:  697b fa2d 41e0 dde5 263f c454 7ecb 1a40  i{.-A...&?.T~..@
	0x0330:  f1c2 5882 37dd 3319 bd51 9efd 89c4 d01e  ..X.7.3..Q......
	0x0340:  cb0c 433c a8c3 7f6a 1e25 7547 21c1 63ba  ..C<...j.%uG!.c.
	0x0350:  8bb4 e501 b5be 58c7 d5b6 3fe0 118f 57b3  ......X...?...W.
	0x0360:  3f64 ba59 992f 535f a6d7 a5f7 691f a6cd  ?d.Y./S_....i...
	0x0370:  6661 26e2 d853 4b93 c5be aadc 2717 564c  fa&..SK.....'.VL
	0x0380:  f887 8748 6b6f 6f3b 6119 26dd b0b0 319b  ...Hkoo;a.&...1.
	0x0390:  a683 e77e 10ec f3a3 9024 34cc 93cc 9a1d  ...~.....$4.....
	0x03a0:  dc59 fc3d 1e2d aae6 0a1a 5194 dab0 352d  .Y.=.-....Q...5-
	0x03b0:  dbdf fa26 323d eea6 714f 6435 4b1b a140  ...&2=..qOd5K..@
	0x03c0:  7458 1376 c325 7473 09a7 b71d 2263 7abd  tX.v.%ts...."cz.
	0x03d0:  8ee3 0de9 8ccb be23 2f25 f127 536b a73e  .......#/%.'Sk.>
	0x03e0:  581b 1350 0c41 4d8e d262 3700 0003 0000  X..P.AM..b7.....
	0x03f0:  0300 1c70                                ...p
^C
2 packets captured
8 packets received by filter
0 packets dropped by kernel

Maybe it is a different codec?

My settings in gimbal_small_3d/model.sdf

        <plugin name="GstCameraPlugin"
            filename="GstCameraPlugin">
          <udp_host>127.0.0.1</udp_host>
          <udp_port>5600</udp_port>
          <use_basic_pipeline>true</use_basic_pipeline>
          <use_cuda>true</use_cuda>
        </plugin>

Setting ‘use_cuda’ to false breaks starting the stream. I in fact to use an nvidia card. Setting ‘use_basic_pipeline’ returns an error in gst-launch-1.0 that it is an invalid payload.

rhys · January 9, 2025, 4:29pm

Is the streaming not visible if you use the default SDF for the GstCameraPlugin and run:

gst-launch-1.0 -v udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

This works on macOS and Ubuntu 22.04. It could be drivers in your system or maybe a missing gstreamer dependency? Can you run the command above at max verbosity and post the terminal output.

phuesler · January 13, 2025, 8:32am

Thank you for the answer. What does SDF stand for? I’ll try again with GST_DEBUG=3 and see what happens.

UPDATE: Nevermind, SDF ist the file. I assume ‘iris_runway.sdf’ correct?

rhys · January 13, 2025, 12:04pm

Using the default values of the streaming plugin in the gimbal camera:

I have not tested the cuda option as I do not have the hardware and there have been reports it is not functioning correctly. The settings in the main branch should work for Ubuntu and macOS and this has been confirmed by other users.