Novice: What are the various ways to store streaming video for playback on UAVs?

General
1

Hello,

I’ve been referred to this group after I posted here

First, I know virtually nothing about UAVs! I also know nothing about streaming video. I’m just a programmer in high level computer languages.

My manager has challenged me to find the answer to the question on the subject line.

All I can do is ask questions and hope to find adequate guidance.

How has streaming video been stored in the past? Are there thousands of ways this can be done? What are the common ways that streaming data has been stored on the UAV file system?

I need to know all the details that I can implement. I have been assigned the task of implementing this capability.

I’m a simple programmer in computer languages such as C/C++. (How would C/C++ fit into the details to do this? What will C/C++ have to do?)

Can anyone tell me where I can get some good sample code of the various implementations? of doing this?

What questions should I beg my manager to answer, questions for the manufacturer of the specific UAV, and questions for the manufacturer of the individual components of the UAV? Who on the planet Earth will help me because I need help.

Does storing streaming video require certain hardware components?

The only other thing that I know of my specific UAV is that it does have a payload. Other than that, I know “nothing” else.

Where else can I go to find out the answer to the question on the subject line?

I don’t know how to start let alone how to program anything using C++. What hardware is involved?

How will hardware data sheets help me? I’ve never used a data sheet before! Will they help me?

Basically, what are the multitude of details that I need to do the work, from step 1 to step N?

Is there an intuitive tutorial somewhere for me to follow? Is there a YouTube video?

What can you tell me about the software to hardware protocols involved? In other words, do you have information about the C/C++ (etc.) procedures involved?

My current assumption is this streaming does not involve streaming to web pages at all. I’m assuming that streaming is done solely to monitor the camera. But, what should I do if the streaming is to web pages? I really don’t know what the streaming involves. What are the details in order to do both ways with streaming to web pages and streaming to monitor a camera without web pages? Is there other types of streaming that I might have to implement?

I basically want to just be able to store streaming video on a UAV.

I have been told that I have to code this with C++. That’s what I have to figure out how to do.

Should I try to contact the manufacturer of the UAV, etc.?

Any help would be greatly appreciated,
Mike

2

I don’t want to say things that are unhelpful or discourage others who are smarter than me from replying by jumping in here first, but I think it really depends on the camera you are using.
For example I fly our fixed wing planes with a runcam (or mobius) action cam which is similar to an earlier model gopro and they record direct to an SD card on their own. I also fly some DJI drones for work and they record video and images direct to an onboard SD card.
If you wish to roll your own from scratch, then I presume whatever camera hardware you pick will come with some API (?) … I don’t know … I’ve never been down this path myself … it wouldn’t surprise me if the camera output some popular video format and all you have to do is write read those bytes from the camera nad write them back out to disk/sd card efficiently.
If you are thinking about a raspberry pi attached sort of camera, then you might take a look at ffmpeg which can encode video in a variety of formats or even send it across a network connection to a remote receiver. I’ve done a lot with opencv (using ffmpeg as the video in/out backend) over the past few years and if you are running a linux-based system, this can be wonderful (although not always as speedy as I’d like.)
I’m sure there are infinite variations and combinations that I have never considered.
Good luck, it’s always tough to start from scratch, but stick with it and I’m sure you’ll find a path through to do what you need to do.

3

I don’t want to say things that are unhelpful or discourage others who are smarter than me
from replying by jumping in here first, but I think it really depends on the camera you are
using.

What should I know about UAV cameras in general? I’d like to learn as much as possible about them.

Can you give me more detail about the below possibilities that you list? I’d like to learn as much about them as is possible. But, I don’t know where to look. Is there an intuitive web site, book, or YouTube video?

For example I fly our fixed wing planes with a runcam (or mobius) action cam which is
similar to an earlier model gopro and they record direct to an SD card on their own. I also
fly some DJI drones for work and they record video and images direct to an onboard SD
card.

Am I right in thinking, you can take the SD card out of the camera and plug it into other hardware for software to access?

If you wish to roll your own from scratch, then I presume whatever camera hardware you
pick will come with some API (?) … I don’t know … I’ve never been down this path myself
… it wouldn’t surprise me if the camera output some popular video format and all you
have to do is write read those bytes from the camera nad write them back out to disk/sd
card efficiently.

What is the nad?

If you are thinking about a raspberry pi attached sort of camera, then you might take a
look at ffmpeg which can encode video in a variety of formats or even send it across a
network connection to a remote receiver.

Can you give me more information about the ffmpeg option?

I’ve done a lot with opencv (using ffmpeg as the video in/out backend) over the past few
years and if you are running a linux-based system, this can be wonderful (although not
always as speedy as I’d like.)

What else can you tell me about opencv?

Thanks for trying to get me started in learning about streaming video on UAVs, anyway, how do I learn this technology?
Mike

4

UAV cameras are just cameras. Often they are lighter and smaller with plastic lenses and smaller sensors so you often end up with lower mega pixels and/or lower image quality compared to a nice high end digital SLR camera.

Yes, websites, youtube, google, wikipedia, if you have a specific question I’m happy to do my best to answer it, but I don’t have a drone photography how-to from first principles power point presentation burning a whole in my pocket (and I wouldn’t know a lot about the deep internals anyway.)

Yeah, they are just cameras like any other standalone camera. You can pop out an sd card and plug it into another computer, or often you can plug them in via usb cable and transfer your pictures that way and sometimes that is how you charge them.

Exception: many people fly FPV cameras that only stream analog video out and if you wanted to record that you would need some receiving device with a built in recorder. I have a receiver that presents itself to my computer as a web cam, but I haven’t done much with it yet.

typo, meant and I think.

google/wikipedia ‘ffmpeg’ and start reading about it? Far better info already exists than what I can type off the top of my head here.

It’s an open-source computer vision / image processing library with a whole lot of capability. Again, if you start reading and working with it and have a specific question, I’m happy to try to try to answer if I know anything relevant. Most of my work is with the python interface to opencv, I’ve done very little in C++ with opencv. (I’ve done a lot of C++ for other projects like FlightGear and my own flight controller eco system, but I code like a 90’s C++ programmer so don’t copy me!)

This last question is way above my pay grade. It’s always a struggle trying to tackle something complicated from scratch. I tend to nibble around the edges from a few different sides until I can find a small finger hold to grab onto. From there it’s just imaginging what you want to do and google around to find out how (or if) you can do it. Sometimes you can dive in and explore the code itself, sometimes there’s good docs.

I’m sure there are books on some of this stuff, but usually when I buy a book it’s a sure sign I’ll never crack it open because in the 5 days it takes for the book to arrive I’ve found the answer online and moved on to other things.

I can tell you what I know in response to specific questions, but I don’t have any books or courses or tutorials to specifically recommend on any of this stuff … what I know is just what I’ve picked up on the side over the years from doing various work and hobby projects.

Some of this could be relevent to this forum if you are itnerfacing the camera to a flight controller (for pointing or triggering or zooming, etc.) but a lot of this is more generic and outside the scope of ardupilot. It’s a lifetime of stuff to learn and really get good at. I’m just scratching the surface of all this stuff myself.

Good luck,

Curt.

5

Hi,
I’ve been a software project manager for over 40 years and with really complex projects, so let me just point you in the right direction. The first thing that you need to know is what are the client requirements and expectations of the project. There are literally thousands of implementations when video streaming is involved and as Curt already pointed out there are some excellent open source solutions you can use.
But to pick the right path you must have a clear goal. If you don’t, it’s like shooting in the dark, you could hit the target but the odds are close to nill.
So, my advice to you is to demand from your manager (or your customer) at least a clear goal of the project. Otherwise you could end up with a solution which nobody wanted and a waste of time. And guess who’s fault it would be.
Just my 2C.

1 Like
6

Hi,
as Igor has mentioned, I would ask more specific questions initially as your questions raised here are already confusing:

  • Usually video is being streamed from the UAV to a ground station, a monitor or something like a smartphone or even FPV goggles.
  • Whist there are sometimes (dependend on system in use) options of storing it on the UAV itself this can cause problems with resources that usually are required by the flight controller. --> Hence, a better idea would be using an idenpendent system…I.e. just attach a dash cam, GoPro or soemthing similar to UAV. In that way you’ll have a independent device with its own memory card and a camera that is already designed for fast motion and an auto-focus system to suit.
  • If you are in fact streaming down to a ground station then recording can take place on that device. But often this is done in combination with telemetry data and hence the common GS software (QGroundControl,Mission Planner) are setup to icnlude live vision from UAV as well as important telemtry data.
  • If such system is being used than various ways of recording video footage are available. I.e. a simple screen recorder like “Simple Screen Recorder” can be use to record everything on screen if required. But keeping in mind that when using something like Mission Plannner the telemetry data will usually be independently recorded and saved. …In that case you would usually only record video on its own.
  • A basic consideration has to be: Is the vision required for live view in order to check on UAV’s actions / progress ? Is it required for additional safety (and perhaps insurance reasons) in the event of a crash? (…in which case storing it on UAV would be uselless ). Or is it simply required to capture good, high grade video footage ? (…I.e. just attach a good independent video camera to UAV - recording straight to a SD card).