ArduPilot and BlueOS for Companion Computers


For my new aerial photography copter design I’m working on, I want to provide all the features that users have come to expect including OTA firmware updates, automatic upload to the cloud of logs, images and videos, visual-follow-me, high altitude non-GPS in case of GPS failure, remote control over LTE telemetry, etc. For many of these the ArduPilot team has already developed solutions but they require a companion computer so we’ve struggled to roll them out to users.

This led me to BlueOS which was originally written for submarines and boats but which @williangalvani and other developers have been happy to extend to support other vehicle types. It already includes many of the features we need including the Cockpit GCS but perhaps more importantly it supports Extensions (based on Docker) which allows us to safely and somewhat easily extend its functionality.

To test it out I wrote a simple Camera Download extension which takes AP’s existing Siyi and XFRobot download scripts and makes them available within BlueOS so any users can now install them via the BlueOS Extensions page.

He’s a picture of the downloader in action while connected to a Siyi A8.

How to get Involved

If you’re interested in getting involved I highly recommend reading BlueOS’s Extensions documentation and get familiar with Docker.

Here are some BlueOS extensions that I’m keen to have developed and so I can mentor developers and/or help them get through the development, review and release process:

Hardware

Re hardware, at the top you’ll see a simple wiring diagram of the major components of my vehicle and they’re all connected via ethernet

Installing BlueOS on RPI4 or RPI5

If you’re interested in setting up something similar yourself, Official RPI4 CM setup instructions are here but below is also how I do it:

  • Purchase an RPI4 CM or RPI5 CM with wifi and EEMC

  • Purchase an RPI4 I/O board or RPI5 I/O board

  • Install rpiboot and rpi-imager on your Ubuntu or Windows PC

  • On the RPI I/O board

    • Mount the RPI4/5 CM on the RPI I/O board
    • Added jumper so the RPI CM essenetially goes into bootloader mode, “Fit jumper to disable eMMC Boot”
    • Connected I/O board to PC via USB
    • Powered up the I/O board (if using the RPI5 I/O board this step is probably done by the above step)
  • On your PC

    • Open a web browser to the BlueOS release page, select a recent release, open “Assets” and download “BlueOs-raspberry-linux-arm64-v8-bookworm-pi5” (if this image isn’t available, pick a different release)
    • run rpiboot:
      • Ubuntu users should open a terminal and enter “rpiboot”
      • Windows users should open the start menu and run “rpiboot-CM4-CM5 Mass Storage Gadget”
    • run rpi-imager:
      • Ubuntu users should open a terminal and enter “rpi-imager”
      • Windows users should open from start menu run “Raspberry Pi Imager”
    • from within RPI Imager:
      • Choose Device: “Raspberry Pi 4” or “Raspberry Pi 5”
      • Operating System: Use custom, select downloaded BlueOS .img file
      • Storage: select RPI drive (should have appeared after rpiboot was run)
      • Select “No” when asked to apply special settings

Configuring BlueOS

After the above installation is complete this is how I perform the initial setup of BlueOS

Configure RPI/BlueOs to connect to your local Wifi:

  • Leave the RPI4/RPI5 on the I/O board initially
  • Make sure a Wifi antenna is attached to the RPI4/RPI5 CM
  • Remove the Jumper and power-down/power-up the I/O board so the RPI boots normally
  • Connect a PC via ethernet to the I/O board’s ethernet port
  • Set the PC’s IP address to manual with IP 192.168.2.100 (the RPI’s default IP address is 192.168.2.2 and we want the PC to be within the same ethernet subnet)
  • Open a Chrome browser to http://192.168.2.2, BlueOS should appear
  • A setup wizard should appear, select “Other vehicle setup”
    • Vehicle Name: MyCopter
    • MDNS Name: “blueos” (default)
  • On the top right, select the wifi icon and connect to your local Wifi

Configure RPI/BlueOs’s Ethernet IP to be the same subnet as the autopilot and camera:

  • Push the buddha icon on the top right and change to “Pirate mode”
  • Select the networking icon
    • select “eth0”
    • select “ADD STATIC IP” and enter 192.168.144.20 (this will be the RPI’s IP address on the vehicle)
    • select “DISABLE DHCP SERVER”
    • beside “192.168.2.2 Static IP” push the trash can icon. At this point you will lose contact with BlueOS but you can re-gain contact by changing the PC’s manual IP address to 192.168.144.100 and reconnecting with the web browser to http://192.168.144.20
    • restore PC’s IP address to automatic

Connect RPI/BlueOs to Autopilot and Camera:

  • Follow AP wiki instruction to setup DroneNet. This should result in these IP addresses for other components:
    • DroneNet/CubeNodeETH: 192.168.144.14
    • Autopilot: 192.168.144.15
    • Siyi a8: 192.168.144.25
  • On the autopilot, create an addtional UDP Server connection for BlueOS to connect to:
    • NET_P1_TYPE = 1 (UDP server)
    • NET_P1_PROTOCOL = 2 (MAVLink2)
    • NET_P1_IP0 = 0
    • NET_P1_IP1 = 0
    • NET_P1_IP2 = 0
    • NET_P1_IP3 = 0
    • NET_P1_PORT = 14560
  • Install the RPI on the Ochin carrier board
  • On the PC open a Chrome browser to http://blueos-avahi.local/ (BlueOS should appear)
  • Open BlueOS’s Autopilot Firmware screen
    - select “Master Endpoint”
    - Connection Type: UDP Client
    - IP/Device: 192.168.144.15
    - Port/Baudrate: 14560
    - push “Save Changes”
  • Check top-right Hearbeat icon has gone white meaning the RPI and autopilot are communicating via MAVLink over ethernet

Setup Cockpit Video Streaming

  • open BlueOS’s Video Streams page
    • under “Redirect Source” select, “ADD STREAM”
    • after a few seconds the live video thumbnail should be displayed
  • Open Cockpit, push arrow on right-middle of screen to open left menu
    • Select Settings, Video
      • under “Streams mapping”, set “External name” to “Siyi A8” (set above)
      • under “Allowed WebRTC remote IP Addresses” delete IP address and leave “Enable auto-retrieval of allowed IP addresses” checked
    • re-open left menu, select “Edit Interface”
      • from bottom scroll bar, drag-and-drop “Video Player” onto the map
      • video should appear but if not:
        • find “VideoPlayer” from top-left menu, select gear icon
        • set “Stream name” to “Stream 1”

Known Issues

It’s still early days for BlueOS with Copter so there are a number of known issues we need to resolve including:

  • Routing the camera’s live video stream to a regular ground station (QGC, MP) hasn’t been confirmed to work yet
  • Cockpit’s missing features that QGC, MP have including camera gimbal controls, EKF status viewer and many mission commands

Videos

Willian’s BlueOS intro

Rafael Lehmkuhl’s Cockpit intro

Patrick Pereira’s MAVLink Tools

Thanks for reading this far and special thanks to BlueRobotics and the BlueOS development team!

12 Likes

Thanks for the awesome work!

1 Like

This is something Arudpilot really needs, we have missed out on a lot of the new tech advancements just due to a lack of onboard processing. linux controllers have a lot of performance but lack IO.

What I would like to see in the next generation of flight controllers is having the linux soc integrated onto the flight controller its self. using the original concept for the pixhawk where the io chip should be able to fly if the main navigation controller crashes. its been almost a decade and things have advanced a lot so we now could have a “flight controller” chip to replace the io chip that is essentially a single chip f405 or h743 ardupilot controller with the high performance linux “navigation controller” running blueos and all the new tech. By keeping the reliable chibios flight controller for all the IO means it can still rtl and fly the vehicle manually and there is already all the failsafes for companion computers built in.

The pixhawk 2 had the right idea, its just a shame they went with the intel edison chip.

It should be possible to add ROS as a docker container.

2 Likes

Hi @geofrancis,

BlueOS has ROS(1) and ROS2 extensions already.
We also just integrated a foxglove extension as well.

2 Likes

Nice, I will get around to testing this soon its has a lot of interesting features.

2 Likes

@geofrancis,

Thanks for the positive feedback, I’m very excited about using BlueOS extensions.

Re having a linux flight controller, I also think that this is likely the future but the only autopilot I’ve seen recently like this is the modal voxl2 and it uses a Qualcomm chip with the QURT OS so it’s difficult to work with and doesn’t yet support Ethernet nor CAN (although we could add this fairly easily). This is why I went with the RPI5 CM. I’m also working on a custom carrier board with AM32 ESCs included. I’m not saying this is the solution for everyone but this was as close as I could get to what I think you’re suggesting. Anyway, I’ll pass on your suggestion to autopilot manufacturers that I know (e.g. AP Partners like CubePilot, CUAV, Holybro, etc).. no promises as to what they’ll say of course :slight_smile:

2 Likes

I have been getting to grips with a beagle bone blue for the last month so i have been getting familiar with linux ardupilot, as much as I hate the the beaglebone blue it has a lot of advantages compared to a standard micro controller. its a super computer with the amount of ram it has even if the OS is using half that still leaves 100x more than a h743 free, it could hold every lua script on the ardupilot git hub in ram with ease but the IO is a nightmare when you compare it to a micro controller, their advantages and disadvantages line up perfectly so it makes sense to use them together.
the beaglebone is almost 10 years old, it was a 1ghz single core with 512mb ram and 4gb flash. These days for the same $30 and package you get a 1.6ghz quadcore with 2gb of ram, 8gb flash, AI and video accelerators.

another idea would be to move all the high level cpu and memory intensive ardupilot tasks like networking and over to a docker container on the main " navigation controller" and have it treat the “flight controller” with all its IO and data shared with the linux process as the io controller that can take over.

the flight controller would just need to run a very stripped down version of ardupilot that handles any realtime tasks, like RC, IO, and can take over for manual flight modes, RTL and landing. and the linux process wouldn’t have to run in real time since its just telling the flight controller where to go. we already have most of this in existence using sitl. it would essentially be turning sitl in to hitl. with the advantage of if linux crashed it could still fly and RTL.

With all the extra ram and linux it would fix a lot of problems, things like the object database could be bigger, smart RTL could be further, more waypoints, ntrip client, networking, USB storage, even USB Ethernet adapters, cameras, USB wifi for long range video.

You could also have multiple flight controllers for redundancy. or have hybrid vehicles with one as a boat and one as a plane

When you think about it, the most successful RTF aircraft to use ardupilot were the parrot quad and plane and they both used linux controllers, I suspect we dont see more RTF because there is nowhere for manufacturers to add their own code.

Most of the new RiscV SOCs being released also have AI cores built in, we could have things like AI autotuning.

stick an APP store on it to bring some money and devs in lol turn ardupilot into android.

Years ago I used a cube carrier from Dronee for a plane, That had a pi3 compute module under it but I havent seen anything like it for pi4,

2 Likes

@geofrancis,

Thanks for the extra ideas and insight. I very much agree we can do a lot with the extra CPU and memory. My personal plan starts with developing these BlueOS extensions with help from other devs:

  1. log, image, video upload to the cloud (see WIP branch here)
  2. Visual Follow Me
  3. Precision Landing using
  4. High Altitude Non-GPS maybe using Optical flow or something else
  5. Publish position to cloud using BlueOS extension (allows multiple vehicles to be seen from a single GCS)

BTW BlueOS has a store and it even allows the user to add a URL to another non-BlueOS store so it’s very flexible.

1 Like

Sorry for the brain dump, it’s just that I have literally had a white board on my wall about this subject for the last 3 weeks on using small linux soc’s for docker and how to interface them with ardupilot and have them compliment each other. I was excited to see this pop up with 90% of what I was looking into.

3 Likes

I have this Pixhawk 6x based baseboard from Holybro, running a CM4 to process RF using a LimeSDR and GNU Radio. Connects to the Pixhawk via Ethernet. I only read data from the FC but that’s just because of my application.

2 Likes

Hi @ReadyKilowatt,

Thanks for the link. That looks like a good small package and it probably also works for the RPI5 CM in case extra CPU power is required. I strongly suspect it’ll work with BlueOS!

1 Like

Power is an issue. The CM is powered by USB C connector off the distribution board. Kind of clunky and I always think it’s going to pop out but so far it hasn’t.

1 Like

@geofrancis Really like this braindump, i would also appreciate such a quite radical way, but as you explain also smart way of using cheap powerful chips, forward.

I was trying out the OBAL Linux approach quite some time ago and it works like a charm. Was already back then wondering why there is that little usage of linux based FCUs in actual vehicles (didnt know that parrot was using it!) out there:

@rmackay9

i always wanted to try out blueos, your guide and new plugins give me some more reasons to try it , thanks :slight_smile:

2 Likes

There are a few linux controllers using pi boards but I think it adds a lot of risk if you want to start running other things on the pi because if you crash or even just lag it out for a second by doing something like launching too many programs and using up all the ram, then you would probably crash the aircraft. By letting the microcontroller do the core flying means it will just go into a degraded state rather than fall out the sky so it would loose all its advanced features but manual flight modes and RTL/landing would work. it would also get around any driver problems on linux associated with sensors and all the different buses you need to get working since you just need to get a serial link working to the flight controller.

I also think having to use a separate flight controller and external computer will put people off setups like this as you need a fairly large aircraft or rover to move a full setup with a pi + flight controller onboard compared to if it was all on the small single board.

2 Likes

At least partially some problems could be mitigated on linux systems by using a stripped down linux with less applications whcih can cause problems and additionally using core and memory pinning to avoid APM loosing control of its ressources.

These measures are a bit more advanced, so maybe not for everyone. But i could imagine that some predefined images for popular platforms like raspberry would help people.

2 Likes

@julled @geofrancis,

I also very much appreciated @geofrancis’s brain dump :-).

Like you two I would also love to see more flight controllers which give us both fast/real-time performance and access to lots of CPU and memory for expensive algorithms (visual follow-me, ROS2, AI, etc). I think the closest we have to this at the moment is the ModalAI VOXL2 but the QURT OS is not very popular and comes with a somewhat open source unfriendly license. I suspect we will get more flight controller which can do both well in the future, I hope so anyway!

1 Like

@rmackay9 how dependent on raspberry pi or raspian is blueos? could it be ran on other pi like boards like orangepi?

Hi @geofrancis,

I don’t know for sure but I suspect it will mostly work. In related news, we’re going to try and get BlueOS running on an NVidia Orin board in the future. The BlueOS guys have already tried and gotten it partially working at least.

1 Like

A couple of updates:

  • Ethernet connectivity to the autopilot is included in BlueOS 1.5.0-beta
  • Live video streaming from a Siyi A8 to Cockpit has been verified to work and I’ve updated the instructions above.

I still don’t have live video streaming to MP or QGC (issue is here)

1 Like

Hi!

I’ve been documenting my experiments with other boards here:

3 Likes