Autonomous soaring – Artificial Intelligence on the fly

Speaking of autonomous soaring (kudos to rctestflight aka Daniel Riley for the great video!), the Microsoft Research blog features a great article written earlier this month by researcher Andrey Kolobov of the Adaptive Systems and Interaction Group at Microsoft. You may recall previous work on ArduSoar, mentioned in this earlier blog post where Andrey and Samuel Tabor co-won the Ardupilot “Best Contribution of the Month” award for their work on ArduSoar

In the blog entry, Andrey describes project FrigateBird, more specifically the most recent initiative within the Microsoft Research project aimed at improving upon ArduSoar, integrated within ArduPlane. The idea is to use AI techniques, more specifically Reinforcement Learning, to take better advantage of thermals for upward lift, which leads to much longer autonomous glider flight times.

At the heart of the team’s new initiative lies POMDSoar, a Bayesian reinforcement learning-based soaring algorithm that the Microsoft team designed and implemented. They ran it on top of Arduplane on a Pixhawk4, and tested it on a 1.5 meter wingspan Snipe 2 motor-less sailplane.

No small feat: As Andrey describes, some birds like Frigatebirds or Albatross instinctively know how to glide effortlessly, and glider pilots also know how to take advantage of thermals by combining “local knowledge, ground features, observing the behavior of birds and other sailplanes, and other cues, in addition to instrument readings, to guess where thermals are”. But modeling such decision making is a difficult problem, even more so when it needs to be implemented on a micro controller or small companion computer with limited computing power.

Yet the Microsoft team started to do just that, and have already demonstrated that the use of the AI based POMDSoar algorithm significantly improved performance and flight times compared to ArduSoar, this in a series of live tests, a first compared to previous related research limited to simulations only.

Impressive results by the research team! Said team includes, in addition to Andrey, Iain Guilliard, a Ph.D. student from the Australian National University in Canberra (Canberra? Hmm … this place rings a bell! Oh wait! :slight_smile: ), and Samuel_Tabor. The team is also further researching thermal prediction partnering with the ETH Zurich’s Autonomous Systems Lab.

The team’s project code is all available on github, for anyone of us in the Ardupilot community and beyond interested in running it on our favorite flight controllers. What’s more, it includes a SITL (Software in the Loop) implementation using the Silent Wings flight simulator, a version of which is free to evaluate for up to two weeks.

I followed the SITL build instructions and had no issue installing and running Silentwings, and building the code and running it (cygwin environment) on a windows machine*. The code allows for a network connection that can be established between a ground station (I used Mission Planner with a Taranis radio as joystick) and Silent Wings, typical of simulation environments, and the project includes a sample plane parameter file for one of the glider models, along with a sample autonomous mission file flying near Starmoen in Norway.

I wasn’t successful on first try at actually running the mission, however, probably due to the fact that the parameter file I used corresponded to another model not available in the demo version. To be investigated … Below is screenshot of where I got so far.

For more information on this impressive research initiative be sure to head over to Andrey’s rich and much more detailed blog post than this post summary!

*Note: I had to “chmod a+x waf” (or “chmod 777 waf”) in cygwin as waf was not executable and running it resulted in an error.


Nice write-up @OlivierB

The Microsoft team is doing a lot to push forward the soaring capabilities with SilentWings and the POMDP method.

Basically, the Microsoft FrigateBird fork uses the same soaring framework as the current ArduPlane versions, including the mode switching, variometer and thermal position/strength estimators. If you’re running ArduPlane on an aircraft, you already have all this available.

However, while the current ArduPlane releases use the usual loitering navigation controller while thermalling, the FrigateBird fork uses a more complex planning approach to select bank angles to improve the thermal estimates and/or maximise height gain. This seems to improve performance in low-altitude messy thermals, at the cost of more CPU cycles and complexity.

I’m hopeful that in time a hybrid approach that gets most of the POMDP benefits with similar complexity to the current system can be devised. This would be suitable for future ArduPlane releases.

The soaring feature is under active development, and there are currently four related pull requests open with testing from @marco3dr .