Voice recognition used for hands-free flying

In this video I demonstrate how I can control the flight of my Arduplane equipped plane using voice commands alone.

The solution is implemented in my FlightZoomer apps, which serve as GCS (FlightZoomer Cockpit 3) and via 3G as a companion computer (FlightZoomer Companion 3). In the video you can see the visual appearance of the app exactly as it looked during the flight.

Of course instead of using voice control, the autopilot can also be controlled conveniently using the GUI or, as another novel feature, using a simple rotary controller (Surface Dial).

FlightZoomer implements its autopilot modes (which simulate the Boeing 787 autopilot) as convenience layer by commanding the required basic flight vector from ardupilot (which runs in GUIDED mode).

The grammar of the voice command interface is this (in EBNF):

  • digit = “zero” | “one” | “two” | “three” | “four” | “five” | “six” | “seven” | “eight” | “nine”;
  • activateAutopilot = (“activate autopilot” | “switch autpilot on”);
  • deactivateAutopilot = (“deactivate autopilot” | “switch autpilot off”);
  • turnRight = “turn right”, digit, [digit], [digit];
  • turnLeft = “turn left”, digit, [digit], [digit];
  • setSpeed = [“set”], “speed”, [“to”], digit, [digit], [digit], [digit];
  • startDescend = “descend”, [“to”], [“altitude”], digit, [digit], [digit], [digit], [digit];
  • startClimb = “climb”, [“to”], [“altitude”], digit, [digit], [digit], [digit], [digit];
  • tuneILS = (“tune” | “set”), “I L S”, [“receiver” | “frequency” | “to”], digit, digit, digit, “dot”, digit, digit;
  • activateAPPmode = (“activate” | “arm”), [“the”], (“approach” | “I L S”), [“mode”];
  • readBackAccepted = (“affirmative” | “accepted” | “positive”);
  • readBackRejected = “negative”;
  • correction = “correction”;

Meaning of the notation:
format: ruleName = chain of text portions, separated by commas
[“some text”]: can be spoken or left away
(“text a” | “text b”): one-of text portion
"text": terminal string

The FlightZoomer 3 Cockpit-app, that offers these capabilities will very soon be released and also offer bunch of other features.

More details can be found also here:


Very impressive, really great!

So cool.
Fantastic work, congrats!

Thanks for the replies!
@Randy, you would probably not remember but here is a thread, in which you have provided very helpful input for my project:

1 Like

im actually interested in your project. Can I ask for the data sets you used in your voice commands? :slight_smile:

What do you mean with data sets?
The app uses the UWP speech recognition API and the used grammar is pretty much explained in the opening post (the only difference is, that the app uses the rules in SRGS form).