Servers by jDrones

Voice recognition used for hands-free flying


(flightzoomer) #1

In this video I demonstrate how I can control the flight of my Arduplane equipped plane using voice commands alone.

The solution is implemented in my FlightZoomer apps, which serve as GCS (FlightZoomer Cockpit 3) and via 3G as a companion computer (FlightZoomer Companion 3). In the video you can see the visual appearance of the app exactly as it looked during the flight.

Of course instead of using voice control, the autopilot can also be controlled conveniently using the GUI or, as another novel feature, using a simple rotary controller (Surface Dial).

FlightZoomer implements its autopilot modes (which simulate the Boeing 787 autopilot) as convenience layer by commanding the required basic flight vector from ardupilot (which runs in GUIDED mode).

The grammar of the voice command interface is this (in EBNF):

  • digit = “zero” | “one” | “two” | “three” | “four” | “five” | “six” | “seven” | “eight” | “nine”;
  • activateAutopilot = (“activate autopilot” | “switch autpilot on”);
  • deactivateAutopilot = (“deactivate autopilot” | “switch autpilot off”);
  • turnRight = “turn right”, digit, [digit], [digit];
  • turnLeft = “turn left”, digit, [digit], [digit];
  • setSpeed = [“set”], “speed”, [“to”], digit, [digit], [digit], [digit];
  • startDescend = “descend”, [“to”], [“altitude”], digit, [digit], [digit], [digit], [digit];
  • startClimb = “climb”, [“to”], [“altitude”], digit, [digit], [digit], [digit], [digit];
  • tuneILS = (“tune” | “set”), “I L S”, [“receiver” | “frequency” | “to”], digit, digit, digit, “dot”, digit, digit;
  • activateAPPmode = (“activate” | “arm”), [“the”], (“approach” | “I L S”), [“mode”];
  • readBackAccepted = (“affirmative” | “accepted” | “positive”);
  • readBackRejected = “negative”;
  • correction = “correction”;

Meaning of the notation:
format: ruleName = chain of text portions, separated by commas
[“some text”]: can be spoken or left away
(“text a” | “text b”): one-of text portion
"text": terminal string

The FlightZoomer 3 Cockpit-app, that offers these capabilities will very soon be released and also offer bunch of other features.

More details can be found also here:
https://flightzoomer.com/


(rmackay9) #2

Very impressive, really great!


(Olivier Brousse) #3

So cool.
Fantastic work, congrats!


(flightzoomer) #4

Thanks for the replies!
@Randy, you would probably not remember but here is a thread, in which you have provided very helpful input for my project:
https://groups.google.com/forum/#!profile/drones-discuss/APn2wQeEAoB0Erbsy42oJsvHb4Lhc-bORPmmZa8lfs2Ps5MurZQ9eOlWMXfC0avEQyYgoY1iNXTg/drones-discuss/hBccNbRBG_A/rAEOxvHaBgAJ