Soubhi Hadri's hand gesture recognition and control of an ArduPilot multicopter

Blog
, ,
1

This is a video demonstration from Soubhi Hadri of an AI based gesture recognition system that can be used to control an ArduPilot multicopter (in this case it is a simulated quadcopter).

The system is open source so if you want to look further into how it’s done:

  1. the code is here on github: https://github.com/SubhiH/HandGesturesDroneController
  2. the dataset used to train the AI engine: https://github.com/SubhiH/HandsDataset
  3. Autopilot control (uses Dronekit): https://github.com/SubhiH/Autopilot

Nicely done Soubhi!

By the way, Soubhi is graduating from University soon so if you’re looking for a talented engineer you can find more info on him here!

4 Likes
2

Great!

  1. The system is tested with CPU and works in real-time.

What CPUs has this been tested on? Will this run on the likes of a raspberry, or does it need Intel-like power to run? I’m referring to inference obviously, not training.

1 Like
3

Thank you! I tested it on 2.7 GHz Intel Core i5. I believe it should work but have not had the chance to run in on raspberry. I think the number of frames per sec will be less but still have the real-time behaviour. If you have the chance to test it, let me know :slight_smile:
P.S. to load the model, it needs 2-3sec but you do that just once when you run the application.

4

Excellent and exciting. Greatly done (as your CV web site too by the way). Wow !

1 Like
5

did you use linux operating system or windows? Because i really liked your project and want to do that project but i’m not that good.