Introduction
AirSim is an open-source, cross-platform simulator for drones and cars, built on Unreal 3D Engine.
It provides physically and visually realistic simulations with external flight controllers such as PX4 or it’s own built-in controller simple_flight using either Software-In-The-Loop (SITL) or Hardware-In-The-Loop (HITL). It is generally used for testing software based on Computer Vision, Deep Learning, etc. & for generating large amounts of visual data which is essential for tasks such as Reinforcement Learning & Deep Learning.
During this summer as my GSoC 2019 project, I will endeavour to add active support for AirSim for ArduPilot SITL, specifically Copter platform. AirSim as a more advanced & realistic simulator (more resource hogging as well) can provide a basis for working on VIO algorithms & Obstacle Avoidance systems in Ardupilot within a simulation environment. I feel that this project will increase the adoption and usage of Ardupilot in cutting-edge research areas such as Computer Vision, Deep Learning & Reinforcement Learning for autonomous drones and vehicles, which are becoming increasingly popular these days.
The following is a list of features which I would like to add or support during this project (probably will be added in the same order chronologically) -
1. Support Normal SITL interface
This will involve initially adding support for the older Solo interface (see this for the Airsim side) in the Ardupilot SITL. Then modifying the AirSim code as well the previous SITL code to add lock-step scheduling for proper timing. Furthermore, the entire sensor packet layout will be changed to a JSON or XML-based one which would be easily extendable to add new fields, sensors, etc.
2. Manual Flying using RC
This would be smaller modification to add support for flying using a RC transmitter with a USB interface as seen in the video. Who wouldn’t like to fly in such beautiful & detailed environments available for Unreal Engine 
3. Add support for Lidar Sensor
AirSim has Lidar sensor with a large number of configurable settings which can be seen here.
Possibly even test the Copter Object Avoidance
4. API Support, ROS Integration
AirSim has it’s own Python & C++ APIs for interfacing with the drone as well as Image API for accessing the visual data being generated. Furthermore, there are also ROS packges for the same (see here & here)
Adding support for these APIs & ROS integration will enable developers to write and test their algorithms in simulation before running them on hardware.
Documentation
The documents and reports that will be created along the way will be added to the wiki, specifically:
- Blog Posts on discuss.ardupilot.org for progress reports and to get feedback from the community
- New page in the Wiki for the AirSim setup and usage with Ardupilot similar to the existing ones for the other simulators
Sample programs, guides for testing things like the Copter Avoidance will also be added
Addditional
There are a few other things which I would like to add but which probably won’t be able to get finished in the summer. But these will definitely be worked on after the period ends in that case
1. Multi-Vehicle Simulation
AirSim allows multi-vehicle simulation and has ROS tutorials for the same
2. ArduRover Support in ArSim
AirSim has Car Vehicle as well but doesn’t have any external controller support till now. Adding support for using AirSim as a simulator for Rover will be great addition!!
3. Deep Learning or Computer Vision using Ardupilot & AirSim
Trying out Deep Learning using Ardupilot & AirSim, maybe blog posts for something like Monocular Depth Estimation, Optical Flow for estimating velocity using a downward-facing camera, so many possibilities!
Progress
I have opened a PR for the initial interface, there’s also a Readme with links for setting up AirSim and using with Ardupilot if anyone’s interested. The status of the project will be updated frequently on this blog as well as on any upcoming ones
