I have been working to adapt a ESP32 AI project i found for use with Ardupilot. It basically uses a ESP32-CAM module to capture video that is then sent to an android app for processing and commands are then sent back to the ESP32-CAM to drive the motors.
It can drive manually, object identification, object tracking and lane keeping. In order to get it compatible with Ardupilot I have modified it to output mavlink commands that Ardupilot can then act on rather than using brushed motor outputs.
I have made 3 versions so far but I haven’t been able to test them yet outside.
The first version sends Mavlink guided commands so that it should behave like the video when guided mode is selected in Ardupilot.
The second version sends Mavlink RC override commands so it should work in all modes.
The third version is an experimental idea I had for lane keeping, when the camera detects the edge of the lane it inserts an object into the avoidance database that ardupilot proximity can then avoid rather than needing to override the flight controller.
The one thing I have not been able to make any progress on using with ardupilot is the object detection, the android app doesnt send the identity of the object back to the esp32. I want to be able to use the camera to identify things like people or pets that get too close so I can adjust the speed of the rover to slow it down before the proximity lidar has to make evasive manoeuvres. I have tried compiling the app in android studio but its over my head.
There are dedicated AI cameras out there that can probably do all this better but none of them come close to the price of a ESP32-cam.