Here’s a link to a blog post from the OpenCV AI (aka OAK-D, aka Luxonis) regarding @rishabsingh3003’s recent work to integrate a OAK-D cameras into ArduPilot’s precision landing and object avoidance features (e.g. Simple avoidance, BendyRuler path planning)
We’ve only been working with these cameras for a couple of months but we are already quite excited about their potential. They are relatively light weight, provide 3D depth information and can run AI pipelines right on the camera itself. We’ve also found that interaction with the developers goes very smoothly because they are knowledgeable, helpful and responsive.
Rishabh is the real expert on this topic but from my point of view, there are two parts to successful object avoidance. The “estimation” portion (i.e. where are the obstacles?) and the “control” portion (i.e. how do we avoid them?). Within ArduPilot we have put a lot of effort into the control side and it works fairly well but the estimation has always been a challenge. Most autopilot CPUs are optimised for real-time processing and can’t do a lot of data/image processing which has led us to use either relatively simple filtering and/or has forced us to rely on companion computer algorithms which are difficult to setup. I suspect these OAK-D AI cameras are going to be an important tool in helping improve both the setup and reliability of the “estimation” problem.