Servers by jDrones

ArduBee, a Ready-To-Fly Micro drone for Education and Swarming

We used a simple global shutter USB Camera from leopard imaging (https://leopardimaging.com/product/usb30-cameras/usb30-camera-modules/li-usb30-m021/).
Also, worth looking at is the NanoPi Neo Air LTS (https://www.friendlyarm.com/index.php?route=product/product&product_id=151) as it supports CSI directly.

1 Like

Also, Jevois (http://www.jevois.org/) is another smart camera which works but is painful to work with.

1 Like

To me the Himax HM01B0 camera seems a very good option, gives 320×240 grayscale @ 60fps, is ultra low power (<4 mW) and very small / light (but has a rolling shutter).

2 Likes

Some consideration (free thinking):

  • other then choosing the right camera and the right SBC they must be compatible both HW and SW (driver) - CSI vs USB vs ETH connection and driver (obvious)
  • here we don’t have to consider the best option in absolute but constrained by the weight and size for ArduBee
  • we can consider two solution: one with higher CPU specs (NanoPi variant or RPi3 Compute Module) plus global shutter camera; the other with lower CPU specs but with CNN acceleration (K210 or GAP8/PULP) plus tiny camera (HM01B0 or similar); this second solution is the one with lower power consumption, lower weight and size.

[Edit]
The second solution of the last point, as I said in a previous post, don’t consent us to do computational intensive applications such as Visual Odometry and SLAM but it could be used to make interesting experiments with convolutional neural networks, such as object detection and tracking and, as @guglie said, advanced navigation tasks using DroNet.

2 Likes

How much payload capacity is reasonable for camera + processing power?

1 Like

I would love to contribute with a video.

4 Likes

I think the sipeed products could be the right solution for this project.

The K210 is a powerful alternative to the current STM32 with CMSIS-NN products. I have worked with OpenMV which are powered by F7 and H7 and are less performant than K210.
Here an academic paper comparing the machine vision options available.

The biggest actual limitation to the K210 is the ram size (8MB), but this could be exploited via adding more PSRAM. The K210 can also be overclocked to 800MHz which gives the possibility of VGA at 60 fps.

Someone has already tried to implement a NN for drone autonomous flying by using Mobilenet Classifier here for INAV.

Moreover, I think that it could be possible to port DroNet for this micro and I will try to do so.

Also consider the Audio and FFT accelerator, this could enable predictive failure detection, by listening to the motors and propellers sounds.

4 Likes

I would say over the K210, NanoPi+Coral USB is much more powerful and more usable at a cost of about 10g more weight.

3 Likes

I am testing RPi4 + Coral USB with TensorFlow Lite (obviously), it is a really powerful pair. For now tested only on ready model for object detection with mobilenet_ssd_v2 and pose estimation with PoseNet.

For object detection I see 20ms per inference with Coral and 250ms per inference without Coral.

Here a little video of the output PoseNet:

2 Likes

Anyway for dimension, weight and power consumption, boards based on K210 are winner IMHO.
I have one of this in my hand, I don’t think is more the 5 grams (I don’t have a precision balance) and power consumption max 1W (10$ included the camera).

I was able to run the DroNet model on the above linked board. I Had to convert from the original Keras model to TensorFlow then to TF Lite then to kmodel (for K210) and I obtained a model working at 0.9fps, but I think there is large space to optimizations.

3 Likes

Anyway as stated above

Higher specs solution could be based on he NanoPi Neo Core 2 with optionally an accelerator as Coral USB or Intel NCS2 and a global shutter camera.

Lower specs solution based on … I think you know what.

2 Likes
Servers by jDrones