Wheelchair guidance

ArduRover Miscellaneous
1

The project goal is cooperative human/machine navigation and user interface development.

Plan a route along a paved nature trail, AI behaviors can interrupt or modify the mission plan, any input from the onboard joystick overrides anything else and is obeyed immediately. Eventually I want the AI to recognize common navigation situations and generate a suggested mission plan. Example: enter through a doorway, follow the edge of a road, remain centered in a hallway, reverse into an empty elevator.
I use a wheelchair to limit weight bearing and shearing of my injured lumbar spine. This requires being reclined so far that I have difficulty seeing forward to navigate. An overhead touchscreen, forward camera, joystick, and seating buttons are my interface to the world.

Seeking overall project guidance to remain compliant with developer code of conduct while employing ArduRover on a human occupied wheelchair.

https://ardupilot.org/dev/docs/developer-code-of-conduct.html#our-pledge

  • ArduPilot is NOT certified for use in applications where ArduPilot is effectively in control of human lives. Members of the development team must not knowingly assist in projects where ArduPilot will be in control of human lives. “In control of human lives” includes but isn’t limited to manned aircraft.

At no point will ArduPilot be “In control of human lives”. I will be either onboard ready to take over manual control in an instant, or the chair will be unoccupied (verified by a seat sensor).
I am experienced building robots and wheelchairs, but new to Ardupilot. I explicitly hold harmless ArduPilot, developers, and forum members who assist in this noncommercial personal project.

A Beagle Bone Blue is on its way to serve as FC, let me know if I should be using something else. CC is a 4Gb Jetson Nano with Intel Realsense D455 depth camera and ultrasonic distance sensors for obstacle identification, and avoidance. CC is attached to a portable touchscreen which will be the main user interface besides the wheelchair joystick.
The wheelchair is a 2020 Permobil M3 which uses R net (CAN) control system and 24v battery. It has been successfully remotely controlled via WiFi using GitHub - redragonx/can2RNET: This repo has code and documentation to control power-wheelchairs with R-Net electronics. python software.
When a GCS is needed, it will be either a R Pi 400 w/touchscreen, or a Samsung S9 running in Dex mode on a touchscreen.

Community input is appreciated to help select most appropriate GCS, configure onboard driving interface, and write the necessary code to interface with can2rnet library.

I’m so new at this I don’t even have firm questions yet, but I know from searching “wheelchair” on the forum that the ethical issue would come up. While waiting for my FC to arrive I hope we can tackle that and agree that this is an Ok project. If anyone thinks this type of project would violate the developer code of conduct, lets talk about that please.

Thanks.

Dustin Maki
slomobile

2

I am not a dev team member, but I am quite familiar with Rover automation as well as common community practices.

The moment that a human occupies a vehicle where ArduPilot is in control, the code of conduct is potentially violated (see Randy’s post below).

That doesn’t stop you from proceeding with your project. What it does mean is that the development team may distance themselves from your project and hesitate/refuse to provide help, while likely cautioning you that ArduPilot was never intended to operate a vehicle where a human is onboard.

For what it’s worth, I think it’s a very interesting concept that’s worth pursuing, but ArduPilot may not be the best autopilot firmware choice.

I’ll defer to the dev team for further guidance, in case I’m missing a nuance. @rmackay9 leads Rover development and may wish to weigh in here.

3 Likes
3

That is unfortunate, but understandable.

One consequence of moving forward with projects like this is that they may eventually be tested by FDA or other official bodies. Which could result in ArduPilot becoming certified. It would be nice if ArduPilot had a subproject where human rating were the eventual goal.

For what its worth, in this project all of the chair’s OEM safety systems remain in tact, unmodified, and functional at all times. The only control authority ArduPilot would have is emulating a physical joystick movement. All of the acceleration and speed limits applied by chair manufacturer and approved by FDA and Medicare remain in place.

ArduPilot in this case would serve only as a drivers aid.

Is there a different autopilot firmware which would be more appropriate?

4

You can chech with CubePilot, they have certified version of the CUBE at a commercial price.

1 Like
5

Hey Dustin! I sincerely hope you find the resources continue with your project. Besides your personal situation, your work could help many others with mobility issues. It’s a worthwhile project and I like it.

I can also understand the Development Teams reluctance to help. I can easily see this resulting from the fear that a brilliant kid would climb onto a big drone he made and fly away…with tragic results.

Meanwhile, I can offer no expertise in the control system over the basics. I’m just learning here.
However, if you have any mechanical/electrical issues, I’d be happy to voice an opinion elsewhere. Best of Luck. Onward!

2 Likes
6

I think the wording in the CoC is what we go by, “ArduPilot is NOT certified for use in applications where ArduPilot is effectively in control of human lives”. In the case of flying vehicles, it’s very clear that a human onboard is violating this but it’s a grey area when it comes to rovers and boats especially slow moving ones.

I think if there’s a button you can easily push that takes AP out of the loop and stops the chair it could be OK.

Thanks @Yuri_Rage for pinging me.

3 Likes
7

Thank you. I appreciate that. Onward!

1 Like
8

Glad to hear there’s a bit more gray area than I expected!

1 Like
9

Hi Dustin

Welcome and don’t feel alone.

I am in the process of building a rover that will be navigated by a low cost sub decimeter navigation device as part of the process of proving the accuracy of the device.

I am using a BeagleBone Blue as FC and am experiencing a number of problems but am getting there.

One of the greatest challenge over here is components as very little is available locally

All the best

Anton

1 Like
10

Thank you, it feels good to be part of the community.

My first stumbling block is figuring out how to write the CAN driver for the Rnet device.
The raw functionality already exists in can2Rnet, I just need to put it in a form useful to ArduPilot.

I’ve seen Welcome to the ArduPilot Development Site — Dev documentation

  • UAVCAN* - Lightweight protocol designed for reliable communication in aerospace and robotic applications via CAN bus. ArduPilot is using the Libuavcan, which is a portable, cross-platform library written in C++ with minimal dependency on the C++ standard library.

That links to https://opencyphal.org/ which links to a guide, which says

The guide intentionally leaves out the details pertaining to the implementation libraries and other specific software products, focusing on the more abstract matters instead. To find documentation for specific software, please consult with the front page at opencyphal.org .

One familiar with v0 will have no trouble migrating to v1 because, in essence, it relies on the same core concepts just having a few bits shifted around. The specification of Cyphal/CAN is hardly five pages long, so implementing it from scratch should be generally a no-brainer and it would barely take more than a few hundred lines of code but even that is unlikely to be necessary given that there are portable MIT-licensed implementations available.

So the guide appears to be circular in nature,
relies on knowledge of the previous implementation which it heavily criticizes,
and manages to be insulting to those unfamiliar by calling it easy.
Can someone just point me toward the codebase of an existing CAN peripheral driver?
For use as an example.

11

Hello

Looking at this github GitHub - redragonx/can2RNET: This repo has code and documentation to control power-wheelchairs with R-Net electronics. (that you posted) it may be possible to write an interface for the controler.

I suggest you first try the Raspberry pi example with the CAN interface board.
If it work successfully it might be possible to implement it using MavLink commands or reading (Decoding) PWM motor control signal from Flight Controler.

I did something similar with Wheel Motors using ARDUINO interfaced to PWM on FC and controlling the CAN/PPS Motor controler

12

I’ve implemented the Raspberry Pi / SocketCAN board and it works. Since the flight controller has a CAN interface and the chair controller is CAN, I wanted to eliminate PWM/PPS entirely. I was looking for the AP driver source for something like Toshiba CAN ESCs — Plane documentation
My Google fu is failing me and I cannot seem to locate it.

A similar approach could enable the CAN motor controllers frequently used in FIRST Robotics Competition. I have Talon SRX on hand, so that may be the the first driver I write.

Edit: I just came across this page which explains a lot.
https://ardupilot.org/plane/docs/common-uavcan-peripherals.html

After extended discussions within the UAVCAN consortium it was decided that the best solution was to continue development of UAVCAN v0 under the name DroneCAN.

Then confuses me again. DroneCAN Setup — Plane documentation

Detailed description of (DroneCAN) protocol can be found at https://uavcan.org/

Which of course redirects to opencyphal.org which is UAVCAN v1, not v0

13

Maybe I could be implemented as a LUA script, @iampete might comment on this

14

Whats up with this?

AP_CANManager: remove ToshibaCAN support

committed 03:08AM - 10 Jun 22 UTC

rmackay9 rmackay9

15

That one got removed, because it was no longer used, but You can use KDECAN as basis, or you can use the removed one and change it.

You want to add a new one anyways, right?

1 Like
16

Please correct me if misunderstand the process.

I clone my local build environment.
Copy contents of ardupilot/libraries/AP_KDECAN at master · ArduPilot/ardupilot · GitHub into something like AP_CAN2RNET.
Modify AP_CAN2RNET.cpp and AP_CAN2RNET.h to provide the functionality of can2rnet.
Follow the Toshiba removals as a roadmap of where to add #includes and references to AP_CAN2RNET as option 12
Test, test, test, …
Fork ArduPilot
Merge my changes
Submit pull request.

17

first fork, than do the rest!

1 Like
18

A Lua based CAN driver might be easier to implement. I am considering doing similar for a client project that uses some very complex motor controllers with a CAN interface. Unfortunately, I don’t think I’ll be able to share the details of that beyond simply reporting whether or not I achieve success.

1 Like
19

This is all a bit out of my comfort zone, but a Lua based CAN driver is completely foreign to me. Still willing to attempt, I understand why it is the better option, just an unfamiliar language for me. I’m assuming I start here

https://ardupilot.org/copter/docs/common-rcoutput-mapping.html?highlight=lua

Are there any existing open source Lua CAN drivers to look at?

20

Here’s the example from the AP_Scripting library:
ardupilot/CAN_MiniCheetah_drive.lua at master · ArduPilot/ardupilot (github.com)

It appears that the example just demos control by sweeping that motor’s position command across a range of values.

For motion control, one approach would be to physically connect the autopilot’s throttle servo outputs to nothing, and then use Lua to read the commanded throttle PWM values. Translate/map those values to an appropriate CAN message, and send it to the motor controller.

The script could provide updates as fast as about 100Hz, so lag time would be nearly imperceptible, assuming the motor controller could consume and process the messages just as fast.

You’ll not likely find much in the way of useful open source drivers that exist outside the ArduPilot repository, since the Lua engine that ArduPilot runs is rather unique.

Here are a couple of videos I made regarding ArduPilot Lua scripting - I hope they are helpful:
ArduPilot Lua Scripting Basics - YouTube
ArduPilot Lua Demos (streamed live, 16 Apr 2022) - YouTube

1 Like