Apsync for Jetpack 4.2 on Jetson TX2

mtbsteve · May 25, 2019, 10:06am

I started to make some modifications to the Apsync code in order to make it run on the latest Jetpack 4.2 release along with Ubuntu 18.04.
While my focus is on the Jetson TX2, it should also work on a Nano. However for the Nano, it is still to be determined how to communicate with the FC.

With Jetpack 4.2, no further kernel patches are needed for eg the Auvidea J120 board and the install process is significantly simplified compared to the 3.x versions of Jetpack.

Here is the link to the install instructions on my github:

github.com

mtbsteve/companion/blob/next-tx2/Nvidia_JTX2_JP42/Ubuntu/1_create_base_image_tx2_JP42.txt

# TX2 setup script for use as companion computer with Jetpack 4.2 release

# STEP1: install NVidia image onto JTX2 (skip this step if you already have a factory image created for current release of Jetpack)

# Download the SDKManager to your host machine and follow all the instructions here: 
# https://developer.nvidia.com/embedded/jetpack

# STEP2: backup the TX2 kernel before installing APSync so that you can roll back in case something breaks or messes up

# Follow the steps as explained here: https://elinux.org/Jetson/TX2_Cloning
# The TX2 must be connected to the host machine over USB and entered into recovery mode. Then enter on the host machine:

cd home/nvidia/nvidia_sdk/JetPack_4.2_Linux_P3310/Linux_for_Tegra
sudo ./flash.sh -r -k APP -G mybackup_Jetpack42.img jetson-tx2 mmcblk0p1
# for "mybackup_Jetpack42" add your desired name for the backup image file. This will take about 30 minutes.


# STEP 3 : install apsync packages on the TX2:
# power on the TX
# log onto TX with your credentials as created during the installation of Jetpack 4.2.

This file has been truncated. show original

The code is not perfect and not fully tested yet. Before I start creating pull requests to Apsync master I would appreciate if someone would be interested in testing the installation.
@peterbarker maybe you also could take a look.

nrotella · June 19, 2019, 3:59pm

Thank you for taking the time to make this image. I will likely give it a try soon since I’m having really strange issues with my TX2 + J120 + ArduPilot setup (see MAVLink communication issues (TX2 + J120 Carrier) for details) in which I can’t establish stable communication via MAVProxy or MAVROS.

EDIT: Please see below for comments during installation:

Small bug in sudo 3_wifi_access_point_JP42.sh - “rdupilot” should be “ardupilot”
Everything installed successfully! The wifi hotspot “ardupilot” comes up automatically on boot, and I can ssh from another machine. I have not tested any other functionality.
I ran this on a patched kernel of JetPack4.2 using Auvidea’s J120 patch, otherwise the USB would not work after flashing 4.2
Unfortunately this APSync image does not solve my strange MAVLink communication issues, any suggestions would be welcome

mtbsteve · July 29, 2019, 10:47am

New version of apsync for the Jetson TX2 including now APStreamline

Some more improvements of apsync for the TX2 running Jetpack 4.2. https://github.com/mtbsteve/companion/tree/next-tx2/Nvidia_JTX2_JP42/Ubuntu
I also modified APWeb for use with the TX2: https://github.com/mtbsteve/APWeb/tree/video_streaming

Major changes:

APStreamline included (thanks to @shortstheory)
Lots of fixes (thanks for everyone contributing to testing)
Legacy code removed
Screens for APWeb adopted for the TX2 options

@nrotella I am still using the Jetson Development Board where everything is working smoothly. My J120 is still waiting for deployment - I will come to this once I managed the ZED camera install

yesbeekay · November 20, 2019, 10:49am

Has anyone tested this with the Aetina 310 Carrier board? I am not able to establish Mavlink communication with that board.

mtbsteve · November 20, 2019, 1:05pm

Your board needs to have a serial port enabled. Check under
ls -l /dev/ttyTHS1 (eg Auidea J120) or /dev/THS2 (eg the original TX2 development board)

Btw, the Firmware for the Aetina 310 as listed on their webpage is based on Tegra 32.1. I am using the latest Tegra 32.2.1 version on my TX2.

aman · January 23, 2020, 10:03am

Hi Steve,
Thanks for your detailed instructions on the installation of APSync on the JetPack 4.2 for TX2.
I have come across a roadblock while trying to interface the ZED camera to the TX2 through APSync. The purpose behind the use of APSync is to use the camera for object avoidance.
I noticed the ZED setup files for the TX1 but could not find any such ZED files for the TX2. Is there a possibility for such computation, if yes could you share the setup files for the ZED camera with TX2.
Any help for the ZED camera with the TX2 would be great.
Thanks!

mtbsteve · January 23, 2020, 7:40pm

Hi Aman,
no problem
I do have a ZED fully operational with the TX2
You need to download the ZED sdk and the ZED-ROS wrapper from the stereolabs website.
I have also compiled a full set of instructions + code for my version of the Nvidia Redtail project. It contains also an install script for everything including the ZED configuration.

Don’t hesitate to ask if you have any questions.

rbachtell · March 16, 2020, 6:15pm

@mtbsteve I have narrowed down the issue but I need a bit of help solving. When installing mavlink-router, flogger and apweb I get the following error and consequently the script is not completed.

make
GEN include/mavlink/ardupilotmega/mavlink.h
Traceback (most recent call last):
File “./modules/mavlink/pymavlink/tools/mavgen.py”, line 16, in
from pymavlink.generator import mavgen
File “/home/apsync/GitHub/mavlink-router/modules/mavlink/pymavlink/generator/mavgen.py”, line 12, in ‘module’
from future import standard_library
ImportError: No module named future
Makefile:1615: recipe for target ‘include/mavlink/ardupilotmega/mavlink.h’ failed
make: *** [include/mavlink/ardupilotmega/mavlink.h] Error 1

this where it ends and that is why nothing more is written to the rc.local file. The file ‘future’ is in place where it should be on the TX2 as referenced on a fully working TX1 apsync.

thanks for the help

mtbsteve · March 17, 2020, 6:43am

@rbachtell That indicates your Python environment is either broken or incomplete. To me, it looks like that the compatibility layer between Python 2 and Python 3 is missing and/or some Python environment settings are wrong.
Google finds plenty of solutions, like:

or this one:

Disclaimer: I am not a Python expert - you may get back to the specialist forums to seek for further support.

rbachtell · March 17, 2020, 3:49pm

Thank you for the suggestions, I will follow with those as I have time. I am not a developer or a programmer or fluent in Linux, I am however experienced as a beta tester for the Ardupilot community and as I have said before successfully implemented @peterbarker apsync/openkai using the exact procedures. Please don’t take offense to my reply, I am continually trying to learn as much as I can and that is why I continue with questions.

Here is my understanding of the install procedure:

flash proper tegra OS followed by install nvidia SDK (4.2.2)
follow the install procedure listed in the mtbsteve/redtail wiki
the scripts will install all essential dependencies and program files
If anything fails to load it should not be a problem resident in the TX2 rather a problem in the repo as nothing else is introduced to the environment.

It is my belief that as an end user and one willing to be a part of the process we can expect any feature open to the community will perform as it should. If not then it is our responsibility to help with the diagnosis to the limits of our ability. It is with that goal that I continue to push this issue, an eagerness to learn more and help refine the product.

Thank you for your time and effort

Randy

mtbsteve · March 18, 2020, 1:47pm

Thanks, all good
Just to put things into perspective - I am an end-user too, and I am not part of or related to the Ardupilot dev team. So what I am posting on my Github is purely the result of my hobby, in the attempt to share it with others so that they can leverage from it - as I am benefiting from others in the community when I am leveraging their stuff.

From my learnings over a year now since I purchased a TX2 I can tell that its for sure more complex to deal with than lets say a Rasperry. Every new version of Jetpack introduces new stuff and may obsolete previously existing stuff. Every breakout board comes with its own kernel patches and eventual changes. There are cross-dependencies between libraries and versions from different sources. So without some knowledge about Linux, Jetpack and the underlying development tools it will be challenging to implement.

I very welcome PR’s and suggestions, but I cannot remote debug nor anticipate any possible setup issue - for Jetpack, you could write a book.

Therefore it usually makes sense to first search for a solution on the Internet - as I am doing it when I try to resolve your issues (and mine as well)

To conclude - if you want to implement Project Redtail you will deal with a much broader complexity than APsync. Redtail uses basically the full spectrum of the Nvidia AI stack plus ROS. And my documentation covers only the bare minimum.

rbachtell · March 18, 2020, 2:16pm

Thank you for the detailed response. I truly appreciate that you have taken the time to help me resolve my issue. I know that this project is perhaps beyond my abilities but that I guess is my attraction to this type of thing. If I find solutions I will share.
For now the weather here has been good and there are a couple of epic trails that need to be ridden

Randy Bachtell

rbachtell · March 20, 2020, 3:10pm

Steve, I found the solution. Python 2.7 is not installed with the script only an empty folder. I never thought to open the folder. I installed 2.7 and the installed Py2 future and mavlink-router, dflogger, and apweb installed without error and apsync works perfectly.

Cheers, RB

andriots · April 16, 2020, 7:03pm

Hello,

First of all, thanks for your guide! It has been huge help. I’m trying to set this up on Jetson TX2 and Connect Tech Spacely carrier board.

Secondly, I can confirm that Ubuntu installs well on newer Jetpack 4.3.

All of the software installs without errors however when I start doing the testing procedure I do not see the expected results from “sudo screen -list”:
apsync@apsync:~$ screen -list
There are screens on:
6154.mavlink-router (16.04.2020 21:34:58) (Detached)
6139.cherrypy (16.04.2020 21:34:58) (Detached)
6129.DataFlashLogger (16.04.2020 21:34:58) (Detached)
3 Sockets in /run/screen/S-apsync.
apsync@apsync:~$ sudo screen -list
No Sockets found in /run/screen/S-root.

As I’m completely new to this I’m guessing this is apweb issue. However the webserver is running:

From here I see that MavLink baudrate is not defined and no data is coming in from gyro or accelerometers etc. Setting it manually does not resolve my problems:

I killed the apweb process that autostarts
started it again by issuing: $ sudo web_server -p 80 -f 14755 -b 1500000

Now the interface shows that MavLink baudrate is set. However no data appears.

I’m using PixHawk as flight controller and it has been set up according to this guide(up until TX2 setup):
https://ardupilot.org/dev/docs/companion-computer-nvidia-tx2.html

Also is there a way to determine, which /dev/ttyTHS[1/2/3] is the correct one? I’m only asking because I’m not using J120 as the carrier. I understand this is serial connection, so I tried connecting to either three of these with screen and see which one responds (screen /dev/ttyTHS1 etc) but they all yield in blank black screen(which according to google means that they are in use).

I’m willing to put in the hours, but I need to know which part of the software is malfunctioning to debug it.

Any help would be greatly appreciated,

Andri

mtbsteve · April 23, 2020, 6:39am

Hi Andri,
thanks for your questions and sorry for my late reply.
Your screen -list output is correct.
Can you please run ps -ef | grep [w]eb_server
If you get something like:
root 6145 6080 0 07:48 ? 00:00:00 ./web_server -p 80 -f 14755
apweb is up and running as well. There is no need to manually set the baudrate, however it doesn’t harm to do so. The baudrate is set in the file config.env and used during the install process.

Since you are not getting any values for uptime and voltage as well as for other values from the FC, I assume the serial connection between your Pixhawk and the Jetson is incorrect.

I cant tell you how to determine the right serial port since I have used only the original dev board and the Auvidia J120.
To test if everything is basically working, I would suggest to use the original Development Board. With that, he right TELEM_SERIAL_PORT=/dev/ttyTHS2 (you would need to reinstall apweb). Please check also the right configuration of the serial interface on the Pixhawk side, eg if you are using Serial2:

SERIAL2_PROTOCOL = 1 (the default) to enable MAVLink on the serial port.
SERIAL2_BAUD = 1500 so the Pixhawk can communicate with the TX2 at 1,500,000 baud.
LOG_BACKEND_TYPE = 3

Please check also the cabling, the right connection between the Pix and the Development Board is described here:

I hope its not an issue with Jetpack 4.3 - I did not try this version. But knowing the many (often incompatible) changes between the different Jetpack releases I would not rule this out.
Pls let me know how it goes.

andriots · May 7, 2020, 4:23pm

Thank you mtbsteve for looking into it.

I reverted back to jetpack 4.2 just in case, but the problem persists so I assume Jetpack 4.3 is also okay.

I debugged the serial connection, I’m 99% sure /dev/ttyTHS2 is TX/RX pins 12/14 on ConnectTech Spacely. I connected a radio telemetry device to these pins using the same connector, and after connecting with ‘sudo screen /dev/ttyTHS2’ and typing random stuff - there is data moving in sync with my keypresses. When I start the software bundle, I can also see the data moving.

Also for future reference for any googlers - Spacely carrier board pin 2/4 TX/RX are linked to /dev/ttyS0 according to my tests. Due to the fact that there is some getty process hogging it, I decided to use the UART2 pins.

I’m completely new to drones and Pixhawk - I did not know that I needed Ardupilot Firmware to even enable these parameters. This part is fixed by now.

I also connected the radio telemetry device to Pixhawk TELEM2 port and there is some data coming from it. So I assume the data is moving both ways but not recieved/processed correctly on either end(I suspect Jetson side).

I’ve monitored the logs for a bit now, and something seems off in mavlink router screenlog:
./mavlink-routerd -c mavlink-router.conf
Open UDP [4] 127.0.0.1:14655
Open UDP [5] 127.0.0.1:14556 *
Open UDP [6] 127.0.0.1:14755
Open UDP [7] 127.0.0.1:14765
Open UDP [8] 10.0.1.255:14550
Open UART [9] /dev/ttyTHS2 *
UART [9] speed = 1500000
Open TCP [10] 0.0.0.0:5760 *
3 messages to unknown endpoints in the last 5 seconds
6 messages to unknown endpoints in the last 5 seconds
5 messages to unknown endpoints in the last 5 seconds
5 messages to unknown endpoints in the last 5 seconds

When I set the log level to debug I see that UART messages are sent, but I think the addresses? are wrong.
UDP: [8] wrote 21 bytes
Endpoint [9] got message to -1/-1 from 254/17
Known endpoints:
Endpoint [9] accepted message to -1/-1 from 254/17
UART: [9] wrote 21 bytes
UDP: Got 10 bytes [6]
UDP: Got 1 bytes [6]
UDP: Got 2 bytes [6]
Endpoint [4] got message to 1/0 from 67/72
Known endpoints:
Endpoint [5] got message to 1/0 from 67/72
Known endpoints:
254/17
57/57
Endpoint [6] got message to 1/0 from 67/72
Known endpoints:
67/72
Endpoint [7] got message to 1/0 from 67/72
Known endpoints:
Endpoint [8] got message to 1/0 from 67/72
Known endpoints:
Endpoint [9] got message to 1/0 from 67/72
Known endpoints:
Message to unknown sysid/compid: 1/0

This seems as if sysid/compid is wrong? In ground control Mavlink ID is set to 255.

I’m sorry I didn’t specify my setup:

I’m using Pixhawk2 Cube, with ProfiCNC Hex. It’s just sitting on my table most of the time powered by laptop USB port which I also use to connect with QGroundControl. There is no actual drone connected to Pixhawk, since I’m just trying to get the Jetson TX2 connected with it.

Due to my setup I was suspcious that telem packets are being sent over microUSB when I power it with laptops USB port, so instead I powered it with 5V adaptor straight from the power outlet, just to be sure but it did not yield results.

tl;dr; version
It seems that both ends of the serial connection are sending data but nothing is recieved. It might be something to do with sysid/compid.

mtbsteve · May 8, 2020, 6:46am

It looks like an issue with the serial connection to me.
I am getting the exact same log results (“xx messages to unknown endpoints …”) when I disconnect the serial connection to the Cube.

The TX2 is waiting for the heartbeat message from the Pixhawk over mavlink in order to set the communication parameters.
The sysid for the vehicle is set to 1. The sysid for groundstations like QGC is set to typically 255. See here how it works:
https://ardupilot.org/dev/docs/mavlink-basics.html
or:
https://mavlink.io/en/

It looks like you never receive the heartbeat message from the cube.
Therefore I suggest you start with the original Jetson developer board - there we know that it works

andriots · May 8, 2020, 7:32am

Unfortunately I don’t have a developer board but I’ll try to get this working with Rpi3 manual setup in order to have a working solution to compare with. Will report back my progress.

darkratio · August 5, 2020, 4:55am

hi @mtbsteve i tried to follow your repository for companion , i followed the exact step , but still I am having error . whenever I try to use the command sudo screen -list , it says no socket found but on screen list it shows all the 3 sockets as shown in your documentation. please help me with this, I am clueless here as to what went wrong here.

mtbsteve · August 7, 2020, 8:07am

@darkratio for some reason sudo screen -list doesn’t show the root processes anymore with the newer Jetpack releases. Thats ok as long as the respective apweb processes are up and running.
Lets take a step by step approach.

On the TX2, please run: head -n 1 /etc/nv_tegra_release and paste the output.
On the TX2, what does screen -list display? You should get:

apsync@apsync:~$ screen -list
There are screens on:
	5959.cherrypy	(12.03.2020 08:40:23)	(Detached)
	5870.DataFlashLogger	(12.03.2020 08:40:23)	(Detached)
	5759.mavlink-router	(12.03.2020 08:40:23)	(Detached)
3 Sockets in /run/screen/S-apsync.

If can not see those 3 screens, please run cat /etc/rc.local and paste the output.

Check for apweb process: on the TX2, please run ps -ef | grep [w]eb_server and paste the output.
mavlink:
Can you please run:
ps -ef | grep [r]outerd and paste the result, then run:
head -n 20 ~/start_mavlink-router/screenlog.0 and paste the output.