Servers by jDrones

System periodically gets delayed commands

We have a UGV set up with a Pixhawk Cube Black running Rover 4.0 (Chibios) connected to a companion computer via USB serial (Ubuntu 18.04) at 961400bps. The companion computer runs ROS/mavros. In ‘manual’ mode there is a ROS node that processes output from a joystick (over CAN bus) and publishes to the /mavros/rc/override topic. This has been working just fine but all of a sudden we have encountered episodes where the commands from the joystick are delayed by up to 10 seconds. We are confident that the delay is not in the ROS system but are lacking tools to diagnose the mavlink mavproxy<->ardurover connection. Something is delaying the processing of these commands and they are getting buffered somewhere and we would like to know the cause. Using tcpdump we can see that the delay is after mavros transmits to mavproxy. Has anyone else encountered this? Can anyone suggest tools that could be used to see what is happening?

Servers by jDrones