GCS Failsafe does not work (for me)

Hi, I am flying my drone with an onboard companion computer. The computer is connected to the Pixhawk with a serial link and uses Mavlink to send a heartbeat once a second.

I run my program to use GUIDED mode to takeoff then fly about 15 metres away and then I kill the program. i.e. there are no more heartbeats going to the drone for >5 seconds. This should cause a GCS failsafe because I have these params set:

I tried FS_OPTIONS set to both “Continue in pilot controlled modes” and to “Disabled” but neither changed the situation.

Sadly the drone just stops in mid air and loiters. The failsafe behaviour should do an RTL. I initiate an RTL myself with the transmitter and after landing I try to take off again but I cannot because the GCS failsafe has not been cleared.

This is looking to me like a bug but I wanted to check first with the experts. Dataflash here (password is my username):

https://www.dropbox.com/s/o43vibqs31wlral/gcs_no_failsafe.zip?dl=0

Edit: sorry for all the edits, but in case it is useful info I have one of the serial ports sending telemetry to my FrSky receiver (X-8R) which transmits it back to the transmitter for display on the LCD, however I believe this is just a one way protocol.

Thanks for any tips.

P.S. I searched the forums for others with the same issue before I posted.

1 Like

Are you sure that the computer is identified as ground station (and not as component of the vehicle)?

https://ardupilot.org/dev/docs/mavlink-routing-in-ardupilot.html

1 Like

I have just read over this:

https://ardupilot.org/copter/docs/common-frsky-telemetry.html

And I do not believe that the Frsky Telemetry is using Mavlink. I appreciate the input all the same.

The problem is also present in SITL so anyone can verify this as long as they set “FS_GCS_ENABLE” first because the SITL default setting is for it to be disabled. I just take off and hover at 10m then stop my program which is transmitting a heartbeat. 10 seconds later the drone is still hovering in place.

How are you doing timesync?

I just use mavlink_msg_timesync_encode() to send a timesync packet and time how long it takes to receive it back. Is that what you mean? I don’t see how it is relevant.

Our GCS failsafes started to work once we correctly configured the timesync according to the ardupilot wiki. But I do not remember any other details.

1 Like

Timesync doesn’t do anything by itself though does it? It is just a ping message that returns the unique ID that you sent to the autopilot.

In my case, I send a timesync, wait for the response, and use that as the round trip time. I then receive a systime message from the autopilot (time on autopilot is set by GPS) and subtract half the round trip time to allow me to set the companion time. In this way, the companion time will be close to the autopilot time.

I will look at the arducopter source code shortly to investigate this unless you have any other ideas, and then I will submit a bug report.

Cheers!

The problem is this in the Arducopter source:

GCS_MAVLINK::handle_heartbeat() is called once per second even if no GCS is attached which calls sysid_myggcs_seen() which updates _sysid_mygcs_last_seen_time_ms (i.e. last communication with the GCS).

With SITL, I see that handle_heartbeat() is called in 2 places (about once per second for each):

  1. MAVLink_routing::check_and_forward()
  2. GCS_MAVLINK::handle_common_message()

So somewhere in SITL a heartbeat is being generated and appears as a GCS communication. Anyone know why this is happening?

My guess is that the heartbeat is generated by the virtual drone and somehow there is confusion in the Arducopter source code about it being a GCS heartbeat when in fact it is a drone-generated (Arducopter) heartbeat.

EDIT UPDATE:
The SITL drone gives me sysid = 1 and compid = 1.
My program uses sysid = 1 and compid = 0.

When I am not running my program, the SITL console tells me it is receiving heartbeat messages with sysid = 255 and compid = 230.
I then run my program to talk over Mavlink with the SITL drone and see my messages correctly triggering a call to handle_heartbeat(), but I also see the 255/230 messages.

The source of these messages remains a mystery to me.

I believe the problem is caused by Mavproxy.

It seems to change the sysid/compid of the heartbeat sent by Arducopter into different values that are then bounced back to Arducopter which sees them as coming from a new GCS.

i.e. a GCS failsafe is impossible if you are using Mavproxy.

@arduouspilot,

I think that SYSID_MYGCS can be used to limit which system ids the GCS failsafe monitors so if all the programs on the companion computer are using different system ids than the ground station then I think the GCS failsafe should work.

1 Like

Thanks a lot. GCS failsafe now works for me in SITL, and I expect it will work with the actual drone using Mavproxy too. I’m surprised I missed that param when searching for “GCS” but I’m grateful for you stepping in. Here are the params that worked for me:

Edit: for anyone reading, this is also possible a way to get things working in SITL at least: How can I trigger GCS failsafe in SITL?

1 Like

I’ve just been flying and GCS behaviour in case of failsafe still does not RTL, although once I have landed, I cannot takeoff because of “GCS failsafe”.

I have noticed also that my drone does not RTL with critical battery either so the problem is probably connected.

Hi @arduouspilot,

I’ll probably need to see an onboard log to help too much more.

Is it possible that the companion computer is changing the autopilot’s mode? Maybe the vehicle is going into RTL but is then being switched out of RTL?

Also remember that the vehicle needs a position estimate (aka GPS lock) in order to RTL. If it can’t RTL then it will normally Land if the GCS triggers.

1 Like

Apologies, should have posted a log. Here it is with my username as the password for the zip:

https://www.dropbox.com/s/nmmv4ccls6vz7tp/2022-05-06%2012-49-34.bin.zip?dl=0

Params are here:
params.param (17.0 KB)

The program is running on a companion computer (C++) and is connected by a serial link to the Pixhawk. The mission was basically:

12:49:35 Takeoff to 20.0m.
12:49:57 Fly to (-10.0m, 20.0m, 20.0m) relative to home position.
12:50:13 I kill the Mavlink program so no more heartbeats sent to Pixhawk after this time.
12:50:27 The GCS failsafe is not causing a RTL so I decide to manually change the mode from GUIDED to RTL with the transmitter. (You see a brief switch to PosHold because I have to use a 3 way switch to get to RTL on my transmitter)

Thanks a lot for looking at this.

P.S. I’m not sure if it’s related to this but my critical battery behaviour does not RTL either when the battery gets low.

Okay, I have another log. It is simply a set_position_target followed by killing the GCS program at 14:44:56 that is running on a companion computer.

I changed SYSID_ENFORCE to “Not enforced” since the previous screenshot of my params and am now on Copter 4.2.1.

I hasten to add that Radio Failsafe works fine (when I turn off my Tx). Just having no luck with doing an RTL when the Companion Computer stops talking Mavlink. :-/

Thanks.

Do you have a mavlink gimbal?

1 Like

No; I have made a diagram to better explain my system:

Edit: I am wondering whether my use of S-PORT FrSky telemetry passthrough is causing a bug in Arducopter. Unfortunately it is hard for me to disconnect it as it is inaccessible without lots of unscrewing/ungluing.

Edit 2: I have SYSID_MYGCS as 255 and the actual ID of my companion is 0. I’m unsure of the use of this variable but I suppose setting it to 0 is what I want?

I had a similar problem too. There was no failsafe when working in Sitl. The solution that worked for me is very simple. I set the FS_GCS_TIMEOUT parameter to 1 and did a Force Save. From now on, when I close GCS, vehicle enters failsafe mode, and when I turn GCS back on, failsafe is cleared.

1 Like