Mavlink to FrSky S.Port Passthru Converter for LRS or PX4 Pro

Eric_Stockenstrom · September 14, 2020, 8:16am

Here is the entire function, and they do delete buf lower down:

int WiFiUDP::parsePacket(){

  if(rx_buffer)

    return 0;

  struct sockaddr_in si_other;

  int slen = sizeof(si_other) , len;

 char * buf = new char[1460];

  if(!buf){

    return 0;

  }

  if ((len = recvfrom(udp_server, buf, 1460, MSG_DONTWAIT, (struct sockaddr *) &si_other, (socklen_t *)&slen)) == -1){

    delete[] buf;

    if(errno == EWOULDBLOCK){

      return 0;

    }

    log_e("could not receive data: %d", errno);

    return 0;

  }

  remote_ip = IPAddress(si_other.sin_addr.s_addr);

  remote_port = ntohs(si_other.sin_port);

  if (len > 0) {

    rx_buffer = new cbuf(len);

    rx_buffer->write(buf, len);

  }

  delete[] buf;

  return len;

}

Eric_Stockenstrom · September 14, 2020, 9:23am

Hi Reinhard

Could you try V2.62.2. I added a UDP.flush() after each write. It seems better but I have run out of time today.

Target0815 · September 14, 2020, 9:41am

Hi Eric,

yes with pleasure, I will do later …

Target0815 · September 14, 2020, 10:35am

Same test as above with v2.62.2 …

No UDP connection is established and after a while a reboot is performed:

Starting MavToPass_v2.62.2 …
Display support activated
EEPROM initialised successfully
NOTE! ALL SETTINGS IN EEPROM SET TO COMPILE_TIME DEFAULTS!
EEPROM settings read and adopted
Target Board is ESP32 / Variant is Heltec Wifi Kit 32
Ground Mode
Battery_mAh_Source = 3 - Define battery capacities in the LUA script
RSSI Override for testing = 70%
Mavlink Bluetooth In
Mavlink WiFi Out - WiFi mode is STA/AP
Protocol is UDP IP-Targeted
Bluetooth master mode host RHS_CF is trying to connect to slave Crossfire 0277. This can take up to 30s
Bluetooth done
S.Port on ESP is inverted on tx pin = 14
Wi-Fi mode set to WIFI_AP
AP IP address: 192.168.4.1 SSID: CF_AP
UDP started, listening on IP 192.168.4.1, port 14555
Web support active on http://192.168.4.1
hb_count=1
hb_count=2
hb_count=3
Mavlink good!

[Here the connect of a client is recognized:]

dhcps: send_offer>>udp_sendto result 0
dhcps: send_offer>>udp_sendto result 0
dhcps: send_offer>>udp_sendto result 0
dhcps: send_offer>>udp_sendto result 0

abort() was called at PC 0x401c67ef on core 1

Backtrace: 0x40092874:0x3ffd4860 0x40092aa5:0x3ffd4880 0x401c67ef:0x3ffd48a0 0x401c6836:0x3ffd48c0 0x401b9033:0x3ffd48e0 0x401b9126:0x3ffd4900 0x401b90dd:0x3ffd4920 0x400dc933:0x3ffd4940 0x400d82ed:0x3ffd4980 0x400d84b3:0x3ffd49d0 0x400d94c9:0x3ffd4a10 0x400e3621:0x3ffd4a70 0x4008ef89:0x3ffd4a90

Rebooting…
ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1216
ho 0 tail 12 room 4
load:0x40078000,len:9720
ho 0 tail 12 room 4
load:0x40080400,len:6364
entry 0x400806b8

Starting MavToPass_v2.62.2 …

Eric_Stockenstrom · September 14, 2020, 2:31pm

My apologies, I should have said to test STA mode through your LTE router, but I just fixed AP also. Same V2.26.2. Thanks you!

Target0815 · September 14, 2020, 8:17pm

I have now tested again in STA mode. Crashes I can’t provoke any more, even if several clients request data.

But reading the parameters does not fully work. It works 50-70% with normal speed, then it stops, nothing happens for minutes. Then again a little bit further, stalling … and at some point the loading is finished. But it can also take 15-20 minutes.

Which debug message should be activated so that you can see in the monitor log what happens if the loading is not continued?

Edit:
I can probably see on Mission Planner that “Already got parameter [param_name]” is reported when it slows down.

Edit2:
In QGC at Mavlink Link Status it shows: Loss rate 98%.

Target0815 · September 14, 2020, 9:06pm

I think I got it… The param_index is always reset. These are the places where “Already Got param” is reported:

Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=435
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=10
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=11
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=11
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=436
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=437
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=10
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=10
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=438
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=439
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=10
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=440
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=441
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=11
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=10
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=442
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=10
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=0
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=1
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=2
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=3
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=4
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=5
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=9
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=10
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=444
Mavlink to FC: #20 Param_Request_Read: gcs_target_system=1 gcs_req_param_id= gcs_req_param_index=445

Edit: By the way, this is the same with TCP/IP, so not a pure UDP problem.

Eric_Stockenstrom · September 15, 2020, 6:45am

No it’s not a network problem at all. It’s a consequence of multiple GCSs.

So what’s happening is that each GCS request all parameters, but the FC is confused by the second request, and starts sending again. The two (or 3, 4, 5) GCSs are then out of sync, as far as parameter delivery is concerned. When the parameter download ends, each GCS determines which are missing and asks for those individually. Eventually it settles down as far as I can see, if you are prepared to ignore the inefficiency. This is a Mavlink issue, perhaps overcome by each GCS having a unique identity in the header.

Has the crashing stopped, or do I need to take more measures? I have an idea to change the UDP class library, to reserve the rather large UDP parse buffer during setup and not every time it is demanded, but that is not a good long-term solution.

Target0815 · September 15, 2020, 10:25am

In this test Mav2PT gets the data from a Crossfire via Bluetooth, which gets the data from a FC. Mav2PT then outputs the data to a WiFi and is the only GCS with Mission Planner active.

The output in the monitor log goes from Mav2PT to the Crossfire/FC. Is your explanation still correct then?

As I said, I could not reproduce the chrashing. So far it looks very good.

I will test a few more combinations …

Eric_Stockenstrom · September 15, 2020, 12:59pm

Are you really, really sure that one of the other GCSs didn’t also connect.

Target0815 · September 15, 2020, 3:16pm

No …

(twenty characters …)

Target0815 · September 15, 2020, 3:55pm

Note from another user: if Wifi-Channel changed in WebGui this change is reverted at reboot.

Eric_Stockenstrom · September 16, 2020, 6:51am

Ok, thanks Guys …

EDIT: Fixed. I’ll publish 2.62.3 shortly

Target0815 · September 16, 2020, 5:43pm

Hi Eric,

there are probably still problems …

First I tested if the parameter for the wifi channel is taken over in the WebGUI after changing it. This seems to work.

Then I couldn’t recognize a “Mav Good”. That was because the name of the Bluetooth slave is shortened by one character (instead of “Crossfire 0277” “Crossfire 027” is displayed in the WebGUI and in the monitor log).

So the define Reset_Web_Default is set. Now comes a “Mav Good”, but also then …:

Starting MavToPass_v2.62.3 …
Display support activated
EEPROM initialised successfully
NOTE! ALL SETTINGS IN EEPROM SET TO COMPILE_TIME DEFAULTS!
EEPROM settings read and adopted
Target Board is ESP32 / Variant is Heltec Wifi Kit 32
Ground Mode
Battery_mAh_Source = 3 - Define battery capacities in the LUA script
RSSI Override for testing = 70%
Mavlink Bluetooth In
Mavlink WiFi Out - WiFi mode is STA/AP
Protocol is UDP IP-Targeted
Bluetooth master mode host RHS_CF is trying to connect to slave Crossfire 0277. This can take up to 30s
Bluetooth done
S.Port on ESP is inverted on tx pin = 14
Wi-Fi mode set to STA sucessfully
Trying to connect to RHS_FPV_MOBIL.
WiFi connected!
Local IP address: 192.168.4.23
WiFi RSSI:-61 dBm
UDP started, listening on IP 192.168.4.23, port 14555
Web support active on http://192.168.4.23
hb_count=1
hb_count=2
hb_count=3
Mavlink good!
Bluetooth disabled to free up SRAM for web support
Guru Meditation Error: Core 1 panic’ed (LoadStoreAlignment). Exception was unhandled.
Core 1 register dump:
PC : 0x4008ff96 PS : 0x00060633 A0 : 0x8008e90c A1 : 0x3ffd5a80
A2 : 0x64726fbf A3 : 0x0000abab A4 : 0xb33fffff A5 : 0x00000001
A6 : 0x00060620 A7 : 0x0000cdcd A8 : 0x0000abab A9 : 0x3ffd5a80
A10 : 0x00000003 A11 : 0x00060623 A12 : 0x00060620 A13 : 0x00000000
A14 : 0x00000002 A15 : 0x3ffd48f0 SAR : 0x00000010 EXCCAUSE: 0x00000009
EXCVADDR: 0x64726fbf LBEG : 0x4000c349 LEND : 0x4000c36b LCOUNT : 0x00000000

Backtrace: 0x4008ff96:0x3ffd5a80 0x4008e909:0x3ffd5ab0 0x4014623d:0x3ffd5af0 0x401462fe:0x3ffd5b10 0x401435d1:0x3ffd5b40 0x400dac83:0x3ffd5ba0 0x400dadad:0x3ffd5bc0 0x4008ef89:0x3ffd5c00

Rebooting…

The Bluetooth is switched off to make room for the WebGUI I understand … But it’s bad if Bluetooth is the input for Mavlink …

The partition scheme is set correctly.

Target0815 · September 16, 2020, 5:49pm

Recompiled, flashed and tested again. Now the connection works and when you call the WebGUI this result is shown:

Mavlink good!
UDP client identified, remote IP: 192.168.4.41, remote port: 14550
abort() was called at PC 0x401c66fb on core 1

Backtrace: 0x40092874:0x3ffd46e0 0x40092aa5:0x3ffd4700 0x401c66fb:0x3ffd4720 0x401c6742:0x3ffd4740 0x401b8f3f:0x3ffd4760 0x401b9032:0x3ffd4780 0x401b8fe9:0x3ffd47a0 0x400dc83f:0x3ffd47c0 0x400d8341:0x3ffd4800 0x400d8507:0x3ffd4850 0x400d93d5:0x3ffd4890 0x400e352d:0x3ffd48f0 0x4008ef89:0x3ffd4910

Rebooting…

I can still test it with v2.62.2 if it helps you.

Eric_Stockenstrom · September 17, 2020, 11:16am

Yes we face some difficult choices here because on most cheap boards we are out of SRAM when we want to do BT and WiFi and Web support simultaneously.

But the solution is not as bad as it looks.

If you go into the web support page, and after updating or flashing, we ALWAYS reboot and then BT starts again. All well and good, but …

if a login screen is triggered (someone browses to our root ) we disable BT, and you do nothing on the web side, mav2pt still tries to process bt telemetry while you are waiting, and crash. So I have done this:

Only if the web interface proceeds to the setting page, then I disable BT and free the RAM. I also then #undef btBuiltin so that no more BT I/O is allowed until after the reboot.

Eric_Stockenstrom · September 17, 2020, 11:18am

Fixed …

Target0815 · September 17, 2020, 11:41am

You are quite clever … . I will continue testing later …

Target0815 · September 17, 2020, 5:54pm

If a connection exists and a client, here Mission Planner, is connected via UDP and then the WebGUI is called, there is a restart:

Mavlink good!
UDP client identified, remote IP: 192.168.4.41, remote port: 14550
Bluetooth disabled to free up SRAM for web support. YOU MUST PROCEED TO REBOOT for BT to be restored!
assertion “res == coreID || res == portMUX_FREE_VAL” failed: file “/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/portmux_impl.inc.h”, line 105, function: vPortCPUAcquireMutexIntsDisabledInternal
abort() was called at PC 0x40112773 on core 1

Backtrace: 0x40092874:0x3ffd5a10 0x40092aa5:0x3ffd5a30 0x40112773:0x3ffd5a50 0x4008ffb1:0x3ffd5a80 0x4008e909:0x3ffd5ab0 0x401462b9:0x3ffd5af0 0x4014637a:0x3ffd5b10 0x4014364d:0x3ffd5b40 0x400dacff:0x3ffd5ba0 0x400dae29:0x3ffd5bc0 0x4008ef89:0x3ffd5c00

Rebooting…

This seems normal to me because Bluetooth is switched off. Better would be a message that the connection to the FC has been lost, but it will probably not be that easy.

Personally I can live with it if the WebGUI should only be called when there is no connection. Normally you don’t reconfigure all the time, so that’s no problem.

Then I have tested further:
3x connection established via TCP from Mission Planner = ok
3x connection established via UDP by Mission Planner = ok
3x connection via UDP established by QGC = ok

It is noticeable that it is much slower with UDP, because the message “Already Got param x” is constantly coming. This happens during the whole loading process, but especially at the end, which takes minutes …

TCP is faster, “Already Got param x” also appears, but practically only at the end.

So because of “Already Got param x” you probably have to look.

Otherwise IMHO looks good and runs stable.

Eric_Stockenstrom · September 18, 2020, 6:45am

This looks like a heap fault, probably memory related judging by the diagnostics, but perhaps someone else has seen this before

Could I ask why you need Bluetooth plus WiFi to be activated together? My recommendation would be to avoid compiling in BT support unless it is needed.