GPS instability/reinitializiation during flight (BN880/MambaF405)

Dear all,

i built a new copter using a MambaF405 flight stack and thus i need to use the current 4.1-dev version. Setup went fine, it flies nearly as it should. There is only one issue i never came across before. The GPS shows some instability and delay.

The GPS (BN880Q - Ublox8) is initialized at startup:
[MAV 001:1] GPS 1: detected as u-blox at 230400 baud

it shows 18+ satellites and good accuracy. During flight the GPS reinitializes several times. The message “GPS 1: detected as u-blox at 230400 baud” appears several times during flight and the copter looses its position.

During flight there is also a lag of several seconds in the update of GPS data. Mostly visible in GPS-speed after fast stop.

This behavior is not present with the same GPS on another copter with an OmnibusF4 running AC 4.0.7 stable. I tested three BN880Q and a BN880 all show the same behavior. With AC 4.0.7 the GPS uses only 56k baud rate.

Is there any possibility to limit the baud rate? SERIAL3_BAUD is ignored…
Internet search reveals two links: one with the same problem, unsolved and a development thread on inav - here the solution is to limit baud rate to 36k…

Thanks

Can you connect to a DMA-enabled serial port? It might help.

Thanks, currently i use RX3/TX3 - Thus, I solder the GPS to RX6/TX6 and have a try?

Actually UART3 is already DMA enabled so that’s not the issue

@THKoelnMech,

Thanks for the report. Any chance you could provide a log?

Hi.
I’m have problem with GPS on this board and 4.1 b4, b5 b6 fw -
With DMA on port 6 or 3 - have many Unhealthy GPS Signal - i’m see GPA.delta logs and see that some time i’m have 330ms response.
I’m disable DMA in FW for 6 uart , compile again and test again - more better but again about 1-3 messages in 1 minute.
And some time - GPS reinitializes.
And i’m have many bugs with 4.1 latest -
it feels like some innovations were causing the very unstable operation of the uarts and maybe the entire system as a whole - just like that imu reset - downloading logs from a flash drive - imu reset, connecting via usb - imu reset, etc.

4.0.x dont have this problems - where problems?
Why fw lost gps data ? How debug this?

This is above my knowledge but perhaps you might find some information in this thread? Yuri's 4.1.0-beta Update (GPS yaw - now beta3)

Maybe try setting the gps rate to update at 5hz if it is not already at that?

Did this in ucenter, trying reset to default and configuring again, try disabling auto config in AP - all this not fix this problem…
I’m think need debug this and find where problem…

What else do you have connected? Are you using GPS on a DMA-enabled UART?

1 Like

r9m connected to rcin and sport to serial1 tx.
gps connected to serial6.
i tried with dma and without dma.
esc telemetry connected to serial3

Oh… I’m cant understand - how other peoples flying? everyone is okay?

I’m fix GPS problems but not CRSF rf mode changes -


I’m revert some commits -

diff --git a/libraries/AP_HAL_ChibiOS/UARTDriver.cpp b/libraries/AP_HAL_ChibiOS/UARTDriver.cpp
index 48a1f4b42e..eece68dafc 100644
--- a/libraries/AP_HAL_ChibiOS/UARTDriver.cpp
+++ b/libraries/AP_HAL_ChibiOS/UARTDriver.cpp
@@ -65,19 +65,19 @@ static const eventmask_t EVT_TRANSMIT_DATA_READY = EVENT_MASK(2);
 static const eventmask_t EVT_TRANSMIT_UNBUFFERED = EVENT_MASK(3);
 
 #ifndef HAL_UART_MIN_TX_SIZE
-#define HAL_UART_MIN_TX_SIZE 512
+#define HAL_UART_MIN_TX_SIZE 1024
 #endif
 
 #ifndef HAL_UART_MIN_RX_SIZE
-#define HAL_UART_MIN_RX_SIZE 512
+#define HAL_UART_MIN_RX_SIZE 1024
 #endif
 
 #ifndef HAL_UART_STACK_SIZE
-#define HAL_UART_STACK_SIZE 320
+#define HAL_UART_STACK_SIZE 256
 #endif
 
 #ifndef HAL_UART_RX_STACK_SIZE
-#define HAL_UART_RX_STACK_SIZE 768
+#define HAL_UART_RX_STACK_SIZE 1024
 #endif
 
 UARTDriver::UARTDriver(uint8_t _serial_num) :
@@ -241,7 +241,7 @@ void UARTDriver::begin(uint32_t b, uint16_t rxS, uint16_t txS)
 
     if (sdef.is_usb) {
         // give more buffer space for log download on USB
-        min_tx_buffer *= 2;
+        min_tx_buffer *= 4;
     }
 
 #if HAL_MEM_CLASS >= HAL_MEM_CLASS_500
diff --git a/libraries/AP_HAL_ChibiOS/hwdef/common/halconf.h b/libraries/AP_HAL_ChibiOS/hwdef/common/halconf.h
index 1867345df7..a7a26c6c53 100644
--- a/libraries/AP_HAL_ChibiOS/hwdef/common/halconf.h
+++ b/libraries/AP_HAL_ChibiOS/hwdef/common/halconf.h
@@ -430,7 +430,7 @@
 #if defined(STM32H7)
 #define SERIAL_BUFFERS_SIZE     512
 #else
-#define SERIAL_BUFFERS_SIZE     256
+#define SERIAL_BUFFERS_SIZE     512
 #endif
 #endif

I’m not sure the F405 has enough memory for those changes, but @tridge would be able to tell you more

4.0.7 have 1024 for TX…
And after 15 min at the table i’m have imu reset without this changes…

And with this patch i’m have again 2 this messages after 20 min… and again silence
I’m shure 100% that this not GPS but UART driver problem… And maybe related to new commits (after 4.0.7)

Why downloading logs from flash - make FC stuck , imu reseting, GPS glitch and so on?

01.08.2021 12:03:51 : CRSF: RF mode 2, rate is 152Hz
01.08.2021 12:03:47 : Radio Failsafe Cleared
01.08.2021 12:03:47 : Radio Failsafe - Disarming
01.08.2021 12:03:46 : CRSF: RF mode 1, rate is 157Hz
01.08.2021 12:02:58 : EKF3 IMU0 MAG0 initial yaw alignment complete
01.08.2021 12:02:58 : EKF3 IMU0 tilt alignment complete
01.08.2021 12:02:56 : EKF3 IMU0 initialised
01.08.2021 12:02:56 : EKF3 IMU0 forced reset

Maybe some thread blocking uartdriver?

Downloading logs is a well known issue - the thread hogs the CPU and everything else gets delayed

Ok - after I’m change mem size - some times I’m have gps warnings and after downloading logs within 3-4 minutes .
this means that periodically something blocks the com port processing thread and I see these messages and failures in the gpa.delta logs. As I understand it, these unknown failures sometimes lead to an internal error. How can you track this?

Typically these kinds of issue are either because a thread is hogging the CPU or because the DMA lock cannot be obtained. There are a few things you can try:

  • In mavproxy do ftp get @SYS/dma.txt -, the first time enables the second time gives stats for the intervening period so do the second time a few minutes after the first - this should give you insight into any DMA contention
  • You can build with --enable-stats and then ftp get @SYS/threads.txt - will tell you the thread CPU usage, it should also tell you if any thread is hogging for longer than its allotted time (shows *)

Ok thanks I’m try debugging

Thanks for all replies. I could now do another test flight.

It seems the flightstack with the FC and the 40A ESC has some electrical insuffencies. The 5v and also the 9v feeding for the VTX show strong disturbencies.
I added a 470uF capacitor to both and the voltage is now acceptable stabile. at least for one battery, 20 min flight time, it seems to work. I will test it further.

1 Like

i’m getting dma log -

DMAV1
DMA=1:0 TX= 18 ULCK= 1 CLCK= 2 CONT=50.0%
DMA=1:2 TX= 222807 ULCK= 55356 CLCK= 0 CONT= 0.0%
DMA=1:3 TX= 6486 ULCK= 801 CLCK= 0 CONT= 0.0%
DMA=1:4 TX= 0 ULCK= 801 CLCK= 0 CONT= 0.0%
DMA=1:5 TX= 135514 ULCK= 16832 CLCK= 3 CONT= 0.0%
DMA=1:6 TX= 0 ULCK= 16831 CLCK= 1 CONT= 0.0%
DMA=1:7 TX= 222808 ULCK= 55355 CLCK= 0 CONT= 0.0%
DMA=2:2 TX= 761568 ULCK= 94610 CLCK= 0 CONT= 0.0%
DMA=2:3 TX= 0 ULCK= 94608 CLCK= 0 CONT= 0.0%

Tasks -

TasksV1
Copter::rc_loop MIN= 7 MAX= 20 AVG= 7 OVR= 0 SLP= 0, TOT= 0.2%
Copter::throttle_loop MIN= 12 MAX= 28 AVG= 14 OVR= 0 SLP= 0, TOT= 0.2%
AP_GPS::update MIN= 12 MAX=564 AVG= 64 OVR= 12 SLP= 0, TOT= 1.0%
OpticalFlow::update MIN= 2 MAX= 83 AVG= 3 OVR= 0 SLP= 0, TOT= 0.2%
Copter::update_batt_compass MIN=192 MAX=224 AVG=198 OVR= 24 SLP= 0, TOT= 0.6%
RC_Channels::read_aux_all MIN= 2 MAX= 3 AVG= 2 OVR= 0 SLP= 0, TOT= 0.0%
Copter::arm_motors_check MIN= 2 MAX= 3 AVG= 2 OVR= 0 SLP= 0, TOT= 0.0%
Copter::auto_disarm_check MIN= 3 MAX= 9 AVG= 3 OVR= 0 SLP= 0, TOT= 0.0%
Copter::auto_trim MIN= 2 MAX= 3 AVG= 2 OVR= 0 SLP= 0, TOT= 0.0%
Copter::read_rangefinder MIN= 13 MAX= 32 AVG= 15 OVR= 0 SLP= 0, TOT= 0.1%
AP_Beacon::update MIN= 2 MAX= 92 AVG= 5 OVR= 2 SLP= 0, TOT= 0.7%
Copter::update_altitude MIN= 26 MAX= 31 AVG= 27 OVR= 0 SLP= 0, TOT= 0.1%
Copter::run_nav_updates MIN= 2 MAX= 28 AVG= 3 OVR= 0 SLP= 0, TOT= 0.1%
Copter::update_throttle_hover MIN= 1 MAX= 20 AVG= 2 OVR= 0 SLP= 0, TOT= 0.1%
ModeSmartRTL::save_position MIN= 6 MAX= 11 AVG= 7 OVR= 0 SLP= 0, TOT= 0.0%
Copter::three_hz_loop MIN= 4 MAX= 8 AVG= 5 OVR= 0 SLP= 0, TOT= 0.0%
AP_ServoRelayEvents::update_even MIN= 2 MAX= 64 AVG= 3 OVR= 0 SLP= 0, TOT= 0.1%
AP_Baro::accumulate MIN= 2 MAX= 9 AVG= 2 OVR= 0 SLP= 0, TOT= 0.0%
AC_Fence::update MIN= 4 MAX= 25 AVG= 6 OVR= 0 SLP= 0, TOT= 0.0%
Copter::fourhundred_hz_logging MIN= 2 MAX= 91 AVG= 6 OVR= 10 SLP= 0, TOT= 0.8%
AP_Notify::update MIN= 8 MAX= 71 AVG= 14 OVR= 0 SLP= 0, TOT= 0.2%
Copter::one_hz_loop MIN=999 MAX=999 AVG=999 OVR= 2 SLP= 0, TOT= 0.3%
Copter::ekf_check MIN= 4 MAX= 62 AVG= 20 OVR= 0 SLP= 0, TOT= 0.1%
Copter::check_vibration MIN= 7 MAX= 62 AVG= 13 OVR= 1 SLP= 0, TOT= 0.0%
Copter::gpsglitch_check MIN= 3 MAX= 24 AVG= 4 OVR= 0 SLP= 0, TOT= 0.0%
Copter::landinggear_update MIN= 2 MAX= 37 AVG= 4 OVR= 0 SLP= 0, TOT= 0.0%
Copter::standby_update MIN= 1 MAX= 45 AVG= 5 OVR= 0 SLP= 0, TOT= 0.1%
Copter::lost_vehicle_check MIN= 4 MAX= 40 AVG= 6 OVR= 0 SLP= 0, TOT= 0.0%
GCS::update_receive MIN= 12 MAX=198 AVG= 19 OVR= 1 SLP= 0, TOT= 2.5%
GCS::update_send MIN= 19 MAX=398 AVG= 44 OVR= 0 SLP= 0, TOT= 5.7%
AP_Mount::update MIN= 2 MAX= 88 AVG= 4 OVR= 3 SLP= 0, TOT= 0.1%
AP_Camera::update MIN= 3 MAX= 94 AVG= 9 OVR= 9 SLP= 0, TOT= 0.1%
Copter::ten_hz_logging_loop MIN= 26 MAX= 43 AVG= 30 OVR= 0 SLP= 0, TOT= 0.1%
Copter::twentyfive_hz_logging MIN= 5 MAX= 11 AVG= 6 OVR= 0 SLP= 0, TOT= 0.1%
AP_Logger::periodic_tasks MIN= 7 MAX=100 AVG= 15 OVR= 0 SLP= 0, TOT= 1.9%
AP_InertialSensor::periodic MIN= 2 MAX= 94 AVG= 4 OVR= 19 SLP= 0, TOT= 0.5%
AP_Scheduler::update_logging MIN= 40 MAX= 40 AVG= 40 OVR= 0 SLP= 0, TOT= 0.0%
Copter::rpm_update MIN= 2 MAX= 87 AVG= 10 OVR= 0 SLP= 0, TOT= 0.1%
Copter::compass_cal_update MIN= 5 MAX= 94 AVG= 9 OVR= 0 SLP= 0, TOT= 0.3%
Copter::accel_cal_update MIN= 2 MAX= 81 AVG= 6 OVR= 0 SLP= 0, TOT= 0.0%
AP_TempCalibration::update MIN= 8 MAX= 11 AVG= 8 OVR= 0 SLP= 0, TOT= 0.0%
AP_Gripper::update MIN= 2 MAX= 19 AVG= 3 OVR= 0 SLP= 0, TOT= 0.0%
AP_Winch::update MIN= 2 MAX= 8 AVG= 3 OVR= 0 SLP= 0, TOT= 0.1%
AP_Button::update MIN= 2 MAX= 6 AVG= 2 OVR= 0 SLP= 0, TOT= 0.0%
AP_Stats::update MIN= 6 MAX= 7 AVG= 6 OVR= 0 SLP= 0, TOT= 0.0%
AP_RunCam::update MIN= 2 MAX= 7 AVG= 2 OVR= 0 SLP= 0, TOT= 0.0%
AP_Vehicle::update_dynamic_notch MIN= 9 MAX=112 AVG= 16 OVR= 0 SLP= 0, TOT= 2.2%
AP_VideoTX::update MIN= 2 MAX= 19 AVG= 5 OVR= 0 SLP= 0, TOT= 0.0%
AP_Vehicle::send_watchdog_reset_ MIN= 3 MAX= 3 AVG= 3 OVR= 0 SLP= 0, TOT= 0.0%
AP_ESC_Telem::update MIN= 5 MAX= 6 AVG= 5 OVR= 0 SLP= 0, TOT= 0.0%
AP_Vehicle::publish_osd_info MIN= 3 MAX= 5 AVG= 4 OVR= 0 SLP= 0, TOT= 0.0%
fast_loop MIN=490 MAX=999 AVG=624 OVR= 0 SLP= 0, TOT=81.0%

Threads -

ThreadsV2
ISR PRI=255 sp=0x20000000 STACK=1248/1536
ArduCopter PRI=182 sp=0x20000600 STACK=5368/7168
idle PRI= 1 sp=0x200129A8 STACK=144/352
UART_RX PRI= 60 sp=0x2001D988 STACK=1024/1360
OTG1 PRI= 60 sp=0x2001C2A8 STACK=272/656
monitor PRI=183 sp=0x2000F048 STACK=480/848
timer PRI=181 sp=0x20010268 STACK=1488/1872
rcout PRI=181 sp=0x2000F958 STACK=472/848
rcin PRI=177 sp=0x2000F3D0 STACK=440/1360
io PRI= 58 sp=0x2000E6C0 STACK=1488/2384
storage PRI= 59 sp=0x2000FCE0 STACK=832/1360
UART3 PRI= 60 sp=0x2001B490 STACK=376/656
UART6 PRI= 60 sp=0x2001A9B0 STACK=376/656
I2C0 PRI=176 sp=0x200196B0 STACK=912/1360
UART1 PRI= 60 sp=0x20014978 STACK=256/656
OSD PRI= 59 sp=0x20015730 STACK=680/1360
log_io PRI= 59 sp=0x20016080 STACK=1056/1656
SPI1 PRI=181 sp=0x200135D0 STACK=576/1360
FTP PRI= 58 sp=0x1000B830 STACK=1432/2896

Serial -

UARTV1
SERIAL0 OTG1 TX = 6302068 RX = 43250 TXBD= 1692 RXBD= 253
SERIAL1 UART1 TX*= 125883 RX*= 0 TXBD= 736 RXBD= 0
SERIAL2 UART3 TX*= 0 RX*= 0 TXBD= 0 RXBD= 0
SERIAL3 UART6 TX = 0 RX = 1139904 TXBD= 0 RXBD= 1644

I’m cant understand what normal what no - for this i’m need learn all ardupilot code…
This is normal data from logs?

It looks ok. To get thread CPU usage you need to build with --enable-stats