Internal error 0x4000 l:215 spi_fail

[aaaa (Jaeuk Bae)]

Sporadically, internal error 0x4000 l:215 spi_fail occurs.

The environment looks like this

Pixhawk 6X Pro (Pixhawk6X 002C0025 30335101 33383936)
ArduCopter V4.5.6 ([7ce11b41])
ChibiOS: 6a85082c
IOMCU: 410 2003 411FC231

The devices associated with FC are listed below.

Benewake TF02-Pro _____ (IIC)
Holybro UM982 _________(GPS1 port - USART1 & IIC)
Companion computer ____(FMU Debug Port - USART3)
Tmotor Datalink _________(Telem1 port - UART7)
ELRS receiver _________ (Telem2 prot - UART5)
Siyi A8 mini ____________(GPS2 port - UART8)
Holybro PM06D ________ (Power1 port)
Motor ________________ (I/O PWM OUT)

I can’t find the exact cause, but it seems to have started happening after I changed the Benewake tf02-pro lidar from UART to IIC or connected a companion computer to the FMU debug port.

I also tried changing the IIC power to an external BEC, but that didn’t help.

Critical: PreArm: Internal errors 0x4000 l:215 spi_fail

I’m having this error pop up after arming and disarming every time. One first reboot, everything is green until I arm. After I arm the rover, I get Critical: Internal Errors 0x4000. When I disarm and try to rearm, I get the error in the title. I have two rovers with Mayteksys F405-VTOL with Mayteksys m10q-5883 GPS units onboard. We’ve tried flashing different versions of Ardupilot with no success; I’m at a loss. So, please help me, Obiwan. You are my only hope.

This is the same symptom as in the post above, but the solution didn’t help, and I didn’t see anything else written about this symptom, so I’m posting it here.

Does anyone have any ideas on this issue? Any response would be appreciated.

[Juergen-Fahlbusch]

SPI and IIC are different bus systems. So as the message shows “spi_fail” I would first check the spi connections and devices.

@aaaa

but it seems to have started happening after I changed the Benewake tf02-pro lidar from UART to IIC or connected a companion computer to the FMU debug port.

Did you tried to disconnect the lidar and the companion computer. Is than the failure disappear?

[aaaa (Jaeuk Bae)]

I know that SPI and IIC are different buses. It was just part of the debugging process.

My understanding is that the only part of the pixhawk that SPI is used by is the IMU and the external spi port, and the IMU seems to be recognized as normal even when SPI fails. (I’m using a mini board, so no external spi port)

Debugging is going to be a very hard and long job as the symptoms are intermittent, but I’ll try removing the lidar and companion computer first.

[Juergen-Fahlbusch]

I think also the SD-card might be on a SPI-port and the SD card as not 100% fixed device might can have connection problems due to vibrations.
Just some ideas, no knowledge

[andyp1per]

This is a pretty low-level issue and usually indicates hardware failure. I think first you need to try and identify which SPI device is causing the issue - there are not that many of them and on my Pixhawk 6X pro it looks like its only the IMUs. So I would first try disabling the IMUs one by one using INS_ENABLE_MASK and see if the problem goes away. You may have to make some config changes if your primary IMU no longer exists.

[aaaa (Jaeuk Bae)]

It’s probably a good idea to consider all the possibilities and check them one by one.

[aaaa (Jaeuk Bae)]

Based on your comments, it seems like the only place to have an SPI_fail issue is with the imu on the Pixhawk 6x.

Because of this, the first thing I checked when I got the SPI_fail warning was the gyro and accelerometer values, and in the mission planner the 3 acceleration and gyro values came in fine.

Is it possible to get the imu values to come in normally even with the SPI_fail issue? I’m a little confused. I’m also wondering under what circumstances exactly SPI_fail is triggered. For example, SPI timeout or something…

[andyp1per]

It comes from here: ardupilot/libraries/AP_HAL_ChibiOS/SPIDevice.cpp at e4de4c9b41f933f906100b123a12c8d1c3977850 · ArduPilot/ardupilot · GitHub

The SPI transaction timed out - that doesn’t mean they all will…

[aaaa (Jaeuk Bae)]

After upgrading to ArduCopter 4.5.7, I removed the rangefinder and companion and reproduced the internal error.

This confirms that the companion and lidar are unrelated; unlike before, I cannot receive data from IMU 1 in case of a spi fail. There are also some changes to the warning wording that occur. (internal error 0x4000 l:215 spi_fail → internal error 0x4000 & gyros not healthy)

But another strange thing is that, fortunately or not, it has never happened in flight.

So my current thinking is that the following factors could be causing this issue

1. a malfunction due to heating of the heater during a long bench test

2. not connecting the RC (ELRS)

3. hardware issue with the Pixhawk

I thought powering via USB might be the problem, so I also tried bench testing with batteries, but the same issue occurred when powered by batteries.