EKF IMU in-flight yaw alignment complete and copter rolled to nearly crash. (Log and Video)

Hi, I have been facing a weird issue of in-flight yaw alignment complete
I was flying in an open field in the middle of sugarcane field. Most of the flight were fine, copter was stable and easy to control.

Here are the log: https://drive.google.com/file/d/1BGRiShIKKZpbkwUXZjrC76x-eWi8wahJ/view?usp=sharing

First failed flight (Bad flight), this one has Data log, Telemetry log, and video here: https://youtu.be/tIEBXqtQvi0
I armed and took off the copter in Loiter as usual(GPS sat count and HDOP were fine). After it took off for a few meter, I pushed the copter forward. It goes forward for a few meter. Then, Mission Planner says: in flight yaw alignment complete. The copter LED blinks orange-blue. Then, the copter make a hard roll to the right without my command. After I notice that I cannot control my copter in Loiter, I switched to ALT HOLD at about 0.09 in the video. I was able to stop the roll, but it’s still feel very hard to control. I managed to take the copter back and landed safely.

Second successful flight (Good flight), this one has Data log.
Then, after the problematic flight. I rebooted the copter and checked that everything is fine GPS, HDOP, Altitude. I tried arming and took off again. Everything was fine. The flight was smooth and easy to control.

Third failed flight (Worse flight), this one has Data log, and Telemetry log.
I changed the location to fly the drone, it is still an open area in a middle of a sugarcane field. I prepared the drone as usual. Plugged-in, booted, and check the status from the Mission Planner and everything looks fine. I armed and took off. I pushed the copter up for a few meter and pushed it forward. Then, it went heywire. Mission Planner starts to sends out a lot of error, horizontal position, compass, terrain EKF Fail. The copter makes a hard roll to the left. I changed the mode to ALT HOLD and tried to regain control. But the copter starts to yaw like crazy. MP tells me that the copter changes EKF frequently during this chaotic moment. The copter was nearly uncontrollable. I wrestle with it for a minute and finally landed it safely.

All flight uses 1 compass: external one. I checked the vibration and it looks fine. GPS on every flight were fine.
The copter has a LIDAR installed for terrain following.

I looked at the log from both problematic flight, both has a problem with heading since boot up. The heading was incorrect and it was off by nearly a hundred degree. (You can see this in the video of flight 1 that the heading that I took off was not the same as in mission planner telemetry log.) Compass and Yaw (from log) was not giving the same value.

Have anyone experience this EKF IMU in-flight yaw alignment complete problem? Has anyone crashed with this error? What am I dealing with?

No one has any idea??

It looks like a failed magnetometer… perhaps the power on initialization is intermittently failing on the sensor.

Have you installed two GPS,I mean GPS1 and GPS2?

I installed only 1 GPS.

I think I have found the problem. It is with the anti-spark XT90 that I used. It sometime interfere with the compass initialization process.

emmmm,Sensors need to stay away from large current lines

Yes, all sensors are quite far away from the current line.
The problem I was talking about came from the construction of the anti-spark plug.

Anti-spark plug has 2 level of contacts (in the positive line) that is separated by a nylon spacer.
The first level has the anti-spark circuit which included a resistor and a capacitor.
When the first contact is made, power is supplied to the power module (and 5V to pixhawk and GPS/compass) through the anti-spark circuit. After that, when we pushed the connector further to make a firm connection, the connector will be disconnected from the first level and become in contact with the second level. But before the second level there is a little gap. During this gap, power module does not get power and it does not have a capacitor large enough to supply the peripherals. So, the 5V power drops out for a few millisecond.

This 5V drops out occurs while the compass was booting or initializing, and when 5V was reapplied the booting / init sequence was corrupted.

The solution is to put a large capacitor over the 5V line, or just use a standard, non anti-spark plug.

4 Likes

Very important hint for me, thank you for reporting!

I had a crash today in AUTO flight mode that was preceeded by the dreaded message “EKF2 IMU0 yaw aligned to GPS velocity”. Before that, everything was smooth, after that the plane went crazy and spiraled into the ground. What may have caused the alignment in the first place? And why does Arduplane 4.05 react so radical? What can I do to prevent this on future flights?
(Pixhawk PX4, Arduplane 4.05, Airspeed sensor present and good GPS coverage)
DF Log is here: https://1drv.ms/u/s!AoQqekY0fhannDqCMLXxxBrGbqK5?e=oh2WAr

Did you find the reason of issue…?

Only indirectly. At the time had throttle nudge active and the glitch was probably connected to the plane being at the edge of the radio reception and the receiver simply caught a bad signal. This may have set things in motion.
I was never able to recreate the exact situation though. (Good for me…)
Therefore it remains unclear if this really had to do with “ EKF2 IMU0 yaw aligned to GPS velocity” or if that was just coincidence paired with bad luck.

was there no vibrations that time?

I wonder the reason… anyway, thank you for your detailed answer.

Hi @tecnic08

I have an issue with the heading when changing from alt holt (Qloiter or any hover mode) to FBWA, it seems to loose the north. Do you think it could be due to the antispark issue that you mention, some problem with the compass inizialization. Could that just be solved just by calibrating the compass after initialization? It is weird because I have another device build the same and it does not have this error. I have tried update the firmware, review the parameters and calibration but it keep happening.

Best
Pablo