Rolled inverted during 'TakeOff Mode'

So, I had a pretty crazy maiden flight this evening. I don’t have much experience with Arduplane but I do have one other airframe I’ve been flying without any issues. I’ve been using Auto Takeoff mode from the very beginning and haven’t had any issues before so today on this new plane I took off in ‘Auto Takeoff mode’. Well about 10 secs into the flight the plane rolled inverted and back again then went straight up until I was able to take it out of that mode. I was able to fly it after the craziness and land safely but I’m trying to figure out what happened before I fly again. I’m not exactly sure how to read the logs, so I’ll post it here and hopefully someone with experience with logs can tell me what happened. I’m running 4.2 Beta6.

Here is a video of the first few secs.

Looks like you took off before the EKF was ready:

2022-04-30 19:02:56.759 Armed AUTO, xaccel = 4.2 m/s/s, waiting 0.1 sec
2022-04-30 19:02:56.860 Triggered AUTO. GPS speed = 1.3
2022-04-30 19:02:56.879 Takeoff to 70m at 200.0m to 29.6 deg
2022-04-30 19:02:58.438 EKF3 IMU0 yaw aligned using GPS
2022-04-30 19:02:58.439 EKF3 IMU1 yaw aligned using GPS
2022-04-30 19:02:58.638 EKF3 IMU0 is using GPS
2022-04-30 19:02:58.638 EKF3 IMU1 is using GPS
2022-04-30 19:02:58.639 AHRS: EKF3 active

I wait for the “using GPS” message before arming.
But you have ARMING_CHECK=1 which should run all the prearm checks and prevent arming before that. Not sure why you were able to arm before the EKF was ready.

I was pretty sure I waited for the “using GPS” message from Yaapu, but you are saying I didn’t? The fact that it let me arm and takeoff is an issue with this beta6 build? I guess I should of posted in the 4.2 Beta thread. Thanks for looking into it and the reply!

Yeah, if I’m reading the log correctly, this looks like a bug in the prearm checks.
@iampete Do you agree?

Yeah, because you took off before the EKF3 was active, the arming check only checked DCM which is almost always happy.

We should add a arming check that were using the configured AHRS source.

The arming check does in fact check, I’m not sure how you got past it.

Is there any other data I could give you to help track down how this happened? I figured if Yaapu was reporting the FC was armed after I flipped the arm switch, I was good to go. Clearly that’s not the case.

I can’t see anything in the log to help, so I posted a link to here on the 4.2beta6 thread.

@Troy_Daigrepont can you please post the tlog? The log you posted show it as being armed when the log starts (see STAT.Armed), so it somehow got armed before that.
The arming checks do check that EK3 has started, so I suspect you may have done a forced arm?

on looking a bit more deeply, the real reason for the crash had nothing to do with arming when EK3 wasn’t ready, the problem was that your primary IMU, the ICM20602, has a failed X gyro:

once EK3 activated it quickly switched to the backup IMU, the MPU6000.
The ICM20602 is clearly faulty. You should disable it by setting INS_USE=0
Or you should replace the flight controller.

I definitely did not do any kind of “forced arm”. Does the gyro issue have something to do with how I was able to arm without EK3 started? I just want to make sure I don’t do this again. If I’m not connected to MP when I’m flying, I will not have tlogs correct? Thank you so much for looking into this!

it is a hardware fault. It came back upright after EK3 initialised as EK3 has the ability to change IMUs when one is inconsistent with other sensors. The old DCM estimator can’t do that.

right, so unfortunately we can’t tell how you managed to arm with EK3 not initialised

@tridge Since the ICM20602 X gyro reported correct values before and after that brief interval during takeoff, what kind of hardware issue might be the cause? Not mechanical breakage, since it looks good afterward, so something at a higher level: maybe a glitch on the SPI bus?

I suspect something at the MEMs level, like stickage