I’ve recently picked up a well-used cinematography octocopter from a friend that I’ve been restoring to flying condition. It’s an Octo+ with 15x5.5" props on T-Motor MN4012s and what appears to be Hobbywing Platinum Pro 40A ESCs (may be knockoffs) running BLHeli. I’ve replaced the FC with a Pixhawk 2.4.8 from BG and added a BN-880 GPS/Compass.
Did a few test flights and things seemed to be in order, Stabilize, PosHold and Loiter all seemed to be working as advertised with no significant issues. Yesterday I brought it out for a more involved test (more then a couple of feet off the ground and faster than a dead crawl) took off in AltHold switched to PosHold back to AltHold with everything seemingly normal. Almost immediately after a final change to PosHold the copter pitched away from me and started climbing didn’t seem to respond to transmitter inputs before reaching the apex of it’s climb, rolling upside down and crashing down in a field 50M away.
I’m somewhat new to ArduPilot and am having a hard time figuring out exactly what went wrong, the DataFlash log shows EKF primary “changed to 1”, “failed to init” and an “EKF yaw reset” immediately before it all went wrong followed by it triggering it’s failsafe shortly after things were going tits up. Of the different location plots in the MP log browser, only the blue GPS plot actually matches the true location during the failure with the red POS plot off in its own world for most of the incident.
I’m hoping someone with more experienced eyes then mine can help me figure out what went wrong here and how I can stop that from happening again.
Just noticed something, it seems that IMU1 suffered a large glitch and then became offset from IMU2 for the remainder of the flight. Is that a possible cause?
To summarise, I think there’s a hardware fault indicated by sudden onset of vibrations. Maybe props break or are slipping on the shaft??
There’s a definite yaw bias or weight imbalance evident in your RCout for the motors, that’s not part of the incident though, just an observation. Also it’d be good if you could set up current monitoring, there’s a few extra tuning parameters you can use then.
8:57:15 motor outputs almost look great, but not what you normally have, indicating some problem, maybe a weight shift.There’s a slight problem with pitch/roll and Desired pitch/roll
8:57:19 vibrations really start and and there’s clipping, motor outputs starting to go crazy trying to maintain control
8:57:23 huge drop in battery voltage and motor outputs are all driven to near maximum, seems almost total loss of thrust - particularly Motor 4. It’s odd that the opposite motor isnt driven to minimum, but I’d say this is because the the flight controller thinks it needs altitude because of the vibrations causing bad position/attitude data.
8:57:26 pitch/roll and Desired pitch/roll have really departed, all control lost
There’s numerous other things in logs that show the effect if the loss of control, like number of sats and HDOP all go bad once the craft is out of control. That can be expected and doesn’t look to be the cause. It even looks like the IMU glitches occur at the same time as the vibrations start, so vibrations could still be the root cause.
To add to what Shawn posted (indeed numerous other things). After you switched to PosHold and the EKF events your throttle drops to zero. Then another EKF event triggers Land. Motors go to Max and it’s climbing until you switch to Stabilize. Throttle is low so a Rapid Return to Earth. Some puzzling things there. You also had a Vibe FS but maybe no surprise as things have already gone bad.
Thank you very much Shawn & Dave for your help. A prop break seems like a very plausible explanation. They had been somewhat abused during the earliest attempts to fly and I am still missing half a prop (3 broke, 2 adjacent to eachother, 3rd on the opposite side).
I’m going to assume that operator error played a role in this as Dave points out, switching to stabilize with a cutoff throttle certainly did not help the situation.
Are tuneables that I can look at to help Ardupilot recover from such a failure in the future?
In either case new current sensing ESCs were already in the mail and I think a more involved pre-flight check is in order for future flights.
I meant to say Vcc is a bit low too, the 5 volt supply for the flight controller - I dont think it’s a contributor here, but something to check and maybe fix.
If you can get current measurement working then you can also run the Compass/Motor calibration to improve compass performance.
For 15 inch props and 6 cell LiPo, check and set these:
After everything is fixed and all these settings taken care of, just do some very basic hover and flight testing in Stabilise and Althold. Recheck logs for further issues.
If all OK, then run Autotune again, maybe one axis at a time. That way at least you know the attitude control will be good.
