Hello Folks,
This DJI S1000 platform with a Pixhawk fc has done about 20 uneventful flights ranging from manual takeoffs to at least 10 full auto takeoff to landing missions. On the flight of the crash all preflights were performed as normal and no alerts or errors were shown in mission planner. Upon launching using an auto mission takeoff command, the aircraft immediately started violently pitching and rolling and yawing in all directions. It ascended up for a few seconds before the GCS operator triggered RTL, which caused the aircraft to begin climbing while violently oscillating (up to 90 degrees in each direction). At about 15 seconds after launch the aircraft was in RTL mode and violated a polygon geofence with the RTL/Land action enabled. It kind of keeled over to one side and dropped straight down ~75 feet onto the ground. The airframe is a loss and unfortunately the dual sensor EO/IR gimbal is also destroyed.
What I am trying to figure out at this point is why the flight controller caused this action. All the motors were spinning and the props were fully unfolded and torqued to spec.
The aircraft had flown around 5-6 flights with this ~800g payload without any anomalies.
After looking at the on-board logs it looked like the aircraft was violently yawing, while also oscillating on the pitch and roll axis. The camera feed, although stabilized on a gimbal did confirm the yawing motion. Any idea what would cause this from the logs? We have launched from this same location many times without any issue.
One note is that with our Pixhawk equipped S900 and S1000, sometimes when doing auto mission execution from a remote GCS (arming and do action -> mission start), the aircraft will have one or two motors spool up a tiny bit ahead of the other motors which will sometimes cause it to lurch and wobble once or twice as it climbs above the ground. It has always stabilized and corrected for these sudden lurches on launch until this flight. This flight the aircraft was unable to correct for the lurch and just made it worse over time.
If required I have a cellphone video of the wobble I am talking about on Auto launch. If anyone has any idea how to resolve that it would be greatly appreciated as well!
Cheers!
EDIT: Here is a replay visualization of the aircraft from the onboard logs. The visual observers have confirmed that this is what they saw the platform do.
Hi, don’t have any answers for you, but just wanted to say sorry for your crash! Really sucks when something like that happens, especially when you’re unlucky enough to lose expensive equipment.
WRT one or two motors spooling up first, have you calibrated your ESCs? That should take care of slight variances in each ESC, and also if the attitude of the craft isn’t perfect, or the accel calibration is slightly off and it thinks it’s not level, it will spool some motors up sooner than others to correct it. As well as making sure you calibrate the accels on a really good level surface, I always do an autotrim in very still weather which helps as well.
Also worth noting that if you have an weird uncontrolled behaviour like this, your go-to mode should be Stabilize, not RTL. Might also be worth considering motor emergency stop - I always have this on a switch somewhere - it guarantees a crash if you’re in the air, but it at least can be in your control, so at a lower altitude, maybe in some softer landing, and without power to the props.
I used Mission Planner’s ‘Auto Analysis’ feature on the on-board log, which flagged the following:
Test: Compass = FAIL - WARN: Large compass offset params (X:-290.00, Y:12.00, Z:-190.00)
WARN: Large compass offset in MAG data (X:-290.00, Y:12.00, Z:-190.00)
Large change in mag_field (65.94%)
Max mag field length (874.30) > recommended (550.00)
This suggested a compass problem, which appears to be confirmed by manually looking through the logs. Immediately after takeoff, there was a significant divergence between Mag 1 and Mag 2 in their Z axis readings:
The aircraft then appears to have began continuously spinning counterclockwise, eventually resulting in EKF failure due to excessive vibration right before the crash (vibration before this point was in the acceptable range).
That’s the limit of what I can glean from the logs, hopefully someone else can chime in with more detail. The only potential explanation I can think of is maybe your gimbal payload included a yaw motor or some other source of magnetic interference relatively close to the Pixhawk, which once airborne interfered with the internal magnetometer.
is it possible some propellers were mounted incorrectly/swapped or something ?
you have pretty high vibrations, they increase a lot in the final 10 seconds, (Z is ~60m/s/s) which is far beyond ok. - maybe something were loose as well ?
Throttle demand goes to 100% during that time, that could look like more than one failure.
Thanks for your input. We did have a 800g gimbal on it, but we had flown four times with the same gimbal connected in the past without any issues. At this point anything is a possibility.
This is a very good observation. I have AS150 connectors for the battery, going to an Attopilot180A current sensor. These are located underneath the bottom plate on the S1000. The Pixhawk is located above the S1000s built in PDB, so there would be a thin carbon sheet and the PDB between the connectors and the pixhawk.
Now that I say that it is obvious that high current fluctuations from the battery / PDB could absolutely effect the flight controller since it rests directly above the PDB without shielding.
Any idea why we would be able to launch and fly the craft without any issues during more than 10 flights previously without getting any mag health errors or anything?
The platform was extensively tested in all flight modes via radio. I only moved over to flying from GCS due to radio failsafe issues from losing LOS. The telemetry link in this case was much more reliable.
Looking back on it, this failure might have been recoverable via stabilized mode. Things happened very quickly. I think that removing the pilot with the RC is where the industry is going, assuming you have a reliable platform. I guess 3 hours of cumulative air testing time on a platform is still not enough to prove reliability.
This flight was performed without the RC transmitter turned on (Using RC_OVERRIDE from the GCS).
I too am trying to figure out why the desired yaw value was so off. Like I said, this flight was an auto mode mission which had been flown 5+ times previously (the exact same mission, no changes to waypoints or anything). To my knowledge, there is no way for a GCS operator to command such a yaw movement, let alone during an auto mode “takeoff” command.
There was nothing wrong with the Physical preflight checks of the aircraft. No modifications or changes occurred since the last 10 flights of the platform.
The vibrations are likely because the aircraft was oscillating more than 150 degrees in each direction while rotating as shown in this log replay https://www.youtube.com/watch?v=88EBcumfYYg
That’s a useful feature for aligning the vehicle with the wind or whatever by hand.
IF your telemetry logs show that the GCS was sending RC_OVERRIDE messages indicating a very low value on the yaw channel then this probably explains the demanded yaw.
IF the demanded yaw was the problem (and the motor outputs seem to indicate it was), then reducing the max yaw rate would probably avoid the problem.
Why did you feel it necessary to use RC_OVERRIDE from the GCS to accomplish anything? AFAIK you should be able to fly without a transmitter and without RC_OVERRIDE (whether flying without a transmitter is a good idea is another matter).
