first of all, I have been reading this wonderful forum for years, and I learned a lot from the community already, thanks!
The reason I created this topic, is that I run out of options, since reading log files is new to me it would be very nice, if somebody could help me understand what happened with our agricultural spraying drone.
We were flying in auto mode and suddenly the drone crashed, I have been trying to understand the log files, but the only info i was able to understand, is that the EKF had some kind of issue, so it switched to IMU2 right before the crash. Can anyone please help to understand?
here is a link to the log file:
There was an odd glitch in the Motor Outputs dropping them to 0 briefly. When it recovered from this the motors on the right side of the craft were commanded to max or close to it. The motors on the left side were dropped to stabilize the craft and down it went. It would have been interesting to look at battery current at this time but itâs not logged. Does what you witnessed make sense form this observation?
thanks for your answer, your explanation is perfectly accurate. I also wanted to check the battery current, Im not sor why it is not logged. Do you have any idea why this happened? We had an error message: EKF PRIMARY: OCCURED OR FAILED TO INITILIASE.
Also, can you tell me which log reader do yo use?
Hi KĂĄroly- From the timing in the log it looks like the EKF error was generated after it lost stabilization or when it crashed. Check out the attached error messages with respect to altitude. These graph are generated from APM Planner 2.
I have to admit I have never seen an RC Output go to zero. To minimum PWM yes, but Iâm not sure what 0 means frankly. Logging didnât stop because there is other data being recorded during this event. I have looked at other logged data that might correlate but Iâm not seeing anything.
However, this is rather troubling as I thought weâd fixed the safety switch issues earlier in the 3.6 series. It was found that some safety switches were activating in-flight - particularly when water was involved.
The parameter â safety buttonâ what is the default Value when using the latest firmware? Also, â activewhenarmedâ thanks
Besides the basic pre arm checks, and other essential settings, is there a cheat sheet out there that one can check against critical parameter settings besides the know how one may have.
if the IOMCU reset then it would have come up with the safety enabled. We donât have IOMCU logging in this old version of the code, so we canât prove if that happened
if the GCS sent a MAVLink message to request safety enable. That bypasses the BRD_SAFETY parameters. We need the tlog to see if that happened.
if we have (or had in 3.6.10) a bug related to safety button handling
@UAVSkies there are several parameters which affect the safety switch.
The default for the BRD_SAFETYOPTION switch is to allow the switch to enable the safety and disable the safety - but NOT when the vehicle is armed.
AFAIK thereâs no cheatsheet on what to check. However, one of the first things we do when checking logs is check the ARMING_CHECK bitmask. If it is zero we start to frown.
A suitably motivated person could probably find a way to crash a vehicle with most of our parameters
Thanks for the answers, here is the link for the tlog. Can you tell me how water affects the safety? Im asking because there was some water involved, since we were spraying.
thanks, I canât see anything in there that indicates a mavlink commanded safety state change.
Iâve also been working to test if an IOMCU reset can cause an issue with master, and it can. We had code to try to cope with it, but testing it this evening shows it wasnât reliable.
Iâve opened a PR here to fix it:
Hi, thanks for your answer, but dont really understand the problem, why would the IOMCU reset? And what issue can it cause with the master (also what master means in this case?)? And can you explain to me the PR you opened? We are quiet new to this, and Im not sure what should we do to avoid this problem next time.
Also @peterbarker, If i understand it correctly, your solution is a bit different, can you tell me anything based on the tlog? Or if water has anything to do with it?
I think it has something with water, I was looking for correlating anomalies, and it seems that right before the safety switch engaged, the VCC power was dipped (not much, but it got all times low, during the flight). Is it possible that the water/spray got to the flight controller or cabling ?
Hi, thanks for your answer, and yes, it is absolutely possible, since we were testing a new spraying system, but It should not be problem, since we are using a drone made for spraying, and also moderate rain is ok, based on the factory guarantee policy. But if it was the water, is there any software solution? Im asking, because @tridge and @peterbarker also thinks it was a software issue, if i understand correctly.
If it is from China, then ârain is ok, based on the factory guarantee policy.â is basically zero guarantee. Unless you have conformal coating on key elements, and Silicone sealing on non movable parts⌠I see and serviced a couple of spraying drones from china (Joyance, TTA, ectâŚ) and all of them were crap
The IOMCU would reset either if their was a bug in the IOMCU firmware or if there was a transient power spike which caused it to reset. It is very rare (I think this is the first time an aircraft has been lost where we think it might be an IOMCU reset) so hard to really attribute causes. We certainly have fixed bugs in the IOMCU firmware since 3.6.10.
By âmasterâ I mean the latest development branch of ArduPilot, which will become the 4.1.x release in the future. That is called âmasterâ in the source code control system we use.
The PR I opened fixes the FMU firmware to automatically disable the safety under the following conditions:
it detects an IOMCU reset (as time went backwards in the status from the IOMCU)
it knows that safety was disabled in the last 20Hz update of status from the IOMCU
the vehicle is armed
If those conditions are true then it forces the safety off to give you a chance to continue flying.
Sorry for the confusing terminology. Basically I think you hit a very rare condition. I canât be sure if the issue is a fault in your hardware or a software bug, but I do know that ArduPilot could have done better in handling the condition, which is why I opened that PR, so if this does happen to someone else in the future it will force the safety off and should keep flying.
Cheers, Tridge
