Apparent in-flight reset, AC 4.0.4

ArduCopter Copter 4.0
1

I was flying my research quadrotor on Nov. 2, and appear to have experienced a mid-flight reset of the Cube Black. This is a configuration which has been quite reliable for months, made from a Tarot Iron Man frame and Tiger Motor propulsion, and flown pretty much daily. I was especially interested in its performance that day because the prevailing winds were roughly 10-15 m/s and I wanted to know if I could count on this airframe in a coastal situation which has similar winds.

My normal flight profile is to make repeated climbs and descents over the same location, between 5m and 130m. I’ve instrumented the quad to measure physcial quantities and encode them into MAVLink packets which the flight controller forwards to the ground; you can see them as data16 and data32 packets. Also, a PingRX ADS-B receiver is aboard, connected to GPS2 at 57600 baud.

In this flight, the quad did one complete up/down profile, and had just started down from the top of its second ascent. The observed behavior was that all motors throttled to zero and the craft fell to the ground. The impact was sufficient to unmate a battery connection, so it was impossible to determine the quality of that connection after the fact.

The dataflash log abruptly ends, showing no indication of any in-flight problem. The telemetry log, though, shows the data coming to a complete halt at 9:12:47, followed by a “Calibrating barometer” text message, and then most of the traffic was data stream requests and, crucially, more of my data packets and some ADS-B receiver health reports, at least until impact, after which the only packets recorded were being emitted by Mission Planner. This suggests that the entire aircraft had no discontinuities in its battery power delivery, at least until impact.

No more dataflash log files were generated from that flight. In doing a post-crash assessment, I connected the Cube Black to Mission Planner to check it for damage, but found it to be nominally OK; nevertheless, there was no mention of any watchdog reset history.

I noticed another topic discussed a vulnerability related to Serial5 which was patch in 4.0.5-rc2, and I’m left wondering if excessive ADS-B packets from the PingRX receiver, especially at the flight apex where its RF horizon was probably some 150 miles, might have been a factor:

Dataflash log: https://drive.google.com/file/d/1Bx3LvCrNupOug5zhvG72F1QmA6UyTzma/view?usp=sharing
Telemetry log: https://drive.google.com/file/d/1mLE_vMAkufBIq1arMUwNR4g1tKWcgp5A/view?usp=sharing

2

I also wonder if 4.0.5 ArduCopter stable release would have prevented this.

3

@KerwoodDerby,

Thanks for the logs. I looked through them but I don’t see evidence of a watchdog reset which would indicate a software problem (see wiki).

Do you have the log file immediately after the one provided (i.e. 00000054.BIN)?

1 Like
4

Yes, although it’s complicated by the fact that LOG_DISARMED was 0, so I had to set that parameter, at which point the 00000054.BIN file was created (without cycling power). Should I have cycled power to then create a 00000055.BIN file in order to potentially generate the evidence of a watchdog reset?

At any rate, also here is the telemetry log I obtained from direct connection to the Cube Black immediately after the crash, in case that might be useful.

00000054.BIN: https://drive.google.com/file/d/1883EA9chLCELXsFf5bYDgJtDH7Cxl1JL/view?usp=sharing
2020-11-02 12-40-05.tlog: https://drive.google.com/file/d/1E0mwCaKVnUipQgQbfrnlGMVxHuDPBN78/view?usp=sharing

5

OK, I finally connected an Openlog logger to my PingRX ADS-B receiver and, just as a first test, stood outside about 3m above ground with it held by my hand, logging for about 30 seconds. (This is in Maryland, USA, along a busy East coast air travel corridor.) Analyzing the logged raw data, I could see that it was producing 50 ADS-B reports per second by the time I stopped logging.

Such a report rate corresponds to about 40% of the maximum rate which the receiver’s 57600 baud port was able to carry. My current suspicion is that when the quad was at its highest point (about 130m AGL), the number of packets it was reporting brought the data port to near its maximum capacity, and may have triggered the 4.0.4 DMA bug and caused a watchdog reset. I have no proof of this, and since I didn’t enable LOG_DISARMED, I couldn’t catch the watchdog message after reboot.

Nevertheless, for the near term I think I’ll fly without the ADS-B receiver connected to the flight controller (upgraded to 4.0.5), and just do independent logging of the PingRX for my data purposes. Once my quad is flyable again, I’ll send the logger up with it to altitude and get an actual report-rate measure.