I2C Bus Failure Mid Flight. Log File Included

Hey Guys,
We were running a mission today and the plane was flying fine. After the mission ended we flew around a bit longer and during the part of the flight I received a Compass Error then GPS Data error on the ground Station.

After landing we are noticing nothing is working on the I2C bus even if we plug-in individual components directly to the I2C port on the plane.

After reviewing the logs there is a “flag power_status_t” where it jumps up at the same time as the loss of GPS, Airspeed, ect.

What happened? Here is a download link to the file. It is to big to upload. Thanks.

birdsiproductions.com/ft/i2c_loss.tlog

That link isn’t working, but what version flight code are you using?

Sorry about that.

Correct link: birdsiproductions.com/ftp/i2c_loss.tlog
We are using Arduplane 3.2.3
What appears to have happened was the 5v power coming off the i2c bus is dead. We used a multimeter and checked the voltage and it was reading 0.

Josh

Hi BirdsEyeProd,

Have you solved it or found the reason of the problem? Watching your tlog, I see I had exactly the same problem! But after it, I never found my plane to do a diagnostic of the problem, only the tlog…

Thanks!

I have lost the i2c communication on two PHs now; the first after two months and the second one after 11 months. Both time the compass goes and is not able to hold a stable calibration.

Since it is a total of 13 months 3DR will not replace the PH as they did with the first one. They did offer a miserly 15% discount for my “inconvenience”.

It seem the Q/C at 3DR is failing. The problems with the Solo is consuming them.

Considering the quality and the fact they won’t stand behind having a PH last even a year I wonder if there is any good reason not to buy a Fixhawk instead.

electronics, once working does not tend to fail unless electrically or physically abused, the fact that you had I2C failure two times, (on a simple 2-wire bus) - makes me wonder what is causing it.

However, if you really want to blame “quality” - then you should be able to see some bad solder flow, cold soldering, or loose component. can be seen with good magnification.

I agree with you in theory. Plugging and unplugging batteries is such an inelegant approach to turning the PH on; that is the only electrical abuse I can see being done. Physically the PH is pretty well protected in a CF frame and the MR has taken no hard hits.

Why does a “simple two wire bus” knock out the internal PH compass rather than only impacting devices connected to the i2c board?

different components can have different tolerances for out-of-spec conditions.
I am attaching an thermal image of APM wit too high I2C voltage , that destroyed magnetometer, and while other components work, the magnetometer is the second hottest component after the main microcontroller, while it would usually not be any hotter then the rest. Note that this is not enought thermal difference to feel it due to component’s small surface, but does show up on thermal.
the 3,3v bus had become more like 4.4v - maybe more before testing.

There are lots of crappy GPS’s and other devices that , for example, run GPS and magnetometer on 5v or with really bad regulator between, or even only a resistor - being capable of supplying 5v to teh 3v3 I2c bus. (remember that many I2C devices operate fine at both 3,3v and 5v - some don’t.)

I have a i2c port come off the board while removing a plug. It could be that the connection was weakened by plugging / unplugging of the magnetometer. It’s unfortunately a characteristic of the DF13 plugs.

yes, and no.
DF13 is not really made for very much plugiing/unplugging cycles, nor is it made for the way “poeple” tend to use them.
Do not pull by cable, rather insert a xacto knife blade between male/female part into the long edge of the connector, and twist a little left/right /(forcing connector parts directly apart from each other) - or there is a video how to do that with a nail, made by Pixhawk developers…