Inverted MAG.MagZ readings and “toilet bowl” events

ArduCopter Copter 4.0
,
1

One of my quad-copters has been having compass problems for a while now. These events are rare, you can have 5 or 10 successful flights in a row. Then a new flight, in the same location, on the same day as a prior successful flight, with a good compass calibration and you get a severe “toilet bowl” event.

When I first started to analyze the logs of these flights I saw the obvious characteristics of an “toilet bowl” event. You can fly the drone in “Stabilized” or “AltHold” mode without any problem, and it’s when you put the copter in “Loiter” or “Auto” that it starts to drift strongly to one side and loses control completely. You can also see how the compass innovations (IMX, IMY, IMZ) go to extreme values.

Then I noticed that in all successful flights the readings of the MAG.MagZ are always positive, indicating that the drone is upside up, with a small variation due to the angle of inclination of the drone on the roll and pitch axes during the flight.

Example of a positive reading in a successful flight (Sruvey Mission):

Capture2
Capture2.PNG1276×371 50.3 KB

But in the flights that suffered the compass issues, the readings of MAG.MagZ are always negative. This is obviously incorrect because a negative reading must only appear if the copter was upside down.

Example of a negative reading in an unsuccessful flight (“toilet bowl” landed in althold):

Capture4
Capture4.PNG1281×386 25.9 KB

The problem is that the compass readings are fine most of the time, but every 5 or 10 flights when the drone is boot up these readings are reversed, and the craft becomes unflyable. When this happens, the compass can be recalibrated. The COMPASS_AUTO_ROT function will recognize the incorrect orientation of the compass during its calibration and correct it by giving it an orientation of for example Roll180Yaw90.

Then you’ll be able to fly without a problem. But when you reboot the drone these compass readings can change and become correct again, making the craft unflyable again due to the wrong calibration with the orientation offset than is no longer needed.

Has this happened to anyone, too? Do any of you know why this happens?

Ardupilot version: AC 3.6.11 ChibiOS and AC 3.6.6 NuttX (UPDATE: Also tested in AC 4.0)
Hardware: Pixhawk1 and external GPS+Compass module (HMC5883)

2

Try Copter 4.0 and/or provide binary log files

3

I’m already using 4.0, I haven’t flown much yet but so far no problems. We’ll see with few more flights.

Here I upload logs of two flights, same location few days of difference.

https://drive.google.com/open?id=12JvCLcb2hqlQimb0XHczLAsdjT-5_tzB

4

I’ve flown 4 flights today, the problem has happened again. Since the last time the firmware has been updated and now is running AC 4.0, also the external GPS+ Compass module has been replaced by a completely new one of the same model.

Same behavior as described above. In some bootups, the MAG.magZ reading becomes negative, (now I’m monitoring this parameter through MP and don’t fly if I see it being negative). I reboot the copter several times until the MAG.magZ reading turns positive then and only then I fly it.

As I have not armed the copter when the error was present I have no logs of the event. Only of the satisfactory flights when the reading is good.

I upload here one of the four logs of successful flights, (Also in my opinion the magnetic innovations are a little high at first, usually are better)
https://drive.google.com/open?id=1OV99Gm9sbrHig2fy2t4vPWi8Y-54U3Ge

I’ll try to keep testing.

If anyone has any clues as to what might be happening here, please comment below.

5

Can I ask which flight controller you are using?

I have seen this issue with an older CubeBlack + Here GPS (from perhaps 2 or 3 years ago). The issue for me only appeared when I quickly unplugged and then plugged back in the vehicle’s battery. If I left the vehicle powered down for more than 6 seconds the problem never occurred. We investigated the issue quite a bit but found that the issue was within the compass itself and from the ArduPilot side we could not detect if the compass was in this bad state or not.

6

In this copter I am using Pixhawk1 and an external 3DR GPS+compass module. I have other units with the same configuration but with the CubleBlak + HERE1 gps, and they don’t seem to suffer from this problem.

At the moment I can’t relate to the fact that it only happens when you reboot the drone in a short period of time. It seems to happen when the reboot takes a long time, too. However, I’m going to try this out more.

I also thought the problem would be in the compass itself, but I replaced the external compass module with a brand new one and this still happens.

7

@AngelAlbeniz,

Re the reboot time, I think it’s clear but just in case, when I saw the problem it happened when I unplugged the battery and then plugged it back in again within 6 seconds.

It could be an unrelated problem though, another possibility is that the auto rotation code is detecting a different rotation but this would only happen if you were re-calibrating the compass. … and normally it’s not necessary to re-calibrate the compass unless the frame has been modified in some way (i.e. metal equipment has been attached or removed from the frame)

8

I finally found the source of the problem!!!

It is not related to the software but to the hardware instead. Another modification that had been done to this copter before it started to give these problems was the installation of a GNSS PPK module. This module is a standalone module, and don’t share almost any connection with the rest of the drone. That’s why I ruled it out from the start as a possible cause of the problem (I was wrong). I said that it almost doesn’t share almost any connection because yes it shares one.

For the PPK module to work it needs to have feedback from the camera, in order to capture the trigger events. Considering that it is necessary to implement this cable between the camera hotshoe and the PPK module, I approve and make a “Y” connection carrying this signal to the Pixhawk1 too.

For this “Y” connection to work it needs to have a pull-up resistor to a 5v line (around 1Kohm). The connection works properly, and both the Pixhawk1 and the PPK module capture the camera events. But for some reason the Pixhawk1 doesn’t like to hve this connection with the pull-up resistor on an AUX pin. And in some bootups this cause the compass readings to be inverted!!

This problem seems to affect only Pixhawk1, since I have several CubeBlack + Here GPS copters with the same GNSS PPK module and the same “Y” connection running without any problem.

For now, I have left the connection directly between the PPK module and the camera. Leaving the Pixhawk1 unconnected to this camera feedback, and the problem has been solved.

Anyone have any idea how this is possible? How can this connection affect the startup process and reverse the compass reading?

9

How is the PPK module powered ? Does it get power from the Pixhawk or it has a separate 5V power. If it has a separate power and you put a pull up resistor to a common signal then it can affect the power ramp up time which can cause an issue. (It is only a theory, need a scope to analyze and prove it)

1 Like
10

The servo rail of the Pixhawk1 is powered by an external 5v BEC and the PPK module is powered by this servo rail, so the 5v line is common for Pix and for the PPK module.

11

I agree that it’s likely a power ramp up problem and I would guess that it’s the GPS/compass that is reacting badly (i.e. not the Pixhawk itself). The difference between the Cube and the Pixhawk1 is interesting… the Cube is a newer design so I can imagine it does handle power better.

Txs for the follow up.

12

I have been testing this a little more.

I try to replicate the issue by placing resistors of different values between the 5v and GND lines in the servo rail, forcing this power ramp up and trying to reverse the compass readings at startup. But I couldn’t get it this way. The only way to force the problem is by placing the resistor between the 5v line and an AUX port of the servo rail.

In your opinion, this may still be a " power ramp up time" problem? Any idea how I can keep testing this furthermore?

13

It could be a power sequencing issue which cause ramp up problem. Try to remove power from servo rail. Power up the pixhawk, then apply power to the servo rail.
If this solves the problem then you can use a bec which has a delayed power on (https://www.readymaderc.com/products/details/matek-ubec-duo-esc-4a-12v-5v for example)
The difference between Pixhawk1 and Cube is that cube does not take main power from the servo rail !

2 Likes