Ardupilot Cube Orange+ IOMCU companion CPU resets mid flight 0x1000

Hi there,

Im lately stuck with a specific problem where i cant find any solution.
I recently updated my Cube Orange+ with from Arducopter v4.3.8 to a new vehicle with Arduplane v4.5.7. Now I’ve done already a few test runs but there is a weird error where i cant find any solution for. I’m occasionally errors of the IOMCU and also 2 times I’ve got the error 0x1000 mid flight where the IOMCU reset itself(checked it with this tool). This error is something new that didnt happened on the same board with Arducopter 4.3.8.

with the new arduplane version i always get at least one IOMC error(Nerr2), often a few thru out the flight and sometimes(but rarely) after a 6 hour log an IOMCU reset 0x1000 because the RSErr has become bigger than 20. This is something that wasnt on Arducopter 4.3.8.

My setup is as basic as it can get. Normal PWM outputs on Main1-4, no B-SHOT or BLHELI, SBUS RCIN. just a bit Lua scripting. 30% memory free and 10% CPU usage.

Also did a test run on Arduplane 4.7.0 but the same thing there. Always at least one Nerr2 error and sometimes also Nerr, Nerr2, RSErr.

In addition tried brand new Cube Orange+ - same behaviour.
All the threads I’ve read had a voltage drop on the servo rail or used Bdshot and didnt helped.

I’ve tried to understand from the code and this line is new compared to the Copter 4.3.8 which causes the reset but still those errors where not there in the old Copter version. also not in such long logs.

My problem is that i cant reproduce the error because it takes ages to run the Pixhawk this long and also then it happens rarely.

Does somebody else also experienced this bug? what happens to the Main1-8 when the IOMCU resets? I didnt experienced anything but is this critical and how long does it take to reboot.
Are those constant failures between the FMU and IOMCU normal? And why didn’t this happened in the old Copter version?

Unfortunately I’ve accidentally deleted the logs where this happend and weren’t able to reproduce the error until now.

How can i trace this error back or does somebody also experienced this?

Any help, idea or experience would be much appreciated.
James

Its quite common to get one or two IOMCU errors when the vehicle first boots. Maybe we should make the error check a rate rather than a absolute number. 20 in a half a hour flight is obviously a lot worse than 20 in 6 hours.

Yes this would be nice and solve this. But why with the Copter 4.3.8 I didn’t got any error message across dozens of logs and now with the arducopter I get at startup constantly one error and here and now on the new plane random up to 10 errors across 6 hours.

Was the way the Pixhawk counts the error’s changed between the versions or why do i see the increase of the errors.

This n addition how critical would you say is a reset of the IOMCU while flying. How long does it take for it to reboot?

If the fw 4.3.8 did not perform this specific test, it can therefore not issue that error.

Newer firmware versions perform this test and detect the error.

So the issue is probably in the old firmware, but that firmware can not detect it.

Just update to 4.6.3 the root cause of the issue is probably fixed there or on 4.7.0

To be 100% sure I would need to have access to your FC.

No, I’m asking if newer version perform this check different? I couldn’t find anything except the 20 read_status_error limit which also rarely gets triggered.

But all the Nerr and Nerr2 errors weren’t triggered in Arducopter 4.3.8 and now they are. And these error counters „should“ be the same in both versions. Is there anything else that changed from the old versions related to that or why suddenly I get these errors in the new versions?

There were two counters on older firmware versions that where hardcoded to zero.

Are these the ones?

Probably. If I will check and update here.

Nerr and Nerr2 never changed on Copter 4.3.8

Checked it in the code. These errors aren’t hard coded and also check the same thing as in the new version. The only difference is that in the new version the reset was added after 20 status read errors.

So I still don’t know why with the new version I get these error and with the old not. It just explains the reset of the iomcu I’ve experienced.

Just to resolve this thread. I didn’t managed to eliminate the errors on the Plane 4.5.7. but i did some tests and found out that the 0x1000 error is sent by the main processor but the TS(timestamp) of the iomcu doesn’t resets and also the pwm outputs keep changing normally - the IOMCU isn’t reseted and keeps working normally as if the reset request didn’t was sent. Even 8 hours after the 0x1000 error was logged.

On version 4.7.0 the error doesn’t occurs because I don’t get there the status errors. Only the Nerr and Nerr2. So 4.7.0 version is more stable in terms of the IOMCU. But also at Plane 4.5.7 the IOMCU isn’t reseted and keeps functioning normally without any measurable outage at timers and PWM output. Even though the error is thrown.