after tuning my airplane I continue to test the RTL function, durring this I observe a issue in the altitude. I had the feeling that the plane circles higher then expected (80 m). After analysing the log files it seems that the altitude has reset two times immediately after the launch and with this the RelAlt was wrong durring the whole flight:
[attachment=1]Alt_Error.png[/attachment]
I tried to find out what can cause this problem without success. It seems that both the GPS.RelAlt as well the BARO.Alt was reset. I didn’t observe this problem in arduplane FW version 2.x.
A clue is that when these height resets occur, the EKF appears to undergo a very specific reset that only occurs when the FMU goes from disarmed to armed. This is consistent with the baro height reset, because when disarmed, the home altitude and position is set equal to the current altitude and position. It appears that your FMU has momentarily disarmed on these occasions. What I can’t tell is what would have made the FMU think it was disarmed.
Another strange thing is that the Vcc and servo rail voltage in the POWR log are zero as is the Vcc voltage in the CURR log. Are you able to post the .BIN file from your SD card? - it should have the missing data.
Probably unrelated, but you have some large spikes on your airspeed sensor and the ARSPD_RATIO of 3.8 is unusually high indicating either a leak in your pitot tube connection or a pitot tube that is badly positioned. This will make your airspeed measurement noisy and inaccurate.
I do notice the latest firmware have some issue in altitude. It seems that when switch to auto mode from an altitude lower than way point 1, it will not climb until the last moment when it almost reach that way point.
My air speed ratio also set at 3.7 or so. But it was right in front of the nose with no obstructed object. Ill check and replace the air hose and see if thing improve.
I toke also look to the tlog and I can’t see that the FMU was disarmed before. The attached ZIP file contains the tlog and the bin file for analysis. I’ve no clue why the vcc and servo rail isn’t logged. It seems that the log bit mask is set to log all (LOG_BITMASK=65535).
From the image below you can see, that my pitot tube is mounted in the nose of my plane. It’s a Eagletree AS sensor.
[attachment=1]FPV_Raptor.jpg[/attachment]
Maybe I’ve in the next days time to look to the code, what can cause a atitude reset.
I continue the debugging of this issue. I review a lot of logs from FW version 3.x to find similar behaviors. I deeply analyzed a log where the altitude drops immediately after the takeoff from ~120m to zero. The attached diagram shows that only the Baro-Alt drops down to zero.
[attachment=1]dia.png[/attachment]
Thus I looked to the code what can be the root cause for this. Here is what I’ve found:
The AP_Baro lib has two methods to reset the altitude ‘AP_Baro::calibrate’ and ‘AP_Baro::update_calibration’. The ‘AP_Baro::update_calibration’ is never called from the ArduPlane code and the ‘AP_Baro::calibrate’ method is called only once from ‘init_barometer’ in ‘ArduPlane/sensors.pde’. The ‘init_barometer’ from ‘ArduPlane/sensors.pde’ is also called only once from ‘startup_INS_ground’ in ‘ArduPlane/system.pde’. If ‘startup_INS_ground’ from ‘ArduPlane/system.pde’ is called, the Airspeed should also be set to zero since this method is called immediately after the’ init_barometer’ function. So, why is the barometer altitude is set to zero? The barometer pressure looks also good.
Questions:
Is there another way to reset the barometer altitude on PX4 or APM?
What I’ve seen, the altitude is calculated in the APM code not in the PX4IO code. Is this correct?
Is it possible that there is a stack/heap issue which overrides some values?
Find attached the tlog and the log file which shows this issue. The reset occurs at ~20:00:08.
I’ve slightly changed the logging code to log also the arm state every 100ms.
Additional to this I run the APM 3.0.3 PX4 code in a HIL environment to check whether this behavior occurs also there (thus I’ve built a HIL target).
The altitude reset occurs also in the HIL and it’s very reproducible!
After looking to the logs it seems that there was no disarm while the altitude was reset.
The issue occurs in the tlog at ~10:59:54 and in the log file at line 102576 or timestamp 264473.
I have alse attached the PX4 firmware files for HIL and normal.
Can someone help me to move forward with this issue or point me to someone who can hel with this?
I moved forward and continue debugging in the HIL environment. After debugging thru the arduplane code it has shown, that the APM was disarmed and armed randomly. Thus I’m continue and focusing on PX4Firmware code. Here I’ve:
[ol]
[li] Enable debugging in the PX4IO classe in 'PX4Firmware.org\src\drivers\px4io\px4io.cpp.[/li]
[li] Add logging output to the PX4IO::io_handle_status to check when ‘safety.safety_off = false’.[/li]
[li] Fly in HIL environment.[/li][/ol]
It has shown, that the safety_off state occures randomly within 5-10minutes. This explains why the arduplane gets armed and disarmd. So the issue seems to be in the PX4Firmware code.
After this I’ve compared the PX4Firmware version from APM release 2.78 against the version 3.0.3 and found some major changes.
[ol]
[li] Commited on 25. March. pwm: added PWM_SERVO_SET_FORCE_SAFETY_OFF ioctl.[/li]
[li] Commited on 14. January. Fixed up init sequence of all sensors - we can publish in interrupt context, but not advertise! All advertisements now contain valid data.[/li]
[li] Some other minor changes.[/li][/ol]
For the first change I add debug output there to check whether this occures or not and it never occures.
To evaluate the second change I’ve merged the PXFirmware from release 2.78b into version 3.0.3 and flew in HIL. The flight was 2 hours without any issues. Durring this flight I found also a minor issue in the Missionplaner (take a look to ‘timeInAir’ value on the screen shot)
[attachment=0]screenshot.png[/attachment]
Some questions to the developers:
[ol]
[li] Is it possible that the code changes from 14. Jan disturbs the serial communication between Cortex M4 and Cortex M3 under adverse conditions?[/li]
[li] Has anyone else observe this kind of issue on PX4FMU+PX4IO+Arduplane 3.0.3?[/li][/ol]
I’ve reviewed the code again and it seems that I2C is used on PX4FMU for interprocessor communication. This is different to the PX4 v2 version, where UART is used.
Is it possible that the increase of messages which are exchanged between IO controller and FMU can cause a message drop on the FMU side if the I2C connectio is used?
And again. Can this IO publish blocking cause side effects to the ISR handling in the I2C code on the FMU side?
here is the debug output from ‘PX4Firmware\src\drivers\px4io\px4io.cpp’ with debugging enabled:
<px4io> io_reg_get(4,0,15): data error -110
<px4io> io_reg_get(3,0,8): data error -110
<px4io> io_reg_get(4,0,15): data error -110
<px4io> io_reg_get(3,0,8): data error -110
<px4io> io_reg_get(4,0,15): data error -110
<px4io> io_reg_get(3,0,8): data error -110
<px4io> io_reg_get(3,0,8): data error -110
<px4io> io_reg_get(3,0,8): data error -110
<px4io> io_reg_get(1,2,6): data error -110
<px4io> io_reg_get(3,0,8): data error -110
[px4io] check for IO reset - force it back to armed if necessary
[px4io] safety.safety_off = false
<px4io> io_reg_get(1,2,6): data error -110
It seems to me that it is a communication issue between Cortex M3 and M4. I’ve seen, that the serial line on PX4FMUv2 is configured with 1.5MBits and on PX4FMUv1 the I2C is configured to 400KHz. Might this be an issue? Is the bandwith to small?