Copter-3.6.0-rc12 available for beta testing

rmackay9 · October 22, 2018, 5:58am

Corrado,

Sorry to keep asking you to do more work but could you try this with Copter-3.5.7? I’m afraid that it’s very likely this issue will need to go to @tridge and I’d like to narrow down on the possibilities before taking his time to ask him to look into this.

I guess this is a pixhawk compatible flight controller or something else?

anon67614380 · October 22, 2018, 6:02am

Black cube 2.1
Yes it does it with 3.5.7 too.

I built myself a little device that waits 10 seconds before feeding sbus as a workaround and it works.

olliw42 · October 22, 2018, 7:01am

This must be the issue re mavlink messages being handled incorrectly in 3.6. … It sounds like this issue doesn’t actually cause problems with the physical movement of the gimbal? I guess it’s the SToRM32 gimbal we are talking about.

that’s one of it, yes. It does. I can’t recall which way around it was, but for one of the messages it leads to 100x smaller gimbal movements (that’s how I found it). It’s unrelated to STorM32.

I’m particularly concerned about any regressions.

well, honestly, I find it hard to understand why one wouldn’t want to add a if which would solve a longstanding issue without any side effects, but instead prefers to theoretize about what the best solution could be. Similarly with the slow update rate in some modes. (both points are unrelated to STorM32 too).

otherwise I like AC3.6, thx, sir

andyp1per · October 22, 2018, 7:11am

Hi Rob, so should we use it?

kknet · October 22, 2018, 10:48am

Hi!

We have had some problems reliably getting datastreams over mavlink to our on-board computer after upgrading to 3.6. Specifically, when we arm the vehicle, a stream requested to 50 Hz will drop to 5 Hz or even less.

After removing some unused features from the scheduler (Optical flow, beacons, etc), it helped a little. Switching to ChibiOS further helped, and we’re reliably getting data in the 30-40 Hz range.

Suspecting that this check has something to do with it?

github.com

ArduPilot/ardupilot/blob/014022de03791078f547fd6628c05f90af3229f1/ArduCopter/GCS_Mavlink.cpp#L280


}


#if HIL_MODE != HIL_MODE_SENSORS
// if we don't have at least 250 micros remaining before the main loop
// wants to fire then don't send a mavlink message. We want to
// prioritise the main flight control loop over communications


// the check for nullptr here doesn't just save a nullptr
// dereference; it means that we send messages out even if we're
// failing to detect a PX4 board type (see delay(3000) in px_drivers).
if (copter.motors != nullptr && copter.scheduler.time_available_usec() < 250 && copter.motors->armed()) {
    gcs().set_out_of_time(true);
    return false;
}
#endif


switch(id) {


case MSG_EXTENDED_STATUS1:
    // send extended status only once vehicle has been initialised
    // to avoid unnecessary errors being reported to user

Its been in the codebase for a while, but might the processor be more busy in this release?

I’ve tested various low-hangig mitigations such as lowering this bound, lowering the max-time of the task and increasing the execution frequency. Resulted in some improvements, but I don’t want to mess with a tested scheduler to much.

Regards,
Kristian

rmackay9 · October 22, 2018, 10:51am

Hi @kknet, re the mavlink timing, could you provide a log file or a parameter file? In particular I’d like to check which streams and also the update rate of the main loop. I’ve certainly seen in Rover cases where the message update rate is far lower than it should be but I hadn’t seen this issue with Copter.

kknet · October 22, 2018, 11:18am

Sure!

Attached are three logs. The first two is with my alterations (only change is disabling features in APM_config.h), and the last is the original (rc 11) where we first discovered this. In the last one, we are requesting raw IMU, RC and attitude at 50 Hz, position at 10 Hz.

Running on a brand new Cube.

Note: The build has some other minor configuration changes, mainly just our own build target to allow for custom default settings. This should not affect the timings, so I was hoping someone else sees the issue on a regular build.

35_mavlink_stream_copter36_chibiOS.BIN (332.6 KB)
36_mavlink_stream_copter36_nuttX.BIN (303.0 KB)
38_mavlink_stream_copter36_nuttX_original_highrate.BIN (575.7 KB)

rickyg32 · October 22, 2018, 12:59pm

Hi Randy thanks for getting back to me.
I am going to hold off on this issue for now as I am finding a number of power problems I need to resolve. However I can tell you that I was getting a green with an HDOP of 100 and in the basement…
The hud said I had a gps…so status was 1 but it was definitely giving me a green led. That said.I need to settle this power issue as I need to rule out power issues causing strange behavior…

once thats resolved I will try again and if the green led shows I will let you know…logs and all.

rickyg32 · October 22, 2018, 4:12pm

Randy
Here is a couple recent logs…they are about 30 minutes old. Also interested in any other things anyone see’s in the log.

7 1979-12-31 7-00-00 PM.bin (216.6 KB)
5 1979-12-31 7-00-00 PM.bin (233.2 KB)
6 1979-12-31 7-00-00 PM.bin (169.1 KB)
8 1969-12-31 7-00-00 PM.bin (254.5 KB)
4 1979-12-31 7-00-00 PM.bin (573.8 KB)

Cipta · October 22, 2018, 5:23pm

Hello Varun and Mackay,

Built-in IMU temperature automatic compensation system

from https://store.cuav.net/index.php?id_product=8&id_product_attribute=0&rewrite=pixhack-v3-autopilot&controller=product&id_lang=1
I use also Pixhack V3 with 3.6 RC… and BRD_IMU_TARGTEMP,45 but never seen such high temperature. I ordered the Pixhack from CUAV
The only thing I what I don’t understand is that fast IMU sampling is not working.
Quad, Frsky, oneshot, GPS, Servo Gimbal…

andypiper · October 22, 2018, 5:59pm

@rmackay9 here is the log. Thanks for your help

rmackay9 · October 23, 2018, 12:09am

@rickyg32, ah, it’s a pixracer! We have a fix for the incorrect LED colours that will go out with 3.6.0 (or the next release candidate if we have to have another). You can give the fix a try by going to the MP’s Install Firmware screen and pressing Ctrl-Q (to load latest) and then push on the multicopter icon. No pressure, to test it though, it’ll be out as a beta or a “soft release” of 3.6.0 later this week.

rmackay9 · October 23, 2018, 12:13am

Roland,

Re the IMU fast sampling, I think @tridge answered this question before saying that the the Pixhack v3’s IMU is an MPU6000 and is apparently not capable of the fast sampling. From the pixhack website it mentions the MPU9250… I wonder if they’ve changed it.

rmackay9 · October 23, 2018, 1:00am

@andypiper,

I’ve had a quick look at the logs and I will ask @priseborough if he also has an opinion on what’s happening.

I don’t see anything really obvious wrong. The 2nd compass (internal) is not good and so it’s good that you’ve set COMPASS_USE2 = 0.

Is there any reason you’re using EKF3? Normally we recommend to stick with EKF2 unless there’s some advanced sensor fusion being used (like Beacons, visual odometry, etc).

rickyg32 · October 23, 2018, 1:13am

A bug…nooooooooo.
Ok cool thanks. If you have a chance to look in the logs I would love to know if anything jumps out at you.

Thanks again by the way…It’s greatly appreciated.

rmackay9 · October 23, 2018, 2:24am

@kknet,

Thanks for the logs. I’ve created a new topic to discuss this. In short I’ve reproduced the issue although my setup is a little different. I can see that Copter-3.6.0-rc12 using Nuttx has a slower mavlink output rate than Copter-3.5.7 but Copter-3.6.0 with ChibiOS is a tiny bit faster than 3.5.7.

Cipta · October 23, 2018, 5:53am

Hello Randy,
Yes that’s true but after sending him specs and log I have heard nothing. By the way I switched to EK3 and maybe I feel the copter stabilize performance is a bit better.
Thanks

rmackay9 · October 23, 2018, 6:45am

Roland,

So if indeed the Pixhackv3 is using the MPU6000 IMU then the fast logging won’t be possible I’m afraid.

Re choosing between EKF2 and EKF3, Paul Riseborough has always said that the EKF2 is more reliable and recovers faster in cases where it has gone into a bad state. He’s said that most people should stick with the EKF2 unless they have a specific need for the EKF2. You’re free to do as you choose of course, it’s your vehicle and it’s open source.

andyp1per · October 23, 2018, 7:12am

@rmackay9 - thanks. I switched to EKF3 because I was getting random dropouts in the compass innovations I may be able to dig up an EKF2 log with similar behaviour (and I see others complaining of random compass switching), although the weather has suddenly turned wintry here so if I can’t it may be a while before I fly again. I also wanted to see whether EKF3 coped better with my high X vibe and high compass interference.

Why would compass 2 be so much worse than compass 3? They are basically in the same place on the pixracer. One calibrates well the other does not (see my post a few up).
Could it be the EKF is looking at compass 2 anyway in some edge case?

andypiper · October 23, 2018, 6:22pm

@rmackay9 here is a similar log from EKF2. No compass switching that I can see but the second IMU innovations are missing until midway through the flight.