Arducopter 3.6 SITL with gazebo unstable flight

I recently upgraded my arducopter version from 3.5 release to 3.6 rc12. My student team has been using the SITL with the arducopter gazebo plugin extensively to test our code, and it works great! Now, upgrading to 3.6 the drone is unstable in gazebo and eventually crashes even using the same parameters we had in 3.5 and using gps. We noticed this bug in april when some of us forgot to checkout 3.5 once we cloned the firmware, but it seems to persist. I’m really not sure what changes might have caused this. Any help would be appreciated.
Thanks!

Eric,

Thanks for the report. I’ll see if one of the dev who uses Gazebo can have a look.

2 Likes

Hello,

I did a test yesterday, and it is working well on master … I will try with 3.6.
Could you explain how did you get to a crash ? What the sequence and what mode you are using ?

I will update the pid parameters and instructions soon, it should improve flight on slow computer and all manual modes.

1 Like

I did a test with 3.6 and everything is fine on my side …

1 Like

I have tested our setup on a couple of different computers and had a friend do a sanity check on me. Unfortunately could not find the problem.

Here is our setup
ubuntu: 18.04
gazebo: 9
arducopter gazebo plugin: https://github.com/SwiftGust/ardupilot_gazebo checking out the gazebo9 branch

running sim

cd ~/ardupilot/ArduCopter/ && sim_vehicle.py -f gazebo-iris --console -I0 

run gazebo

cd ~/ardupilot_gazebo/gazebo_worlds
gazebo iris_irlock_demo.world 

The drone takesoff and starts to oscillate varying its position by over a meter in radius.
If we use our parameters from 3.5 the drone will crash.Here are our parameters from 3.5
https://github.com/Texas-Aerial-Robotics/Controls-Other/blob/master/Params/Sim_master.param

My suspicion, is either a parameter change in 3.6 from 3.5, or 3.6 with gazebo 9 issue. Interesting thing is that our setup works beautify in 3.5.

Hi Eric

I am using the same setup and I was also facing the same issue. In my case setting AHRS_EKF_TYPE=10 fixed the problem. (This was previously suggested to me by @tridge here when I had similar issues on Ubuntu 16.04)

Hi. Been meaning to pitch into the conversation.

Using the same configuration as @sprained_ankle above (Ubuntu 18.04, Gazebo 9, SwiftGust ardupilot_gazebo (gazebo9 branch)).

In this folder, it’s the exact same setup, but one is checked out to ArduPilot branch Copter-3.5, and the other is ArduPilot branch Copter-3.6. https://drive.google.com/drive/folders/1pT9FG61l8VkEya8lUsyR0Nj0VV9YZpae?usp=sharing

Willing to help try to fix if there’s any ideas where this could be coming from.

Hi @umer936 I tried implementing lock-step scheduling since that was the suspected reason for unstable flight, but that didn’t fix it. Maybe I may have made some mistakes while implementing. It will be great if you can fix this because without stable position control it is quite difficult to do any sort of autonomous planning with precision. There is an open issue here too OctomapPlanner Issue 3

@AyushGaud Hmm. Not sure I know how to implement that. Nor am I sure that’s where the problem would be. It’s just odd that it’s great in 3.5, but not in 3.6. I would think if it needs scheduling, it would’ve needed it in 3.5 as well.

@khancyr I was wondering if we could get a list of what your setup is? As I’ve said above, ours is Ubuntu 18.04, Gazebo 9, SwiftGust ardupilot_gazebo (gazebo9 branch).

1 Like

Hello, I’m fine with my copters with gazebo 7 or 8 but not sure about 9.
It would be nice if you can share the dataflash log, log would be in ~/ardupilot/ArduCopter/logs if you always used same command launch.

My guess is something to do with gazebo itself rather than ArduPilot firmware since my repository uses lots of gazebo supported dynamics such as liftdrag plugin, Open Dynamics Engine which are subject to change by Gazebo, ODE developers and I see tremendous changes in gazebo 9 recently, lots of minor versions upgrade have been made.

P.S. Is it fine with my gazebo 9 dev branch??
some of users are reporting that has trouble with installing
but I couldn’t get my hands on since I have no clean desktop to test it.

Hello, I am on ubuntu 18.04 too, but with my version of the plugin (I check and there aren’t major differences with Swiftgust one’s beside of the init). But I have modified the world init to give more time on each sim step.

It works now. The problem was with the step size. By changing the following lines in our world file we were able to get the setup to function correctly.

from

 <real_time_update_rate>0</real_time_update_rate>
 <max_step_size>0.0025</max_step_size>

to

 <real_time_update_rate>-1</real_time_update_rate>
 <!--<max_step_size>0.0020</max_step_size>-->

I am still curious as to what changed between 3.5 and 3.6 to make this break, since this world works fine with gazebo 9 + arducopter_gazebo plugin + arducopter 3.5

Anyways, thank you all for taking the time to work through this problem! Your input was super helpful and we couldn’t have solved it without ya’ll.

2 Likes

@SwiftGust just so you know, your plugin with gazebo 9 checked out builds perfectly fine. It also works perfectly fine with the update rate/step size solution from above.

It is great you managed to handle problem.
and glad to hear my repository is OK.

Those two parameters are for physics engine (ODE by default).
What I’m guessing from your work is that,
max_step_size was not enough to converge solution for dynamics calculation behind.
But unclear how it is affected by ArduPilot firmware.

This maybe helpful, http://gazebosim.org/tutorials?tut=modifying_world
to understand those two parameters, but it may need to look deep down in gazebo,
since gazebo API documentation does not reveal enough information for how does those parameters actually work.

Or it could be a computation load issue by a slight chance,
you can launch sim_vehicle.py with argument -j # of thread for multi-threading
i.e sim_vehicle.py -v ArduCopter -f gazebo-iris -j 8

Plus, I recommend you to use ArduPilot plugin instead of ArduCopter plugin,
ArduPilot plugin is general interface for any firmware behind(Plane, Copter, Rover, etc…),
while the later is specific interface for ArduCopter, remained for compatibility.

hello,

-j parameter is not for launching SITL with multi-threading, it is just for compilation !

The ArduCopter plugin shouldn’t be use anymore as the ArduPilot one is far better !

I think the issue is with the computing power, requesting max_step_size gives hard time contraintes that normal computer generally cannot achieved … since the timing aren’t achieved on gazebo part, SITL is burning to much time without update that lead to timing issue and instability on the controler.
My patch only said to computer to use as much computation speed as available for gazebo, the lockstepping is thus constraint by SITL update.

It recommand to not build SITL in debug mode if you don’t need it, as you will reduce computation need for SITL by 10 times without debug !