I’ve been fiddling with this LeddarOne thinking it was an issue with the sensor itself. I can’t fly it without contstant Bad Lidar Health issues and quite a large disparity between barometer and rangfinder height readings.
So I decided to load this build onto my Main rig. And now it has the same issues with a Lightware SF11/C rangefinder.
The barometer (the reading on the MP hud) on my backup is +/-7-10 ft at any given time in flight, and doesn’t seem to even out sitting on the ground.
My main appears to be doing the same thing. They both begin to bob up and down although the LeddarOne seems to do it FAR worse.
Both rangefinders are spot on, in the telem, I’ve measured both at hove and they’re on the money.
I haven’t loaded the new sensor build onto my main, although I did on my backup with the LeddarOne.
Here are the last several logs from the backup copter. They have both 3.4.2 rc2 and 3.5 sensor update.
Here are the last few from my main, with the 3.4.1 Craft and Theory build, and I loaded 3.4.2 rc2 official tonight, which started showing the problems.
Edit: Crashed it
Here’s a video of the bobbing, prior to the crash if you want to see what it’s doing. I’m at hover, in Pos Hold. This behavior occurs over any surface, just keeping over the grass in case - which still = broken things on crash. Rangefinder Height Issues
I’ve had a look at the “32.bin” and “39 3.5 bad crash.BIN” logs and there are very large jumps (about 10m) in the lidar range. There is still at least one suspicious thing in the LeddarOne driver so it may be a software issue.
Can you point me to the logs where you’re using a different lidar? This will help me determine if it’s a lidar driver issue or a frame/config issue.
The crash in “39 3.5 bad crash.BIN” looks like some kind of motors/ESC issue in that the actual roll just suddenly diverges from the desired. It points towards a failure of the front-right motor because the vehicle rolls right and pitches forward. The only problem with this theory is that normally, if a single motor fails, it’s output in the RCOU message goes to full for that motor and we don’t see this.
What’s worrying is that there are EKF “lane changes” just before the crash and I see large discrepancies between the AHRS2.roll/pitch and the ATT.roll/pitch. I’ve asked Paul Riseborough if he can check.
Sorry Randy I didn’t get a chance to get out this afternoon it was super windy and I didn’t feel confident enough with that specific rangefinder.
It was really strange the way it happened. I’d flown it a few times with that build and it seemed pretty stable other than the rangefinder bobbing thing…which I’m used to because the Lightware did the same thing at first until I got it sorted out. I just can’t seem to sort the leddar out.
I’d rolled left about 10 or 20’ to get over some different terrain, and semi-stabilized it, then did a few test rolls, nothing super crazy. And right as or after I rolled, it just flipped. I had zero chance to pull out and zero warning as I had telemetry on the radio and MP sitting right in front of me.
I noticed no difference in sound from the motors prior, something I’m fairly sensitive to actually.
I’ll get the main out ASAP and get something current up for you. The last log where I was having this issue with my main (lightware rf) is in the folder link above “MainPixhawkRangfinderissues”. #22 I believe although I have several in there. That’s the last time I flew it.
You can also watch the youtube video I linked, if you’re able, to see the bobbing up and down behavior literally 2 mins before it (leddarone equipped) crashed. I don’t think this attributed to it truthfully, it had been doing it already and like I said I was able to keep it controlled enough to test through it. Once it’s moving, it’s pretty stable, but still errors out.
Thanks Randy. I do appreciate all you guys’ work and time. If there’s anything specific you need or think I should look at let me know.
As soon as the copter does a yaw change, the GPS velocity innovations rise rapidly for the second IMU, so it is likely IMU2 that has the wrong orientation setting.
Ok, so this is an AC3.5-dev log so it’s code from a somewhat random point in time. We highly recommend not using master because it’s possible to pick up changes before they’re really ready. In this case we’re doing major changes in the sensor drivers and you’ve likely stumbled into a problem.
I’ll pass this onto Tridge to see if he’s aware of any current issues with the IMU orientation but really, don’t fly master! You just don’t know what you’re going to get.
The bad height sensor caused a lane switch at a point in time where the EKF for IMU2 had managed to recover. Then as soon it switched, closing the loop around an IMU with a 90 degree yaw offset caused the attitude control system to go crazy.
I will check the EKF selection logic, because looking at the innovation test levels, I would not have expected it to switch.
Update:
It looks like there is a race condition that can occur when evaluating innovation test levels for the two lanes. I have reworked the selection logic here and will complete and test on Wed.
I had a quick look at the MainPixhawkRangeFinderIssues’s 22.bin which is using Lightware lidar and the sonar data looks really good. No spikes or anything. I don’t see any bobbing either.
So this definitely points to the LeddarOne as the cause of the bobbing. The spikes must be either caused by problems in the driver or some kind of interference. There are some more improvements to the LeddarOne driver. We’ve had quite a few problems with this driver… I think it might be best to not fly with this lidar until we’ve done more ground testing to ensure it’s ok. https://github.com/ArduPilot/ardupilot/pull/5245.
Sorry once again I failed to properly describe. That log -should- have been throwing out the Bad Lidar Health issues. I pretty much have the bobbing under control on the Lightware.
I’ll make sure I’ve got a log as soon as I can with that error.
Also, on the 3.5 build, I had responded to the “calling all testers” sensor test.
Maybe I’d misread where to download it, but I thought it had said to use the “latest” folder build.
I agree. That’s why I was a little leary of flying yesterday having just rebuilt
I have really tried hard to isolate any possible cause of interference.
I have all DC wires twisted and oversized, I’ve shortened the length of the cable I made for the Leddar to as short as possible to get outside of the footprint of my copter.
I’ve even used some copper shielding tape on various leads and noisy devices (mainly power supplies I use to keep only power module power going into the pixhawk) to attempt to keep any of them from causing any interference - and truthfully I’ve seen nothing either way.
Maybe I’ll try powering off the same source to see. They are sharing the same ground/neg.
Believe me when I say, I ONLY test this thing. It’s unplugged otherwise.
I TOTALLY forgot that I fixed the issue on the main (lightware). Precision Landing was enabled for some reason. When I turned it off the lidar health issue went away.
Sorry about that have way too much happening.
Consider the lidar health issue on the lightware / non-leddar closed, due to config error on MY part.
once again I failed to properly describe. That log -should- have been throwing out the Bad Lidar Health issues. I pretty much have the bobbing under control on the Lightware.