Right, that was what I was going to ask - what is the weather like? The more expensive flight controllers have IMU heaters to avoid calibration divergance. If you are flying in very cold weather I can imagine things shifting. Have you tried insulating the IMU chip?
The weather is coldā¦ But I think itās more than just that. I fly in the cold every dayā¦ The issue is present on other copters in the summer (though not this bad). And, it does it insideā¦
It is a new kakute mini, and I have two of them. They are both like this. All my omnibus nanoās do this too, to varying degrees. (Iāve flow 3 omnibus nanos regularly for years, and now 2 kakute miniās all winter)
Itās only when I do not have GPS. As soon as I see the message about EKF using GPS, itās fine. But I often donāt have GPS on these little microcoptersā¦ Flying in the woods under the tree canopy, flying indoors, etc.
The issue is not present on any of my 250mm and larger copters.
At least, not that Iām aware ofā¦ I guess I always fly those where I have GPS reception.
Things me and a couple other people who have posted about this issue have thought as the cause have been electrical interference, either needing better filtering, or maybe the 4in1 ESC being mounted directly under the FC causing problems. (Iāve tried caps. Canāt do anything about the ESC)
Someone was sure it was related to the load on the processor. (Iām fairly sure thatās not my issue)
I was quite confident vibrations were the cause, but not so much anymore. (Fixing vibration issues on my last kakute mini definitely made it better, but I donāt think I can improve them on this new copter)
Temperature, like you guys are suggestingā¦ But I donāt think thatās itā¦ all these small FCās build up the same gyro bias in any weather. I will try insulating the IMU, though.
Since every one of my small copters does this, Iām surprised more people havenāt noticed the issueā¦ But maybe most people wait for GPS before flying. I know at least 4 or 5 people have posted about the issue, all using sub 250mm copters. But weāve all had different theories to the cause. Iām not sure if the other people solved it or just moved on to other projects.
I just looked at logs from all 5 of these small boards (3 omnibus, 2 kakute), and ONE does not have the ever increasing bias on the X axis. Iām not sure what to think about this at the moment.
-Edit, no, theyāre all about the same. I looked at a lot more logs, and apparently they all occasionally have a flight where it doesnāt increase the bias much.
So a few things to consider.
- My guess would be that the leans while no GPS is because the EKF cannot compensate for the X-axis offset without the additional data that the GPS provides.
- Accels are not calibrated on startup. Have you tried re-calibrating in the field when the temperature has stabilised?
- It would be good to see a log with the raw accel values for comparison. I suspect this will look the same, but would also be good for comparison. Will fill up your logs quickly so beware.
- You are flying in stabilize right? Can you reproduce this with the GPS disabled and in a stable hover? That would be interesting for comparison.
Iām struggling to get a logā¦ By the time I land theyāre 16MB, and when I try to graph things, itās all just a straight line. I think it basically starts overwriting itself and gets corrupt. Iām heading to work but will try again later or tomorrow.
What seemed odd. I took my first flight by quickly arming and flying, w/out any warm up period. It wasnāt leaning badā¦ A bit, but I could deal w/ it. I landed and accidentally saved the wrong log file, then cleared the logs, so no data. Then my next flight it tilted VERY badly. But, the copter had been inside and warming up for a while. Temp is about 40F outside. I had to land because I couldnāt really control it. I just pulled the battery and plugged it back in, took another flight, and the leaning was not too bad again. So that could be pointing to your temperature ideaā¦ But I really donāt think thatās it. I fly all of these in a huge range of temperatures. I was flying in below zero (F) weather a few weeks ago w/ my omnibus copter, after an inside warm up period, and didnāt have any problems.
One thing I noticed about the leaningā¦ It started leaning to the left and got to about 45 degrees. Then I yawed around so forward was pointing to what was the left side, and as I yawed, the copter tilted up. So the left roll became a back pitch. Then it leveled itself and started tilting left again. The direction of the error basically was directional based on the earth, and not based on the copterā¦ or somethingā¦ I was able to repeat that a few timesā¦ (My other copters are probably the same, but the amount they lean is not nearly so bad, and I donāt really notice the several degrees of pitch being off after I yaw around)
Well anyway, off to work. Iāll try logging raw again while doing a stable hover when I get some time.
A log is key and sounds like a little detective work might give us a useable clue - it feels like we are close to some better insight.
Make sure you are using the latest release candidate of copter as it includes my logging fixes which should clear up the overwriting issue you are seeing
I am using rc4ā¦ Still logging issues. I can see legit data in the table for gyr, acc, etc. But both MP and APM Planner are only graphing straight lines.
I guess Iāll upload itā¦
The log looks valid. I think the issues you are seeing are not log related, but perhaps scaling related - so the incredibly large peaks are hiding the actual data on the āflatā line.
The incredibly large peaks are worrying, that should not be happening, that points to some kind of sensor overflow I think.
@tridge any thoughts on the raw sensor data in this log?
Not sure if itās related or an issue, but I often donāt get ātilt alignment completeā and āgps saving configā messages until after I land, after about 6 minutes.
Iāll get another log later to see if those same big peaks are in it.
Insulating the IMU seems to have helped the bias growingā¦ But it still leans the same.
In fact, I had GPS lock, and it still tilted! This is the only copter thatās ever done that to me.
I am actually starting to wonder about a bad FC againā¦ This is the one I originally used when I started the āKakute mini problems (or just problems w/ my build)ā thread. We got side-tracked by all the vibration issues, and I did replace this FC w/ a new oneā¦ and at the time I just focused on the vibrations being the same on the new one as this one. But maybe the leaning has been better because of the new FC. URG, I donāt knowā¦ I tried to get Holybro to RMA it a month or two ago when I first had the issue, and they flat out told me Arducopter isnāt supported. Show them a screen shot of the IMU test in betaflight failing, or else they wonāt RMA it. This issue is too subtle for the IMU test in betaflight to find a problem.
I couldnt find all the parameters (or much else) in you log files. I was looking for:
INS_GYR_CAL 1
And what value do you have for EK2_GPS_CHECK ?
Set this:
LOG_BITMASK 141310
And then get us another log to look at
It will be over mavlink as that board does not have an SD card
Out of curiosity - what makes you think it is corrupt? It will start abruptly because itās a wrapped 16Mb FRAM log and APM Planner wonāt like it because there are ISBH and ISBD messages in there, but I wasnāt seeing any other problems?
INS_GYR_CAL is 1
EK2_GPS_CHECK is 31 (allows arming while moving, which I need)
That is what my log bitmask is atā¦ My last one had IMU disabled because I was collecting raw_imu, but the previous one should have been 141310, and I will go take a flight now w/ 141310 and post the file in a while.
Well, itās raining/snowing. I did get a hover inside with the tilting issue, which might actually be more helpful than a log of me flying around outside.
The log has two hoversā¦ The first one wasnāt badā¦ I am having to slightly compensate for the tilt. Then I landed and the copter sat there for a while. Then the second flight was more typicalā¦ The way I have it trimmed is with some extra right trimā¦ So, I take off and am having to pull it left for a few seconds. Then the issue develops and it levels off and starts pulling left, so I am compensating by giving it right roll.
Iāve just been randomly trying thingsā¦ Tried various gyro and accel filter settings, taking gyro all the way back down to 20. Tried various settings for the ek2_gbias_pnse parameter. Tried GSCL_P_NSE, GYRO_P_NSE, put the loop rate back to 400, tried oneshot instead of dshot. Nothing made a difference in the leaning.
It starts leaning just seconds after I take off. Then after I land, Iām sitting level on the ground, but the HUD in MP shows me at an angle. The HUD starts leveling itself, and after sitting unpowered for several seconds, the HUD in MP shows me level.
It seems like something in the logs would have to show this difference from reality, but I canāt find itā¦ Other than looking at my roll RC input, and seeing me manually compensating for it. And I did notice AHR2 roll and ATT roll are different from each other. The worse flight seems to be more different.
Thanks again for everyoneās help. Iām close to giving in and purchasing another new FC to see if it makes a differenceā¦ (anyone have any contacts at Hollybro to get a development sample? This is getting expensive )
Iāve mentioned before, all my copters w/ these small boards do this, but to different degrees. A couple years ago when I was experimenting w/ the omnibus, I basically had 1 thatās so bad I canāt use it, and three that were OK, but some worse than others. Now itās looking like a similar thing w/ the kakute. This one was badā¦ I replaced it and the next one I was able to get working satisfactorily. (I assumed because of vibration mitigations) But now Iām trying to use the original one again and can not get it working, with even lower vibrations.
So I think you can see what you are talking about in this graph:
The AHR2.Roll is the inertial roll only (I think) whereas the ATT.Roll is the EKFās idea of roll. You can see that they diverge quite dramatically in the second flight. You can also see that the innovations for GPS velocity measurements in the north direction vary dramatically as well (https://ardupilot.org/dev/docs/extended-kalman-filter.html) the docs say these should be small and centered around zero. Yours are anything but - and that seems to be because the GPS started working half-way through the second flight.
Can you try this - physically disconnect the GPS, re-calibrate the accels on site (you still have a large accel bias, but it is constant) and then see if you can reproduce the problem?
I did calibrate accelās this morning before my tests. And, I did it wellā¦ Attached the copter to the inside of a box, and had it level and still on each side as I calibrated.
GPS is likely because Iām indoors w/ a quite bad signal. The issue is definitely present w/ no GPS. I did for a couple flights this morning turn off the serial port (put it to -1) so it wouldnāt talk to the GPS, and those flights still had the issue.
I can get a log w/out the GPS, thoughā¦ I unfortunately overwrote the logs from when I did it this morning. I D/L them, then cleared them. Then messed around w/ the copter for a while (tested vibrations per motor) and downloaded those logs. Since there was no GPS, they had the default file names, and it just overwrote my original logs.
It would be really nice if MP had a save dialog for the log downloaderā¦ Would be helpful for people w/ multiple copters, and apparently for when thereās no GPS data to rename the logs.
Ok, from your memory of the second flight what were the leaning characteristics? You can see the two crossover midway.
FWIW I donāt think you get dated logs with FRAM even if the GPS is working, I looked at trying to make this work and it wasnāt that easy - I agree save as would be nice!