Fast_loop() timing jitter

Hi,

I’ve added some performance monitoring to the code and it looks like the fast_loop timing goes over time fairly often. See graphs below: in EKF2, readIMUData() is supposed to run each 2.5ms but I see lots of spikes inlcuding up to 10ms pretty frequently.
Same thing for NavEKF2_core:UpdateFilter(): it’s supposed to run each 10ms but displays spikes at the same time as readIMU()

This is an issue for us because we are trying to design a custom flight control algorithm but this timing jitter wil create performance problems.

I was curious as to what causes those spikes? Is it an issue at the OS level?
Any idea how I can identify where the issue is coming from and what’s the best way to remove those timing spikes?
I’ve included the log here:

fast_loop() expects an IMU sample to be ready, if it is not it will wait. I’m guessing that is what is going on here. The why may well be dependent on your hardware - if you have a fast IMU and fast STM32 then I would expect this to better. What are you running it on? The converse is also true - something else taking too long and fast_loop() getting late to read the IMU the next time around :slight_smile: SCHED_DEBUG will tell you more.

1 Like

Awesome, I didn’t know about SCHED_DEBUG. Will try it and report.
I’m using a pixracer.

I really recommend you try something faster. Also you could switch off one of the IMUs

I did activate SCHED_DEBUG but I don’t see any new messages in the logs. Is that debug info available through an nsh console like seems to be indicated here: https://github.com/ArduPilot/ardupilot/issues/7986#issuecomment-375263178
If yes, I’m guessing I should access it on a debug port on the pixracer? Is that the JST SM06B connector described here?
Also you recommended trying something faster than Pixracer, is there any autopilot in the same form factor that’s faster? I think the pixracer’s CPU runs at 180MHz.

The console is available if you use mavproxy.

I have a 3" running with an F7 (http://www.holybro.com/product/kakute-f7-mini/) 2hich is 268Mhz I believe. There are several F7 options.

Great, I’ve removed a bunch of custom logging that I had added and the loop time dropped to 10.85ms. Then I deactivated the second IMU and the EKF loop time is now down to 10.01ms … not very far from 100Hz!
The SCHED_DEBUG option gave me some good info on which tasks overrun but not how often they do. That would help me prioritize which task to focus on first. I’ve opened a separate topic as some other people might be interested by scheduler fine-tuning beyond my specific EKF jitter problem.
The new topic is here: Streamlined scheduler statistics?