It basically looks like the helicopter is not stable and over-reacting to control inputs. The RATE messages show a lot of overshoot of the rate in this log, which indicates the VFF is too high for this helicopter.
Getting the rate right is the hardest part of tuning. What I would do is record your rate PID settings, then zero them out for pitch and roll, including the ILMI's. Leave the VFF where it is right now at .11 and hover the heli just using the VFF. Make some sharp stick inputs in both pitch and roll, pull the log and look at the RATE messages. This is the graphs we'll be looking at:
In that graph notice how the actual rate is exceeding the desired rate by a considerable amount, in this case in the roll axis. When you make the sharp stick inputs hovering in Stabilize it will cause peaks in that graph for the desired rate. What you want is for the actual rate to have the same height (amplitude) peaks when you make those sharp stick inputs to pitch and roll the heli. When the actual rate is too high, it causes the heli to respond extremely fast. Which can make it unstable. The lines in the graph will not be exactly on top of one another. What you're looking at is the amplitude of those rate peaks when you make a high-rate request with the sharp stick inputs.
Once you get the rate so it's responding properly, then re-tune the rate PIDS's and reset your ILMI back to the values you recorded, and try it again to see if it's smoother.
Being this is a 450-class and they are notorious for being extremely fast-handling heli's, the VFF values will likely need to be lowered some to get the rate to match desired. But don't short-change it by being gentle with it, or you'll over-tune the VFF. Slam it full-stick back and forth to make the maximum rate request so you can get it tuned properly.