Confirming what Bill had said above already, yes I had tried the filter at 4hz. It helped me get the P term above .02-.03 without really bad tendencies, but ultimately I could not go above .05 before the ossicilations piled up.
Basically I am finding that filter choice, whether it is the in-built low pass filter or the notch bill just helped me implement, determine where the instability will pop up. I notched at X frequency to attenuate the problem areas of the spectrum i detemined by setting the lowpass at 20 hz and seeing where they started. Then i notched the frequencies for pitch and roll as well as setting the lowpass at 4hz and have acheived great resluts. There is now a very weak instability in the 3-4hz region, but it is nothing in comparison to the issues before. So using the filter seems to have moved an instability to a lower frequency. That being said the positive effects of the filter without any rigorus tuning have been nothing short of stellar. It is truly an amazing difference in the handling quality. I immediately felt "at home" on the sticks to coin a phrase and proceeded to fly it like I stole it.
I would be completely content leaving it where I did yesterday but the reality of it is that I started from scratch and only put a few flights on it and nailed it down 100% better than it was before, so im guessing with a closer look at the data and some more tuning flights it can only get better.
My current settings: Low pass filter 4hz both PIT and RLL
Notch filter 5hz PIT 7hz RLL
P term .154 RLL. .13 PIT
I term .18 PIT. .19 RLL
D term .001 PIT. .001 RLL
ILMI PIT .07 ILMI RLL .07
VFF PIT .06 VFF RLL .05
I LEAK RATE .02
IMAX PIT .4 IMAX RLL .4
With these settings if feels very much like a standard FBL unit. I think that is why I am happy with the results so far. I am running lower P term on the PIT axis which is counter intuitive from the norm I guess as from what I have read the RLL axis is usually the first to have issues, but since I have a bit of a bounce I dialed it back for the short term.
I believe Chris has hit the nail on the head in that it is a product of the headspeed I am running. That being said, I am going to try to feather in some D term and see if that helps the bounce a bit? Worse comes to worse I will bump up the headspeed a bit.
I just know that before the filter, adding more headspeed required me to keep lowering the P term from the already low value it was at so I split the difference per se' before.
I am also going to see what happens when the Low pass filter is set to something like 10hz as well. Basically at the moment I am putting aside the end result I have in mind for the sake of data collection. This produced results that are incredibly promising I feel for the traditional helicopter side of Copter, I always have the Octa Copter for work if I need to do a job so the heli is going to be a test platform for the near future now. Turning out to be very good stuff here!