Taming the beast, I mean tuning lol!

I’ve got a big, heavy X8 at 15kg and 28" props. The initial tune seems pretty stable, but the overshoot is a bit confusing. Normally with a new smaller rig I wouldn’t see the overshooting around level, just at the peaks of rotation. And normally I would either lower the p and i or increase the d. These look more to be the mass of the drone itself and not the controller. If I lower the p term, it’s going to get worse, right? I just wanted to get it more sorted before autotuning. I’ll also apply a dynamic notch filter before AT as well. Any thoughts? Here’s a link to a short flight:

Overshoots on the big movements indicate to me (and I could be wrong) too high P term - usually the Angle P. In your case ANG_P is still low so maybe the issues are coming from the relatively high (default) Rate terms.
EDIT - they way I’ve stated this could be backwards and wrong, it’s too early in the morning! I would still go with my recommends below. Probably not enough Angle P term, and reduce Rate terms to avoid inducing other bad effects.

I would try decreasing the Rate terms, and increasing the Angle P terms as follows:

ATC_ANG_PIT_P,6.0
ATC_ANG_RLL_P,6.0
ATC_RAT_PIT_D,0.00275
ATC_RAT_PIT_I,0.11
ATC_RAT_PIT_P,0.11
ATC_RAT_RLL_D,0.00275
ATC_RAT_RLL_I,0.11
ATC_RAT_RLL_P,0.11

and threw in an extra bit of reduction to D terms. You could try 0.003 if they seem too low.

I would also set these for the next flight

BATT_FS_CRT_ACT,1
BATT_FS_LOW_ACT,2 or 3
FENCE_ENABLE,1
INS_ACCEL_FILTER,10
INS_HNTCH_ENABLE,1  // set this then refresh params to see the rest
INS_HNTCH_MODE,1
INS_HNTCH_REF,0.173
INS_HNTCH_FREQ,33
INS_HNTCH_BW,15
INS_HNTCH_FM_RAT,0.7
INS_LOG_BAT_MASK,1
INS_LOG_BAT_OPT,4

If there is instability then reduce the ATC_ANG_x_P terms a bit.
When you have good attitude control try an ascent and descent to check MOT_THST_EXPO

I’ll give it a go when it stops raining. I got to ask though, where did you get the INS_HNTCH_FREQ from? I don’t think I was running the batch sampling, INS_BAT_MASK was 0. Just curious lol.

I get INS_HNTCH_FREQ from an approximation I have based on prop size. The real value depends on much more than just prop size of course, but I’ve got a rough formula that is near-enough as a starting point.

This spreadsheet was the basis for the Initial Params in MissionPlanner and I’ve added a bit for HNOTCH.
The spreadsheet is derived from Leonard Halls tuning guide and other work.