Filter looks good.
I ran the file through the online Magfit and got these numbers for calibration. Both compasses looked but benefit from running current offset (COMPASS_MOT)
COMPASS_OFS_X,99.94135301903044
COMPASS_OFS_Y,-4.835519981104709
COMPASS_OFS_Z,50.250378311671156
COMPASS_DIA_X,1.0117537800791392
COMPASS_DIA_Y,0.9818517132775418
COMPASS_DIA_Z,1.0063945066433189
COMPASS_ODI_X,-0.003602344416835102
COMPASS_ODI_Y,0.040688781800220064
COMPASS_ODI_Z,-0.058146142788499756
COMPASS_MOT_X,-1.985606683577201
COMPASS_MOT_Y,1.410246026038831
COMPASS_MOT_Z,2.523027101894699
COMPASS_SCALE,1.0191731682386649
COMPASS_ORIENT,6
COMPASS_OFS2_X,78.41447421779984
COMPASS_OFS2_Y,43.206690925386646
COMPASS_OFS2_Z,-107.95031784010868
COMPASS_DIA2_X,0.9958673701718391
COMPASS_DIA2_Y,0.9943710827410694
COMPASS_DIA2_Z,1.0097615470870918
COMPASS_ODI2_X,-0.012584929405016348
COMPASS_ODI2_Y,0.008887379839260376
COMPASS_ODI2_Z,0.0033807649604197395
COMPASS_MOT2_X,1.1396117243532504
COMPASS_MOT2_Y,1.3249293227740089
COMPASS_MOT2_Z,-4.580542271601371
COMPASS_SCALE2,0.9986165464255835
COMPASS_ORIENT2,0
COMPASS_MOTCT,2
The quick tune looks functional. I can see why you say itās a bit soft, but I donāt see any oscillations or really crazy overshoots. So Iād say that worked.
The bad news is the vibrations. Before you go any further is you need to sort out the vibrations. Thereās still something going on. Itās on all axis, and thereās even a few clipping events on one of the IMUs. Have a good look at the props. Maybe oneās got some damage or a bend or something. double check all the arms are tight. The X500V2 has an option for a landing gear leg brace that you can 3D print. Check your landing gear isnāt flopping around too much, and if you have access to a printer maybe consider one of those braces. How did you mount the GPS? Make sure that isnāt loose or moving.
So my biggest suggestion is to sort out the vibrations before you do anything else. Everything else looks like itās on track.