S500 Quadcopter strange decent behavior apm2.6 3dr w/telem

Seems to happen no matter what mode I am in. Starting out in stab mode until I have my 10-13 sats then switch to loiter mode. It kicks left and right once or twice then sets in to a nice loiter. I begin to climb up as far as I am able to without giving myself a heart attack :unamused: . I get my aerial footage and then begin a descent. battery is at 11v. While lowering the throttle it kind of falls 2 inches on the left then 2 inches on the right. It does this most of the way down. The faster I want to descend, the more it falls on each side. The strange thing is, is that I am not trying to descend to rapidly at all.