what makes you think that failsafe is kicking in too early?
If I look at your graph of curr.volt it appears to be a “normal” discharge curve, leading you from 16.8V (full battery) to ~14V at the end, at which point the battery failsafe kicks in (as it probably should, 3.5V per cell of failsafe level is what many people use as default I think).
What is noticeable are the momentary negative spikes that get close to 10V (I assume these are max load moments). As far as I know 2.5V per cell is the threshold that must not be gone below as at that voltage the cells take unrecoverable damage.
To me this looks as if you are pushing your battery to the limit with the max current you mention (180A is 45C for a 4Ah battery…).
You might want to consider getting a slightly bigger battery with the same C rating, that should help have less dramatic voltage drops (and let you fly longer at the same time).
How much mAh do you get charged back into your battery after such a flight that terminates with failsafe?
P.S. I would definitively do the voltage calibration with 0% load - 16.8V sounds perfect as idle voltage for a 4S battery.