@asw2000 something that worked for me was reducing mot_thst_expo and then setting Mot_spin_min slightly higher and then recalibrate esc from within mission planner. My take off vibration was gone. But I should mention in my case I set mot_thst_expo at 0.
https://ardupilot.org/copter/docs/motor-thrust-scaling.html
Please check more details regarding thrust expo and set it according your esc and motor configuration. Since its in-house motor and esc you should have your thrust curve and details and you can have easier time than me.
Checkout this discussion too.