The gyros drift (all the time) hence the need to calibrate every boot. If you gyro cal is taking a long time then the craft is not still enough.
The way to disable the gyro cal at boot time is “boat mode” designed to allow boot and launch from a moving platform. You wouldn’t want to use that mode all the time though, only when actually required.
Try installing Rover firmware, then back to Copter and run through all calibrations and set up manually - don’t load any old param files.
If that doesnt make any difference you might have to talk to your supplier about a replacement.
My situation, I was tested with 4 units of same board, different lote time, because I am drone productor, and all X7 board have same problem gyro very sensity, I fell this is calibration problem, because only doing when run motors stopped on the table, is very clearly, I am studing to recalibration, my cause is doing only Plane fw, in copter it didn’t doing.