SITL copter swarm incorrectly impacts ground

I’m trying to setup a SITL copter swarm sim that fly’s into a canyon and has negative relative altitude (from takeoff). Using the single copter it works, but the copter swarm does not. I have tried using absolute and terrain relative altitudes with no luck. When ever the delta altitude goes negative during a swarm mission the drone(s) reports hitting the ground. For the terrain relative path I enabled terrain feature on the drones and I verified I’m well above the ground. The same path when running single copter works just fine. Is there a setup step I’m missing?