Quadplane with no control surfaces?

Hello I just stumbled over this exciting webpage.
I had the following idea: when building a quadplane with four quad motors for lifting, and one motor for forward flight, could you just leave out all the control surfaces? The quad motors should give full control over all axes. Maybe that does not work at higher speeds though?