Software regression testing

I’m interested to know if any of the lead developers for a vehicle type have a documented way of conducting regression testing on their vehicles firmware. Is there a standard set of tests you do before releasing as beta whether it be SITL or actual flight test? Is there any coordinated method for testing once a version is released for beta testing? Or do all of the users just test at their own risk knowing it is beta version. They do their own set of tests that they deem necessary to vet the software.
I realize that there are so many changes that go in to a major release in such a short time that it is difficult to test all the possibilities. Just wondering how the process works and the extent of testing that coordinated or not.

We sure do. Real flight tests and driving is normally done before a release. But we also have some auto testing. Here is the web page - not the pictures at the end.

The directory ardupilot/Tools/autotest has the files that do this - is probably the best place to start.

Thanks, Grant.

Thanks for sharing. That is very interesting. I’ve always used the autotest directory to run SITL but never realized the powerful tools. Do you have set scripts for the real flight tests and what you are looking to learn from it. Test plans?

Hi Bill,
For what you are doing with Helis I would highly recommend you get RealFlight8 and use that with ArduPilot. If you want a demo of what can be done let me know and I’ll share my screen with you on skype or G+ hangouts so I can show you what its like. It really is quite revolutionary for testing the heli code.
Cheers, Tridge