Pixhawk 2 real life performance measurement

While studying info on different autopilots and trying to compare it with Pixhawk 2 I found that almost none of autopilot developers declare autopilot performance specifications. Many datasheets have hardware, software specifications, features, etc, but nothing about real life performance. I don’t mean performance of used sensors - there’s lot of it, but performance of autopilot as entire device.

Does it make sense to measure and declare these specifications? If yes than what characteristics we should define and how to measure it? And then what are real life performance characteristics of Pixhawk 2?

What I think would be useful to measure and declare:

  • estimator attitude, position and speed accuracy based on built-in or recommended sensors in GNSS fix mode an no fix mode.
  • wind speed and direction accuracy

What do you think?

difficult to do this without a reference aircraft and conditions. Too many variables otherwise.

Nobody think it’s easy.

But how can we tell that some autopilot performs better than other? Speaking methodologically we can’t even progress without measuring. We need to have a metric to measure advance. When we talk about features it’s easy to measure. But how can we tell for example, that all improvements to Pixhawk 2 really gave us any progress in performance? So I think we have to think about measuring or at least evaluating Pixhawk performance.