I’ve been running claude and codex together to create an AI review system for ArduPilot for the last 3 months. I’ve been refining the system as we go and I think it is now working very well. I’ve got very positive feedback on the system from other developers and we now use it for each of our dev calls, plus a separate AIReview label for non-devcall reviews.
I’d now like to move this off my personal accounts and onto accounts funded by ArduPilot. I’d like to propose a 4 month trial with x20 Pro for both claude and codex, cost of US$400/month, total cost US$1600.
If this goes well then I will need to do a team vote for future funding.
I’ve recently added a dashboard showing the usage of the system:
you can see the reviews themselves here:
here for example is yesterdays DevCallEU run:
and here is the latest “AIReview” label run:
The system is currently running out of cron on a VM in my house. I’ll likely leave it there initially, then move to a more permanent server later.
I’ll also change it to use a separate dedicated github username instead of using tridge, maybe ardupilot ?
Happy to support a trial, but part of the trial needs to be demonstrating that this is better than alternatives such as CodeRabbit and Watchgoose that are free for public repos/open source.
I think we’ve already demonstrated that. The key difference is this uses two AIs, not one, the review quality is vastly better than you get with a single LLM, at least with current LLMs
Like James I am happy to support a trial but I do not currently see this as better/more effective than something like CodeRabbit which I have used extensively and much prefer. Both CodeRabbit and this tool provably make mistakes (as does AI in general) but the usability of CodeRabbit is far superior. Note that I certainly think we should be doing AI review in general so whatever we finally agree on as a team I will support.