Servers by jDrones

Maverick - Mavlink/Dataflash web log analysis/graphing

analysis
log
companioncomputers
graphs
maverick

(Fnoop) #1

Maverick 1.1.0 ‘Ooh pretty pictures’ Release

The latest Maverick release adds a web based mavlink/dataflash log analysis tool.

  • Drop any bin or text dataflash log file into ~/data/mavlink/inbox and it will be automatically picked up and imported into a nanosecond resolution time-series database (InfluxDB), and a set of links added to the web dashboard (Grafana)
  • Use mavlink-router as your mavlink proxy, and set LOG_BACKEND_TYPE to 3 in your flight controller (or SITL) firmware. This automatically saves dataflash bin logs into ~/data/mavlink/fc (or ~/data/mavlink/sitl), and as soon as you disarm the logfile will be automatically imported into InfluxDB and a set of links added to the web dashboard
  • A wide variety of system metrics are automatically collected (using collectd) and imported into InfluxDB, and are available in realtime in the System Dashboard, including power consumption and thermal status for Joule modules
  • Best of breed opensource components - Collectd, InfluxDB, Grafana
  • Preset dashboards for Flight Data, EKF2, EKF3, EKF2 vs EKF3, and MAVExplorer mavgraphs graph sets
  • Automatatically generated index dashboards for each flight and flight errors
  • Interactive graphs, zoom in and out, select series, fullscreen graph, linked graphs, events, errors
  • In the future, rich custom analysis pages using statistical engines will be included
  • Realtime mavlink graphs will come soon to a browser near you

As always with Maverick, to update a previous installation just do:

maverick self-update
maverick configure

To download and install, please see the documentation:
https://fnoop.github.io/maverick/#/

Documentation on Analysis/Graphing:
https://fnoop.github.io/maverick/#/modules/analysis

Note: 1.1.1 update released with many improvements and bugfixes

As always, feedback, suggestions, testing, bug reports very welcome :slight_smile:

Demo:
http://www.maverick.one/graphs

Login - mav:wingman
Click on ‘Flight Logs Index’ to view the index of sample flight data.
Click on ‘Error Events Index’ to see all error events across all flights
Click on ‘System Dashboard’ to see sample system data (live data from the server)

Screenshots:


Precision Landing Help needed
Dev Call Jun 26, 2017 2300 UTC
Ardupilot Log Analyser
Maverick latest release
Pixhawk 2 RCOUT C3 FLAT-LINED CAUSED CRASH
Can't edit posts
(rmackay9) #2

Looks nice! feel free to make this a blog post! (change the category to Blog)


(Matt) #3

Wow man! That is incredible work! You just got a whole lot of ooohs and ahhhs on the dev cal!


(Olivier Brousse) #4

Looks fantastic, can’t wait to try it out!


(Fabrício Silva) #5

This seems very promising! I couldn’t wait and decided to give it a try.

It seems that Raspberry Pi image is still on previous version (1.0.6). Using this image and running self-update and then configure, resulted in the following error:

Notice: /Stage[main]/Maverick_analysis::Grafana/Exec[grafana-maverickorg]/returns: Error: no such table: main.org
Error: ‘/bin/sleep 10; /usr/bin/sqlite3 /srv/maverick/data/analysis/grafana/grafana.db "insert into main.org values(10,0,‘Maverick’,’’,’’,’’,’’,’’,’’,’’,‘2017-06-20 11:02:55’,‘2017-06-20 11:15:51’)"’ returned 1 instead of one of [0]
Error: /Stage[main]/Maverick_analysis::Grafana/Exec[grafana-maverickorg]/returns: change from notrun to 0 failed: ‘/bin/sleep 10; /usr/bin/sqlite3 /srv/maverick/data/analysis/grafana/grafana.db "insert into main.org values(10,0,‘Maverick’,’’,’’,’’,’’,’’,’’,’’,‘2017-06-20 11:02:55’,‘2017-06-20 11:15:51’)"’ returned 1 instead of one of [0]

The following stages are skipped due to previous dependencies.


(Fnoop) #6

Thanks for the bug report :slight_smile: That looks like possibly an ordering issue - it’s trying to setup the data before the database has been initialised. If try a configure again, does it work? (In general with maverick/puppet, if you get an error, try it again once or twice as it might fix itself, if it’s just an ordering issue. It’s really hard get all the ordering right in complex non-deterministic systems when you’re making a lot of changes!)


(Fnoop) #7

ps - I won’t always update the downloadable images for every release, as they take a while to create and upload. Normally, self-update/configure should be just fine. But while there’s still a lot of flux as I’m still adding major features/functionality, it will break once in a while. Sorry about that, I’ll fix them as quickly as I can when they get reported. You can raise an issue in the github as well if you want.
In the future, I’ll at least test a release from the currently posted images.


(Fabrício Silva) #8

I tried twice. No success. Even after switchin to env=flight.

It’s ok. I’ll be playing with the rest of Maverick for while.

Thanks!


(Fnoop) #9

@FabricioSilva - Sorry for the delay, I’ve just done a 1.1.1 release with a whole bunch of bugfixes and updates. I’ve also tested it doing upgrades from previous releases and also uploaded new ready-to-go images.

For the 1.1.1 release:

  • Added and improved the indexes which jump straight to the timeranges and dashboards for every flight or error
  • Added ‘annotations’ - selectable groups of vertical lines on the graph that tell you about each event when you hover over the little bottom triangle
  • Added EKF3 dashboard for single or multiple IMUs, and an EKF2 vs EKF3 dashboard to compare EKF2 running on one IMU and EKF3 running on another IMU
  • Added a quick port of MAVExplorer ‘mavgraphs.xml’, available through the flight indexes ‘Mavgraphs’ links. Ports of other MAVExplorer graph sets can be done, but if there’s interest in this graphing system it’s probably better to spend efforts at overhauling the graph sets generally.
  • Lots of small bugfixes and improvements

The demo has been updated to 1.1.1.


(Fabrício Silva) #10

Great! I’ve been testing Maverick since then. It’s a very interesting idea. It has great potential.

In RPi 1 (model B), it is quite slow. “maverick configure” takes 11 minutes to run. I even had to replace mavproxy with cmavnode, due to high cpu usage, The switch was flawless: +1 for Maverick.

I experienced some limitations, but it might be because I’m used to configuring things by myself. I’ll let you know when I have a well formed opinion about your project.

I’ll give version 1.1.1 a try.


(William Tolbirt) #12

Why does this not run on Windows?


(Sebastian Schürmann) #13

Hi,

I am trying to run Maverick on a Raspberry Pi 2. The image boots fine at first, maverick self-update works, but maverick configure gives different errors. I copied the output into a textfile:

MaverickConfigureOutput.txt (22.8 KB)


(Fnoop) #14

Hi, that libjpeg package has caused some problems, it’s named different things on different platforms…
Could you please grab the log out of ~/var/log/maverick, and raise an issue on github?
thanks :slight_smile:


(Sebastian Schürmann) #15

Hm, there is no folder maverick/ in /var/log/, or a log of that name.


(Fnoop) #16

In ~/var/log (/srv/maverick/var/log)


(Michael D) #17

Hi @fnoop. This tool looks amazing! Is this still live? both the GitHub page and the www.maverick.one page seem to be offline?


(Fnoop) #18

Hi @Michael_D I can’t edit this post currently for the new links, so here are updated links until I can edit:

New project location:
https://github.com/goodrobots/maverick
Documentation
https://goodrobots.github.io/maverick

GoodRobots is a new community repo to bring together a few interested developers and projects, more to come soon!