Autotest is losing old tlog files

General Website and Documentation errors
1

Old tlog files are getting lost and replaced by PNG files.

If someone wants to check, just try to download any tlog that is older than November.
I have also created a script to check each tlog file available.

#!/usr/bin/env python3
import magic
import posixpath
import requests

from bs4 import BeautifulSoup

ARDUPILOT_AUTOTEST_HISTORY = "https://autotest.ardupilot.org/history/"

def get_all_folders_urls():
    page = requests.get(ARDUPILOT_AUTOTEST_HISTORY)
    soup = BeautifulSoup(page.content, 'html.parser')
    dates_href = [date['href'] for date in soup.find_all('a', href=True) if '2021' in date['href']]
    return [posixpath.join(ARDUPILOT_AUTOTEST_HISTORY, date_href) for date_href in dates_href]

def get_all_tlogs_urls(folder_url):
    page = requests.get(folder_url)
    soup = BeautifulSoup(page.content, 'html.parser')
    artifacts_href = [artifact['href'] for artifact in soup.find_all('a', href=True) if artifact['href'].endswith('.tlog')]
    return [posixpath.join(folder_url, artifact_href) for artifact_href in artifacts_href]

for folder in get_all_folders_urls():
    for tlog_url in get_all_tlogs_urls(folder):
        data = requests.get(tlog_url)
        print(f"{tlog_url} is a {magic.from_buffer(data.content)}")

https://autotest.ardupilot.org/history/./2021-05-01-07:05/HeliCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                       
https://autotest.ardupilot.org/history/./2021-05-01-07:05/Rover-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                            
https://autotest.ardupilot.org/history/./2021-05-01-07:05/ArduPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                        
https://autotest.ardupilot.org/history/./2021-05-01-07:05/APMrover2-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                        
https://autotest.ardupilot.org/history/./2021-05-01-07:05/Soaring-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                          
https://autotest.ardupilot.org/history/./2021-05-01-07:05/ArduSub-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                          
https://autotest.ardupilot.org/history/./2021-05-01-07:05/QuadPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                        
https://autotest.ardupilot.org/history/./2021-05-01-07:05/BalanceBot-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                       
https://autotest.ardupilot.org/history/./2021-05-01-07:05/AntennaTracker-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                   
https://autotest.ardupilot.org/history/./2021-05-01-07:05/ArduCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                       
https://autotest.ardupilot.org/history/./2021-05-01-16:05/HeliCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                       
https://autotest.ardupilot.org/history/./2021-05-01-16:05/Rover-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced                                            
https://autotest.ardupilot.org/history/./2021-05-01-16:05/ArduPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-01-16:05/APMrover2-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-01-16:05/Soaring-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-01-16:05/ArduSub-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-01-16:05/QuadPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-01-16:05/BalanceBot-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-01-16:05/AntennaTracker-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-01-16:05/ArduCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/HeliCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/Rover-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/ArduPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/APMrover2-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/Soaring-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/ArduSub-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/QuadPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/BalanceBot-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/AntennaTracker-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-00:05/ArduCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/HeliCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/Rover-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/ArduPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/APMrover2-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/Soaring-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/ArduSub-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/QuadPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/BalanceBot-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/AntennaTracker-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-02-08:05/ArduCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/HeliCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/Rover-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/ArduPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/APMrover2-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/Soaring-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/ArduSub-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/QuadPlane-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/BalanceBot-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/AntennaTracker-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-03-00:05/ArduCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-05-01:05/HeliCopter-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
https://autotest.ardupilot.org/history/./2021-05-05-01:05/Rover-test.tlog is a PNG image data, 600 x 600, 8-bit/color RGB, non-interlaced
...
2

Those tlog files are actually simply not present on the autotest server - removed by a cleanup cronjob, I believe.

The Apache “MultiViews” module is assuming that because (e.g.) HeliCopter-test.tlog doesn’t exist you would really like HeliCopter-test.tlog.png instead!

We’ll discuss cleaning the .png files up at the same time we do the tlog

3

Hey @peterbarker, any update ?