Mini bugette

[Adrian Hudson]

I have my station set to roll over at 09:00Z.

I have noticed a minor problem with the daily backup the program makes at the end of the month.

For example, this June just gone:

. On the 30th at 10:00 BST (09:00Z) the program made its daily backup and included the June log file up to that time.
. The next day, on the 1st July at 10:00 BST the program made its backup of the new July log file (saving a single entry for 10:00 BST).

Therefore the period between 10:10 on the 30th Jun to 09:50 on the 1st Jul was not backed up.

I guess, on the 1st of the month, the program ideally should backup the current month AND the previous month log files.

[sfws]

This is way Cumulus 1 worked (except for some early versions), so MX works same way.
What we call "End of Day" is happening after new day has started, so it has already switched dates, and switched months when appropriate. When I and others raised this issue back in those Steve Loft days, he always replied "The purpose of the Cumulus back up is not to back up the data you have collected, that is your responsibility, but to back up files that will allow you to rewind Cumulus to a sensible state for a restart after you have had a problem". In other words, the backup daily folder contains "start of day" states so you can restart a day (or month).
Edit - removed part of answer that depends on which MX version you are using, to avoid future confusion

[Adrian Hudson]

Thanks for that swfs. I have implemented the <currentlogfile> thing.

I realise this isn't the most earth shattering issue but someone might be caught out by it one day. It seems to me that someone thinking logically may decide that copying the "cumulusmx\backup" folder offline would actually be the safest way of copying the data. They may decide that saving the "data" folder might result in data inconsistencies if cumulus were accessing the data as it was being saved. If they did save only the backup folder and later relied on their backup then they might lose up to a day's data.

It would be *really* nice if there were a <currentdatetime> so that the file could be put in the correct folder to,o but hey

Anyway, it seems it has been done to death in the past. Steve wasn't one to be swayed by argument!

[sfws]

someone might be caught out by it one day.

I am sure, many other people, like me, have been caught out by it several times.

If I was skilled enough, I would rewrite several parts of MX to make a product that worked sensibly, but I failed when I tried to rewrite just 3 pages of the admin interface where Mark actually said when they were first released that someone needed to improve them. Mind you, it was not just my incompetence, a lot of the reason for my struggle was that MX uses such obsolete versions of library software, a lot of functionality in current documentation was denied to me.



Another suggestion made many times in old days was that Cumulus should log the maximum and minimum since the last log line update each time it writes another line to the standard log file. By almost doubling the items written (quite a few don't get lowest derived, e.g. rain, wind, heat index) you can easily recalculate daily, monthly, annual or all-time figures without missing the actual extreme. Currently, you only log a small sample of all the measurements that Cumulus processes and the chance of extremes being logged is tiny. I don't have enough fingers and toes to count how often that was raised, it was considered for Cumulus 2 (although that never had all intended functionality), and so I expected it to arrive with MX, but no for compatibility with Cumulus 1, the chance of improving design when MX came out was missed.

It would be *really* nice if there were a <currentdatetime> so that the file could be put in the correct folder to,o but hey

I suspect the problem is that end of day uses several threads and the date-time from that dealing with the backup, could not be easily made available to that dealing with the extra web files processing, but Mark knows his code better than I do to answer your question.

But, Adrian, my end of day PHP script (initiated by Cumulus in that EOD program option) does work out the folder name too. But it can't amend the extra web files action and it needs to wait for Cumulus to finish doing its writing of the files to the back-up folder, so in my script I include a delay of 35 seconds so this access to the folder happens when it definitely will exist and will not be locked.
$cumulus is the folder above the backup. This script worked with Cumulus 1 for almost a decade, and carried over to MX when I swapped last April.

Code: Select all

$directoriesFound = 0;
$pathName			=	$cumulus . "/backup/daily/";
$directoryName		= $today_ini_date . $rollOver . "*/";
foreach(glob($pathName . $directoryName) as $testName)
{
	$newPathName[$directoriesFound]	=	$testName;
	$directoriesFound++;
}

That can sometimes find 2 directories (hence putting folder into array), but generally finds one which is the backup folder, which script can then use! My script puts out a message that manual action is needed if 2 found!

It is perhaps worth adding that when I learnt PHP in my retirement I wanted to test every feature, so I was so glad when I found "glob" was very useful!

[mcrossley]

I'll add the previous months log files to the daily backup on the first of the month.