Cumulus Support

• Excess CPU usage by Cumulus

• Daily rollover not happening every day

• Issues with Month.ini and today.ini randomly being set to zero bytes

• The times when the month.ini drops to zero bytes correspond to nothing obvious in the log. It just appears to be a normal write. However once it is zero bytes it still gets "updated". ie the date of the file changes but nothing is written, which makes me think that the values CMX is storing internally are the problem rather than the actual writing to the file.

• The time the ini drops to zero happens at different times. ie I thought it was something to do with the rollover given the backups had the zero-byte files but it happens at other times during the day and the backup just reflects the file it had at the time.

• The month.ini is written to every minute even though nothing is changed. But sometimes there can be gaps of several minutes where no writes occur. For example, on the 8th December between 22:50 and 23:21

• Stopping CMX using Ctrl-C and restarting it sometimes triggers a proper write of the month.ini. For example, on the 9th December at 9:08

• Even though I am running b3215 and now have my station logging interval set to the same 10 mins as CMX, it still reads thousands of archive records from the station at startup. Unless I am misunderstanding the release notes for b3215.

• Some high CPU threads come and go within a few minutes. Others seem to stick around. It's the ones that stick around that appear to be the problem. May I suggest that when a new background worker is started that it writes a debug log entry so we can see when the threads are started?

• The style of logging varies considerably in the code. eg Some log statements include the calling method, while others don't, which makes it hard to track what is going on. Other times what gets written to the debug log is a different message than what gets written to the console even though they follow each other in the code. I'm sure this is on your TODO list to fix.

• A lot of things trigger a write to the month.ini. To narrow down where the issue is, it may be worthwhile adding a log entry in the code just before calling WriteMonthIniFile to show why it was called. See example below

if (OutdoorHumidity > ThisMonth.HighHumidity.Val)
	{
		ThisMonth.HighHumidity.Val = OutdoorHumidity;
		ThisMonth.HighHumidity.Ts = timestamp;
		LogDebugMessage("New High Monthly Humidity value" + OutdoorHumidity)
		WriteMonthIniFile();
	}

A few things I've noticed.
The times when the month.ini drops to zero bytes correspond to nothing obvious in the log. It just appears to be a normal write. However once it is zero bytes it still gets "updated". ie the date of the file changes but nothing is written, which makes me think that the values CMX is storing internally are the problem rather than the actual writing to the file.

The time the ini drops to zero happens at different times. ie I thought it was something to do with the rollover given the backups had the zero-byte files but it happens at other times during the day and the backup just reflects the file it had at the time.

Even though I am running b3215 and now have my station logging interval set to the same 10 mins as CMX, it still reads thousands of archive records from the station at startup. Unless I am misunderstanding the release notes for b3215.

Some high CPU threads come and go within a few minutes. Others seem to stick around. It's the ones that stick around that appear to be the problem. May I suggest that when a new background worker is started that it writes a debug log entry so we can see when the threads are started?

The style of logging varies considerably in the code. eg Some log statements include the calling method, while others don't, which makes it hard to track what is going on. Other times what gets written to the debug log is a different message than what gets written to the console even though they follow each other in the code. I'm sure this is on your TODO list to fix.

A lot of things trigger a write to the month.ini. To narrow down where the issue is, it may be worthwhile adding a log entry in the code just before calling WriteMonthIniFile to show why it was called. See example below

I need to see why you have batches of writes every so often. The code needs optimising and some of the writes consolidating at a higher level - it will help SD disk wear for rPi users too!

	WriteMonthIni = False;
	if (OutdoorHumidity > ThisMonth.HighHumidity.Val)
	{
		ThisMonth.HighHumidity.Val = OutdoorHumidity;
		ThisMonth.HighHumidity.Ts = timestamp;
		LogDebugMessage("New High Monthly Humidity value" + OutdoorHumidity)
		WriteMonthIni = True;
	}
	... all the other checks along the same lines ...
	if WriteMonthIni 
	{
		WriteMonthIniFile();
	 	WriteMonthIni = False
	} 

	WriteMonthIni = False;
	DoApparentTemp(timestamp);
	DoFeelsLike(timestamp);
	DoHumidex(timestamp);
	DoCloudBaseHeatIndex(timestamp);
	if WriteMonthIni 
	   {
	   	WriteMonthIniFile();
	   	WriteMonthIni = False
	   } 
	

One thing I would be interested in trying. If you get a zero-length file, try deleting it and not restarting CMX. It may recreate the full file.