Dayfile Mean Temp Wrong if MX not running 24H

[ARootBear]

While setting up my new website with Cumulus MX, I noticed that my NOAA heating degree days data didn't look right. For example, on February 3rd, the first full day of data I have, the recorded mean temperature for the day was 8.6, but the heating degree days recorded was only 2.3 - heating/cooling degree day base is set to 18.3, and the temperature was below this value all day (ie. no cooling degree days recorded).

I started digging into this by pulling up the Feb21log file in excel and calculating the average temperature for the day to see if something strange was going on, and to my surprise, taking the average of the recorded 5-minute data points for February 3rd gave me 3.9°C. If the number was within a few decimal points of 8.6°C I would not be concerned, but a difference of 4.7°C is too much to chalk up to variability in the 1-minute live data vs 5-minute logged data.

After taking a look at other days this month, my conclusion is that the problem appears when Cumulus is not running live for the entire 24-hour period. The biggest error I found was on the 3rd - I was still trying to get my Weatherlink live to communicate properly with cumulus at this point, and most data for that day was downloaded from Weatherlink rather than recorded live. The temperature was close to freezing for much of the day, but warmed up to about 10°C at night, which is when I got Cumulus running, and lines up with the recorded mean temp of 8.6°C being so much higher than the average of the 5-min data (the cold temperatures earlier in the day are either not being weighted the same as the data recorded live, or are being ignored).

I am not entirely sure what is causing this problem. Looking back through the backup/daily folders, most days which had folders created at 12:00 AM (ie. Cumulus was running overnight) have mean temperatures and NOAA degree days which seem correct, but days with folders that were created later in the day (for example, if Cumulus was not running overnight and was started at 9:00 AM) have mean temps which don't match up with the log files and Heating Degree Day totals that are significantly lower than other days.

I have attached my Feb21log file, dayfile, and a screenshot of backup file creation times. Please note that a few of the folders with 12:00 folder creation times did not end up having Cumulus running for 24 hours due to overnight PC updates/restart. February 7th was a day which ran without interruption which could be used for comparison. Please let me know if any other info or files are required to figure out why this is happening.

[sfws]

Have you read https://cumuluswiki.org/a/Average_temperature?

[mcrossley]

I cannot see where this may be going wrong, but from your files it obviously is.

On catch-up from the logger it adds 'n' number of samples to the averaging calc per archive record, where n is the logger archive interval in minutes. Then when live running adds one sample per minute. So all values should have equal weighting.

I think I need to see the MXdiags log file that cover the 3rd/4th of Feb please.

[ARootBear]

Thanks Mark. I was thinking something like that had to be going on in the background to make the averages work out. Unfortunately, I have restarted Cumulus so many times in the last few days while customizing my website that the MXdiags file for the 3rd/4th no longer exists (the oldest one in the folder is from yesterday )

I will shut Cumulus down tonight, start it up again tomorrow morning and upload the diags files that result.

[mcrossley]

Ta. Could you start Cumulus with the -debug option tomorrow please? You can switch it off again via the dashboard once it has run for a while after the catch-up.

[ARootBear]

Just did a quick experiment and shut Cumulus down for a while then started up again (with debug enabled). I copied the temperature section of today.ini immediately after Cumulus had shut down, and again after it was restarted and displayed "normal running" in the console. Neither the temperature total or sample count was updated. The MXdiags file for this instance is attached, along with copies of the temperature section of the today.ini file before and after the shutdown.

Just as confirmation that I didn't simply copy the info before Cumulus had a chance to finish loading up, I also shut Cumulus off for about 30 minutes earlier today and copied the today.ini data:

1:26

[Temp]
Low=-5.2222222222222214
LTime=01:05
High=0.27777777777777779
HTime=11:24
Total=-2578.0000000000041
Samples=804
ChillHours=222.48333333340253
HeatingDegreeDays=11.991481481481458
CoolingDegreeDays=0

2:00

[Temp]
Low=-5.2222222222222214
LTime=01:05
High=0.27777777777777779
HTime=11:24
Total=-2578.5000000000041
Samples=806
ChillHours=222.51666666673589
HeatingDegreeDays=12.017245370370345
CoolingDegreeDays=0

I restarted Cumulus around 1:58, so by the time I copied the today.ini data, two more 1-minute live data points had been recorded - but there was nothing added for the 30+ minutes cumulus was shut down. I can do the same thing overnight if that would help, although I think it would produce exactly the same result, just with more historic data points.

[mcrossley]

Ah! You are using a WLL, not a VP 2 logger!

Yes, there is a bug in the WLL historic data catch-up for the average temp calculation, it is fixed in the next release.

[ARootBear]

That's great, looking forward to the release. Yes, it's been a few years since I've updated my profile info on here, if that's what you were going by. Thanks for the help.

[mcrossley]

Well, it was this thread that prompted me to look at the WLL code and spot the issue there after studying the VP2 code and not seeing a problem.

[ARootBear]

Did that fix make it into build 3117? I still seem to be having the same issue unfortunately.

Edit: Just did a 20 minute shutdown and the average temp is updating correctly. It is heating degree days which is not catching up for me (and presumably cooling degree days, but we don't get those here in March).

[mcrossley]

Doh! Yes, I missed the chill hours and degree days. Added for the next release.