Cumulus Support

The BOM in Australia (love it or hate it) report the rainfall from 9am-9am but the temperature from midnight to midnight. Can there be a way to reflect this in Cumulus please for the daily reports. Cheers

The main issue is displaying today minimum temp on my website - looks good until 9am and then the minimum temp is then only whatever the minimum temp from 9am is - which to someone reading the data it looks a bit odd.

I was quite surprised reading these different observation times and can't reconsile it with what the BOM says about temp measurement.
Rain observation is similar.

I conclude both observations are done at 9 am.

rick044 wrote: ↑Wed 06 Apr 2022 11:37 pm The main issue is displaying today minimum temp on my website - looks good until 9am and then the minimum temp is then only whatever the minimum temp from 9am is - which to someone reading the data it looks a bit odd.

Your request re ressigning date for lowest temperature and rainfall, has been made several times before, in respect of Cumulus 1. Steve Loft planned back then (if you look up the references - EDIT 1 - adding cross-references that I meant - viewtopic.php?t=14597 , viewtopic.php?p=30156#p30156 and viewtopic.php?p=44626#p44626 ) to make Cumulus MX right for UK (and Australia), but it never happened!

Several Cumulus users have found a way to manipulate Cumulus output to resolve this, including me. Put simply, to match BOM, you just need to define a new web page EDIT 2: on that (in Cumulus web tag terms) report max temp using <#tempTH> <#tempunit>, minimum temperature using <#tempYL> <#tempunit>, and rainfall using <#rfallY> <#rainunit>. That will show you exactly what you want, after 9 am. If you are able to write scripts, as I can, then you can make adjustments to what is shown, depending on whether time is before or after 9 am and make the new web page even more sensible.

Here in Britain, I work on 9am to 9am UTC meteorological days, reporting the maximum temperature within that day, but the minimum temperature from what Cumulus considers as metdateyesterday. That pair of numbers gets stored in my database table against the date for 9 am. (This does mean I have coded my own web pages, I don't use the MX default ones; I have coded my own scripts for MX to process, I don't use the provided .json; I have designed my own daily summary database table, I don't use the MX standard SQL to update it).

Thus for a typical calendar day, the minimum is reported in the early hours of that day, the maximum is reported in the afternoon of that day and my reported temperature range, during any day, is calculated from subtracting that early morning minimum from the latest maximum (Cumulus temperature range is ignored by me EDIT 3: I forgot to end bracket ")" here. Equally, the WMO standard, but less accurate temperature mean, calculated from adding maximum and minimum and then dividing by two, uses that pair of measurements; [obviously I also record the more accurate Cumulus mean now calculated in MX (if I understand the development of MX correctly for my weather station type) from every 6th instance of measurements that MX reads at 10 second intervals].

My manipulation of Cumulus outputs puts me in line with not only how your BOM defines temperature measurement, but also the UK Met Office, and many other national services.

rick044 wrote: ↑Tue 05 Apr 2022 12:17 pm the temperature from midnight to midnight.

On some days, my lowest temperature is just before midnight, I still assign it to the date for the next 9am (in my case UTC or GMT), and still use it for WMO type mean and temperature range calculations. I think (like HansR) BOM would, so you are wrong to suggest changing temperature day at midnight. You can find other posts in this forum about people who do use midnight to midnight reporting, who swap minimum temperatures temperatures just before midnight to next day e.g. here and manually revise minimum temperatures to be those just after midnight e.g. here. There are several other posts.

You also mention rainfall, again it is simple for a new web page to report yesterday's rain. I have a novel way of accounting for precipitation, that I won't describe, but again it ignores some Cumulus outputs and adds some others I find useful.

You could easily use custom SQL in MX to write yesterday's rain and lowest temperature against today's date in a database table that has today's maximum temperature in same row. Database recording was discussed in viewtopic.php?p=118676&hilit=minimum+te ... re#p118676.


I know Freddie who hosts this forum, breaks a 24 hour day into two 12 hour periods for some of his reporting in a spreadsheet using UK Met Office format, see link to his web site from any of his posts for full details of what reported for when.

HansR wrote: ↑Thu 07 Apr 2022 2:53 am I was quite surprised reading these different observation times and can't reconsile it with what the BOM says about temp measurement.
Rain observation is similar.

I conclude both observations are done at 9 am.

Yes I see it’s confusing. This is the statement I am trying to replicate with the cumulus data - most notably the minimum temperature:
“The highest temperature for the 24 hours leading up to the observation is recorded as the maximum temperature for the previous day, while the lowest temperature over the previous 24 hours is recorded as the minimum temperature for the day on which the observation was made”

So I want the minimum temperature of the day (almost always recorded before 9am) to be the minimum for the current day - not “yesterday” as it’s reset at 9am.

I hope that makes sense … trying to explain the best I can. Cheers.

I see. Well, I leave that to @mcrossley to comment then.

I'll come back to this once the next release is out. Remind me if I forget!

Wahoo thanks so much. This is the last piece of the puzzle for me for this software being perfect.

Cheers

mcrossley wrote: ↑Mon 11 Apr 2022 7:07 pm I'll come back to this once the next release is out. Remind me if I forget!

Since it is possible both @rick044 and @mcrossley have forgotten...


Consider yourself reminded.

tbh I had a dose of "this may be harder than it looks".

All that has been mentioned so far is shifting the minimum temperature and rainfall total to the following day.

But if the minimum temperature for the previous 24 hrs belongs to the current day, what about the values derived from it like minimum dewpoint, feels like, apparent, humidex etc? But I guess they also use humidity and wind which are not shifted! So does the shift only affect those two values?

The dayfile contains the days minimum temperature and a time, that time would now apply to the previous met day, not the dayfile records met day. This would just "have to be understood" unless the dayfile structure is altered.

Which raises another question, the web tags and records would show the minimum temp and rain values shifted by one day, is that always wanted, would you not want to know the "real" date/time of minimum temperatures and total rainfalls as well?

Does anyone else want to comment on whether MX should handle temperature extremes and rainfall differently now?
This timing correction request has been made several times in the past, I rather suspect those people have given up hoping for a change, and are coping with how Cumulus works now, but if any of you still look at forum posts, here is your chance to contribute again.

rick044 wrote: ↑Mon 18 Apr 2022 11:46 am Wahoo thanks so much. This is the last piece of the puzzle for me for this software being perfect.

Cheers

@rick044 - you need to respond now, if you don't MX won't change!

mcrossley wrote: ↑Sat 21 May 2022 12:58 pm But if the minimum temperature for the previous 24 hrs belongs to the current day, what about the values derived from it like minimum dewpoint, feels like, apparent, humidex etc? But I guess they also use humidity and wind which are not shifted! So does the shift only affect those two values?

You are confused. Dewpoint, wind chill, feels like, apparent, humidex are all calculated from spot values at instants of time of the various measurements, feeding into their calculations.
So if there is a requirement for lowest feels like to be reported as experienced the previous 24 hrs (and whether there is that requirement is for others to say), it is never combining values from different days.

The timing of extremes is not interlinked in same way, so just because the lowest temperature typically occurs in early morning before the 9am rollover, and highest temperature in mid afternoon, does not actually mean the minimum/maximum dewpoint (or any other derived value) has to occur at same times. Which should relate to day ending and which to day starting at rollover is a separate question.

@freddie - you hinted at these issues in this post, any response to Mark's comments?

mcrossley wrote: ↑Sat 21 May 2022 12:58 pm The dayfile contains the days minimum temperature and a time, that time would now apply to the previous met day, not the dayfile records met day. This would just "have to be understood" unless the dayfile structure is altered.

I don't think anyone is seeking a change to that file structure.

As posts in this support forum (and the wxforum) reveal, there are already a lot of just "have to be understood" aspects to MX.

You operate on a midnight to midnight day, and perhaps forget that for those operating a 9am-9am day already the "day start/dayend" is already different for fields like sunshine hours to other fields in that file.

mcrossley wrote: ↑Sat 21 May 2022 12:58 pm Which raises another question, the web tags and records would show the minimum temp and rain values shifted by one day, is that always wanted, would you not want to know the "real" date/time of minimum temperatures and total rainfalls as well?

Once again, as you use midnight rollover, date/time details already have a different interpretation for those of us using 9am rollover. I leave it to others to respond any further on this, as in my implementation I have scripts that can manipulate date/time when needed.

Half a decade ago there was a proposal for a Cumulus 4 that by use of modularisation would separate the extraction of weather station data into storage, from whatever processing is done taking it out of storage for the various standard outputs. I suspect if that multiple package approach is ever implemented it will not only make the software easier to maintain, but it would also make tailoring processing/outputs for the needs of different users easier. I use MX purely as a way to get data from my weather station into some database tables that don't follow your column structures, so in effect I have implemented that Cumulus 4 proposal.

My requirements for MX are very simple, and I only use a tiny fraction of the functionality currently present, ignoring almost all the provided output.
I don't upload to any external sites, MX does process two templates for me at real time interval, and another two templates at rollover, but each has been reduced to contain a bare minimum of web tags. MX does launch a PHP script for me at the real time interval, but it only does anything if it is raining as then it updates a database table that lets me track rainfall tips as my weather station records them so I record rain uninterrupted by rollover.
Otherwise, at the end of each day MX runs several queries via the Custom SQL, and then launches a PHP script that validates what MX did in its custom SQL and extracts some extra information from Cumulus files that web tags don't cover, this means once a day there is an update to two database tables (the daily table covers various daily derivatives, not just those in dayfile.txt; the monthly table covers single months, and monthly-all-time), and my web pages can select what they display from what is in these database tables.

All I know is I'd like a web tag to show the minimum and maximum that resets at midnight, while still keeping the rain showing from 9am which it already does.

Currently as the temperatures reset at 9am it shows minimum only from 9am, which if you look at the website anytime after 9am in the morning it isn't what you would expect "today's" minimum to be.

Same with Maximum too really, because looking at the website before 9am in the morning shows the maximum temperature from yesterday.

Im happy with how all the current reporting is done.

Thanks

I hope I have explained what functionality would greatly benefit me - any chance of this being implemented? Keep 9am rain reset, but reset min/max at midnight.

Cheers.

Or even just a web tag that has the min/max since midnight - that way there is no impact to the program reporting etc? @mcrossley