Welcome to the Cumulus Support forum.
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4019) - 03 April 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4019) - 03 April 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
9am rain reset with midnight temp reset
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
9am rain reset with midnight temp reset
The BOM in Australia (love it or hate it) report the rainfall from 9am-9am but the temperature from midnight to midnight. Can there be a way to reflect this in Cumulus please for the daily reports. Cheers
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
Re: 9am rain reset with midnight temp reset
The main issue is displaying today minimum temp on my website - looks good until 9am and then the minimum temp is then only whatever the minimum temp from 9am is - which to someone reading the data it looks a bit odd.
- HansR
- Posts: 5958
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: 9am rain reset with midnight temp reset
I was quite surprised reading these different observation times and can't reconsile it with what the BOM says about temp measurement.
Rain observation is similar.
I conclude both observations are done at 9 am.
Rain observation is similar.
I conclude both observations are done at 9 am.
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
-
- Posts: 1183
- Joined: Fri 27 Jul 2012 11:29 am
- Weather Station: Chas O, Maplin N96FY, N25FR
- Operating System: rPi 3B+ with Buster (full)
Re: 9am rain reset with midnight temp reset
Your request re ressigning date for lowest temperature and rainfall, has been made several times before, in respect of Cumulus 1. Steve Loft planned back then (if you look up the references - EDIT 1 - adding cross-references that I meant - viewtopic.php?t=14597 , viewtopic.php?p=30156#p30156 and viewtopic.php?p=44626#p44626 ) to make Cumulus MX right for UK (and Australia), but it never happened!
Several Cumulus users have found a way to manipulate Cumulus output to resolve this, including me. Put simply, to match BOM, you just need to define a new web page EDIT 2: on that (in Cumulus web tag terms) report max temp using <#tempTH> <#tempunit>, minimum temperature using <#tempYL> <#tempunit>, and rainfall using <#rfallY> <#rainunit>. That will show you exactly what you want, after 9 am. If you are able to write scripts, as I can, then you can make adjustments to what is shown, depending on whether time is before or after 9 am and make the new web page even more sensible.
Here in Britain, I work on 9am to 9am UTC meteorological days, reporting the maximum temperature within that day, but the minimum temperature from what Cumulus considers as metdateyesterday. That pair of numbers gets stored in my database table against the date for 9 am. (This does mean I have coded my own web pages, I don't use the MX default ones; I have coded my own scripts for MX to process, I don't use the provided .json; I have designed my own daily summary database table, I don't use the MX standard SQL to update it).
Thus for a typical calendar day, the minimum is reported in the early hours of that day, the maximum is reported in the afternoon of that day and my reported temperature range, during any day, is calculated from subtracting that early morning minimum from the latest maximum (Cumulus temperature range is ignored by me EDIT 3: I forgot to end bracket ")" here. Equally, the WMO standard, but less accurate temperature mean, calculated from adding maximum and minimum and then dividing by two, uses that pair of measurements; [obviously I also record the more accurate Cumulus mean now calculated in MX (if I understand the development of MX correctly for my weather station type) from every 6th instance of measurements that MX reads at 10 second intervals].
My manipulation of Cumulus outputs puts me in line with not only how your BOM defines temperature measurement, but also the UK Met Office, and many other national services.
On some days, my lowest temperature is just before midnight, I still assign it to the date for the next 9am (in my case UTC or GMT), and still use it for WMO type mean and temperature range calculations. I think (like HansR) BOM would, so you are wrong to suggest changing temperature day at midnight. You can find other posts in this forum about people who do use midnight to midnight reporting, who swap minimum temperatures temperatures just before midnight to next day e.g. here and manually revise minimum temperatures to be those just after midnight e.g. here. There are several other posts.
You also mention rainfall, again it is simple for a new web page to report yesterday's rain. I have a novel way of accounting for precipitation, that I won't describe, but again it ignores some Cumulus outputs and adds some others I find useful.
You could easily use custom SQL in MX to write yesterday's rain and lowest temperature against today's date in a database table that has today's maximum temperature in same row. Database recording was discussed in viewtopic.php?p=118676&hilit=minimum+te ... re#p118676.
I know Freddie who hosts this forum, breaks a 24 hour day into two 12 hour periods for some of his reporting in a spreadsheet using UK Met Office format, see link to his web site from any of his posts for full details of what reported for when.
Last edited by sfws on Tue 12 Apr 2022 7:05 am, edited 3 times in total.
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
Re: 9am rain reset with midnight temp reset
Yes I see it’s confusing. This is the statement I am trying to replicate with the cumulus data - most notably the minimum temperature:HansR wrote: ↑Thu 07 Apr 2022 2:53 am I was quite surprised reading these different observation times and can't reconsile it with what the BOM says about temp measurement.
Rain observation is similar.
I conclude both observations are done at 9 am.
“The highest temperature for the 24 hours leading up to the observation is recorded as the maximum temperature for the previous day, while the lowest temperature over the previous 24 hours is recorded as the minimum temperature for the day on which the observation was made”
So I want the minimum temperature of the day (almost always recorded before 9am) to be the minimum for the current day - not “yesterday” as it’s reset at 9am.
I hope that makes sense … trying to explain the best I can. Cheers.
- HansR
- Posts: 5958
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: 9am rain reset with midnight temp reset
I see. Well, I leave that to @mcrossley to comment then.
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
Re: 9am rain reset with midnight temp reset
sfws wrote: ↑Thu 07 Apr 2022 6:01 amThanks - this is the approach I have taken for now to get the website to display correctly. A couple of php if time is before 09:00am statements etc. At least it "looks" correct on the website. It would be great if this function could be added, similar to the raintags I have have a "rain since 9am" and a "rain since midnight" version of the same data.rick044 wrote: ↑Wed 06 Apr 2022 11:37 pm
Several Cumulus users have found a way to manipulate Cumulus output to resolve this, including me. Put simply, to match BOM, you just need to define a new web page EDIT: on that (in Cumulus web tag terms) report max temp using <#tempTH> <#tempunit>, minimum temperature using <#tempYL> <#tempunit>, and rainfall using <#rfallY> <#rainunit>. That will show you exactly what you want, after 9 am. If you are able to write scripts, as I can, then you can make adjustments to what is shown, depending on whether time is before or after 9 am and make the new web page even more sensible.
- mcrossley
- Posts: 12756
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: 9am rain reset with midnight temp reset
I'll come back to this once the next release is out. Remind me if I forget!
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
Re: 9am rain reset with midnight temp reset
Wahoo thanks so much. This is the last piece of the puzzle for me for this software being perfect.
Cheers
Cheers
- mcrossley
- Posts: 12756
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: 9am rain reset with midnight temp reset
tbh I had a dose of "this may be harder than it looks".
All that has been mentioned so far is shifting the minimum temperature and rainfall total to the following day.
But if the minimum temperature for the previous 24 hrs belongs to the current day, what about the values derived from it like minimum dewpoint, feels like, apparent, humidex etc? But I guess they also use humidity and wind which are not shifted! So does the shift only affect those two values?
The dayfile contains the days minimum temperature and a time, that time would now apply to the previous met day, not the dayfile records met day. This would just "have to be understood" unless the dayfile structure is altered.
Which raises another question, the web tags and records would show the minimum temp and rain values shifted by one day, is that always wanted, would you not want to know the "real" date/time of minimum temperatures and total rainfalls as well?
All that has been mentioned so far is shifting the minimum temperature and rainfall total to the following day.
But if the minimum temperature for the previous 24 hrs belongs to the current day, what about the values derived from it like minimum dewpoint, feels like, apparent, humidex etc? But I guess they also use humidity and wind which are not shifted! So does the shift only affect those two values?
The dayfile contains the days minimum temperature and a time, that time would now apply to the previous met day, not the dayfile records met day. This would just "have to be understood" unless the dayfile structure is altered.
Which raises another question, the web tags and records would show the minimum temp and rain values shifted by one day, is that always wanted, would you not want to know the "real" date/time of minimum temperatures and total rainfalls as well?
-
- Posts: 1183
- Joined: Fri 27 Jul 2012 11:29 am
- Weather Station: Chas O, Maplin N96FY, N25FR
- Operating System: rPi 3B+ with Buster (full)
Re: 9am rain reset with midnight temp reset
Does anyone else want to comment on whether MX should handle temperature extremes and rainfall differently now?
This timing correction request has been made several times in the past, I rather suspect those people have given up hoping for a change, and are coping with how Cumulus works now, but if any of you still look at forum posts, here is your chance to contribute again.
So if there is a requirement for lowest feels like to be reported as experienced the previous 24 hrs (and whether there is that requirement is for others to say), it is never combining values from different days.
The timing of extremes is not interlinked in same way, so just because the lowest temperature typically occurs in early morning before the 9am rollover, and highest temperature in mid afternoon, does not actually mean the minimum/maximum dewpoint (or any other derived value) has to occur at same times. Which should relate to day ending and which to day starting at rollover is a separate question.
@freddie - you hinted at these issues in this post, any response to Mark's comments?
As posts in this support forum (and the wxforum) reveal, there are already a lot of just "have to be understood" aspects to MX.
You operate on a midnight to midnight day, and perhaps forget that for those operating a 9am-9am day already the "day start/dayend" is already different for fields like sunshine hours to other fields in that file.
Half a decade ago there was a proposal for a Cumulus 4 that by use of modularisation would separate the extraction of weather station data into storage, from whatever processing is done taking it out of storage for the various standard outputs. I suspect if that multiple package approach is ever implemented it will not only make the software easier to maintain, but it would also make tailoring processing/outputs for the needs of different users easier. I use MX purely as a way to get data from my weather station into some database tables that don't follow your column structures, so in effect I have implemented that Cumulus 4 proposal.
My requirements for MX are very simple, and I only use a tiny fraction of the functionality currently present, ignoring almost all the provided output.
I don't upload to any external sites, MX does process two templates for me at real time interval, and another two templates at rollover, but each has been reduced to contain a bare minimum of web tags. MX does launch a PHP script for me at the real time interval, but it only does anything if it is raining as then it updates a database table that lets me track rainfall tips as my weather station records them so I record rain uninterrupted by rollover.
Otherwise, at the end of each day MX runs several queries via the Custom SQL, and then launches a PHP script that validates what MX did in its custom SQL and extracts some extra information from Cumulus files that web tags don't cover, this means once a day there is an update to two database tables (the daily table covers various daily derivatives, not just those in dayfile.txt; the monthly table covers single months, and monthly-all-time), and my web pages can select what they display from what is in these database tables.
This timing correction request has been made several times in the past, I rather suspect those people have given up hoping for a change, and are coping with how Cumulus works now, but if any of you still look at forum posts, here is your chance to contribute again.
@rick044 - you need to respond now, if you don't MX won't change!
You are confused. Dewpoint, wind chill, feels like, apparent, humidex are all calculated from spot values at instants of time of the various measurements, feeding into their calculations.mcrossley wrote: ↑Sat 21 May 2022 12:58 pm But if the minimum temperature for the previous 24 hrs belongs to the current day, what about the values derived from it like minimum dewpoint, feels like, apparent, humidex etc? But I guess they also use humidity and wind which are not shifted! So does the shift only affect those two values?
So if there is a requirement for lowest feels like to be reported as experienced the previous 24 hrs (and whether there is that requirement is for others to say), it is never combining values from different days.
The timing of extremes is not interlinked in same way, so just because the lowest temperature typically occurs in early morning before the 9am rollover, and highest temperature in mid afternoon, does not actually mean the minimum/maximum dewpoint (or any other derived value) has to occur at same times. Which should relate to day ending and which to day starting at rollover is a separate question.
@freddie - you hinted at these issues in this post, any response to Mark's comments?
I don't think anyone is seeking a change to that file structure.
As posts in this support forum (and the wxforum) reveal, there are already a lot of just "have to be understood" aspects to MX.
You operate on a midnight to midnight day, and perhaps forget that for those operating a 9am-9am day already the "day start/dayend" is already different for fields like sunshine hours to other fields in that file.
Once again, as you use midnight rollover, date/time details already have a different interpretation for those of us using 9am rollover. I leave it to others to respond any further on this, as in my implementation I have scripts that can manipulate date/time when needed.
Half a decade ago there was a proposal for a Cumulus 4 that by use of modularisation would separate the extraction of weather station data into storage, from whatever processing is done taking it out of storage for the various standard outputs. I suspect if that multiple package approach is ever implemented it will not only make the software easier to maintain, but it would also make tailoring processing/outputs for the needs of different users easier. I use MX purely as a way to get data from my weather station into some database tables that don't follow your column structures, so in effect I have implemented that Cumulus 4 proposal.
My requirements for MX are very simple, and I only use a tiny fraction of the functionality currently present, ignoring almost all the provided output.
I don't upload to any external sites, MX does process two templates for me at real time interval, and another two templates at rollover, but each has been reduced to contain a bare minimum of web tags. MX does launch a PHP script for me at the real time interval, but it only does anything if it is raining as then it updates a database table that lets me track rainfall tips as my weather station records them so I record rain uninterrupted by rollover.
Otherwise, at the end of each day MX runs several queries via the Custom SQL, and then launches a PHP script that validates what MX did in its custom SQL and extracts some extra information from Cumulus files that web tags don't cover, this means once a day there is an update to two database tables (the daily table covers various daily derivatives, not just those in dayfile.txt; the monthly table covers single months, and monthly-all-time), and my web pages can select what they display from what is in these database tables.
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
Re: 9am rain reset with midnight temp reset
All I know is I'd like a web tag to show the minimum and maximum that resets at midnight, while still keeping the rain showing from 9am which it already does.
Currently as the temperatures reset at 9am it shows minimum only from 9am, which if you look at the website anytime after 9am in the morning it isn't what you would expect "today's" minimum to be.
Same with Maximum too really, because looking at the website before 9am in the morning shows the maximum temperature from yesterday.
Im happy with how all the current reporting is done.
Thanks
Currently as the temperatures reset at 9am it shows minimum only from 9am, which if you look at the website anytime after 9am in the morning it isn't what you would expect "today's" minimum to be.
Same with Maximum too really, because looking at the website before 9am in the morning shows the maximum temperature from yesterday.
Im happy with how all the current reporting is done.
Thanks
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
Re: 9am rain reset with midnight temp reset
I hope I have explained what functionality would greatly benefit me - any chance of this being implemented? Keep 9am rain reset, but reset min/max at midnight.
Cheers.
Cheers.
-
- Posts: 13
- Joined: Sat 15 Dec 2012 11:58 pm
- Weather Station: vantage pro 2
- Operating System: mac osx
- Location: brisbane
Re: 9am rain reset with midnight temp reset
Or even just a web tag that has the min/max since midnight - that way there is no impact to the program reporting etc? @mcrossley