Welcome to the Cumulus Support forum.

Latest Cumulus MX release 3.9.0 (build 3095) - 09 October 2020 (please see announcement regarding releases since 3.5.0)
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of the NOAA report and Snow Index drop-down menus to 2030)

Use this Wiki link to Download the Software (Cumulus MX / Cumulus 1 and other related items).

Weather Diary (C1 and MX Differences) - Resolved already in MX

A Forum to archive Cumulus MX development suggestions that have been rejected or solved by other means.
Post Reply
sfws
Posts: 930
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: --
Operating System: --

Weather Diary (C1 and MX Differences) - Resolved already in MX

Post by sfws »

I moved from Cumulus 1 to MX, and this post has my observations re Weather Diary.

While MX can read most Cumulus 1 log files, it cannot read the Cumulus 1 log.xml file and transfer its contents to diary.db. Luckily, there has not been much snow here since I moved and restarted my Cumulus 1 last January, so in my case I manually copied my 3 entries across. I don't know whether an automated transfer would be useful for anyone else, but for me it might be easy to write a PHP script to do that.

To me the key Weather Diary functionality difference is that in Cumulus 1 you can select the time at which the snow settings rollover (Cumulus.ini, Station section, SnowDepthHour=9), and I am guessing MX does not support that (as there is no mention in build 3046/3047 announcements) although I have not experimented yet to test accurately when the web tag <#snowdepth> is updated in MX.

Another difference is that Steve's design permitted multiple entries on a single day (all edits) to be retained, but Mark's editor design only has a date picker, (all entries are recorded at time of 00:00:00) so any edit overwrites previous entry.

Anyway, my schema for my daily summary table in my MySQL database (written a decade ago for use with Cumulus 1) includes columns for all that is now in the sqlite database as well as reading dayfile.txt and some other logs. For MX, my minor change to my database table update script (that normally runs when Cumulus is processing the rollover) is so it reads the sqlite db when forming the INSERT or UPDATE queries for my daily summary table. I use the"PHP Data Objects (PDO)" commands to access the sqlite database and then "foreach" constructs to read the one row for each date and assign the values to the relevant MySQL columns. I'm guessing the MX SQL facilities can not do this?
Last edited by sfws on Wed 15 Apr 2020 8:56 pm, edited 1 time in total.

User avatar
mcrossley
Posts: 6998
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Weather Diary

Post by mcrossley »

Thanks for this. I really don't have much of an idea about the how the diary was used in Cumulus 1. The code is MX was largely put in place by Steve, but commented out. I just implemented a user interface and enabled the code.

So the gist of it is, in Cumulus 1...
  • You can add multiple diary entries per day
  • The values for the web tags <#snowdepth>, <#snowlying>, <#snowfalling> are taken from the last diary entry for today? Today taking the SnowDepthHour into account.
  • The snow "day" can have an independent rollover time from the station met day
OK, the good news, CumulusMX still reads the value "SnowDepthHour" from the Cumulus.ini file, and uses it with the above web tags

Multiple entries per day should be doable, I'll take a look. Adding multiple entries is easy, but I'll have to think about how to handle editing them.
They would have to logged against actual date, but Cumulus internally converts the day using the SnowDepthHour just like it does for met day hour.
My question is, how critical is it to have multiple entries vs. a single editable entry?

Adding snow data via the builtin SQL would be possible, but you would have to use a custom SQL statement rather than the default. I'd have to check when the custom SQL is run in relation to the rollover of the snow day.
Of course there would be a mismatch between the met day and snow day if they were different. For example if you had a midnight met day, and 9am snow day, the SQL entry would be written at midnight, part way through the snow day, to resolve that I think you are back to customised routines.

sfws
Posts: 930
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: --
Operating System: --

Re: Weather Diary

Post by sfws »

Mark, thank you for the reply.
I did not realise that Steve had made provision for this feature by adding code.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
OK, the good news, CumulusMX still reads the value "SnowDepthHour" from the Cumulus.ini file, and uses it with the above web tag
That is a relief.


I will make some suggestions based on your other points, you decide subsequent action. I hope this covers everything you need from me, and my answers are not shot down by others.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
The snow "day" can have an independent rollover time from the station met day
The reason for this, I believe Steve said sometime, was so those with midnight rollover, did not have to go out at midnight to manually record details for the diary.
Those who follow professional meteorologists by using 9am/10am for normal weather recording would follow them by using 9am for snow (they would apply that year round hoping no snow in summer time so does not matter it does not swap to 10am)
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
I'd have to check when the custom SQL is run in relation to the rollover of the snow day.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
the SQL entry would be written at midnight, part way through the snow day, to resolve that I think you are back to customised routines.
These relate back to previous answer, I think it is reasonable to expect standard routines to work for matching times, and customised routines to be needed if times do not match.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
Of course there would be a mismatch between the met day and snow day if they were different.
In my opinion, the default should be that the snow day matches the met day, including the 9am to 10am switch if that is selected, although we hope that the chance of snow is low in period when summer time operates. I believe we should try to treat everything we report on same time basis. This would apply to any extra parameters like lightning, soil temperatures ... as well.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
The values for the web tags <#snowdepth>, <#snowlying>, <#snowfalling> are taken from the last diary entry for today? Today taking the SnowDepthHour into account.
Only the first is a web tag and it does reset at the SnowDepthHour, the second can be deduced by checking if first is zero or not. Cumulus 1 also outputs the Snow Index this is basically the sum of all the snow depths in the selected period (month, year), Steve Loft said Philip Eden defined it as the sum of all significant snow depths at 9am, so I was a bit surprised his weather diary did not implement time-stamping the multiple entries during the day, and I never managed to prove whether he ignored all but last entry for day.

There is nothing provided in Cumulus 1 to let a web page know if snow is falling. You have to do what I and others have done (the routine was actually developed by Miloš Jirík, it was written in Czech and posted in the support forum) and use a "$xml = simplexml_load_string($converted);" and "foreach ($xml->xpath('//ROW') as $item)" approach in PHP. Then you have to check the date in the row and see if it matches the day you are displaying on the web page before updating output.

I'm unsure what the other people who process snow records do, I think Miloš did take his output from last diary entry for day, but I take the worst case from all the rows for that day, i.e. I report the highest depth, snow falling if any record had snow falling ticked etc.

My suggestion for MX is that it should provide the full range of web tags, not just <#snowdepth>, and treat like any other 'Today' tags, report what the database has at that time,
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
You can add multiple diary entries per day
True for Cumulus 1, my personal opinion is that one entry per day is better (to remove the worst case question), but I would store the time as the time given in the value "SnowDepthHour" from the Cumulus.ini file.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
Multiple entries per day should be doable, I'll take a look. Adding multiple entries is easy, but I'll have to think about how to handle editing them.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
My question is, how critical is it to have multiple entries vs. a single editable entry?
My personal opinion is don't bother, but if you do it would be way down in anybody's priorities.
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
Adding snow data via the builtin SQL would be possible, but you would have to use a custom SQL statement rather than the default.
There was a recent question about lightning SQL, we really desperately need some documentation on built-in SQL and custom SQL in MX.

freddie
Posts: 896
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux Ubuntu 20.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Weather Diary

Post by freddie »

sfws wrote:
Sat 28 Mar 2020 5:31 pm
Those who follow professional meteorologists by using 9am/10am for normal weather recording would follow them by using 9am for snow (they would apply that year round hoping no snow in summer time so does not matter it does not swap to 10am)
The definitions used by professional meteorologists (at least, in the UK) are these:
  • Snow day runs from midnight to midnight (UTC). If snow falls in this time then the date is recorded as a "day of snow".
  • A "day of snow" is futher split into two categories: (1) A day in which a mix of snow and liquid precipitation (rain and/or drizzle) occurs simultaneously (commonly called sleet); and (2) A day in which snow occurs but not simultaneously accompanied by liquid precipitation (rain and/or drizzle). If (1) and (2) both occur in the "day of snow" then it is categorised as (2) - i.e. the "worse" category.
  • Snow depth is measured at 0900 UTC. A "day with snow lying" is recorded if there is snow cover of at least 50% at 0900 UTC.
These are the offical definitions used.

Note that a "day of snow" is more likely in the non-winter month of April than it is in the winter month of November (in most locations).
Freddie
Image

sfws
Posts: 930
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: --
Operating System: --

Re: Weather Diary

Post by sfws »

freddie wrote:
Sat 28 Mar 2020 6:51 pm
Snow depth is measured at 0900 UTC. A "day with snow lying" is recorded if there is snow cover of at least 50% at 0900 UTC.
I got that right, the Philip Eden that Steve Loft quoted was working for an independent company at the time he invented snow index, but he took account of above.
freddie wrote:
Sat 28 Mar 2020 6:51 pm
Snow day runs from midnight to midnight (UTC)
That makes life complicated for implementing the weather diary, two different days 9am GMT (UTC) to 9 for most weather records, plus snow depth; midnight to midnight for snow falling

Your rain records separate 9am UTC to 9pm and 9pm to 9am, so presumably rain days are not midnight to midnight.

That basically stops Mark from being able to do any standard end-of-day action. I leave him to work out whether he could change his interface to implement the UK system. I recall lots of articles in this forum about the daily periods Australasia's BOM uses being different depending on parameter.

You are a mine of information, Freddie, meteorological and computational.

freddie
Posts: 896
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux Ubuntu 20.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Weather Diary

Post by freddie »

sfws wrote:
Sat 28 Mar 2020 7:21 pm
freddie wrote:
Sat 28 Mar 2020 6:51 pm
Snow day runs from midnight to midnight (UTC)
That makes life complicated for implementing the weather diary, two different days 9am GMT (UTC) to 9 for most weather records, plus snow depth; midnight to midnight for snow falling
It's the same for days of hail and gale too. Fog is similar to snow depth, in that it is the existence of fog (visibility less than 1000m) at 0900 GMT that makes a "fog day". Snow depth and fog day are measured at an instant in time (i.e. 0900 UTC) whereas all the other elements are measured over a period of time.
In fact, for temperatures it is complicated too. The definition of maximum temperature is that it is attributed to the date on which your meteorological day begins - which sounds reasonable. For minimum temperature, however, it is attributed to the date on which your meteorological day ends. So Cumulus doesn't have it strictly correct. For example - my weather station (via Cumulus) reported at 0900 this morning that the maximum temperature was 13.1, and the minimum temperature was -1.2. Following the definitive rules, the maximum temperature is attributed to the 27th March, but the minimum is attributed to the 28th March. But in the Cumulus "dayfile" they will both be attributed to the 27th. As maximum temperature typically occurs mid afternoon and minimum temperature around sunrise, then you can see why this strange-sounding rule originated.
sfws wrote:
Sat 28 Mar 2020 7:21 pm
Your rain records separate 9am UTC to 9pm and 9pm to 9am, so presumably rain days are not midnight to midnight.
No, rain days are 0900 to 0900 (UTC). The date the rainfall total is attributed to is the date at the start of the meteorological day - this Cumulus does get correct.
sfws wrote:
Sat 28 Mar 2020 7:21 pm
You are a mine of information, Freddie, meteorological and computational.
I have worked in both industries for nearly 35 years, so I hoped to have picked up a thing or two during that time :D
Freddie
Image

User avatar
PaulMy
Posts: 2236
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2+ Cumulus1 / CummulusMX
Operating System: Windows8 / Windows10
Location: Komoka, ON Canada
Contact:

Re: Weather Diary

Post by PaulMy »

I have been using the Cumulus1 weather diary for snow fall and depth. Now also using the CumulusMX for the same. CumulusMX is a better for me as it allows decimal measurement. I do refer to the snow index in Cumulus1 on occasion, but developed my own from my manual records http://www.komokaweather.com/weather/snowfall-log.pdf

I also do a daily manual reading and reporting of rainfall, snowfall and snow depth for CoCoRaHS http://cloud.cocorahs.org/wys/2018-2019 ... -2019.html and they ask for readings to be at the same daily time as much as possible, between the hours of 7:00 and 9:00 am. I use this CoCoRaHS reporting for the weather diaries. CoCoRaHS objective is to have an ongoing measurement of precipitation which is used by various organizations including conservation authorities and others who use that for draught and flooding decisions. Automated readings are not accepted by CoCoRaHS.

Enjoy,
Paul
Last edited by PaulMy on Sun 29 Mar 2020 1:50 pm, edited 1 time in total.

sfws
Posts: 930
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: --
Operating System: --

Re: Weather Diary

Post by sfws »

freddie wrote:
Sat 28 Mar 2020 8:20 pm
The definition of maximum temperature is that it is attributed to the date on which your meteorological day begins - which sounds reasonable. For minimum temperature, however, it is attributed to the date on which your meteorological day ends. So Cumulus doesn't have it strictly correct. For example - my weather station (via Cumulus) reported at 0900 this morning that the maximum temperature was 13.1, and the minimum temperature was -1.2. Following the definitive rules, the maximum temperature is attributed to the 27th March, but the minimum is attributed to the 28th March. But in the Cumulus "dayfile" they will both be attributed to the 27th. As maximum temperature typically occurs mid afternoon and minimum temperature around sunrise, then you can see why this strange-sounding rule originated.
I was aware of this conflict with Cumulus, and do understand the dawn and mid-afternoon aspect. There are some now very old posts where users and Steve argue about whether he got his design right, he claimed Cumulus 1 did have it strictly correct for UK Met O, but if people disagreed they should do what he did," write your own software". Prior to my use of Cumulus when I made manual weather measurements when I could, (and first school then work made 9am GMT hard to achieve), I did when comparing with official measurements make the flip with temperatures.

I have to add, my personal opinion was that when MX was being developed, it should have abandoned its need to be compatible with Cumulus 1 in these few respects and made some of these adjustments to better suit meteorological rules.To me they are more significant when comparing with official figures at month or year level, because quite often there are sharp changes around the period when one month ends.
PaulMy wrote:
Sat 28 Mar 2020 9:22 pm
CumulusMX is a better for me as it allows decimal measurement.
Paul, in my implementation, I entered integer figures into Cumulus 1 and divided by 10 for output, so I am sticking to same and not yet using MX decimal ability.

I am sorry to hear that you have organisations who make droughts and flooding, round here flooding was a big issue last month (the height of water in rivers and the distance away of flooded properties broke even 2000 flooding records), and (not that far away) many people who were flooded last month are now confined to their temporary homes with no repair work going on in their flooded premises.

I have moved from having a garden that was exceptionally dry (sand and gravel on a steep hill) to one that was waterlogged (clay near a brook) for my first few months! I have already added nearly 4 cubic metres of gravel to the front garden, and raised part of the back garden.

User avatar
mcrossley
Posts: 6998
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Weather Diary

Post by mcrossley »

sfws wrote:
Sat 28 Mar 2020 5:31 pm
mcrossley wrote:
Sat 28 Mar 2020 4:25 pm
The values for the web tags <#snowdepth>, <#snowlying>, <#snowfalling> are taken from the last diary entry for today? Today taking the SnowDepthHour into account.
Only the first is a web tag ...
The others were added in 3.1.1 - b3054

sfws
Posts: 930
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: --
Operating System: --

Re: Weather Diary

Post by sfws »

mcrossley wrote:
Sun 29 Mar 2020 1:46 pm
The others were added in 3.1.1 - b3054
I missed that, and neither you nor I edited https://cumuluswiki.org/a/BETA_webtags#Miscellaneous

Post Reply