Welcome to the Cumulus Support forum.

Latest Cumulus MX release v3.4.6 (build 3070) - 23 March 2020
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014

See the Wiki for downloading either version.

Cumulusutils

Other discussion about creating web sites for Cumulus that doesn't have a specific subforum

Moderator: daj

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Sun 05 Jan 2020 11:34 am

sjmcinness wrote:
Sun 05 Jan 2020 9:53 am
Ahh, dry heat is good - it makes sweating worthwhile, instead of just making one wet. I much prefer warm over freezing; we rarely have a day under 5°C.
It's a different world. Looking at the graphs and tabulated daily data from the three Australian stations, I can tell you this half year of CumulusUtils and looking at all Utils stations daily has taught me a lot.
sjmcinness wrote:
Sun 05 Jan 2020 9:53 am
Bushfires aren't good, though, and over 6 million hectares have been burnt in Australia since the current bushfire season started in August (lots more detail on Wikipedia) which, to give some perspective, is nearly 1.5 times the size of Netherlands.
I know. You're all over the news and discussion with Phil (NSW) in September made me decide to speed up development. You may read my blog on this issue.
sjmcinness wrote:
Sun 05 Jan 2020 9:53 am
I've added the other pages to the link list at the bottom of each page now and I added the map to the bottom of the FWI page. There's probably some more formatting to do and some more consistency to fix but it will do for now (I should probably change to CumuluxMX as well, since it's being developed again; I stopped checking when Steve stopped developing and feature parity was still poor back then).
OK, looks great. Saw that beteljuice gave you some advice above. You may want to follow his advice as he's responsible for the beteljuice-interface you're using ;) If necessary I'll make some changes.
sjmcinness wrote:
Sun 05 Jan 2020 9:53 am
I wonder whether using the data from dayfile is the best option for temperature and humidity - it does give you the highest temperature and the lowest humidity but it doesn't give you the humidity at the same point in time as the highest temperature - which FWIcalc did (well, it preferred to use midday but provided the option to update based on a higher temperature later in the day).
Ah, well, the timing on the day is more or less irrelevant. I had several discussions on this with Phil (NSW) and I regard the process as seen from the forest. Moisture related processes in forests are not very fast, neither in absorbing moisture as in releasing it. So an hourly estimation of FWI is not really necessary and neither is it important to take the highest temperature point specifically. It's a different approach from FWIcalc.
sjmcinness wrote:
Sun 05 Jan 2020 9:53 am
Oh, the fact that Phil's and mine are relatively close but have vastly different conditions is a great thing - more points of data is usually a good thing and it can only make the model better 8-)
It is. It actually is the most stunning lesson of my CumulusUtils exercise (I run daily through all stations participating if I can, weather is stunning)
sjmcinness wrote:
Sun 05 Jan 2020 9:53 am
Thanks for the prompt fixes :) I see that you're running the updated version and YADR looks good now but you have removed the % symbol from the RH column on the FWI page - if it's possible, would you please be able to add an option so that the RH values could be displayed as percentages? I find RH easier to read that way and then it makes it consistent with RH on all of the other pages.
You're welcome. Thanks for the observations!
With respect to the RH (and other items on the pwsFWI page): I will keep it in fractions, while on all other pages RH will be in percentages. The reason is that the pwsFWI values are those actually entered in the equations. So even if you would be on imperial system, you would see the values in degree Celsius while on the YADR page you would see Fahrenheit. That's how it is, it's for analysis and information. The same holds for the resulting csv-file.

Cheers,
Hans
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Sun 05 Jan 2020 2:09 pm

@sjmcinness:
[...] Saw that beteljuice gave you some advice above. You may want to follow his advice as he's responsible for the beteljuice-interface you're using ;) If necessary I'll make some changes.
Reading backwards I noticed the beteljuice remark on the website was aiming at @hills so, ignore ;)
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
laulau
Posts: 547
Joined: Tue 13 Oct 2009 10:52 pm
Weather Station: WeatherDuino Pro2
Operating System: Win 7
Location: Meyenheim, Alsace, FR
Contact:

Re: Cumulusutils

Post by laulau » Sun 05 Jan 2020 3:28 pm

Hi,
Is there a way to ignore the (old) dayfile empty fields ?
I have this DOS display :
Capture.JPG
log :
2020-01-05 16:14:34.285 CumulusUtils : Line read from dayfile.txt added to MainList:
15/03/04,23.8,183,14:30,4.1,07:26,13.5,16:06,1027,18:26,1032,08:59,0,09:00,0,8.8,0,9.4,12:36,36,15:56,86,07:16,0,0,18.6,14:26,17,15:26,2.1,03:45,0,00:00,3.8,03:45,5.2,13:46,1.5,04:05,196,7.2,0,0,00:00,0,00:00,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2020-01-05 16:14:34.285 DayfileValue constructor start
2020-01-05 16:14:34.285 Dayfile : ReadLine Start
2020-01-05 16:14:34.285 CumulusUtils : Line read from dayfile.txt added to MainList:
16/03/04,21.6,213,15:45,4,07:28,17.3,15:15,1030,17:56,1033,08:59,0,09:00,0,10.7,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2020-01-05 16:14:34.286 DayfileValue constructor start
DayfileValue.SetValues fails: Le format de la chaîne d'entrée est incorrect.
DayfileValue.SetValues fails: Need latest dayfile.txt filled correctly with data
2020-01-05 16:14:34.300 Dayfile destructor: Closing file and ending program
2020-01-05 16:14:34.302 Ini Flush cache modif:False, cumulusutils.ini
Thanks
You do not have the required permissions to view the files attached to this post.
Laurent

Image

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Sun 05 Jan 2020 3:43 pm

@laulau:
Is there a way to ignore the (old) dayfile empty fields ?
No there is not but I may be able to adjust the reading of the file such that it is possible to continue. Otherwise you will have to remove to erroneous lines yourself.

Send me your dayfile.txt and cumulusutils.log (PM) and I'll analyse it. Also give me your sysinfo and cumulus version.

Cheers, Hans
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

hills
Posts: 684
Joined: Sat 26 Dec 2009 8:52 am
Weather Station: Fine Offset WH1091
Operating System: Raspbian Buster
Location: Crafers, South Australia
Contact:

Re: Cumulusutils

Post by hills » Sun 05 Jan 2020 10:40 pm

Wow!! for 5/1/2020, your software prediction was 348.83. then after my dayfile was created at 9:00am the actual was 341.15. I'm very happy with that accuracy for a prediction!! :clap:

A colleague of mine uses cumulus and lives at Surrey Downs, I'll ask him to implement cumulus utils so we have another close station for comparison.

hills
Posts: 684
Joined: Sat 26 Dec 2009 8:52 am
Weather Station: Fine Offset WH1091
Operating System: Raspbian Buster
Location: Crafers, South Australia
Contact:

Re: Cumulusutils

Post by hills » Mon 06 Jan 2020 3:36 am

hills wrote:
Sun 05 Jan 2020 10:40 pm
A colleague of mine uses cumulus and lives at Surrey Downs, I'll ask him to implement cumulus utils so we have another close station for comparison.
He is still in the process of tidying it up, but initial install is here: http://wood-ward.com:8080/weather/pwsFWI.html

gti138
Posts: 1
Joined: Wed 16 Oct 2013 10:48 am
Weather Station: Fine Offset
Operating System: Windows Server 2012R2
Location: Australia

Re: Cumulusutils

Post by gti138 » Mon 06 Jan 2020 4:19 am

hills wrote:
Mon 06 Jan 2020 3:36 am
hills wrote:
Sun 05 Jan 2020 10:40 pm
A colleague of mine uses cumulus and lives at Surrey Downs, I'll ask him to implement cumulus utils so we have another close station for comparison.
He is still in the process of tidying it up, but initial install is here: http://wood-ward.com:8080/weather/pwsFWI.html
Hi there - as Phil said, i've got the basics up and going here:

http://wood-ward.com:8080/weather/pwsFWI.html
http://wood-ward.com:8080/weather/maps.html

I'll look to have a static file created soon.

Cheers.
Michael

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Mon 06 Jan 2020 7:50 am

@hills: Thanks! Spread the word! :clap: :D :arrow:
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Mon 06 Jan 2020 7:58 am

hills wrote:
Sun 05 Jan 2020 10:40 pm
Wow!! for 5/1/2020, your software prediction was 348.83. then after my dayfile was created at 9:00am the actual was 341.15. I'm very happy with that accuracy for a prediction!! :clap:
That precision is of course kind of chance knowing the precision of weather predictions ;) .
But I did make a short analysis and thanks to meteosangonera and PaulMy a better insight into the effect of prediction could be made.
Last edited by HansR on Mon 06 Jan 2020 8:03 am, edited 1 time in total.
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

griffo42
Posts: 46
Joined: Thu 10 Dec 2015 6:41 am
Weather Station: Davis Vantage Pro2
Operating System: Win 10
Location: Brisbane, Queensland, Australia
Contact:

Re: Cumulusutils

Post by griffo42 » Mon 06 Jan 2020 7:59 am

Hi all

I have taken an alternative approach in displaying the Bushfire Danger level on my website. It, in macro, is to take the forecast bushfire danger by the Australian Bureau of Meterology for my area (which happens to be suburban Brisbane) as an XML file from the BOM website and show it on my Saratoga PHP Scripts /Cumulus website - URL is https://www.kstwx.net/index.php

If anyone is interested in the detail of how I have done it, I am happy to share it here.

My approach is: if the experts at the BOM have already done the assessment why try to reinvent the wheel! Having said that, it clearly is not applicable to sites outside AUS.

Best wishes,
Keith
Davis Vantage Pro2 Model #6152AU - CumulusMX - Win10 - Saratoga Scripts
https://www.kstwx.net/index.php
https://www.kstwx.net/cu/index.htm
Image

hills
Posts: 684
Joined: Sat 26 Dec 2009 8:52 am
Weather Station: Fine Offset WH1091
Operating System: Raspbian Buster
Location: Crafers, South Australia
Contact:

Re: Cumulusutils

Post by hills » Mon 06 Jan 2020 9:47 am

Good question Keith.

My reason is because my firedanger warning area - Mt Lofty Ranges, covers a lot of different types of terrain and to a degree different climates. I live in a particularly damp area as the hills behind us cause more precipitation here than most other locations in the Mt Lofty Ranges, so I wanted to be able to compare my "micro climate" to the rest of the area. In fact this is also the reason I got my own weather station in the first place and having previously been in the CFS, it was something I was toying with doing myself. Luckily someone with a lot more programming skills than I have beat me to it. ;)

That said I still pay attention to the BOM rating and therefore am interested in what you've done.

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Mon 06 Jan 2020 9:59 am

griffo42 wrote:
Mon 06 Jan 2020 7:59 am
My approach is: if the experts at the BOM have already done the assessment why try to reinvent the wheel! Having said that, it clearly is not applicable to sites outside AUS.
There are several reasons - as a meteorology amateur - as why to have the local Fire Weather Index for PWSs on your site (without even trying to devalue the BoM work, I know what's behind it and what they do!):
  1. The BoM predictions and warnings are for a vast area and may not directly apply to your own situation. The discussion on Crafers is clear on that issue.
  2. Dealing with the warnings yourself makes you think more conscious about the process of fire and the dangers involved (making you probably more careful)
  3. Having a local warning on your PWS can be seen as a trigger to look at the BoM messages more careful.
Fireweather will become more and more important. Amateur meteorologists deal with everything and measure everything (from sunrise to rain to air quality) so why do that? Why not just say: oh somebody else (the government) does it already, why would I do it? Thinking about phenomena in the environment is good, It enriches your mind to rethink.

And indeed, the world is bigger than AUS (I am Dutch). And everywhere they use similar but different and very complex warning systems. Mine is uniform, cheap and I claim validity everywhere. Usefulness is different: having a meteostation with pwsFWI in the middle of the Sahara or in the centre of Australia is clearly not useful though my guess is that the warning level would be extremely high.

Anyway: local check does not stand against the BoM warning. So you can use both beside each other. Check one after the other and wonder: why the difference. To do so you have to understand how both warning systems work. So you have to study, think about the science behind the warning, about the causes of fire. If my warning and accompanying blogs trigger that, I am successful.

Edit: and what @hills says above, he was first ;)
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Mon 06 Jan 2020 6:33 pm

@gti138:
Hi, missed your reaction because of the sudden discussion. Thanks for using pwsFWI. I did put you in the list as well. Looking forward to your updated site.
Clearly you have other surroundings than Northgate, closer to the hills but also urban effects, please do provide some context in the header. It would not surprise me if there were also effects of placement of instrumentation as well (close to a building, sheltered of wind) which may have effect on the calculated value of pwsFWI. Difficult to know from here.
Cheers, Hans
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Wed 22 Jan 2020 3:07 pm

Version 2.1.0 is online.

Although the user probably won't find significant changes, there are two places where the data handling got a big overhaul: reading the dayfile.txt file and in creating the top10 list. In the latter I used single statement LINQ queries i.s.o. the more low level approach I used from my prior C-implementation (the top10 list was the very first part of CumulusUtils). The dayfile.txt reading has been cleaned from redundancies and made more efficient.
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 630
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Wed 22 Jan 2020 8:52 pm

I have a question for the users (as a consequence of the issue by @laulau above):

So far several problems have occurred with the data in dayfile.txt, three types:
  1. Illegal format for the time of a value (e.g. '7:00' i.s.o. '07:00')
  2. illegal value for a number, which triggers an overflow event
  3. An empty value
I implemented an error handling where the assumptions are:
  1. The user will be informed of all errors and their exact location through a message in the logfile for all errors detected;
  2. If there is more than one error in a line, only the first error is reported after which the next line is read; there is no inline recovery (so far, maybe I'll find a way for that and show all errors in a line in one pass). This may lead to problems further down the road and may require serious editing of the dayfile. Does there exist a dayfile editor?
  3. It is up to the user to eventually correct the errors or live with the skipping of the line / automatic corrections.
The challenge is to continue the program and present the data anyway, supposing the good data will be the majority.
Knowing the field in which the error occurs, I think I have the following options:
  1. Abort execution on encounter of the first error (default)
  2. ignore the line with the erroneous data completely and continue with the next line (which is the option currently implemented)
  3. Choose a value for that field which would blend in with the other values (e.g. take the value for the erroneous field of the day before. Incorrect but still there is a result to show). If you have a series of many days with errors, that may lead to strange graphs but at least you know you are missing something.
So my question to the users: how would you want recovery to work or would you skip CumulusUtils because correcting dayfile would be too much work.
And how do other dayfile readers deal with errors like this?

NOTE: this does not mean CumulusUtils is a generic dayfile checker because it does not use all the fields.
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

Post Reply