Welcome to the Cumulus Support forum.

Latest Cumulus MX release v3.4.6 (build 3070) - 23 March 2020
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014

See the Wiki for downloading either version.

Cumulusutils

Other discussion about creating web sites for Cumulus that doesn't have a specific subforum

Moderator: daj

User avatar
kocher
Posts: 106
Joined: Sat 19 Apr 2014 7:57 pm
Weather Station: Davis Vantage Pro2+
Operating System: Windows 10
Location: San Sebastian - Spain
Contact:

Re: Cumulusutils

Post by kocher » Thu 27 Feb 2020 5:26 am

You're right Beteljuice

I think it is because each page of the template has a usual structure:

Code: Select all

<html>
  <head>
  <meta http-equiv="content-type" content="text/html; charset=utf-8">
  <meta name="generator" content="PSPad editor, www.pspad.com">
  <title></title>
  </head>
  <body>
  	<?php include 'menu.php';?>
  </body>
</html>
But within the <body>, there is another instruction:

<? php include 'menu.php';?>

which in turn (menu.php) also has said html structure, so this duplicity results
Image

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Thu 27 Feb 2020 9:47 am

Version 2.2.1 is online

Note that all months of March shopuld display now without manual interference ;)
Also note that I made a first optimisation here, I did test of course, but please make rough checks on validity of output.

Also note, that this appears to be one of the heaviest modules so far with respect to performance. A data set of 8-10 years takes roughly 30 seconds on my Windows machine, 5.5 seconds on my RPi. This is due to the fact that for each record it runs several times through the list of data to find the record value for the specific item. I will look for more optimization here, apparently I chose the worst loops of all loops in Linq. Must be able to do better (remember top10 which was lightning fast when I did it in C :? ).
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Thu 27 Feb 2020 12:30 pm

Have been looking at performance issue of the records module and modified code such that full table scans (in relational database terms) are dramatically reduced.
Did some timing on it (using the Stopwatch from system diagnostics) and came to the following for my own dayfile and dayfile of tZandt which has 8 years:
  1. On RPi overall performance for my own small datafile went from 5.5 to 0.244 sec. (factor 23)
  2. On Win10 for datafile tZandt performance went from 28 seconds to 0.973 seconds (factor 29 :!: ).
The difference for the larger file (which I can't measure on my RPi or at least did not want to set up that environment :| ) must be sought in difference in memory management and different overhead.

Anyway, good enough, I will create version 2.2.2 and release it.
Overall timing of record selection (not generation of HTML) remains in the logfile code, so you can see for yourself how much it takes on your system.
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Wed 04 Mar 2020 3:30 pm

Version 2.2.5 is online.

Two modules have been added (NOAA reports reader and a Day Record table).
Also where selection can be made for years/months from a drop down list, invalid months are greyed out.

So, where is this all leading to? Starting with a top10 because I did not like the existing PHP solution, and a brand new Fire Weather Index for PWSs because I thought it a challenge and was missing it in any existing software, I came up in half a year with a program which generates building blocks for meteo websites with Cumulus (1 and MX) as a data source. I enjoy doing it but now I am getting to a point where I think: what's next? So I thought a little and came up with the following path:
  1. Statistics - I really want to have true statistics on some measurements with standard deviation plotted in the graphs. NOTE: that for true statistics you need at least two years of data, preferably three (I have less than one year unfortunately :( .
  2. Previous point would probably lead to other graphs, more extended graphs. Most likely some other ideas. That may happen.
  3. A completely newly designed website, not so much from the user experience (I like the Saratoga template very much) but from a technical point of view to get rid of all those php and auxiliary files.
  4. The CumulusUtils as it stands now creates one or more reports/functionalities at the request of the user. That functionality will remain. But I would also like it also to generate all those modules incorporated in a website with dashboard, menu etc... which adjusts automatically depending on the viewport.
  5. Some other existing (maybe even non-existing) functionality may be turned into a module. Or modify some existing functionality.
Oh, and though requests will be considered, I do this principally for myself so there is no rush.
I am not in a hurry so I don't think it will be done tomorrow nor the day after, but some day....
I'll keep you posted. :)
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Thu 05 Mar 2020 9:01 am

Version 2.2.6 is online.

Alas, 2.2.5 was buggy and crashed (see releas notes) :oops: .
2.2.6 should fix it. Some confidence on my side, keep me posted :? .
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Sun 08 Mar 2020 12:06 pm

Version 2.3.0 is online.

Thanks to Javier of kocher.es, I started a climate module, you might want to look at the preview.
Comments as always are very welcome (no guarantee for change or implementation ;) ).
Bug reports - if any of course 8-) - are also very welcome.
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

sjmcinness
Posts: 8
Joined: Wed 27 Nov 2013 12:37 pm
Weather Station: Davis VP2+
Operating System: Windows 10 x64
Location: Adelaide, SA, AU
Contact:

Re: Cumulusutils

Post by sjmcinness » Sun 08 Mar 2020 11:59 pm

Thanks Hans, as always :)

I've added the climate module here but I haven't added it to the links bar yet. Whilst the yearly temperature is fairly obvious, I'm not sure what the yearly rain statistics is telling me - is it also showing the daily range of rainfall or some other interval? It may be worth adding a short description (~10 words?) under the dropdown box, to explain what the graphs show.

I've also been meaning to ask - would you be able to add previous/next buttons either side of the year dropdown on the YADR page, please? It will make it much easier to flick between pages of the same metric, particularity when comparing year-to-year.

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Mon 09 Mar 2020 8:08 am

sjmcinness wrote:
Sun 08 Mar 2020 11:59 pm
Thanks Hans, as always :)
With pleasure Scott :D
sjmcinness wrote:
Sun 08 Mar 2020 11:59 pm
I've added the climate module here but I haven't added it to the links bar yet. Whilst the yearly temperature is fairly obvious, I'm not sure what the yearly rain statistics is telling me - is it also showing the daily range of rainfall or some other interval? It may be worth adding a short description (~10 words?) under the dropdown box, to explain what the graphs show.
I am not so happy with the rain graph either, I think it is the scaling. But basically you see the same as in the temperature graph (daily average in line, range and StdDev in column. I'll think about improving the graph and may get some explanation somewhere (maybe a popup on the title or something). *Future Development* ;)
sjmcinness wrote:
Sun 08 Mar 2020 11:59 pm
I've also been meaning to ask - would you be able to add previous/next buttons either side of the year drop-down on the YADR page, please? It will make it much easier to flick between pages of the same metric, particularity when comparing year-to-year.
I understand. I'll see what I can do. :idea:
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
PaulMy
Posts: 2076
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis Vantage Pro 2+ w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Cumulusutils

Post by PaulMy » Tue 10 Mar 2020 9:17 pm

Hi Hans,
A lot has happened over the past few weeks... I have updated to v.230 and all seems fine.
All modules added to my www.komokaweather.com/komokaweather-ca

Enjoy,
Paul
Davis Vantage Pro 2 - Windows 8 - Cumulus v1.9.4 b.1099 - CumulusMX/WiFiLogger
www.komokaweather.com
www.komokaweather.com/cumulusmx
www.komokaweather.ca
www.komokaweather.com/pws
Image

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Tue 10 Mar 2020 9:36 pm

Thnx Paul, good to see you back, Everything OK I assume? Hope you like the additions. Don't hesitate to comment.

Best, Hans
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Thu 12 Mar 2020 8:05 am

Version 2.4.0 is online

Many big changes though it looks small. Notably the interface change (as requested by Scott for buttons in the yadr module) has been applied over all modules which could use the change. Furthermore, I removed the Climate module (that was short lived :? ) and integrated it with the Graphs module with the possibility to switch on/off graphs. See the README for more info. The EVT graphs are now standard, if you don't have a Davis they are useless, so better switch those off then.

I reviewed the tables because there were always problems in some templates. That should be solved now: Piotr, could you try the txt-files without iframe or renaming to prove it works.

Anyway, hope it does not have too many problems, especially having year and month drop down together with prev/next buttons provided a slight headache.
Graphs may change in future but I am not sure yet which coding road I take first, first a small spring break and outdoor excursion.
Weather seems to get better over here.
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
PaulMy
Posts: 2076
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis Vantage Pro 2+ w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Cumulusutils

Post by PaulMy » Thu 12 Mar 2020 4:08 pm

Hi Hans,
First, feeling much better now, thanks.

Installed v2.41 and like the changes (I have not tried v2.40) - use of All, NOAA selections...

The only issue is that the graphs are now not showing www.komokaweather.com/komokaweather-ca/wxGraphs.php
The graphs file is being updated www.komokaweather.com/komokaweather-ca/graphs.txt
The log file is quite large but did find any error issues in it.

Any suggestions what else I can check?

Enjoy,
Paul
Davis Vantage Pro 2 - Windows 8 - Cumulus v1.9.4 b.1099 - CumulusMX/WiFiLogger
www.komokaweather.com
www.komokaweather.com/cumulusmx
www.komokaweather.ca
www.komokaweather.com/pws
Image

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Thu 12 Mar 2020 4:44 pm

Hi Paul,

First great you like the changes :)

That's a pity you don't see the graphs. I had already seen it with the French sites but can't debug on my tablet. When I checked on the PC it appeared the jQuery was not included. I include it in all other modules but forgot it in Graphs. :oops: Apparently my template includes it standard somewhere and I did not notice :(

Sorry for this, version 2.4.2 is coming soon.

A quick fix would be to include the jQuery library in your template:

Code: Select all

 <script src="https://code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>);
[Edit:] I found the cause: I have the Beteljuice Sun-stuff loaded and that uses the jQuery lib :!: So it was already there. Apparently loading it twice does not hurt (apart from some performance)

Hans
Last edited by HansR on Fri 13 Mar 2020 8:02 am, edited 1 time in total.
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

User avatar
beteljuice
Posts: 2878
Joined: Tue 09 Dec 2008 1:37 pm
Weather Station: None !
Operating System: ex XP SP3 - now W10 laptop grr
Location: Dudley, West Midlands, UK

Re: Cumulusutils

Post by beteljuice » Thu 12 Mar 2020 5:01 pm

@Hans ...

A problem that is starting to appear with older scripts and new 'cutting edge' scripts is that for quite some time now the '"latest" jquery script request is actually redundant and stuck in the past at something like ver 1.9.1.

JQuery is of course still in various cdn librarys but you have to ask specifically for which version you want.
It also recommended that you store a local copy rather than use a cdn.

The 'latest' JQuery inclusion is:

Code: Select all

<script src="https://code.jquery.com/jquery-3.4.1.min.js" type="text/javascript"></script>
Image
......................Imagine, what you will KNOW tomorrow !

User avatar
HansR
Posts: 642
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Cumulusutils

Post by HansR » Thu 12 Mar 2020 5:12 pm

beteljuice wrote:
Thu 12 Mar 2020 5:01 pm
@Hans ...

A problem that is starting to appear with older scripts and new 'cutting edge' scripts is that for quite some time now the '"latest" jquery script request is actually redundant and stuck in the past at something like ver 1.9.1.

JQuery is of course still in various cdn librarys but you have to ask specifically for which version you want.
It also recommended that you store a local copy rather than use a cdn.

The 'latest' JQuery inclusion is:

Code: Select all

<script src="https://code.jquery.com/jquery-3.4.1.min.js" type="text/javascript"></script>
Thanks for the warning, I'll think about it (when I start reorganising and unify the code 8-) ).
https://meteo-wagenborgen.nl / Cumulus (3.4.3 b3067)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 10 (buster) / 4.19.97-v7+ armv7l
Mono JIT compiler version 6.8.0.105 (tarball Tue Feb 4 21:51:48 UTC 2020)

Post Reply