Welcome to the Cumulus Support forum.

Latest Cumulus MX release 3.28.5 (build 3282) - 23 February 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Cutils 6.23.0 beta 1

Discussion of the Cumulusutils tool and website generator.

Moderator: HansR

AndyKF650
Posts: 673
Joined: Thu 20 Aug 2020 8:35 am
Weather Station: Davis VP2 Plus
Operating System: RPiv4 64bit Linux 12 Bookworm
Location: Jersey Channel Islands
Contact:

Cutils 6.23.0 beta 1

Post by AndyKF650 »

Hi Hans

Looks like you have been busy in the background again.

I loaded to latest download as normal and ran the website command and noticed it ran a lot quicker than normal. On checking the website most things looked good but the customlogs charts were blank. Looking at the utilslog for that run I can see that the date field is asking for a different format against that which worked well previously.

I have attached the log file and a temperature data file for information.
You do not have the required permissions to view the files attached to this post.
User avatar
Dador
Posts: 158
Joined: Thu 24 Nov 2011 2:22 pm
Weather Station: Davis VP2 Plus & Ecowitt
Operating System: Windows 10
Location: Rybnik, Poland
Contact:

Re: Cutils 6.23.0 beta 1

Post by Dador »

If I may chime in,
I would like to report that the ExtraSensors Top10 Graphs CustomLogs modules are working fine in the latest release.
User avatar
HansR
Posts: 5731
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Cutils 6.23.0 beta 1

Post by HansR »

AndyKF650 wrote: Mon 04 Dec 2023 2:42 pm Looks like you have been busy in the background again.

I loaded to latest download as normal and ran the website command and noticed it ran a lot quicker than normal. On checking the website most things looked good but the customlogs charts were blank. Looking at the utilslog for that run I can see that the date field is asking for a different format against that which worked well previously.

I have attached the log file and a temperature data file for information.
I think I found it. In the opriginal tekst of the CustomLogs Wiki I had a text saying:
NOTE: The formatting of the datafiles is important as the Custom Logs are basically free format: you can put into the file any text and numbers with any separator you wish. CUtils will NOT handle that Free Format in general. The user has to take care the information in the resulting datafiles is as follows:

1) There MUST be at least one space between the numbers;
2) There cannot be any plain text in the logfile
The date/time will be untouched i.e. the format is: dd-mm-yy;hh:mm;
Unfortunately that text has been modified a lot of times and eventually it disappeared.
The way it works now is : the separator is replaced by a space and then the line is split on the spaces to address the individual members. Unfortunately, the removal of two spaces - you use a comma AND a space as separator - is very time consuming using a regex so I removed that, not realising you use two separators. I would advise you to edit your definition of your custom logs to <webtag>,<webtag> etc... to get a single separator. Note that a single space is also allowed but there can be only one (1) separator between the fields just like in the regular logfiles.

To fix the existing logs just edit them - either switch CMX off or edit between the intervals - by replacing all ", "-sequences with ",".

Than try again.

And of course I will adjust the Wiki.

Dador wrote: Mon 04 Dec 2023 3:09 pm If I may chime in,
I would like to report that the ExtraSensors Top10 Graphs CustomLogs modules are working fine in the latest release.
Thanks :D
Hans

https://meteo-wagenborgen.nl
CMX build 3278 ● Ecowitt GW1100/WS80/WH40 ● RPi 3B+ ● Raspberry OS 6.1.21 ● Mono 6.12.0.200
https://meteo-wagenborgen.nl/CMX4/NET8/ (beta)
CMX build 4003+ ● RPi 4B ● RPi OS 6.1.0-rpi7-rpi-v8 aarch64 ● dotnet 8.0.1
User avatar
PaulMy
Posts: 3732
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2+ Cumulus1 / CummulusMX
Operating System: Windows8 / Windows10
Location: Komoka, ON Canada
Contact:

Re: Cutils 6.23.0 beta 1

Post by PaulMy »

In addition you need to be aware of the following:
The simplification of the logfile reading code is not visible for the user but has major impact if there is a bug in it so please check carefully the results of all CUtils output. There should not be any difference between this version and the previous one. Especially where older series are in play (Yes Paul :!: ;) ) this may have an effect. The separators are checked only once for each logfile type. So any change of locale over the years may have some unwanted effects.
I installed and the website runs seemed to go ok, and the result is fine.
I have it run on two separate CMX:
One uploads to www.komokaweather.com/cumulusmx/index.html and this run took about 2:07 mm:ss where the midnight run on previous version took 6:10 mm:ss
The second uploads to www.komokaweather.com/cumulusmxwll/index.html and this took about 2:51 mm:ss where the previous version took 6:25 mm:ss
These are now both with FTP as I still can't use PHP upload due to ISP issue. The previous version CUtils using php upload, when I was able to use it, took less than a minute.

EVT graphs look nice... and noticed that for 2021 there was huge spike April 13th. Using my CU1 data I found the error in my CMX install where it seems there was a stop and restart and that caused the total ET to drop from 149.78 to 0.00 and then next interval back to 149.78
I think I have the data corrected.

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www .komokaweather.com/cumulusmxwll/index.htm
MX www.komokaweather.com/cumulusmx/index.php
MX www.komokaweather.com/cumulusmx/index.html

Image
User avatar
HansR
Posts: 5731
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Cutils 6.23.0 beta 1

Post by HansR »

PaulMy wrote: Mon 04 Dec 2023 6:25 pm I installed and the website runs seemed to go ok, and the result is fine.
Good :D
PaulMy wrote: Mon 04 Dec 2023 6:25 pm I have it run on two separate CMX:
One uploads to www.komokaweather.com/cumulusmx/index.html and this run took about 2:07 mm:ss where the midnight run on previous version took 6:10 mm:ss
The second uploads to www.komokaweather.com/cumulusmxwll/index.html and this took about 2:51 mm:ss where the previous version took 6:25 mm:ss
That is what I call significant improvement and makes me a happy man 8-) :D
PaulMy wrote: Mon 04 Dec 2023 6:25 pm These are now both with FTP as I still can't use PHP upload due to ISP issue. The previous version CUtils using php upload, when I was able to use it, took less than a minute.
Yes PHP uploads are lightning fast. Hope you can use that again in future.
PaulMy wrote: Mon 04 Dec 2023 6:25 pm EVT graphs look nice... and noticed that for 2021 there was huge spike April 13th. Using my CU1 data I found the error in my CMX install where it seems there was a stop and restart and that caused the total ET to drop from 149.78 to 0.00 and then next interval back to 149.78
I think I have the data corrected.
That is a nice spin off of CUtils, there is always some data improvement :idea:

Thanks Paul, this is a nice result :clap:

Btw... did you look at and get anywhere with the CustomLog charts?
Hans

https://meteo-wagenborgen.nl
CMX build 3278 ● Ecowitt GW1100/WS80/WH40 ● RPi 3B+ ● Raspberry OS 6.1.21 ● Mono 6.12.0.200
https://meteo-wagenborgen.nl/CMX4/NET8/ (beta)
CMX build 4003+ ● RPi 4B ● RPi OS 6.1.0-rpi7-rpi-v8 aarch64 ● dotnet 8.0.1
User avatar
PaulMy
Posts: 3732
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2+ Cumulus1 / CummulusMX
Operating System: Windows8 / Windows10
Location: Komoka, ON Canada
Contact:

Re: Cutils 6.23.0 beta 1

Post by PaulMy »

Hi Hans,
Btw... did you look at and get anywhere with the CustomLog charts?
Not looked at that yet as too occupied in trying to get ISP and host issues resolved (I emailed host just before my annual hosting renewal in case they had a better plan, but they didn't get back) so auto-renew at the existing plan today :bash: ISP technician is stumped and nothing yet from ticket escalation. So call croxyproxy.com nearly every time I want to do a new session on my websites.

Enjoy :bash:
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www .komokaweather.com/cumulusmxwll/index.htm
MX www.komokaweather.com/cumulusmx/index.php
MX www.komokaweather.com/cumulusmx/index.html

Image
User avatar
HansR
Posts: 5731
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Cutils 6.23.0 beta 1

Post by HansR »

@Paul: very sorry to hear that, really hope you get it resolved soon :roll: :?
Hans

https://meteo-wagenborgen.nl
CMX build 3278 ● Ecowitt GW1100/WS80/WH40 ● RPi 3B+ ● Raspberry OS 6.1.21 ● Mono 6.12.0.200
https://meteo-wagenborgen.nl/CMX4/NET8/ (beta)
CMX build 4003+ ● RPi 4B ● RPi OS 6.1.0-rpi7-rpi-v8 aarch64 ● dotnet 8.0.1
User avatar
meteo19
Posts: 88
Joined: Tue 03 May 2016 8:13 pm
Weather Station: Davis VP2+/DP1500 SmartHub
Operating System: Microsoft Windows 7 Pro
Location: Ségur-Le-Château, France
Contact:

Re: Cutils 6.23.0 beta 1

Post by meteo19 »

Hi Hans

New version installed and everything works for me.

Best regards,
AndyKF650
Posts: 673
Joined: Thu 20 Aug 2020 8:35 am
Weather Station: Davis VP2 Plus
Operating System: RPiv4 64bit Linux 12 Bookworm
Location: Jersey Channel Islands
Contact:

Re: Cutils 6.23.0 beta 1

Post by AndyKF650 »

Hi Hans

After a lengthy data cleanse on the data/customlog files all looks good this morning. I must pay attention to spacings when generating new logs :bash:
User avatar
HansR
Posts: 5731
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Cutils 6.23.0 beta 1

Post by HansR »

meteo19 wrote: Tue 05 Dec 2023 5:39 am New version installed and everything works for me.
Good, thanks :D
AndyKF650 wrote: Tue 05 Dec 2023 7:42 am After a lengthy data cleanse on the data/customlog files all looks good this morning. I must pay attention to spacings when generating new logs :bash:
Good and thanks.
And I must pay attention to the Wiki/Instruction/Modifications, sorry for this one :D
Hans

https://meteo-wagenborgen.nl
CMX build 3278 ● Ecowitt GW1100/WS80/WH40 ● RPi 3B+ ● Raspberry OS 6.1.21 ● Mono 6.12.0.200
https://meteo-wagenborgen.nl/CMX4/NET8/ (beta)
CMX build 4003+ ● RPi 4B ● RPi OS 6.1.0-rpi7-rpi-v8 aarch64 ● dotnet 8.0.1
User avatar
Dador
Posts: 158
Joined: Thu 24 Nov 2011 2:22 pm
Weather Station: Davis VP2 Plus & Ecowitt
Operating System: Windows 10
Location: Rybnik, Poland
Contact:

Re: Cutils 6.23.0 beta 1

Post by Dador »

I noticed that just after midnight the Cutils modules start up on their own. I have tasks set in my schedule to run after 1 a.m. Interestingly, this only happens for Cutils, where I have the CustomLogs module. In the second instance (for Ecowitt stations) it works normally.

Hans, where does Cutils get information about the annual rainfall norm? Here there is also a difference between cutils for Davis and cutils for Ecowitt. For the Davis station I have a line with the rainfall standard on the charts, for the Ecowitt station I do not have this line. My Cutils settings for both stations are practically the same. Hmmm....
You do not have the required permissions to view the files attached to this post.
User avatar
HansR
Posts: 5731
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Cutils 6.23.0 beta 1

Post by HansR »

Dador wrote: Wed 06 Dec 2023 9:09 am I noticed that just after midnight the Cutils modules start up on their own. I have tasks set in my schedule to run after 1 a.m. Interestingly, this only happens for Cutils, where I have the CustomLogs module. In the second instance (for Ecowitt stations) it works normally.
Totally impossible: CUtils does not start itself, it must be some residual entry in your scheduler (or in CMX?).
Dador wrote: Wed 06 Dec 2023 9:09 am Hans, where does Cutils get information about the annual rainfall norm? Here there is also a difference between cutils for Davis and cutils for Ecowitt. For the Davis station I have a line with the rainfall standard on the charts, for the Ecowitt station I do not have this line. My Cutils settings for both stations are practically the same. Hmmm....
The Normal Annual Rainfall is calculated as the sum of the NOAA parameters for rain in Cumulus.ini (so e.g. NOAARainNormJan). So if you spot a difference between two configurations, that would be where to look.

If you are missing the reference line on one of the charts, look at the setting of the UseNormalRainReference parameter in cumulusutils.ini. That parameter can have the setting Normal, Station or Both. Note that the temperature has a similar setting!
Hans

https://meteo-wagenborgen.nl
CMX build 3278 ● Ecowitt GW1100/WS80/WH40 ● RPi 3B+ ● Raspberry OS 6.1.21 ● Mono 6.12.0.200
https://meteo-wagenborgen.nl/CMX4/NET8/ (beta)
CMX build 4003+ ● RPi 4B ● RPi OS 6.1.0-rpi7-rpi-v8 aarch64 ● dotnet 8.0.1
User avatar
Stigian
Posts: 57
Joined: Sun 22 Aug 2021 3:56 pm
Weather Station: 6242EU Vantage Vue
Operating System: Debian (Bookworm)
Location: Vaassen (NL)
Contact:

Re: Cutils 6.23.0 beta 1

Post by Stigian »

Hi Hans,

Just installed and looking good :D
Paul

Image
User avatar
HansR
Posts: 5731
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Cutils 6.23.0 beta 1

Post by HansR »

Stigian wrote: Wed 06 Dec 2023 10:32 am Hi Hans,

Just installed and looking good :D
Thanks!
Hans

https://meteo-wagenborgen.nl
CMX build 3278 ● Ecowitt GW1100/WS80/WH40 ● RPi 3B+ ● Raspberry OS 6.1.21 ● Mono 6.12.0.200
https://meteo-wagenborgen.nl/CMX4/NET8/ (beta)
CMX build 4003+ ● RPi 4B ● RPi OS 6.1.0-rpi7-rpi-v8 aarch64 ● dotnet 8.0.1
User avatar
Dador
Posts: 158
Joined: Thu 24 Nov 2011 2:22 pm
Weather Station: Davis VP2 Plus & Ecowitt
Operating System: Windows 10
Location: Rybnik, Poland
Contact:

Re: Cutils 6.23.0 beta 1

Post by Dador »

I don't want to start a new thread, so I'll write here.

Can I ask you to add a translation option for Limit values, Winter, Spring, Summer, Autumn, which are used in Seasons.
Post Reply