Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

For New Users

Discussion of the Cumulusutils tool and website generator.

Moderator: HansR

User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

@staccermaccer: Btw, with a data series from 1997 onwards you probably have one of the longest series in the circuit :clap:
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
staccermaccer
Posts: 24
Joined: Thu 29 Sep 2011 8:37 am
Weather Station: Davis Vantage Pro II, UV, Solar
Operating System: Win10
Location: Antwerp
Contact:

Re: For New Users

Post by staccermaccer »

Hi, Hans

Yes i have been running Cumulus for ages, i was able to start the program but i got this:
2021-05-20 22:11:50.036 Information Generating Heat Map data, doing year 2021
2021-05-20 22:11:50.046 Information Graphs : Start Wind section
2021-05-20 22:11:50.166 Information Graphs : Start writing HTML Style and Menu.
2021-05-20 22:11:50.931 Generate WindRose : Start writing the javascript menu.
2021-05-20 22:11:51.221 Generate Windrun : Start writing the javascript menu.
2021-05-20 22:11:51.466 ReadMonthlyLogs: start.
2021-05-20 22:11:51.706 ReadMonthlyLogs: reading data/apr12log.txt
2021-05-20 22:11:52.066 Information Monthfile : EOF detected
2021-05-20 22:11:52.256 ReadMonthlyLogs: reading data/apr13log.txt
2021-05-20 22:11:53.141 Information Monthfile : EOF detected
2021-05-20 22:11:53.216 ReadMonthlyLogs: reading data/apr14log.txt
2021-05-20 22:11:53.716 Information Monthfile : EOF detected
2021-05-20 22:11:53.811 ReadMonthlyLogs: reading data/apr15log.txt
2021-05-20 22:11:54.236 Information Monthfile : EOF detected
2021-05-20 22:11:54.401 ReadMonthlyLogs: reading data/apr17log.txt
2021-05-20 22:11:54.401 Error ReadMonthlyLogs: Separator Inconsistency in data/copy_apr17log.txt found, FATAL ERROR! Exiting.
2021-05-20 22:11:54.401 Error ReadMonthlyLogs: Please check data/copy_apr17log.txt and the data directory.
i will dig in to this tommorow, time to go to sleep :groan:

I will attach a copy of the log, i found the inconsistancy seems like somehow in time my date separator switched from slash to a dash, that seems to be the HUGE problem if a have to change them all :cry:

Marc,
You do not have the required permissions to view the files attached to this post.
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

@staccermaccer: Yes, Cumulus(MX) makes use of the locale in setting the filename of the monthly logs but also it uses the separators defined by the locale for the data storage. The file naming issue has been partly resolved in the sense that abbreviated month names will be used without a point (if the locale defines it with a point like e.g. in NL "apr." is the official locale name but CMX now uses "apr" for the formation of the filename).

However for the separators that is a whole different story and datafiles will have a different field format for dates, numerics and different field separators.

CumulusUtils determines the separators independent of the locale and detects the files independent of the locale. The directory is not allowed to contain *log.txt files other than the monthly logs and some known "alltimelog" and "AirLink" logfiles.

So, what does this mean for you?

Apparently you changed locale in 2017 - most likely somewhere halfway a month - while I determine formats on the file of 2012. The naming of the monthly logs before 2012 apparently is such that it deviates so CumulusUtils can't find it, otherwise it would have started in 1997. CumulusUtils assumes consistency in filenaming and dataformat over all datafiles.

To prevent CumulusUtils from stopping, just prevent it from reading the Monthly Logs. The monthly datafiles are used for the wind statistics and the solar energy statistics. So if you set the following parameters to false (and - not sure - don't use the ChartsCompiler?) it should continue to the end of the run :

Code: Select all

WindRose=false
SolarEnergy=false
SolarEnergyYearMonth=false
When this works and you feel like it we might try to find a solution for your problem.
The file naming issue simply requires the renaming of the files to the current naming system ( to: "local abbreviated month name without point" + "YYlog.txt").
The internal formatting issue is more problematic. I don't have a ready and go solution to switch to your current format for that.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
staccermaccer
Posts: 24
Joined: Thu 29 Sep 2011 8:37 am
Weather Station: Davis Vantage Pro II, UV, Solar
Operating System: Win10
Location: Antwerp
Contact:

Re: For New Users

Post by staccermaccer »

Hi all!

I got it working here: https://www.akker.be/index.html But only with the exception that i do not have records before may 2021, i had to rename most of them, i tried to change the date separator in Excel but that did not work because it's a text-file? I tried with a formula that works for one cell so i am getting somewhere but that is due to my limited knowledge of Excel. Perhaps an Excel-guru here who can figure that out for me? I can't recall why i changed to a dash, could be that when i switched to a RPI3 for some time i did not notice that change in the configuration!

Hans: Bedankt voor de opvolging!!


Marc,
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

Hi Marc, You're welcome ;), looks good :clap: , at least the realtime things and the pwsFWI are working. That's good.

But it really is a pity to waste such long series (for the climate graphs). I'll start thinking on how to be able to automatically correct such series (if it is possible).
Could you compress the datafiles and PM them to me? I assume the pre-Cumulus datafiles really are different or did you adjust those? Could you comment on that in the message?
I'll see if the data can be modified to fit CumulusUtils.

But we'll take that exercise off this thread, use PM at first instance. When successful I'll make a tool out of it and publish ;)

Regards,
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
staccermaccer
Posts: 24
Joined: Thu 29 Sep 2011 8:37 am
Weather Station: Davis Vantage Pro II, UV, Solar
Operating System: Win10
Location: Antwerp
Contact:

Re: For New Users

Post by staccermaccer »

Hi Hans!

Sometimes you are so busy looking for a solution that you don't see it lying in front of you :bash:

Notepad++ is the solution!

I do not know if you know this, does the alltime.log get a reading from utils also and is that also looking for slash because the date format is different?

Marc,
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

staccermaccer wrote: Sat 22 May 2021 8:23 am Notepad++ is the solution!
Could you elaborate so that others could do the changes as well?
staccermaccer wrote: Sat 22 May 2021 8:23 am I do not know if you know this, does the alltime.log get a reading from utils also and is that also looking for slash because the date format is different?
alltimelog.txt, monthlyalltimelog.txt and all AirLink*log.txt are ignored by CumulusUtils while reading the monthly logs and are not used anywhere else.
I assume the alltimelog files have a uniform format all over the world, if you need to know how and why that would be a question for Mark.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
freddie
Posts: 2434
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: For New Users

Post by freddie »

HansR wrote: Sat 22 May 2021 8:43 am
staccermaccer wrote: Sat 22 May 2021 8:23 am Notepad++ is the solution!
Could you elaborate so that others could do the changes as well?
Notepad++ has a global find/replace function that can use regular expressions. Extremely handy.
Freddie
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

@freddie: but regular expressions are not everybody's taste and liking, not to mention understanding. So some elaboration would come in handy.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
staccermaccer
Posts: 24
Joined: Thu 29 Sep 2011 8:37 am
Weather Station: Davis Vantage Pro II, UV, Solar
Operating System: Win10
Location: Antwerp
Contact:

Re: For New Users

Post by staccermaccer »

Hi Hans!
I will include a screenshot so everybody kan see:

Open file in Notepad++ go to: 'search' and in the dropdown-list goto 'replace' you get a window where you can fill in the separator that should be changed in the first field, in the second field fill in the right separator. Then you can click 'replace' or if you are sure 'repace all' et voila!

Hope this is clear enough?
2021-05-22 (1).png
You do not have the required permissions to view the files attached to this post.
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

Hi Marc,

OK, fair enough. This is not a regular expression, just a replace character operation.

And in that sense - I hope you have a backup because I think you also replaced the minus sign for any negative numeric value (does that appear in the monthly log?).
And second: what locale (lanuage/separators/international settings) do you use at the moment on the Windows machine/CMX? And is that locale the same as you want to use in CumulusUtils? In other words: do new entries appear with dashes or slashes, is it OK now?

(I think your default locale is nl-BE while your CumulusUtils setting at the moment is (default) en-GB. Watch out for the devils of the locale!)
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

Marc: And btw the regular expressions to fill in for the search and replace for the date are (take exactly as displayed):
  1. Search:

    Code: Select all

    (\d+)-(\d+)-(\d+)
  2. replace:

    Code: Select all

    \1/\2/\3
(try one before you do a replace all ;) )

If you need to replace other date formats I assume it is just the separator you need to change in place of the minus.
All numeric minus in the file (if any) will be untouched.
(Though not exact, the Notepad++ manual did show some help, Chua Hock-Chuan has a nice page as well)
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
staccermaccer
Posts: 24
Joined: Thu 29 Sep 2011 8:37 am
Weather Station: Davis Vantage Pro II, UV, Solar
Operating System: Win10
Location: Antwerp
Contact:

Re: For New Users

Post by staccermaccer »

Hi,

Thanks for the pointers, i do have a backup and you are quit right, the minus signs where also replaced, a little to quick after the 'aha' moment :lol:

I changed the locale in the cumulusutils.ini and we'll see...I am going to try the formula you provided me here, i tried the formula but i comes back with an error you can see in the attachment...
formule.png
BTW: where can i change the separator in Cumulus, i can't seem to find it anywhere or is this automatically done by the locale?

Thanks!

Marc,
You do not have the required permissions to view the files attached to this post.
Image
User avatar
beteljuice
Posts: 3292
Joined: Tue 09 Dec 2008 1:37 pm
Weather Station: None !
Operating System: W10 - Threadripper 16core, etc
Location: Dudley, West Midlands, UK

Re: For New Users

Post by beteljuice »

@ staccermaccer

Look at the bottom part of the find / replace pane.

Zoekmethode


Reguliere expressie ;)
Image
......................Imagine, what you will KNOW tomorrow !
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: For New Users

Post by HansR »

staccermaccer wrote: Sun 23 May 2021 8:04 am Thanks for the pointers, i do have a backup and you are quit right, the minus signs where also replaced, a little to quick after the 'aha' moment :lol:

I changed the locale in the cumulusutils.ini and we'll see...I am going to try the formula you provided me here, i tried the formula but i comes back with an error you can see in the attachment...formule.png
Like beteljuice remarked: you have to set that you are using the regular expression (Note that the English are using the Dutch translation and I am using the English term :lol: ; Tower of Babel syndrome)
staccermaccer wrote: Sun 23 May 2021 8:04 am BTW: where can i change the separator in Cumulus, i can't seem to find it anywhere or is this automatically done by the locale?
That is all implicit in the use of the locale by CMX.
What locale are you using in CMX? NOTE: the locale for CMX on windows is either the Windows used international setting or that what you give on the command line (if anything)

If you use a different local in CumulusUtils than in CMX - yes that is possible and you may - than don't worry, you will get into that later and we meet again.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
Post Reply