Page 1 of 3

The new 3.6.0 'WindRose' release

Posted: Sun 17 May 2020 5:10 pm
by HansR
I had some message with problems with the WindRose release stating that:

1) There was a complaint about a logfile and therefore the run with windrose did not succeed. The file could not be read in the editor, but could be read with an iMac (I do not advice editing with an iMac and copying back btw!) . Files were received.

2) In a second run there was an issue with copy_monthlyalltimelog.txt

3) A third run did not complete either and I received the logfile indicating CumulusUtils had stopped because it could not download a file.

----------------

Replies:

1) If CumulusUtils stops with the message it has a probnlem with a logfile, please don't continue running several times, post the problem here with both the file and the logfile (make sure the logfile is from that specific run!)

2) The problem with the 'copy_monthlyalltimelog.txt', I had already expected not to occur but it existed because CU checks for *log.txt files to get the list of monthly logfiles (and bypassing the filenaming issues which differ per locale). Apparently the the release that went out still has the alltimelog files in the list. That will be fixed.

3) The third run dod not complete because CU could not download a file from the internet. That is a serious error in your connection and CU can't continue without internet.


Please report any problems with this beta release below, with cause and logfile and if necessary with datafile - zipped because they are large files.
Thank you, Hans

Re: The new 3.6.0 'WindRose' release

Posted: Sun 17 May 2020 5:34 pm
by sutne
1) There was a complaint about a logfile and therefore the run with windrose did not succeed. The file could not be read in the editor, but could be read with an iMac (I do not advice editing with an iMac and copying back btw!) . Files were received.
There is nothing wrong in editing files on a iMac.
Is this system only for people with Windows computers?

The RPI runs a version of Linux. iMac runs a version of Unix.

What makes Windows better?

The editor I am using (on the iMac) can handle all types of files, and verified that the corrupted file had been tagged as binary (on the RPI).

Re: The new 3.6.0 'WindRose' release

Posted: Sun 17 May 2020 7:15 pm
by HansR
sutne wrote: Sun 17 May 2020 5:34 pm
1) There was a complaint about a logfile and therefore the run with windrose did not succeed. The file could not be read in the editor, but could be read with an iMac (I do not advice editing with an iMac and copying back btw!) . Files were received.
There is nothing wrong in editing files on a iMac.
Is this system only for people with Windows computers?

The RPI runs a version of Linux. iMac runs a version of Unix.

What makes Windows better?
It is not a matter of good or better. But if you have a problem, and edit a file on the mac and than send me the file, it becomes impossible for me to find the problem.
If you detect a problem with a datafile, please send that file as is to me, together with the log. That is the only way how I can find aberrations in the datafiles (which occur for historical reasons or locale deviations or whatever). I need to be able to reproduce the error.

So, Mac is very OK, Windows is very OK, RPi is very OK. But apparently somewhere something is not OK :D
sutne wrote: Sun 17 May 2020 5:34 pm The editor I am using (on the iMac) can handle all types of files, and verified that the corrupted file had been tagged as binary (on the RPI).
If tagged as binary on the RPi? Meaning what? The executable bit is/was set? Or was it downloaded as binary by the Mac?

Regards, Hans

Re: The new 3.6.0 'WindRose' release

Posted: Sun 17 May 2020 8:15 pm
by sutne
The files were corrupted on the RPI. The editor on RPI would not read them, but I could read them with an editor on the iMac. You had the original corrupted files.

Re: The new 3.6.0 'WindRose' release

Posted: Sun 17 May 2020 8:32 pm
by HansR
sutne wrote: Sun 17 May 2020 8:15 pm The files were corrupted on the RPI. The editor on RPI would not read them, but I could read them with an editor on the iMac. You had the original corrupted files.
OK. Thanks, interesting because I can read them just like that, so it was not corrupted on arrival. The question is then what makes the files different on your RPi system?

Re: The new 3.6.0 'WindRose' release

Posted: Sun 17 May 2020 8:33 pm
by HansR
Fortunately 3.6.0 did work on the Segur-le-Chateau site....
So something is good ;)

Re: The new 3.6.0 'WindRose' release

Posted: Mon 18 May 2020 4:12 pm
by HansR
The 3.6.1 release seems better but no!

My installation and Segur-Le-Chateau runs fine but in Norway and Canada there are problems reading the datafiles as if memory gets overwritten. As a result, the system want to write data for the WindRose (where the problem must be) from the year 0. That is pretty much without end. So, no, we're not there yet.

Re: The new 3.6.0 'WindRose' release

Posted: Mon 18 May 2020 8:52 pm
by HansR
Problem seems to be solved: in the data directory was a file okt16log.txt beside Oct16log.txt. Of course both were taken up in the list of files to read. However, the first had completely different separators (.;.) than the rest of the files (/,.). And that is the problem: the software determines the separators in the first file and treat all files accordingly. It can't handle other combinations (quite rightly so), however it does not detect it encounters an erroneous situation either. That I will fix.

But @all users: screen you data directory for old garbage or switches in locale which are always - or most likely - at the origin of [these] data errors. Then you can safely use tools which make use of these files. Cumulus itself included probably.

Re: The new 3.6.0 'WindRose' release

Posted: Tue 19 May 2020 1:19 am
by PaulMy
Cumulusutils v.3.6.1 wind rose is working nicely www.komokaweather.com/cumulusutils/index.html

Enjoy,
Paul

Re: The new 3.6.0 'WindRose' release

Posted: Tue 19 May 2020 10:02 am
by HansR
Thanks Paul (though it seems that now you've got to fix the windmeter :lol: )
We're still working Norway though but getting closer.

Re: The new 3.6.0 'WindRose' release

Posted: Tue 19 May 2020 11:26 am
by HansR
We're now at a point where on some systems (RPi only?) the file 'monthlyalltimelog.txt' is not removed from the file list of monthly log files, but on most systems it is.

Anybody comment on this little piece of C# code?

Code: Select all

      for (int i=0; i<MonthfileList.Length; i++)
      {
        if (MonthfileList[i].Contains("alltimelog") )
        {
          Sup.LogDebugMessage(message: $"MonthfileList removing: {MonthfileList[i]}");
          var foos = new List<string>(MonthfileList);
          foos.RemoveAt(i);
          MonthfileList = foos.ToArray();
        }
      }


Re: The new 3.6.0 'WindRose' release

Posted: Tue 19 May 2020 11:39 am
by mcrossley
Not seeing the code, but wouldn't it be simpler to get a clean list of monthly log files using say a regex like \.*[0-9]{2}log.txt\

Re: The new 3.6.0 'WindRose' release

Posted: Tue 19 May 2020 11:59 am
by HansR
Thought of that, but I ask for

Code: Select all

 MonthfileList = Directory.GetFiles("data/", "*log.txt");
Getfiles does not support regular expressions, any suggestion for an alternative? From what I can find that is not all that simple and besides there is the performance penalty for using regex although that is not too big in this relatively small array/list.

The only thing I need to do is remove the files containing "alltimelog" which I do.

And beside that, it seems to work everywhere, but not on a Norwegian RPi system with an en-GB locale.

Re: The new 3.6.0 'WindRose' release

Posted: Tue 19 May 2020 2:17 pm
by HansR
Removing an entry moves up the list effectively skipping an entry if you don't adjust the index.

So, slight modification:

Code: Select all

foos.RemoveAt(i--);
Will prepare a version 3.6.5

Re: The new 3.6.0 'WindRose' release

Posted: Tue 19 May 2020 2:36 pm
by mcrossley
HansR wrote: Tue 19 May 2020 11:59 am Thought of that, but I ask for

Code: Select all

 MonthfileList = Directory.GetFiles("data/", "*log.txt");
Getfiles does not support regular expressions, any suggestion for an alternative? From what I can find that is not all that simple and besides there is the performance penalty for using regex although that is not too big in this relatively small array/list.
Ah, yes, you'd have to use a ".Where(path => regex)" as well. Can't see it being much of a overhead though, especially weighed against creating a new list, removing items, converting to array, then saving again.

What's odd about Norwegian systems file naming then?