Upgrade to CUtils 6.17.3

[laulau]

Hi,
I'm a little lost with a lot of errors in log file after upgrade!
Please advice
log in attached file

[HansR]

Hi,

You have a very long dataseries which does not fit CMX and therefore not CUtils.

My first reaction is:

1. Run Create Missing, this will add missing info to your dayfile.txt
2. For the dataseries which are very old, they are ignored and give 10 messages per flog file. Sorry for that. Ignore those.
3. There is also a line like:

   Code: Select all

   2023-04-16 17:42:08.665 Information GetUrlData Start: URL - http://api.tameteo.com/index.php?api_lang=fr&localidad=62247&affiliate_id=fi92ew78wgau 

   which I don't understand at the moment. Let's do the Create MIssing first and then see what's next. It seems to have something to do with pwsFWI prediction. I assume the language is the issue. But I am not sure now.

[laulau]

Ok, i'll do that.
I think i was using the same dayfile before the upgrade.
here is the logfile before updating. there are less informations about missing data but something else about FtpWebRequest.

[HansR]

Ok, i'll do that.
I think i was using the same dayfile before the upgrade.
here is the logfile before updating. there are less informations about missing data but something else about FtpWebRequest.

This is something with a lockfile meaning you are using a very old executable.
Lockfile for the usermap have not been used for a long time.
Did you update correct?
The time in the logfile is midnight. Does not seem to be correct.

[laulau]

Previous log was before the upgrade.
Here is the last one after i run CreatMissing.
At the end i can see :

023-04-17 00:00:52.229 Error ReadMonthlyLogs: Separator Inconsistency in data/copy_ftplog.txt found, FATAL ERROR! Exiting.
2023-04-17 00:00:52.229 Error ReadMonthlyLogs: Please check copy_ftplog.txt.

What does it mean ?

[laulau]

It looks like ftp process doesn't work since a while.
the last upload to utils directory is dated from 10/11/2020 !

[HansR]

Previous log was before the upgrade.
Here is the last one after i run CreatMissing.
At the end i can see :

023-04-17 00:00:52.229 Error ReadMonthlyLogs: Separator Inconsistency in data/copy_ftplog.txt found, FATAL ERROR! Exiting.
2023-04-17 00:00:52.229 Error ReadMonthlyLogs: Please check copy_ftplog.txt.

What does it mean ?

ftplog.txt is a file which does not belong in the data directory.
It is erroneously put in the list of files to read. What is the date of that file?
Just delete it. Any FTP logfile is logged to the directory utilslog.

And while we are at it: any file with prefix 'copy_' must be deleted as well from the data directory.

Other observation from the log:

Code: Select all

2023-04-17 00:00:47.704 ReadMonthlyLogs: reading data/août20log.txt
2023-04-17 00:00:47.728 Error MonthfileValue.SetValues fail: Le format de la chaîne d'entrée est incorrect.
2023-04-17 00:00:47.728 Error MonthfileValue.SetValues: in field nr 18 (SolarRad)
2023-04-17 00:00:47.728 Error MonthfileValue.SetValues: line is: 10/08/20,07:55,17.9,92,16.6,0.0,0.0,0,0.0,0.0,1015.6,306.8,22.9,61,0.0,17.9,17.9,0.1,73.485,0.00,0.00,20.1,133,1.5,0,0.0,0.0,19.3

Apparently in august 2020 errors got into the file for the value of the solar radiation. The line above has 73.485 which is a floating point number but the solar radiation is defined as an integer. Therefore it can't read the number.

You will have to edit your file for that field to make it an integer, it is the only file in the log with that error, you may find others when you rerun after deleting the copy_ftplog.txt. I have no idea how that value got there, it must have been a disturbance in CMX which creates and owns the monthly logs.

Now do the next try run.

NOTE: all files with the missing field 19 (i.e. before 2011 apparently) will not be used in CUtils. I will look into how to reduce the nr of messages in the log.

[HansR]

It looks like ftp process doesn't work since a while.
the last upload to utils directory is dated from 10/11/2020 !

That may be related to the ftplog in the data directory (see previous post) as that inhibits the functioning of CUtils.
Apparently this has not been checked for some time.

I suggest we continue with stepwise improvement until successful, so far things are going fine and we are eliminating errors.
We have not seen any FTP transfer in the log so until that happens we will not conclude anything about the current status of FTP.

[laulau]

Hi Hans,
It seem to work better
FTP took place but I have the feeling that noaa.txt and yadr.txt only upload without the thrifty option. Right ?
Will now look at the new options.
THANKS

[HansR]

Good

Now that the infrastructure is working (again) do one run without thrifty. This will regenerate all and upload the new runtime system. When that has been done OK, you can rerun the daily thrifty run again.

Did you install a UserAskedData run like e.g.

Code: Select all

utils/bin/cumulusutils.exe UserAskedData SystemInfo UserReports

Or something similar in the scheduler for the Interval frequency (e.g. 10 or 15 minutes)?

OK, I'll now look at your logs and come back to it.

[HansR]

The 08h34 log has the following error:

Code: Select all

2023-04-17 08:35:04.123 ReadMonthlyLogs: reading data/mars12log.txt
2023-04-17 08:35:04.175 Error MonthfileValue.SetValues fail: Le format de la chaîne d'entrée est incorrect.
2023-04-17 08:35:04.175 Error MonthfileValue.SetValues: in field nr 24 (CurrWindBearing)
2023-04-17 08:35:04.175 Error MonthfileValue.SetValues: line is: 27/03/12,03:50,9.3,37,-4.7,8.4,13.1,12,0.0,0.0,1030.0,1297.0,18.3,43,10.3,8.1,9.3,0.0,0,0.00,0.00,5.1,0,0.0,1927/03/12,07:05,6.3,42,-5.7,0.0,0.0,0,0.0,0.0,1030.0,1297.0,17.8,43,0.0,6.3,6.3,0.0,0,0.00,0.00,3.6,0,0.0,0
2023-04-17 08:35:04.177 Information Monthfile.SetValues : Continuing to read data
2023-04-17 08:35:04.181 Error MonthfileValue.SetValues fail: Le format de la chaîne d'entrée est incorrect.
2023-04-17 08:35:04.181 Error MonthfileValue.SetValues: in field nr 24 (CurrWindBearing)
2023-04-17 08:35:04.181 Error MonthfileValue.SetValues: line is: 29/03/12,03:00,8.2,36,-6.0,0.0,0.0,0,0.0,0.0,1024.0,1297.0,18.9,43,0.0,8.2,8.2,0.0,0,0.00,0.00,5.5,0,0.0,029/03/12,17:55,19.9,15,-7.5,14.4,23.9,348,0.0,0.0,1019.0,1297.0,19.3,43,21.1,19.9,19.9,0.0,0,0.00,0.00,14.2,237,0.0,353
2023-04-17 08:35:04.183 Information Monthfile.SetValues : Continuing to read data

It looks like two days have concatenated lines. You have to take the file into a proper editor (wrt not generating additional characters into the file and be able to save as UTF8) and split those lines. The times are very much apart so I don't really understand how this came about.

The rest of the log looks good.

I have some remarks: I see you generate the CUtils website, do you plan to use that separately like the other websites or do you plan to only use the pwsFWI module (as the link in the map suggests. Because if you only use the pwsFWI module you can do things a bit more efficient.

I'll now check the 09h21 log.

[HansR]

The 09h21 log does not add new issues (apart from the one mentioned in the previous post), everything looks OK.

This means the system is OK afaiac.
IF you wish to use the website, please add the URL to the map iso only the pwsFWI. In addition you could start using the menu (System 2) and the new charts (using the ChartsCompiler).

Any questions, let me know.

[laulau]

Hi,
Some correction done.
i changed the station website :

[Maps]
Website=http://laurentmey.fr/meteo/utils/index.html
Participant=true
DoneToday=2023-04-17T08:35:29

But it's not yet reflected although I relaunched CumulusUtils with the param website!
In the menu there is 'Ajouter' but it should be 'A propos de' in french. I didn't find where to change this traduction.

[HansR]

But it's not yet reflected although I relaunched CumulusUtils with the param website!

The signature file is only sent once per day. You can force a send by removing the parameter DoneToday.
If you do that, then the new info will be on the map at xxh45 i.e. the map is regenerated every quarter before the whole hour.
If you let it go naturally, the new info will be sent once per day and picked up at the next generation of the map.

In the menu there is 'Ajouter' but it should be 'A propos de' in french. I didn't find where to change this traduction.

The language file for your installation is in the CumulusMX directory CUstringsFR.ini (if your language setting is fr-FR which I assume it is). You can enter your translation there. In this case the keyword is 'About' in the section Website.

The shared French language file is created by @meteo19 you can communicate with him or you can update the language file and I will replace it (and communicate that with him).

The original date is 12 may 2020, since then there have been added more language elements so an update could be a good exercise.
As French has some variations in the regions maybe some agreement would be good

Hope this helps.

[laulau]

Hi,
Main CumulusUtils page not working today.
Link : http://meteo.laurentmey.fr/utils/index.html
I don't undestand why !
Please help.