Page 1 of 4
Extra Webfiles Missing
Posted: Mon 02 Dec 2019 9:44 pm
by Phil23
Hi All,
Just looked at my page
http://weather.inverellit.com/today-yesterday.htm & found it hadn't updated since 6:00pm on 02/12/2019.
Looked in the Extra Webfile settings & saw it's entry was missing.
Actually all entries except the first 10 were gone.
Checked the Diags & the only thing that stood out was repetitions of this...
Code: Select all
2019-12-02 17:10:01.977 Extra web file #8 [\CumulusMX\Cumuluswebtags.txt] not found!
Don't know what triggered it other than the fact that I do have task scheduler shutting down & restarting MX at both 6:00AM & PM.
In relation to this, can I suggest that Cumulus.ini be included in the files saved in the various backup directories in a future update.
Cheers
Phil.
Re: Extra Webfiles Missing
Posted: Tue 03 Dec 2019 1:58 am
by Phil23
Just been pulling entries from an older ini from last upgrade.
Only unusual thing I notice is that there is a block of other data after the first 10 entries & then the others continue.
As in the entries are not in a continuious block, other entries are present after entry 9 then it continues with Entry 10 etc.
Code: Select all
[FTP site]
Host=ftp.inverellit.com
Port=21
Username=
Password=
Directory=public_html/weather
AutoUpdate=1
ActiveFTP=1
EnableRealtime=1
RealtimeFTPEnabled=1
RealtimeTxtFTP=1
RealtimeInterval=5000
UpdateInterval=10
IncludeSTD=1
IncludeSTDImages=1
FTPRename=1
DeleteBeforeUpload=0
ResizeGraphs=0
GraphHeight=300
GraphWidth=750
ImageFolder=
ImageCopyRealtime=0
ExtraLocal0=\CumulusMX\web\gauges-ssT.htm
ExtraRemote0=/public_html/weather/gauges-ss.htm
ExtraProcess0=1
ExtraBinary0=0
ExtraRealtime0=0
ExtraFTP0=1
ExtraLocal1=\CumulusMX\web\realtimeT.xml
ExtraRemote1=/public_html/weather/realtime.xml
ExtraProcess1=1
ExtraBinary1=0
ExtraRealtime1=1
ExtraFTP1=1
ExtraLocal2=\CumulusMX\web\sunbirdT.txt
ExtraRemote2=/public_html/weather/sunbird.txt
ExtraProcess2=1
ExtraBinary2=0
ExtraRealtime2=1
ExtraFTP2=1
ExtraLocal3=\CumulusMX\web\index2t.htm
ExtraRemote3=/public_html/weather/index2.htm
ExtraProcess3=1
ExtraBinary3=0
ExtraRealtime3=0
ExtraFTP3=1
ExtraLocal4=\CumulusMX\web\davcon24T.txt
ExtraRemote4=/public_html/weather/davcon24.txt
ExtraProcess4=1
ExtraBinary4=0
ExtraRealtime4=1
ExtraFTP4=1
ExtraLocal5=\CumulusMX\web\davconfcstT.txt_MX
ExtraRemote5=/public_html/weather/davconfcst.php
ExtraProcess5=1
ExtraBinary5=0
ExtraRealtime5=0
ExtraFTP5=1
ExtraLocal6=\CumulusMX\web\davcon24T.txt
ExtraRemote6=/public_html/weather2/davcon24.txt
ExtraProcess6=1
ExtraBinary6=0
ExtraRealtime6=0
ExtraFTP6=1
ExtraLocal7=\CumulusMX\web\davconfcstT.txt_MX
ExtraRemote7=/public_html/weather2/davconfcst.php
ExtraProcess7=1
ExtraBinary7=0
ExtraRealtime7=0
ExtraFTP7=1
ExtraLocal8=\CumulusMX\Cumuluswebtags.txt
ExtraRemote8=\CumulusMX\Cumuluswebtags.php
ExtraProcess8=1
ExtraBinary8=0
ExtraRealtime8=0
ExtraFTP8=0
ExtraLocal9=\CumulusMX\web\graphsT.htm
ExtraRemote9=/public_html/weather/graphs.htm
ExtraProcess9=1
ExtraBinary9=0
ExtraRealtime9=0
ExtraFTP9=1
ExternalProgram=
RealtimeProgram=
ExternalParams=
RealtimeParams=
DailyProgram=C:\CumulusMX\Scripts\DayEnd.bat
DailyParams=
IncludeSolarChart=0
IncludeUVChart=0
IncludeSunshineChart=0
UTF8encode=1
ExtraUTF0=0
ExtraUTF1=0
ExtraUTF2=0
ExtraUTF3=0
ExtraUTF4=0
ExtraUTF5=0
ExtraUTF6=0
ExtraUTF7=0
ExtraUTF8=0
ExtraUTF9=0
Sslftp=0
FTPlogging=0
IncludeGraphDataFiles=1
ExtraLocal10=\CumulusMX\web\kindleT.htm
ExtraRemote10=/public_html/weather/kindle.htm
ExtraProcess10=1
ExtraBinary10=0
ExtraRealtime10=1
ExtraFTP10=1
ExtraUTF10=0
ExtraLocal11=\CumulusMX\web\datasummaryT.htm
ExtraRemote11=/public_html/weather/datasummary.php
ExtraProcess11=1
ExtraBinary11=0
ExtraRealtime11=0
ExtraFTP11=1
ExtraUTF11=1
ExtraLocal12=\CumulusMX\data\dayfile.txt
ExtraRemote12=/public_html/weather/dayfile.txt
ExtraProcess12=0
ExtraBinary12=0
ExtraRealtime12=0
ExtraFTP12=1
ExtraUTF12=0
ExtraLocal13=\CumulusMX\web\today-yesterdayT.htm
ExtraRemote13=/public_html/weather/today-yesterday.htm
ExtraProcess13=1
ExtraBinary13=0
ExtraRealtime13=0
ExtraFTP13=1
ExtraUTF13=0
ExtraLocal14=\CumulusMX\web\gT.htm
ExtraRemote14=/public_html/weather/g.htm
ExtraProcess14=1
ExtraBinary14=0
ExtraRealtime14=0
ExtraFTP14=1
ExtraUTF14=0
ExtraLocal15=\CumulusMX\web\betel_monthlyrecord_mxT.html
ExtraRemote15=/public_html/weather/betel_monthlyrecord_mx.html
ExtraProcess15=1
ExtraBinary15=0
ExtraRealtime15=0
ExtraFTP15=1
ExtraUTF15=0
ExtraLocal16=\CumulusMX\utils\systeminfoTable.txt
ExtraRemote16=/public_html/weather/sysinfoTable.txt
ExtraProcess16=1
ExtraBinary16=0
ExtraRealtime16=1
ExtraFTP16=1
ExtraUTF16=1
ExtraLocal17=\CumulusMX\web\gaugesT.htm
ExtraRemote17=/public_html/weather/gauges.htm
ExtraProcess17=0
ExtraBinary17=0
ExtraRealtime17=0
ExtraFTP17=0
ExtraUTF17=0
ExtraLocal18=
ExtraRemote18=
ExtraProcess18=0
ExtraBinary18=0
ExtraRealtime18=0
ExtraFTP18=0
ExtraUTF18=0
ExtraLocal19=
ExtraRemote19=
ExtraProcess19=0
ExtraBinary19=0
ExtraRealtime19=0
ExtraFTP19=0
ExtraUTF19=0
ExtraLocal20=
ExtraRemote20=
ExtraProcess20=0
ExtraBinary20=0
ExtraRealtime20=0
ExtraFTP20=1
ExtraUTF20=0
Phil.
Re: Extra Webfiles Missing
Posted: Tue 03 Dec 2019 3:02 am
by Phil23
Further investigations has found I also lost the Realtimegauges.txt setting & all my NOAA Temperatures & Rainfall values were missing.
The process of shutting down & restarting MX twice a day has been going on for many months, so can't see it having any relevance to this issue.
Re: Extra Webfiles Missing
Posted: Tue 03 Dec 2019 11:36 am
by mcrossley
Can't help with the missing entires, but the out of order is just because new options have been added to Extra Files feature. They get added after any existing ones.
You *can* clean it up and make it sequential (purely for ease of human reading) by renaming Cumulus.ini to something like Cumulsu.ini.bak whilst MX is running, and then either making a minor change to a config item (and reversing it), or shutting down MX. It will then write out a new Cumulus.ini in a logical order.
Adding the ini file to backup is a good idea.
Re: Extra Webfiles Missing
Posted: Tue 03 Dec 2019 8:07 pm
by Phil23
mcrossley wrote: ↑Tue 03 Dec 2019 11:36 am
You *can* clean it up and make it sequential (purely for ease of human reading) by renaming Cumulus.ini to something like Cumulsu.ini.bak whilst MX is running, and then either making a minor change to a config item (and reversing it), or shutting down MX. It will then write out a new Cumulus.ini in a logical order.
Thanks Mark,
That will be a whole lot easier than the manual editing I was thinking of. Maybe a worthwhile addition to the Wiki.
Incidentally I tried to register on the Wiki some months back & kept failing.
Hoping you'll add the ini to the files stored in the backup folder for others in the future.
Cheers.
Re: Extra Webfiles Missing
Posted: Wed 04 Dec 2019 10:42 am
by mcrossley
One caveat with that ini file process - you will lose any settings that are "ini file only" - i.e. are only ever read by Cumulus and never written.
Options such as CreateWxnowTx, SpecialLog etc, etc
So be careful and check your old file against the new to see if anything has "gone" that you need. These options *must* be added back in to the new with Cumulus stopped.
I guess a safer option would be to get Cumulus to generate the new file - rename that, rename the original back, and cut'n'paste just the FTP section back into the original file.
Re: Extra Webfiles Missing
Posted: Thu 05 Dec 2019 5:20 am
by Phil23
mcrossley wrote: ↑Wed 04 Dec 2019 10:42 am
I guess a safer option would be to get Cumulus to generate the new file - rename that, rename the original back, and cut'n'paste just the FTP section back into the original file.
I'm Assuming it would be a fairly major re-write to have MX Rename the ini file to bak & then write a the new ini file each time in a logical order?
Re: Extra Webfiles Missing
Posted: Thu 05 Dec 2019 9:10 am
by mcrossley
Yes, mainly because there are possibly lots of settings in there that are read-only.
Re: Extra Webfiles Missing
Posted: Thu 05 Dec 2019 10:07 am
by HansR
@mcrossley: You might want to modify the ini-library such, that if a parameter does not exist, it will be created when read by the program with a certain default.
I did modify the library as such and therefore, I am not interested in what is being done by the user. Any deletion will be corrected with an addition of that parameter with a certain default. Especially for read-only parameters (why have these in the ini in the first place?) that would be an easy solution. In other cases the user may have to correct the default again.
Re: Extra Webfiles Missing
Posted: Thu 05 Dec 2019 7:50 pm
by Phil23
Has Happened again, but I got a completely blank default ini file this time.
Not even the station is set.
Stations last update was 12:00AM, just before a scheduled midnight shutdown.
The shutdown is run by the task scheduler which runs a compiled AutoHotKey script.
All that script does is put focus on the MX window & then send it Ctrl-C,
so I'm not doing anything dopey like simply killing a task.
Add to that this has been running for over 12 months with no issues.
Fortunately I'd upgraded to Version 3059 yesterday arvo, so could just retrieve the ini file from a 6:00pm backup.
(Thanks Mark).
Getting the impression it's come about as of a certain update number, but it's still only two random occurrences.
Phil
Re: Extra Webfiles Missing
Posted: Thu 05 Dec 2019 8:23 pm
by Phil23
Ok,
Been looking at the diags logs & looks like my host had a FTP issue just before the 6:00 PM shutdown.
The 6:00pm shutdown wrote the ini correctly,
but the 12:00am one wrote it at zero bytes.
Seems that MX may have been very busy trying to re-establish connections.
I'm assuming at the 6:00 am restart MX wrote the default Ini file having had nothing to read from the blank one written at mindnight.
Just detail on the restarts; I shut MX down routinely to allow weatherlink to download updates & then exit.
MX is then restarted one minute later; 12:01am, 6:01am etc.
But again, that's been working for months....
3 diag files attached.
Cheers.
Re: Extra Webfiles Missing
Posted: Thu 05 Dec 2019 9:37 pm
by Phil23
Don't think this is totally related;
After upgrading to 3059 @ 3:55pm everything was working fine.
Then @ 5:49pm realtime uploads started failing.
But still, the 6:00pm shutdown wrote the ini correctly.
Code: Select all
2019-12-05 17:40:00.908 Writing log entry for 5/12/2019 5:40:00 PM
2019-12-05 17:40:00.922 Written log entry for 5/12/2019 5:40:00 PM
2019-12-05 17:40:00.923 Writing today.ini, LastUpdateTime = 5/12/2019 5:40:00 PM raindaystart = 215.2 rain counter = 215.2
2019-12-05 17:40:01.487 Extra web file #8 [\CumulusMX\Cumuluswebtags.txt] not found!
2019-12-05 17:42:25.666 Reading reception stats
2019-12-05 17:49:03.489 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:08.497 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:13.498 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:18.501 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:23.512 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:28.513 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:28.598 Error connecting ftp - Timed out trying to connect!
2019-12-05 17:49:33.513 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:38.513 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:43.514 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:43.599 Error uploading realtime.txt to public_html/weather/realtime.txt : Timed out trying to connect!
2019-12-05 17:49:48.516 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:53.516 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:58.517 Warning, previous realtime ftp still in progress, skipping this period.
2019-12-05 17:49:58.618 Error uploading web\realtimegauges.txt to public_html/weather/realtimegauges.txt : Timed out trying to connect!
2019-12-05 17:50:00.395 Writing log entry for 5/12/2019 5:50:00 PM
2019-12-05 17:50:00.484 Written log entry for 5/12/2019 5:50:00 PM
FTP errors on realtime files continued till about 6:18pm then came good,
But then reappeared at 21:38 for a couple of minutes.
Then there were not apparent errors for the next few hours up till the midnight shutdown that clobbered the ini file.
Phil
Re: Extra Webfiles Missing
Posted: Thu 05 Dec 2019 10:47 pm
by mcrossley
Thanks for the files, I'll take a look over the next day or so. CMX must be starting the process of writing the ini but terminating before it completes for some reason.
Re: Extra Webfiles Missing
Posted: Sun 23 Feb 2020 10:39 pm
by Phil23
Seem to have lost some Ini settings again.
First notice graph hours back at 24 Vs 168.
Also Upload credentials for other services & NOAA Values.
Tracking back thru the backups ATM, but it seems lit it's been this way since before 22/01/2020, which may correspond with a recent upgrade.
Would need to check when I downloaded recent updates.
Cheers
Phil.
Edit:-
WeatherCloud notified me that it went down 19/02/2020 @ 11:50pm; not sure what time zone that is, but my downloads show I downloaded 3064 on 20/02/2020 @ 5:43am AEST (GMT +11). Not sure if it's related.
Phil.
Re: Extra Webfiles Missing
Posted: Mon 24 Feb 2020 8:19 pm
by Phil23
mcrossley wrote: ↑Tue 03 Dec 2019 11:36 amYou *can* clean it up and make it sequential (purely for ease of human reading) by renaming Cumulus.ini to something like Cumulsu.ini.bak whilst MX is running, and then either making a minor change to a config item (and reversing it), or shutting down MX. It will then write out a new Cumulus.ini in a logical order.
Tried this yesterday; copied my entire MX directory to another PC & carried it out there.
Simply renamed the INI while MX was running then shut MX down.
Interesting when I compare the 2 files with diffMerge, (Thx BetelJuice), there is a very big difference in the placement of the data blocks & also the position of the entries within those blocks. Not sure whether that would impact at all on timings, or potential errors from the previous; that's one only for the programmers.
My original ini still contains some Cumulus 1 entries also that can probably be dispensed with, particularly references to EasyWeather.
Might look at rebuilding my ini file from scratch, as the only notable changes seem to be the omission of the Com3 entry (as the station did not connect)
& the enrty for LastLoggerDownload.
Is the above entry the only one that changes when MX is Started & stopped without config changes, or are there other entries to consider?
Adding the ini file to backup is a good idea.
Unfortunately the added above backup didn't help me in this case as the copy I required had expired but the time I discovered the issue.
Is an ini setting for
number of backups number of days backups are retained feasible, given that MX is often unattended for long periods?
In the mean time I might just schedule a task to copy them out of where they live & let them accumulate in another folder.
(And again search for a decent Win8 backup replacement).
Cheers
Phil.