partial web page updating failure

[archae86]

I imagine my cumulus installation must have become corrupt. Sadly, my primary system backup (which updates changed copies of most directories of my system on another system on my network daily) has already run since the problem began.

Details:

So far as I can tell my postings to Wunderground are running OK. That site is at:
https://www.wunderground.com/personal-w ... NMALBUQ102

However my Cumulus-generated web pages at:
http://pastoll.info/weather/index.htm
Seem currently only to update the now (index), and today pages properly. Gauges, this month, this year, records, monthly records, and trends (which I call graphs) all display a copy now about ten hours old and aging.
The Yesterday page displays on my Firefox browser as blank.

I've zipped up the diagnostic folder and attached it to this post.

Any suggestions as to how I might best recover gratefully received.

[steve]

I can't look at the diags files at the moment with the tech available to me while on holiday, but I can have a look when I return next week. Do I remember correctly that you are using my hosting?

You could have a look in the diags files to see if any errors are logged when the upload occurs, and also turn on ftp logging to see if the ftplog.txt file gives any clues.

If the problem is that the Cumulus files on your PC have been corrupted, you could just reinstall Cumulus.

[archae86]

Yes, I use your web hosting.

There seemed something odd in the popup window when I did a forced web update just after restarting Cumulus, so I think the idea of turning on ftp logging is the first thing I'll try when I get back from piano lesson, along with looking into the diag files.

I actually have no other evidence that my files are corrupted save for the bad results I reported.

I'll report back in a few hours.

[mcrossley]

You are getting a solid FTP error 426 = "Connection closed".
Looking back through the older logs this has been happening intermittently since they start on 23 May, so maybe it is a server issue?
Are you using passive or active FTP?

[archae86]

Summary: it seems my problem is was specifically in the FTP process, as the html files generated by Cumulus on my own machine appear to exist.
After some fiddling documented below, it appears things are working properly again.

Details:

The most recent diag file shows a great many entries during startup, which may possibly reveal a problem but not one I can notice.

Then, it falls into a repeating pattern at the repetition interval of 5 minutes (my chosen web publishing interval). I'll show just one chosen of the middle:

6/14/2016 08:35:00.951 : Writing today.ini, LastUpdateTime = 6/14/2016 8:35:00 AM raindaystart = 2.71000003814697 rain counter = 2.71000003814697
6/14/2016 08:35:36.215 : FTPRequestDone ftpPutAsync Errcode = 426

A comparison with older diag files predating my problem suggests that a normal appearance would be of this nature:

6/10/2016 12:25:00.920 : Writing today.ini, LastUpdateTime = 6/10/2016 12:25:00 PM raindaystart = 2.6800000667572 rain counter = 2.6800000667572

In other words, the line containing Errcode = 426 is new, and is in some sense a symptom of my problem.

As advised, I turned on FTP logging.

Paying attention to a single web update attempt, I see normal-appearing sections relating to the successful FTP transfer of the first two page files (index and today)

Here I show the section relating to the successful transfer of today.tm

Code: Select all

13:33:03:550 ! Next3PutAsync
13:33:03:550 ! HighLevelAsync 0
13:33:03:550 ! HighLevelAsync done
13:33:03:664 Uploading F:\Tools\Cumulus\web\today.htm to today.htm
13:33:03:664 ! HighLevelAsync 0
13:33:03:664 FTP Command: PASV
13:33:03:664 052D7750 TriggerDataSent 612
13:33:03:803 FTP Response: 227 Entering Passive Mode (81,4,106,218,156,3).
13:33:03:803 >|227 Entering Passive Mode (81,4,106,218,156,3).|

13:33:03:803 ! HighLevelAsync 0
13:33:03:803 052D7F10 Socket handle created 1260
13:33:03:803 TWSocket will connect to 81.4.106.218:39939
13:33:03:947 ! Data Session opened (Put)
13:33:03:947 FTP Command: STOR today.htm
13:33:03:947 052D7F10 TriggerDataSent 1260
13:33:03:947 052D7750 TriggerDataSent 612
13:33:04:088 FTP Response: 150 Ok to send data.
13:33:04:088 >|150 Ok to send data.|

13:33:04:088 ! Send first block
13:33:04:088 DataSocketPutDataSent 6513
13:33:04:088 052D7F10 TriggerDataSent 1260
13:33:04:088 DataSocketPutDataSent 0
13:33:04:088 052D7F10 TCustomWSocket.Shutdown 1 1260
13:33:04:369 ! Data Session closed
13:33:04:369 ! Next3PutAsync
13:33:04:369 052D7F10 TCustomWSocket.Shutdown 1 1260
13:33:04:507 FTP Response: 226 Transfer complete.
13:33:04:507 >|226 Transfer complete.|

Immediately following this success there is logged the failed attempt to transfer yesterday.htm, which to the end of the file reads thus:

Code: Select all

13:33:04:507 ! Next3PutAsync
13:33:04:507 ! HighLevelAsync 0
13:33:04:507 ! HighLevelAsync done
13:33:04:664 Uploading F:\Tools\Cumulus\web\yesterday.htm to yesterday.htm
13:33:04:664 ! HighLevelAsync 0
13:33:04:664 FTP Command: PASV
13:33:04:664 052D7750 TriggerDataSent 612
13:33:04:806 FTP Response: 227 Entering Passive Mode (81,4,106,218,64,239).
13:33:04:806 >|227 Entering Passive Mode (81,4,106,218,64,239).|

13:33:04:806 ! HighLevelAsync 0
13:33:04:806 052D7F10 Socket handle created 876
13:33:04:806 TWSocket will connect to 81.4.106.218:16623
13:33:04:949 ! Data Session opened (Put)
13:33:04:949 FTP Command: STOR yesterday.htm
13:33:04:949 052D7F10 TriggerDataSent 876
13:33:04:950 052D7750 TriggerDataSent 612
13:33:35:666 ! Aborting
13:33:35:666 FTP Session Closed
13:33:35:666 ! HighLevelAsync 0
13:33:35:666 ! Abort detected
13:33:35:666 ! HighLevelAsync done
13:33:35:666 052D7F10 TCustomWSocket.Shutdown 1 876
13:33:35:666 ! Data Session closed
13:33:35:666 ! Next3PutAsync
13:33:35:666 FTPRequestDone ftpPutAsync Errcode = 426

To my inexpert eye, the FTP log looks normal right up to the line containing "Aborting".

Clearly there is trouble transferring the needed htm files from my PC to the hosting server, but I am pretty completely ignorant of all things FTP, so have not much of an idea of what might be wrong at my end, in transmission, or at the other end to give this set of outcomes.

As an experiment I connected to the server using FireFTP (an addon to FireFox I've used for non-weather purposes for some years).

It showed yesterday.htm as a zero-sized file on the server with time stamp the same as the (correctly transferred) today and index. It showed the other page htm files with much older time stamps--the time they had last been successfully updated.

I tried commanding yesterday.htm to be deleted by FireFTP, then transferring the current copy using it. The deletion and transfer both worked, and the updated yesterday page was viewable as linked from a link on another page.

So I deleted all the stale files, and transferred their (slightly stale) latest versions by FireFTP.

This all seemed to work. As FireFTP deep down inside seems to use the same basic transfer mechanism as does Cumulus, this seems to point suspicion back at the current state of Cumulus installation on this PC. Or perhaps the "zero-size" yesterday.htm which somehow got on the server was the repeating stumbling block, in which case my deleting it may just have been the healing step.

Before pressing the "submit" button on this post I decided to restart Cumulus and wait through a normal 5-minute update cycle. To my surprise, it seemed to go correctly, and properly updated pages seem to be served out by the server.

So maybe something I did in all this "healed" the problem, which is my excuse for leaving the verbose string of events in this post.

[mcrossley]

I have seen that a few times, when Cumulus is abnormally terminated, or the FTP interrupted it leaves a zero length file on your web server. All subsequent uploads of that file (and any subsequent ones in the same upload session) then fail until you manually delete it off the server.

The problems seems to be the FTP server returning the length of the file that is zero length causes an error condition.

[archae86]

I have seen that a few times, when Cumulus is abnormally terminated, or the FTP interrupted it leaves a zero length file on your web server.

Is it perhaps then prudent to shut down Cumulus by explicit exit from the GUI rather than relying on orderly shut-down by Windows when doing a system restart or shut down? I've not been doing that, and maybe I've just been lucky until now.

[steve]

It's possible that Windows does not allow Cumulus to fully close cleanly if you rely on it doing it as part of a restart or shutdown. I strongly recommend closing programs manually before doing a Windows restart or shutdown.

[archae86]

Steve,

Thanks for the shut down advice, and thanks for the earlier debug suggestions that nudged me in directions which got me up and running again.

However in this specific incident I had not booted the machine since June 6, so however unwise my shutdown procedure may have been it was not the proximate cause. Possibly a transient communication problem somewhere between the application on my PC and the file on the server interrupted things at just the inconvenient moment.