My ftp to the web stalls frequently

[jandrew]

Hi folks

I have a problem with the ftp of my cumulus files to the web, and I am not sure why or what is happening.

The background is that over the last 10 days or so I appear to have been having interruptions to the smooth working of my Hub5 (BT Affinity) although it has worked very well for the previous 3 months.
These interruptions might be faults (or BT working) on the broadband line, or faults with the Hub5 or some malware on my computer. I am slowly researching these alternatives.
On one occasion the Hub5 has completely reset, more often (every two/three days) my IPAddress is changed, and I am sure that there are more interruptions for which there is no permanent evidence.

It seems to me that these interruptions are having a particular effect on the ftp of my cumulus files (but I'm not sure).

The ftp works perfectly for several uploads (they are every 30 minutes) but then I get a failure. I can get back to normal working by closing then immediately restarting cumulus upon which the ftp works again until the next failure.
I am getting failures at the rate of 2 or 3 a day - even when I have no permanent evidence of an interruption on the Hub5 (the IPAddress has not changed during that period) - the cumulus ftp failure might, of course, have nothing to do with the interruptions, but might have an entirely different cause.

It has done it again this morning. Perfect upload at 03.00, failure at 03.30, with no change in the IPAddress.

When the ftp is working properly, the first file uploaded is "cumuluswebtags.txt" followed seamlessly by "realtime.txt".
But when I have a failure the "cumuluswebtags.txt" file may or may not fully upload but the ftp then stalls and does not move on to uploading "realtime.txt".
Instead I get an error message :

FTP error
Already disconnected"

I did an "ftplog" at 20.56 a couple of days ago when the ftp was working, and then an "ftplog" at 9.05 the following morning by which time the ftp was failing. I attach a zip file of that ftplog here :.

ftplog.zip

Does anybody have any clues about what is happening - or even a solution?

[steve]

All I can tell you from the log is that the ftp component in Cumulus timed out the connection because it had no response from the server for 20 seconds. It then attempts to abort the transfer and disconnect. It should then just try again at the next upload, but if you have to restart Cumulus to get it working again, it looks like the connection gets 'stuck' somehow and can't be disconnected automatically.

[OhioRVHawk]

I just found this problem about 5 months ago. maybe this will help

I had Cumulus running along with Yacam. I screwed up with some settings, and my web host told me about it. Sometimes if you have the active set, it will not close properly. and then you will get FTP errors. try to set it up so that it is not using Active FTP, uncheck it. and let it run to see if you have any more errors in your FTP. if you do not then for some reason your FTP is not disconnecting properly. talk to your hosting site to see what they may recommend. it may even be on their end as well. I had to have them se-bot the FTP server for me, and check things out. after all that. I was able to re-check the active FTP again.

hope it helps

[jandrew]

I think that I have sorted my problem - but then I find another

At the start-up of cumulus.exe the file "cumuluswebtags.txt" starts off quite small but then enlarges as data is added.
The ftp to my website works perfectly (with "cumuluswebtags.txt" as the first file to be uploaded) for several hours until that files gets to about 66K then it fails.
Restarting cumulus.exe resets that file to the small size which is why the ftp starts working again.
So far so good I just find a way to ftp large files.
I have asked my provider who says "no problem - change the ftp setting to - FTPES (FTP explicit SSL)".

So now I have a different problem. How do I make this change in Cumulus?

I have done it for my Filezilla uploads. Just change the settings to :
Protocol ====== FTP - File Transfer Protocol
Encryption ==== Require explicit FTP over TLS

I can also do it on Dreamweaver.

Is there an expert out there who can tell me how to do this for the automatic Cumulus uploads?

[steve]

Cumulus doesn't support FTP over TLS. You can use WinSCP to do it, as described in the wiki - http://wiki.sandaysoft.com/a/SFTP

You could also edit the webtags file to remove the ones that increase in size - <#wspddata> and <#wdirdata>, if you're not using them, as a workaround. Did your provider not say what the problem is with their server?

[jandrew]

Thanks. Will look at the WIKI.

The provider says that they accept uploads ONLY by FTPES.

I have now heard that other providers do this as well.

[steve]

Yes, a handful of other Cumulus users have hosting where they insist on FTPS or SFTP (although in some cases, one of those is the default, but you can enable plain FTP). I don't think I've come across a case where plain FTP is supported up to a particular file size before.

The problem with providers not supporting plain FTP is that for developers of free software like myself, apart from the time required to add support for these protocols, typically there are no reliable free or cheap components available. But at some point I will look at adding support to the new version of Cumulus if I can.

[jandrew]

Thanks Steve.

Don't get me wrong, I wasn't suggesting that you make any changes to your software. If I have a problem I expect to have to sort it myself - but as I get older, my brain begins to ache and solutions are not as easily found as they used to be.

I have read the WIKI. I shall need to work hard on that.

The temporary answer is to edit the webtags file as you suggest. I wonder how
Edit:12:07 managed to edit the file.