Upload errors

[freddie]

I'm seeing the following sort of error message in the diags file:

2016-01-25 09:11:34.270 Error uploading winddata.json to /var/www/weather/winddata.json : Timed out trying to read data from the socket stream!

I only see it for the wind speed and pressure JSON files, and also for the realtime.xml file. All of these files last uploaded around two days ago, and are currently listed on the web server as zero byte files:

-rw-r--r-- 1 www www 0 Jan 23 05:06 realtime.xml
-rw-r--r-- 1 www www 0 Jan 23 05:07 pressdata.json
-rw-r--r-- 1 www www 0 Jan 23 05:10 winddata.json

Permissions are okay for an overwrite (or even a delete).

All other files transferring okay. All above data files on the MX box contain data.

Any ideas?

[steve]

As always, for ftp problems turn on ftp logging and look at ftplog.txt, it usually (but not always) gives more information. Posting your ftp settings may also help diagnosis.

[freddie]

As always, for ftp problems turn on ftp logging and look at ftplog.txt, it usually (but not always) gives more information.

I hadn't done this as I thought that a restart (required for the FTP logging to work) would destroy evidence. I was basically asking the question to see if it triggered recognition of similar problems with other users.
I have now switched on logging and restarted MX. As I thought would happen, the problem has now gone away. Nevertheless, I will keep the logging switched on in the hope that the problem will recur.
Is there any way of being able to toggle the FTP logging without a restart?

Posting your ftp settings may also help diagnosis.

These haven't changed for a long time - but I will post them for completeness:

[FTP site]
Host=ext.hosiene.co.uk
Port=21
Username=[REMOVED]
Password=[REMOVED]
Directory=/var/www/weather
AutoUpdate=1
ActiveFTP=0
EnableRealtime=1
RealtimeFTPEnabled=1
RealtimeTxtFTP=1
RealtimeInterval=2000
UpdateInterval=1
IncludeSTD=1
IncludeSTDImages=1
IncludeSolarChart=0
IncludeUVChart=0
IncludeSunshineChart=0
FTPRename=0
DeleteBeforeUpload=0
ResizeGraphs=0
GraphHeight=300
GraphWidth=750
ImageFolder=
ImageCopyRealtime=0
ExtraLocal0=/opt/cumulus/web/cumulusxml.tpl
ExtraRemote0=/var/www/weather/realtime.xml
ExtraProcess0=1
ExtraBinary0=0
ExtraRealtime0=1
ExtraFTP0=1
ExtraLocal1=/opt/cumulus/web/cumulus.xml
ExtraRemote1=/var/www/weather/cumulus.xml
ExtraProcess1=1
ExtraBinary1=0
ExtraRealtime1=1
ExtraFTP1=1
ExtraLocal2=/opt/cumulus/web/realtimegaugesT.txt
ExtraRemote2=/var/www/weather/realtimegauges.txt

[Truncated as none of the other FTP settings are in use]

[steve]

restarted MX. As I thought would happen, the problem has now gone away.

Well, I didn't expect that, given the error message, and I can't think of any explanation for it.

Is there any way of being able to toggle the FTP logging without a restart?

Not currently, but in future - as with a number of other settings currently requiring a restart, it's just a question of finding the time to write the code.

Posting your ftp settings may also help diagnosis.

These haven't changed for a long time - but I will post them for completeness:

I'm not saying that they have, but my point still stands, your FTP settings (whether you're using 'rename' etc) will aid diagnosis of an FTP issue whether they've changed recently or not.

[freddie]

restarted MX. As I thought would happen, the problem has now gone away.

Well, I didn't expect that, given the error message, and I can't think of any explanation for it.

I thought it would, as there was no difference between the attributes/permission of the successful/unsuccessful files at either end, and size didn't appear to be a trigger (there were files that were successfully transferred that were larger than those files that were failing), and it was the same files failing each time. What I had surmised (rightly or wrongly) was that something had happened around 05:06 on Saturday to cause the three transfers to fail, and that MX was unable to perform any further transfers on those files for some reason. FTP transfers for all other data continued to work just fine. This led me to believe there was a resource-releasing type problem which tends to go away when the program is terminated and restarted. I guess we will never know the detail, as FTP logging wasn't enabled at the time.

Is there any way of being able to toggle the FTP logging without a restart?

Not currently, but in future - as with a number of other settings currently requiring a restart, it's just a question of finding the time to write the code.

That will be useful, thanks.

Posting your ftp settings may also help diagnosis.

These haven't changed for a long time - but I will post them for completeness:

I'm not saying that they have, but my point still stands, your FTP settings (whether you're using 'rename' etc) will aid diagnosis of an FTP issue whether they've changed recently or not.

Fair point.