FTP problems, narrowing down

[balt]

Ok, after Cumulus has been running for a couple of days, the FTP upload once again stopped. I didn't have logging enabled, so I enabled it without restarting the app. It never even started to create a log. I then noticed some other peculiar behaviour of the computer pointing at network issues. So I executed a "netstat -a -p tcp" in a console window, only to find out that every single port on the computer had an application listening. I then quit Cumulus, and the ports became free again.

This leaves two options:

1. The Cumulus install I have downloaded has been security compromised and is carrying a trojan load (unknown to the avast virus scanner I have installed)

2. More likely: A programming "feature" is causing Cumulus to listen on all sorts of ports. Note that I have ftp uploads, wunderground as well as PWS uploads enabled.

Cheers

- Balt


P.S. This is "netstat -a -p tcp" output without Cumulus running:

ctive Connections

Proto Local Address Foreign Address State
TCP EeePC:ssh EeePC:0 LISTENING
TCP EeePC:epmap EeePC:0 LISTENING
TCP EeePC:microsoft-ds EeePC:0 LISTENING
TCP EeePC:2869 EeePC:0 LISTENING
TCP EeePC:5800 EeePC:0 LISTENING
TCP EeePC:5900 EeePC:0 LISTENING
TCP EeePC:ssh yeti.atnf.csiro.au:62351 ESTABLISHED
TCP EeePC:netbios-ssn EeePC:0 LISTENING
TCP EeePC:1025 px-in-f101.google.com:http ESTABLISHED
TCP EeePC:1027 a-61-9-193-135.deploy.akamaitechnologies.com:http ESTABLISHED
TCP EeePC:5900 yeti.atnf.csiro.au:50368 ESTABLISHED
TCP EeePC:1030 EeePC:0 LISTENING
TCP EeePC:1053 localhost:27015 ESTABLISHED
TCP EeePC:5152 EeePC:0 LISTENING
TCP EeePC:5354 EeePC:0 LISTENING
TCP EeePC:27015 EeePC:0 LISTENING
TCP EeePC:27015 localhost:1053 ESTABLISHED
TCP EeePC:62514 EeePC:0 LISTENING


And this is the output after 10 minutes of Cumulus running:


Active Connections

Proto Local Address Foreign Address State
TCP EeePC:ssh EeePC:0 LISTENING
TCP EeePC:epmap EeePC:0 LISTENING
TCP EeePC:microsoft-ds EeePC:0 LISTENING
TCP EeePC:1045 EeePC:0 LISTENING
TCP EeePC:1046 EeePC:0 LISTENING
TCP EeePC:1047 EeePC:0 LISTENING
TCP EeePC:1048 EeePC:0 LISTENING
TCP EeePC:1049 EeePC:0 LISTENING
TCP EeePC:1050 EeePC:0 LISTENING
TCP EeePC:1051 EeePC:0 LISTENING
TCP EeePC:1052 EeePC:0 LISTENING
TCP EeePC:1054 EeePC:0 LISTENING
TCP EeePC:1055 EeePC:0 LISTENING
TCP EeePC:1056 EeePC:0 LISTENING
TCP EeePC:1057 EeePC:0 LISTENING
TCP EeePC:1058 EeePC:0 LISTENING
TCP EeePC:1059 EeePC:0 LISTENING
TCP EeePC:1060 EeePC:0 LISTENING
TCP EeePC:1061 EeePC:0 LISTENING
TCP EeePC:1062 EeePC:0 LISTENING
TCP EeePC:1063 EeePC:0 LISTENING
TCP EeePC:1064 EeePC:0 LISTENING
TCP EeePC:1065 EeePC:0 LISTENING
TCP EeePC:1066 EeePC:0 LISTENING
TCP EeePC:1067 EeePC:0 LISTENING
TCP EeePC:1068 EeePC:0 LISTENING
TCP EeePC:1069 EeePC:0 LISTENING
TCP EeePC:1070 EeePC:0 LISTENING
TCP EeePC:1071 EeePC:0 LISTENING
TCP EeePC:1072 EeePC:0 LISTENING
TCP EeePC:1073 EeePC:0 LISTENING
TCP EeePC:1099 EeePC:0 LISTENING
TCP EeePC:1100 EeePC:0 LISTENING
TCP EeePC:1101 EeePC:0 LISTENING
TCP EeePC:1102 EeePC:0 LISTENING
TCP EeePC:1103 EeePC:0 LISTENING
TCP EeePC:1104 EeePC:0 LISTENING
TCP EeePC:1105 EeePC:0 LISTENING
TCP EeePC:1106 EeePC:0 LISTENING
TCP EeePC:1107 EeePC:0 LISTENING
TCP EeePC:1108 EeePC:0 LISTENING
TCP EeePC:kpop EeePC:0 LISTENING
TCP EeePC:1110 EeePC:0 LISTENING
TCP EeePC:1111 EeePC:0 LISTENING
TCP EeePC:1112 EeePC:0 LISTENING
TCP EeePC:1113 EeePC:0 LISTENING
TCP EeePC:1114 EeePC:0 LISTENING
TCP EeePC:1115 EeePC:0 LISTENING
TCP EeePC:1116 EeePC:0 LISTENING
TCP EeePC:1117 EeePC:0 LISTENING
TCP EeePC:1118 EeePC:0 LISTENING
TCP EeePC:1119 EeePC:0 LISTENING
TCP EeePC:1120 EeePC:0 LISTENING
TCP EeePC:1121 EeePC:0 LISTENING
TCP EeePC:1122 EeePC:0 LISTENING
TCP EeePC:1123 EeePC:0 LISTENING
TCP EeePC:1124 EeePC:0 LISTENING
TCP EeePC:1125 EeePC:0 LISTENING
TCP EeePC:1126 EeePC:0 LISTENING
TCP EeePC:1127 EeePC:0 LISTENING
TCP EeePC:2869 EeePC:0 LISTENING
TCP EeePC:5800 EeePC:0 LISTENING
TCP EeePC:5900 EeePC:0 LISTENING
TCP EeePC:ssh yeti.atnf.csiro.au:62351 ESTABLISHED
TCP EeePC:netbios-ssn EeePC:0 LISTENING
TCP EeePC:1085 cds836.lax.llnw.net:http ESTABLISHED
TCP EeePC:5900 yeti.atnf.csiro.au:50368 ESTABLISHED
TCP EeePC:1030 EeePC:0 LISTENING
TCP EeePC:1053 localhost:27015 ESTABLISHED
TCP EeePC:5152 EeePC:0 LISTENING
TCP EeePC:5354 EeePC:0 LISTENING
TCP EeePC:27015 EeePC:0 LISTENING
TCP EeePC:27015 localhost:1053 ESTABLISHED
TCP EeePC:62514 EeePC:0 LISTENING

balt@EeePC /cygdrive/c/Program Files/Cumulus

[balt]

The upload is still not working, even after restarting Cumulus. Here's the FTP log:

14:30:10:698 ! HighLevelAsync 0
14:30:11:291 00FF80E0 TriggerDataSent 1836
14:30:11:776 >|220-Microsoft FTP Service|

14:30:12:244 >| ------|

14:30:12:244 >| DFW1.OPENATC.COM|

14:30:12:244 >| Private Access Only.|

14:30:12:244 >| Please find somewhere else to hack. Thank you!|

14:30:12:244 >|220 ------|

14:30:12:244 ! HighLevelAsync 0
14:30:12:244 00FF80E0 TriggerDataSent 1836
14:30:12:479 >|331 Password required for aly.|

14:30:12:479 ! HighLevelAsync 0
14:30:12:479 00FF80E0 TriggerDataSent 1836
14:30:12:713 >|230 User aly logged in.|

14:30:12:713 ! HighLevelAsync 0
14:30:12:713 ! HighLevelAsync done
14:30:12:713 00FF80E0 TriggerDataSent 1836
14:30:12:947 >|250 CWD command successful.|

14:30:12:947 00FF80E0 TriggerDataSent 1836
14:30:13:166 >|200 Type set to A.|

14:30:13:166 ! HighLevelAsync 0
14:30:13:182 00FF80E0 TriggerDataSent 1836
14:30:28:648 ! HighLevelAsync 0
14:30:28:648 ! HighLevelAsync done
14:30:28:648 ! HighLevelAsync 0
14:30:28:663 ! HighLevelAsync 0
14:30:28:663 ! HighLevelAsync 0
14:30:28:679 ! HighLevelAsync 0
14:30:28:679 ! HighLevelAsync 0
14:30:28:694 ! HighLevelAsync 0
14:30:28:710 ! HighLevelAsync 0
14:30:28:726 ! HighLevelAsync 0
14:30:28:726 ! HighLevelAsync 0
14:30:28:741 ! HighLevelAsync 0
14:30:28:757 ! HighLevelAsync 0
14:30:28:757 ! HighLevelAsync 0
14:30:28:773 ! HighLevelAsync 0
14:30:28:773 ! HighLevelAsync 0
14:30:28:788 ! HighLevelAsync 0
14:30:28:788 ! HighLevelAsync 0
14:30:28:788 ! HighLevelAsync 0
14:30:28:788 ! HighLevelAsync 0
14:30:28:804 ! HighLevelAsync 0
14:30:28:804 ! HighLevelAsync 0
14:30:28:804 ! HighLevelAsync 0
14:30:28:819 ! HighLevelAsync 0
14:30:28:819 ! HighLevelAsync 0
14:30:28:819 ! HighLevelAsync 0
14:30:28:819 ! HighLevelAsync 0
14:30:28:835 ! HighLevelAsync 0
14:30:28:835 ! HighLevelAsync 0
14:30:28:835 ! HighLevelAsync 0
14:30:28:851 ! HighLevelAsync 0

[steve]

Presumably these ports are the ones opened by the ftp component for listening in active ftp mode, but I don't know why it would be opening so many. Are you using active ftp mode? if so, have you tried passive mode? What is your upload interval set to?

Unfortunately the ftp component's logging isn't always very helpful. If you turn on the debug log in Cumulus, that may give more info. Note that it only gets flushed to disc every five minutes and may contain passwords.

[balt]

Hi,

I've tried both active and passive mode. Passive mode for some reason fails (I have no firewalls inbetween, so I don't know why). I was updating in 5 minute intervals.

The solution now would be to have cumulus write the files just to the local directory. I can then automatically scp them onto the server (preferred to ftp anyway).

Does Cumulus write the files locally if no FTP upload is selected? It doesn't seem so.

Cheers

- Balt

[steve]

Does Cumulus write the files locally if no FTP upload is selected?

Remove your ftp host name leaving everything else the same, and it will just create all the files without trying to ftp them.

[balt]

That worked, thanks! We're approaching donation time with things starting to look good. Thanks for providing such a great software and support!

Cheers

- Balt

[steve]

No problem. It's a shame the built-in ftp won't work for you, though.

[balt]

FTP is a bit of a problem child in many respects. SSH/SCP/SFTP is a much easier option for most people, and it doesn't have the firewall issues that FTP does. SSH ports outbound are either open or not.

Cheers

- Balt

[steve]

SSH/SCP/SFTP is a much easier option for most people,

Does the cheap/free web hosting that most people use allow those?

[n9mfk]

Hi Steve e-rice.net does not
Beau

[mcpaton]

Hi

I am also having FTP problems... the files are uploaded, and named with an extension .htmtmp but are staying that way, and not replacing the old files. I have tried the 'delete old files first' option, with no luck...

Any ideas?

Thanks

[steve]

You've selected the 'rename' option, but it looks like your server doesn't support it. You can either turn on ftp logging, which will confirm this if it is the case, or you can just untick the rename option.

[mcpaton]

I had a look through the FTP log... there's a whole heap of this error:

Code: Select all

Cannot create a file when that file already exists

Do you want the whole log?

[steve]

Yes, I'll have a look at the whole thing, please. It's not clear whether your server isn't allowing the upload when the file already exists, which is a bit perverse (but someone else did have this problem with their server) or whether it's not allowing the rename when the file already exists (which I guess is a little more understandable). What do you have the 'rename' and 'delete before upload' options set to when this message is generated?

I suspect the solution will be to untick 'rename' and tick 'delete before upload'.

[mcpaton]

Never mind, turning off both the FTP rename and the delete before upload seems to have gotten it going. Strange, I am sure I tried that before...