Welcome to the Cumulus Support forum.

Latest Cumulus MX release v3.0.0 build 3050 - 1 May 2019.
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014

See the Wiki for downloading either version.

FTP TLS 1.2 upload not working with v3.0.0 build 3045

Discussion of version 3 of Cumulus, which runs on Windows, Linux, and OS X. All Cumulus MX queries in here, please.

Moderator: mcrossley

Post Reply
Dinant
Posts: 61
Joined: Sat 17 Aug 2013 9:11 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 2012 R2
Location: Markelo
Contact:

FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by Dinant » Mon 31 Dec 2018 4:55 pm

Today I replaced b3043 with b3045 because I want to know if the FTP upload with TLS1.2 is working right now.
I got errors during the upload.
In MXdiag is says:

Code: Select all

2018-12-31 17:45:00.668 Error connecting ftp - The handshake failed due to an unexpected packet format.
2018-12-31 17:45:00.715 Error uploading web\graphconfig.json to /cumulus/graphconfig.json : The handshake failed due to an unexpected packet format.
2018-12-31 17:45:00.762 Error uploading web\tempdata.json to /cumulus/tempdata.json : The handshake failed due to an unexpected packet format.
I enabled FTP logging.

Code: Select all

2018-12-31 17:45:00.590 Connecting to ftp.xxx.xx

# Connect()
Status:   Connecting to xxx.xxx.xxx.xxx:21
Response: 220 ::ffff:xxx.xxx.xxx.xxx FTP server ready
Command:  AUTH TLS
Response: 234 AUTH TLS successful
2018-12-31 17:45:00.668 Uploading web\graphconfig.json to /cumulus/graphconfig.json

# OpenWrite("/cumulus/graphconfig.json", Binary)
Status:   There is stale data on the socket, maybe our connection timed out or you did not call GetReply(). Re-connecting...
Status:   Disposing FtpSocketStream...
I also tried to add disableEPS to cumulus.ini:

Code: Select all

[FTP site]
DisableEPS=1
but without success. Anyone a suggestion?

Dinant
Posts: 61
Joined: Sat 17 Aug 2013 9:11 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 2012 R2
Location: Markelo
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by Dinant » Mon 31 Dec 2018 5:11 pm

I've read the release notes of b3045 again and now I see I missed the line:
Now supports SFTP over TLS1.1 and 1.2

I 've tested with FTPS not SFTP! So FTPS won't work.
I'll try with SFTP and let you know the results.

Dinant
Posts: 61
Joined: Sat 17 Aug 2013 9:11 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 2012 R2
Location: Markelo
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by Dinant » Mon 31 Dec 2018 5:26 pm

My provider only permits me to connect to the web server with FTPS with username/password or SSH with a SSH private/public keypair.
Cumulus 3.00 b3045 only supports SFTP with username and password authentication.
So it seems I cannot connect to the web server with Cumulus b3045.

On the Github site of FluentFTP is says:
Full support for FTP, FTPS (FTP over SSL), FTPS with client certificates and FTPS with CCC (for FTP firewalls)
That's confusing. Does Cumulus support SFTP or FTPS?

User avatar
mcrossley
Posts: 5595
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by mcrossley » Mon 31 Dec 2018 7:00 pm

Oops, a slip of the keyboard - it is of course FTPS.

CMX does not support SSH file transfers, nor does it support FTPS with client certificates.

B3045 is working OK in my test environment to my NAS using FTPS (and EPSV).

Code: Select all

Status:   Connecting to #######:21
Response: 220 #### FTP server ready.
Command:  AUTH TLS
Response: 234 AUTH TLS command successful.
Status:   FTPS Authentication Successful
Status:   Time to activate encryption: 0h 0m 0s.  Total Seconds: 0.7225609.

Dinant
Posts: 61
Joined: Sat 17 Aug 2013 9:11 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 2012 R2
Location: Markelo
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by Dinant » Tue 01 Jan 2019 11:04 am

OK, I understand. FTPS is the only protocol supported.
In the meantime I've send an email to my provider. Maybe they can see something in the logfiles of the server why the FTPS upload is not working.

Dinant
Posts: 61
Joined: Sat 17 Aug 2013 9:11 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 2012 R2
Location: Markelo
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by Dinant » Wed 02 Jan 2019 11:47 am

I've tested with my Synology NAS and the FTP upload is working fine.
I see a 15 seconds delay between the files.

Code: Select all

2019-01-02 12:10:01.046 Uploading web\graphconfig.json to /weer/cumulus/graphconfig.json
2019-01-02 12:10:16.046 Uploading web\tempdata.json to /weer/cumulus/tempdata.json
2019-01-02 12:10:31.312 Uploading web\pressdata.json to /weer/cumulus/pressdata.json
2019-01-02 12:10:46.578 Uploading web\winddata.json to /weer/cumulus/winddata.json
2019-01-02 12:11:01.845 Uploading web\wdirdata.json to /weer/cumulus/wdirdata.json
2019-01-02 12:11:17.110 Uploading web\humdata.json to /weer/cumulus/humdata.json
2019-01-02 12:11:32.423 Uploading web\raindata.json to /weer/cumulus/raindata.json
2019-01-02 12:11:47.689 Uploading web\solardata.json to /weer/cumulus/solardata.json
2019-01-02 12:12:02.970 Uploading web\dailyrain.json to /weer/cumulus/dailyrain.json
2019-01-02 12:12:18.251 Uploading web\sunhours.json to /weer/cumulus/sunhours.json
2019-01-02 12:12:33.501 Uploading web\dailytemp.json to /weer/cumulus/dailytemp.json
Does anyone else see such a delay?

The upload to the webserver is still not working. I am still investigating this together with my provider.

User avatar
mcrossley
Posts: 5595
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by mcrossley » Wed 02 Jan 2019 1:38 pm

I have found a problem with the TLS implementation in b3045, it affects the 'normal' periodic uploads using TLS1.2, but not the realtime uploads. I'll fix it in b3046. Sorry about this.

Dinant
Posts: 61
Joined: Sat 17 Aug 2013 9:11 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 2012 R2
Location: Markelo
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by Dinant » Wed 02 Jan 2019 3:33 pm

No problem. I am very happy that you are being able to update the Cumulus software. I tried it myself but I'm not able to understand the whole package but I can help with certain things. So if you need someone to help, test, etc. I'll be there to help.

My provider told me the software is not connecting with TLS 1.1 or TLS 1.2.
In the Cumulus.cs file I think there is an error.

Code: Select all

if (Sslftp)
{
    conn.EncryptionMode = FtpEncryptionMode.Explicit;
    conn.DataConnectionEncryption = true;
    conn.ValidateCertificate += Client_ValidateCertificate;
    // b3045 - switch from System.Net.Ftp.Client to FluentFTP allows us to specifiy protocols
    RealtimeFTP.SslProtocols = SslProtocols.Default | SslProtocols.Tls11 | SslProtocols.Tls12;
 }
I think this should be change to:

Code: Select all

if (Sslftp)
{
    conn.EncryptionMode = FtpEncryptionMode.Explicit;
    conn.DataConnectionEncryption = true;
    conn.ValidateCertificate += Client_ValidateCertificate;
    // b3045 - switch from System.Net.Ftp.Client to FluentFTP allows us to specifiy protocols
    conn.SslProtocols = SslProtocols.Default | SslProtocols.Tls11 | SslProtocols.Tls12;
 }
Is this the error you are talking about?

User avatar
mcrossley
Posts: 5595
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by mcrossley » Wed 02 Jan 2019 3:48 pm

Yep - classic cut and paste, and when I tested it my NAS was allowing fallback to SSL :(
I just retested it now with enforced TLS 1.2 only and it works fine.

Dinant
Posts: 61
Joined: Sat 17 Aug 2013 9:11 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 2012 R2
Location: Markelo
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by Dinant » Wed 02 Jan 2019 7:28 pm

I have installed B3046 and now the FTPS upload is fine.

The first time I enabled ActiveFTP. I got an error saying "500 Illegal PORT command".

Code: Select all

# OpenActiveDataStream(PORT, "STOR /xxx/graphconfig.json", 0)
Command:  PORT xxx,xxx,xxx,xxx,yyy,zzz
Response: 500 Illegal PORT command
Status:   Disposing FtpSocketStream...
After disabling Active FTP mode the upload went well.
In the ftplog.txt file is says:

Code: Select all

# OpenPassiveDataStream(AutoPassive, "STOR /xxx/dailytemp.json", 0)
Command:  EPSV
Response: 229 Entering Extended Passive Mode (|||35047|)
Status:   Connecting to xxx.xxx.xxx.xxx:yyy
Command:  STOR /xxx/dailytemp.json
Response: 150 Opening BINARY mode data connection for /xxx/dailytemp.json
Status:   FTPS Authentication Successful
Status:   Time to activate encryption: 0h 0m 0s.  Total Seconds: 0.0156678.
Status:   Disposing FtpSocketStream...
Response: 226 Transfer complete
Status:   Disposing FtpSocketStream...
Now the 15 seconds delay is also gone. The 12 json files are transferred in less than one second.

User avatar
mcrossley
Posts: 5595
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: FTP TLS 1.2 upload not working with v3.0.0 build 3045

Post by mcrossley » Wed 02 Jan 2019 8:59 pm

Dinant wrote:
Wed 02 Jan 2019 7:28 pm
Now the 15 seconds delay is also gone. The 12 json files are transferred in less than one second.
Good, I suspect it was failing the connection and negotiating down the protocols. The way it is configured the periodic upload shares the realtime connection, it just creates another control channel. So the connection was made originally as TLS then the periodic connection didn't have TLS enabled - confusion reigns!

Post Reply