Welcome to the Cumulus Support forum.

Latest Cumulus MX release v3.4.0 (build 3064) - 19 February 2020.
    Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014

    See the Wiki for downloading either version.

    Another ftp issue

    Discussion of version 3 of Cumulus, which runs on Windows, Linux, and OS X. All Cumulus MX queries in here, please.

    Moderator: mcrossley

    thegasman
    Posts: 36
    Joined: Sun 24 Mar 2013 6:42 pm
    Weather Station: WMR88
    Operating System: Raspbian on Pi2
    Location: Ayr, Scotland, United Kingdom
    Contact:

    Another ftp issue

    Post by thegasman » Mon 30 Dec 2019 5:25 pm

    Hi,

    I have a similar, but different, issue to the current thread on ftp failures.
    My web host also upgraded their server on Christmas eve (why? :o) , and now I can't ftp to it from CumulusMX on a Raspberry Pi. I can connect fine using FileZilla, so I know my connection details are right.

    Cumulus manages to log in, but then seems to fail with the command OPTS UTF8 ON

    Code: Select all

    220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
    220-You are user number 7 of 50 allowed.
    220-Local time is now 17:03. Server port: 21.
    220-This is a private system - No anonymous login
    220-IPv6 connections are also welcome on this server.
    220 You will be disconnected after 15 minutes of inactivity.
    USER *******
    331 User ****** OK. Password required
    PASS <omitted>
    230-This server supports FXP transfers
    230 OK. Current restricted directory is /
    FEAT
    211-Extensions supported:
    UTF8
    EPRT
    IDLE
    MDTM
    SIZE
    MFMT
    REST STREAM
    MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
    MLSD
    PRET
    AUTH TLS
    PBSZ
    PROT
    TVFS
    ESTA
    PASV
    EPSV
    SPSV
    ESTP
    211 End.
    Text encoding: System.Text.UTF8Encoding
    OPTS UTF8 ON
    504 Unknown command
    
    After this error Cumulus seems to upload the realtimegauges file, but nothing else.
    So I've tried unchecking the 'use UTF8' box in internet settings, but Cumulus still seems to be sending OPTS UTF8 ON.
    PS. This is not a secured connection, it's plain ftp.

    Any ideas?

    Martin

    User avatar
    mcrossley
    Posts: 5996
    Joined: Thu 07 Jan 2010 9:44 pm
    Weather Station: Davis VP2
    Operating System: Stretch Lite rPi
    Location: Wilmslow, Cheshire, UK
    Contact:

    Re: Another ftp issue

    Post by mcrossley » Mon 30 Dec 2019 9:13 pm

    Odd, the server lists the UTF8 extension as being active, and the OPTS UTF8 ON is a valid command (it should be on by default on all modern FTP implementions).

    I'd ask the ISP why their server is rejecting the OPTS UTF8 command.

    User avatar
    mcrossley
    Posts: 5996
    Joined: Thu 07 Jan 2010 9:44 pm
    Weather Station: Davis VP2
    Operating System: Stretch Lite rPi
    Location: Wilmslow, Cheshire, UK
    Contact:

    Re: Another ftp issue

    Post by mcrossley » Mon 30 Dec 2019 9:30 pm

    More info: The "Use UTF8" option isn't used by the FTP client, is just for how Cumulus reads and writes the files sent over FTP.
    AFAIK, the FTP always sends an OPTS UTF8 ON command. The reponse is normally either...
    200 OK,... something like... UTF-8 enabled... or... Entering UTF-8 mode
    or
    501 option not supported
    or
    500|504 command not supported

    But the command failing should not stop the rest of the FTP process working.

    You should post the whole FTP log file so we can try and see what is going on. Plus the MXdiags file covering the same period.

    thegasman
    Posts: 36
    Joined: Sun 24 Mar 2013 6:42 pm
    Weather Station: WMR88
    Operating System: Raspbian on Pi2
    Location: Ayr, Scotland, United Kingdom
    Contact:

    Re: Another ftp issue

    Post by thegasman » Tue 31 Dec 2019 3:38 pm

    Hi Mark,
    But the command failing should not stop the rest of the FTP process working.
    Thank you, you are right, that was not the problem.
    However I'm still scratching my head to find what the issue is.
    Initially Cumulus would not connect by ftp at all - see the first mxdiags log. I fiddled with lots of settings, and now it will connect and upload realtime.txt and realtimegauges.txt, but none of the other files including the extra web files. There is no error showing in mxdiags though, or in the ftp log, it just seems not to be trying to upload the files.

    I can't figure out what's going on. It seems like an almighty co-incidence that this should happen the day my web hosts upgrade their systems after years of flawless ftp'ing, yet FileZilla transfers without problem using the same settings as Cumulus.

    Thanks for any help you can suggest,

    Martin
    You do not have the required permissions to view the files attached to this post.

    User avatar
    mcrossley
    Posts: 5996
    Joined: Thu 07 Jan 2010 9:44 pm
    Weather Station: Davis VP2
    Operating System: Stretch Lite rPi
    Location: Wilmslow, Cheshire, UK
    Contact:

    Re: Another ftp issue

    Post by mcrossley » Tue 31 Dec 2019 4:01 pm

    The last log is saying that you no longer have an ftp host defined?

    You are going to have to switch on debug logging to get any useful information, switch it on and let Cumulus run for a while and send the latest log file, and I'll need to see a copy of your Cumulus.ini file (please remove any usernames/passwords from it first).

    thegasman
    Posts: 36
    Joined: Sun 24 Mar 2013 6:42 pm
    Weather Station: WMR88
    Operating System: Raspbian on Pi2
    Location: Ayr, Scotland, United Kingdom
    Contact:

    Re: Another ftp issue

    Post by thegasman » Thu 02 Jan 2020 2:10 pm

    Hi Mark,

    Here are the files.
    Still seems to uploading the realtime files but nothing else. No obvious errors I can see.

    Thanks,

    Martin
    You do not have the required permissions to view the files attached to this post.

    User avatar
    mcrossley
    Posts: 5996
    Joined: Thu 07 Jan 2010 9:44 pm
    Weather Station: Davis VP2
    Operating System: Stretch Lite rPi
    Location: Wilmslow, Cheshire, UK
    Contact:

    Re: Another ftp issue

    Post by mcrossley » Thu 02 Jan 2020 5:31 pm

    I have just realised that you are on a VERY old version of Cumulus MX - build 3038 - from March 2016. I suggest you update as I cannot support that old version.

    thegasman
    Posts: 36
    Joined: Sun 24 Mar 2013 6:42 pm
    Weather Station: WMR88
    Operating System: Raspbian on Pi2
    Location: Ayr, Scotland, United Kingdom
    Contact:

    Re: Another ftp issue

    Post by thegasman » Thu 02 Jan 2020 6:19 pm

    I'm ahead off you...

    I thought you would say that, so I have already upgraded to the latest version. The problem persists :cry:

    Martin

    thegasman
    Posts: 36
    Joined: Sun 24 Mar 2013 6:42 pm
    Weather Station: WMR88
    Operating System: Raspbian on Pi2
    Location: Ayr, Scotland, United Kingdom
    Contact:

    Re: Another ftp issue

    Post by thegasman » Thu 02 Jan 2020 7:14 pm

    Hmm, files such as tempdata.json and winddata.json don't seem to be being updated by Cumulus either, but realtime.txt is... :?

    Martin

    User avatar
    mcrossley
    Posts: 5996
    Joined: Thu 07 Jan 2010 9:44 pm
    Weather Station: Davis VP2
    Operating System: Stretch Lite rPi
    Location: Wilmslow, Cheshire, UK
    Contact:

    Re: Another ftp issue

    Post by mcrossley » Fri 03 Jan 2020 11:05 am

    OK, now you are on the latest version, can you...

    1. Shutdown MX
    2. Edit Cumulus.ini and set debug, data, and FTP logging on

    Code: Select all

    [Station]
    Logging=1
    DataLogging=1
    [FTP site]
    FTPlogging=1
    
    3. Restart MX, and let it run for a while
    4. Zip up and attach the MXdiags and the FTP log file.

    Thanks

    thegasman
    Posts: 36
    Joined: Sun 24 Mar 2013 6:42 pm
    Weather Station: WMR88
    Operating System: Raspbian on Pi2
    Location: Ayr, Scotland, United Kingdom
    Contact:

    Re: Another ftp issue

    Post by thegasman » Fri 03 Jan 2020 8:32 pm

    Zip file attached.
    Realtime files continue to upload correctly, but no sign of the 'standard' files.

    Thanks again for your help,

    Martin
    You do not have the required permissions to view the files attached to this post.

    User avatar
    mcrossley
    Posts: 5996
    Joined: Thu 07 Jan 2010 9:44 pm
    Weather Station: Davis VP2
    Operating System: Stretch Lite rPi
    Location: Wilmslow, Cheshire, UK
    Contact:

    Re: Another ftp issue

    Post by mcrossley » Mon 06 Jan 2020 2:49 pm

    But the FTP log shows the files *are* being successfully transferred to your server, and then renamed successfully too?

    For example - today.php...

    Code: Select all

    2020-01-03 19:46:31.571 Uploading /home/pi/CumulusMX/web/todayT.htmtmp to today.php
    
    # OpenWrite("today.phptmp", Binary)
    
    # GetFileSize("today.phptmp")
    Command:  SIZE today.phptmp
    Response: 550 Can't check for file existence
    
    # OpenPassiveDataStream(AutoPassive, "STOR today.phptmp", 0)
    Command:  EPSV
    Response: 229 Extended Passive mode OK (|||35208|)
    Status:   Connecting to 198.91.81.12:35208
    Command:  STOR today.phptmp
    Response: 150 Accepted data connection
    Status:   Disposing FtpSocketStream...
    Response: 226 0.111 seconds (measured here), 53.46 Kbytes per second
    Response: 226-File successfully transferred
    Status:   Disposing FtpSocketStream...
    
    # Rename("today.phptmp", "today.php")
    
    # GetWorkingDirectory()
    Command:  PWD
    Response: 257 "/" is your current location
    
    # GetWorkingDirectory()
    Command:  PWD
    Response: 257 "/" is your current location
    Command:  RNFR /today.phptmp
    Response: 350 RNFR accepted - file exists, ready for destination
    Command:  RNTO /today.php
    Response: 250 File successfully renamed or moved
    2020-01-03 19:46:32.654 Completed uploading /home/pi/CumulusMX/web/todayT.htmtmp to today.php

    thegasman
    Posts: 36
    Joined: Sun 24 Mar 2013 6:42 pm
    Weather Station: WMR88
    Operating System: Raspbian on Pi2
    Location: Ayr, Scotland, United Kingdom
    Contact:

    Re: Another ftp issue

    Post by thegasman » Mon 06 Jan 2020 5:10 pm

    Sorry, I should have explained, as a workaround I added those files to 'Extra web files', and checked realtime, to get them to upload to the server, restoring some functionality to my website.
    Screenshot (2).png
    Realtime ftp works, but nothing else gets ftp'ed. Nothing set to go at the standard upload interval goes, and the jsons with the trend data are not being updated (tempdata.json etc). I'm thinking a file must be corrupted somewhere, but Cumulus doesn't seem to be giving any error message. What is the sequence at the upload interval? Do the trend files get updated and then all the files ftp'ed? Maybe if a trend file is corrupted the sequence gets interupted and the ftp never takes place? I'm guessing though.

    Martin
    You do not have the required permissions to view the files attached to this post.

    BCJKiwi
    Posts: 889
    Joined: Mon 09 Jul 2012 8:40 pm
    Weather Station: Davis VP2 Cabled with Solar
    Operating System: Windows 10 Pro
    Location: Auckland, New Zealand
    Contact:

    Re: Another ftp issue

    Post by BCJKiwi » Mon 06 Jan 2020 9:31 pm

    Not sure exactly what you are attempting but in the Remote Filename column there is no path except for testT.txt which is being sent to the same folder (web) as testT.txt and testT.txttmp files.

    Typically this would look something like httpdocs/websitefolder/destination filename
    ie, the same destination as you have set in Settings - Internet settings - Directory

    User avatar
    mcrossley
    Posts: 5996
    Joined: Thu 07 Jan 2010 9:44 pm
    Weather Station: Davis VP2
    Operating System: Stretch Lite rPi
    Location: Wilmslow, Cheshire, UK
    Contact:

    Re: Another ftp issue

    Post by mcrossley » Mon 06 Jan 2020 9:42 pm

    He has everything uploading to the FTP root folder, so the paths are correct.

    Post Reply