Welcome to the Cumulus Support forum.
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4019) - 03 April 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4019) - 03 April 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Dayfile.txt uploading to a web server via FTP?
Moderator: mcrossley
-
- Posts: 83
- Joined: Mon 30 Jun 2014 12:02 pm
- Weather Station: Instromet Mk 3
- Operating System: Raspberry Pi Zero & CumulusMX
- Location: Kent
Dayfile.txt uploading to a web server via FTP?
Hi,
I have spent some time trying to achieve this task and am failing so would like some advice.
I am setting up a system using CumulusMX with a Raspberry Pi.
All is working so far with a web server successfully displaying data.
On my existing set up using a PC/Cumulus 1 I have been using Toolbox to upload my DayFile.txt file to the server
every 24 hours to allow online data searching.
I can't seem to find a practical way to do this on the Raspberry Pi set up.
Has anyone been able to do this sort of task and if so how.
Many thanks
Dave
I have spent some time trying to achieve this task and am failing so would like some advice.
I am setting up a system using CumulusMX with a Raspberry Pi.
All is working so far with a web server successfully displaying data.
On my existing set up using a PC/Cumulus 1 I have been using Toolbox to upload my DayFile.txt file to the server
every 24 hours to allow online data searching.
I can't seem to find a practical way to do this on the Raspberry Pi set up.
Has anyone been able to do this sort of task and if so how.
Many thanks
Dave
-
- Posts: 40
- Joined: Sun 16 Dec 2012 11:19 pm
- Weather Station: WH-1080
- Operating System: Windows XP / 7, Ubuntu, Centos
- Location: South Gloucestershire
Re: Dayfile.txt uploading to a web server via FTP?
It should be pretty easy to achieve using either python or a shell script and Cron to schedule your job to do the upload. There shoudl be lots of examples if you google "FTP shell script" for example.
-
- Posts: 83
- Joined: Mon 30 Jun 2014 12:02 pm
- Weather Station: Instromet Mk 3
- Operating System: Raspberry Pi Zero & CumulusMX
- Location: Kent
Re: Dayfile.txt uploading to a web server via FTP?
Thanks for that, I did try and write a script but it seemed that my Pi could not run FTP.
Googling found many folk suggesting using everything except FTP, for security reasons, so not much help.
I thought there would be a 'simple' way forward.
Dave
Googling found many folk suggesting using everything except FTP, for security reasons, so not much help.
I thought there would be a 'simple' way forward.
Dave
- steve
- Cumulus Author
- Posts: 26701
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
Googling for a linux ftp script returns a number of pages; this one, for example - http://www.stratigery.com/scripting.ftp.html
You could use a modified version of the example given there, invoked either by cron or using the Cumulus 'daily command'.
If you don't have the ftp client installed on your Pi: sudo apt-get install ftp
ncftp is an nicer alternative to ftp which makes scripting easier. sudo apt-get install ncftp
You could use a modified version of the example given there, invoked either by cron or using the Cumulus 'daily command'.
If you don't have the ftp client installed on your Pi: sudo apt-get install ftp
ncftp is an nicer alternative to ftp which makes scripting easier. sudo apt-get install ncftp
Steve
-
- Posts: 40
- Joined: Sun 16 Dec 2012 11:19 pm
- Weather Station: WH-1080
- Operating System: Windows XP / 7, Ubuntu, Centos
- Location: South Gloucestershire
Re: Dayfile.txt uploading to a web server via FTP?
Your Pi should be able to run FTP, what error do you see if for example you type in "ftp" without the quotes and hit enter?
-
- Posts: 83
- Joined: Mon 30 Jun 2014 12:02 pm
- Weather Station: Instromet Mk 3
- Operating System: Raspberry Pi Zero & CumulusMX
- Location: Kent
Re: Dayfile.txt uploading to a web server via FTP?
Many thanks Steve once again and Karv for your help.
I had, in my searches, come across that site and had tried to use that script but had failed.
They often say it can be the simpler things that trip you up and in this case, to a Linux and Raspberry Pi rookie, it was.
I has assumed that as my test weather station and CumulusMX were talking to each other and my web pages that FTP
was running on my PI. It Wasn't!
Having actually loaded it with your suggested link Steve and having spend a morning writing a script and getting paths
correct. It all is now working as it should.
I am still puzzled how CumulusMX was able to use an FTP link to my server but I couldn't without installing it but that
is why I am an amateur and you guys are the experts.
Thanks once again and I am learning from my mistakes.
Dave
I had, in my searches, come across that site and had tried to use that script but had failed.
They often say it can be the simpler things that trip you up and in this case, to a Linux and Raspberry Pi rookie, it was.
I has assumed that as my test weather station and CumulusMX were talking to each other and my web pages that FTP
was running on my PI. It Wasn't!
Having actually loaded it with your suggested link Steve and having spend a morning writing a script and getting paths
correct. It all is now working as it should.
I am still puzzled how CumulusMX was able to use an FTP link to my server but I couldn't without installing it but that
is why I am an amateur and you guys are the experts.
Thanks once again and I am learning from my mistakes.
Dave
Last edited by Hobbyist on Tue 13 Oct 2015 4:07 pm, edited 1 time in total.
- mcrossley
- Posts: 12767
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
Cumulus has its own built-in FTP routines, it doesn't rely on the client that you use from the command line.
-
- Posts: 83
- Joined: Mon 30 Jun 2014 12:02 pm
- Weather Station: Instromet Mk 3
- Operating System: Raspberry Pi Zero & CumulusMX
- Location: Kent
Re: Dayfile.txt uploading to a web server via FTP?
Cheers Mark that makes total sense.
Dave
Dave
-
- Posts: 239
- Joined: Sat 13 Jun 2015 5:57 pm
- Weather Station: FineOffset WS3080
- Operating System: Debian Buster - Raspbian - Raspb
- Location: Germany - nearby Kassel
Re: Dayfile.txt uploading to a web server via FTP?
Sorry, when I am late, answering to this topic, but you don't need a "real" FTP Program to upload files to an FTP Server.
Already installed on the Raspberry PI is: curl
Syntax to upload a file to an FTP Server is:
curl -m 60 -T /path_to/source_file ftp://"FTPHOST":"FTPPORT" "FTPDIR"/ --user "FTPUSER":"FTPPASS"
-T, --upload-file FILE Transfer FILE to destination
-m, --max-time SECONDS Maximum time allowed for the transfer
have fun
Jan
Already installed on the Raspberry PI is: curl
Syntax to upload a file to an FTP Server is:
curl -m 60 -T /path_to/source_file ftp://"FTPHOST":"FTPPORT" "FTPDIR"/ --user "FTPUSER":"FTPPASS"
-T, --upload-file FILE Transfer FILE to destination
-m, --max-time SECONDS Maximum time allowed for the transfer
have fun
Jan
-
- Posts: 159
- Joined: Wed 13 May 2009 9:35 am
- Weather Station: Davis Vantage Pro 2 Wireless
- Operating System: Raspbian wheezy
- Location: Nr Ilkley, West Yorks, UK
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
As an extension of this original request, I was also running toolbox to copy the dayfile to my webserver. I have now created a script to upload it using cron from my raspberry pi, but is there a clever way to also update the MySQL server that I have running on my webserver with the contents of the dayfile?
I was using the "remote commands" option in Toolbox to run the ImportCumulusFile.php but there isn't an option to Cron on my webserver, other than to use one of the free utilities to do it.
Just wondering if I can use the "Daily Program" option to do this rather than utilising a third party utility? It is probably blindingly simple to many of you, but I can't fathom it (yet!).
I was using the "remote commands" option in Toolbox to run the ImportCumulusFile.php but there isn't an option to Cron on my webserver, other than to use one of the free utilities to do it.
Just wondering if I can use the "Daily Program" option to do this rather than utilising a third party utility? It is probably blindingly simple to many of you, but I can't fathom it (yet!).
bigmac
Menston Weather
Menston Weather
- steve
- Cumulus Author
- Posts: 26701
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
MX has MySQL updating built in, including one for dayfile.txt, you just need to configure it with your MySQL details and enable it. Or am I misunderstanding what you want to achieve?
Steve
-
- Posts: 159
- Joined: Wed 13 May 2009 9:35 am
- Weather Station: Davis Vantage Pro 2 Wireless
- Operating System: Raspbian wheezy
- Location: Nr Ilkley, West Yorks, UK
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
It's probably my understanding of how this is meant to work Steve! I was playing around with it the other day (when I caused my issue with the port number hanging CumulusMX), but it didn't seem to do anything to the MySQL server that was running on my website.
To help my understanding, the MySQL that sits in CumulusMX, is that an alternative to running it on my webspace? If so, I don't have anywhere near the knowledge required to then get this integrated with my website, so was just looking for an easy way to continue updating the MySQL server that sits on my webspace.
I have for the moment configured one of the free Cronjob utilities to update my webspace MySQL server (seems to work fine), so don't worry too much about detailed instruction and spending time on helping me understand it, you have enough on your plate! I am quite happy to try and figure things out in my own time
To help my understanding, the MySQL that sits in CumulusMX, is that an alternative to running it on my webspace? If so, I don't have anywhere near the knowledge required to then get this integrated with my website, so was just looking for an easy way to continue updating the MySQL server that sits on my webspace.
I have for the moment configured one of the free Cronjob utilities to update my webspace MySQL server (seems to work fine), so don't worry too much about detailed instruction and spending time on helping me understand it, you have enough on your plate! I am quite happy to try and figure things out in my own time
bigmac
Menston Weather
Menston Weather
- steve
- Cumulus Author
- Posts: 26701
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
MX acts as a client to update a table on your server. You have to give it the correct database and table names, and the schema has to match. I used the schema from http://wiki.sandaysoft.com/a/ImportCumulusFile as I thought it was the one most people were using already. There is more information in the announcement for build 3025, when I introduced it. Updates and any error messages are logged to the MXdiags files.
If you are using a different schema, you could do a 'custom' update instead of the default.
If you are using a different schema, you could do a 'custom' update instead of the default.
Steve
-
- Posts: 159
- Joined: Wed 13 May 2009 9:35 am
- Weather Station: Davis Vantage Pro 2 Wireless
- Operating System: Raspbian wheezy
- Location: Nr Ilkley, West Yorks, UK
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
Thanks Steve, very much appreciated. I am using the same schema, but I will re-read as I couldn't figure out how I could do this at a specified time, i.e. I upload the dayfile at 00:15 and then update the MySQL server a few minutes later at 00:30. These options only seemed to allow it to happen just before rollover, shouldn't it be after rollover, once the dayfile has been updated?
But that will be just my ignorance!
But that will be just my ignorance!
bigmac
Menston Weather
Menston Weather
- steve
- Cumulus Author
- Posts: 26701
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Dayfile.txt uploading to a web server via FTP?
If I've said somewhere that it happens just before rollover, that's wrong and needs correcting. It does it when it's added a new entry to dayfile.txt, as part of the rollover.
Steve