Welcome to the Cumulus Support forum.

Latest Cumulus MX V4 release 4.4.2 (build 4085) - 12 March 2025

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

If you are posting a new Topic about an error or if you need help PLEASE read this first viewtopic.php?p=164080#p164080

Upload historical data from text file

From Cumulus MX version 3 build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since, and has recently released Cumulus MX version 4. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

Post Reply
User avatar
GWCTas
Posts: 49
Joined: Sat 22 Mar 2014 1:26 am
Weather Station: Fine Offset WH2901c
Operating System: Mac OS X 13.6.3
Location: Perth, Tasmania, Australia
Contact:

Upload historical data from text file

Post by GWCTas »

I have recently (yesterday) converted my website weather www.lmacrc.com from a Wordpress plugin (abandoned) to Cumulus MX. I have all the daily data from 2021 in text files. Each file covers the data collected over the 24 hour period, every minute. Is it possible to import data into Cumulus MX and if so, what is the required file structure to do so?

TIA
George
freddie
Posts: 2870
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 24.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Upload historical data from text file

Post by freddie »

The Wiki (see link in pink panel above) gives information on required formats. Essentially there are monthly files that contain a month's data at whatever interval you recorded at - one set of recordings per line, and a Dayfile that contains summary data (max, min, rainfall etc.) for each day - one day per line.
Freddie
Image
RogerH
Posts: 11
Joined: Fri 05 Jan 2024 5:59 pm
Weather Station: Ecowitt WS69 + HP3500, GW2000, W
Operating System: Raspberry Pi 5

Re: Upload historical data from text file

Post by RogerH »

User avatar
mcrossley
Posts: 14384
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Upload historical data from text file

Post by mcrossley »

If you can create the monthly log files, then the Create missing utility will construct the day file you.
freddie
Posts: 2870
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 24.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Upload historical data from text file

Post by freddie »

RogerH wrote: Wed 19 Jun 2024 3:26 pm Is this the relevant page?
https://www.cumuluswiki.org/a/Standard_ ... in_3.nn?.0
Yes, that's the one. As Mark said, just do the monthly files and CreateMissing will generate the Dayfile for you.
Freddie
Image
User avatar
GWCTas
Posts: 49
Joined: Sat 22 Mar 2014 1:26 am
Weather Station: Fine Offset WH2901c
Operating System: Mac OS X 13.6.3
Location: Perth, Tasmania, Australia
Contact:

Re: Upload historical data from text file

Post by GWCTas »

Great and thank you for the replies.
Excuse my ignorance, but where do I find the "CreateMissing" utility? It is not on my Cumulus desktop.
User avatar
GWCTas
Posts: 49
Joined: Sat 22 Mar 2014 1:26 am
Weather Station: Fine Offset WH2901c
Operating System: Mac OS X 13.6.3
Location: Perth, Tasmania, Australia
Contact:

Re: Upload historical data from text file

Post by GWCTas »

User avatar
PaulMy
Posts: 4355
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: Upload historical data from text file

Post by PaulMy »

Hi,
Current version CreateMissing is included in the latest CumulusMX zip download. It is in the main CumulusMX folder, and run it from there.

Enjoy,
Paul
VP2+
C1 www.komokaweather.com/komokaweather-ca
MX https://komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX https://komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX https:// komokaweather.com/cumulusmx4/index.htm
Image
User avatar
GWCTas
Posts: 49
Joined: Sat 22 Mar 2014 1:26 am
Weather Station: Fine Offset WH2901c
Operating System: Mac OS X 13.6.3
Location: Perth, Tasmania, Australia
Contact:

Re: Upload historical data from text file

Post by GWCTas »

I see that the wiki here https://www.cumuluswiki.org/a/Standard_ ... n_the_file states the log file format as -
These files are automatically created with names in the form <Month><Year>log.txt.
In UK English locale, the file for August 2009 is called Aug09log.txt.

Whereas, MX is creating a file format as [YYYY][mm]log.txt?
User avatar
HansR
Posts: 6926
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Re: Upload historical data from text file

Post by HansR »

GWCTas wrote: Thu 20 Jun 2024 4:54 am I see that the wiki here https://www.cumuluswiki.org/a/Standard_ ... n_the_file states the log file format as -
These files are automatically created with names in the form <Month><Year>log.txt.
In UK English locale, the file for August 2009 is called Aug09log.txt.

Whereas, MX is creating a file format as [YYYY][mm]log.txt?
Correct. The wiki has not been maintained in this respect.
You need to use the current format [YYYY][mm]log.txt
Hans

https://meteo-wagenborgen.nl
CMX build 4070+ ● RPi 4B ● Linux 6.6.62+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
User avatar
GWCTas
Posts: 49
Joined: Sat 22 Mar 2014 1:26 am
Weather Station: Fine Offset WH2901c
Operating System: Mac OS X 13.6.3
Location: Perth, Tasmania, Australia
Contact:

Re: Upload historical data from text file

Post by GWCTas »

Thank you Hans
User avatar
mcrossley
Posts: 14384
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Upload historical data from text file

Post by mcrossley »

And they now use a fixed format of dot decimals, and comma field separators ;)
User avatar
GWCTas
Posts: 49
Joined: Sat 22 Mar 2014 1:26 am
Weather Station: Fine Offset WH2901c
Operating System: Mac OS X 13.6.3
Location: Perth, Tasmania, Australia
Contact:

Re: Upload historical data from text file

Post by GWCTas »

Thanks Mark, I noticed that in the text file.
By the way, I love the layout of your weather pages. Is that format available for others to use or is there a repository where anyone can view options of different web interfaces? :D

I also see that you use a RPi. At our Launceston Aero Club, I have installed a RPi Zero but it won't accept .NET 8 nor under Mono. I am currently running Cumulus for the club from my home, which is not ideal long term. The newer Rpi Zero 2W has a better processor and will run .NET 8. What Rpi do you use?
Post Reply