Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Converting weatherlink.com csv download files to Cumulus log files

Discussion specific to Davis weather stations
Post Reply
Defenderdogs
Posts: 19
Joined: Mon 19 Apr 2021 6:32 am
Weather Station: Davis Vantage Pro2 Plus
Operating System: RPi3 Buster

Converting weatherlink.com csv download files to Cumulus log files

Post by Defenderdogs »

For various reasons, some self-inflicted via a lack of comprehensive backups and some Davis-inflicted via corrupt data in my logger (anyone else think Davis support is 'sub-optimal'?), I have lost several months of data between 2015 and now. Finding CumulusMX was a godsend given that it allows me to retire the WL PC app for all but a couple of functions but now I want to move all my history across. In order to get a complete history I need to download it from weatherlink.com, where I have a Pro account, but the only option on offer is year by year downloads in csv format.

My tech skills are insufficient for me to be able to manipulate these csv files and convert them into something Cumulus can understand (I have tried but get pretty seriously lost) so I was wondering if anyone had a utility like the log converter discussed in this thread to do something similar with the wl.com csv files?

A long shot I suspect, but thought it worth asking before giving up on getting my history in...
User avatar
mcrossley
Posts: 12685
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Converting weatherlink.com csv download files to Cumulus log files

Post by mcrossley »

I don't believe such a beast exists. I haven't played with the data export, but another complication is that format of the report depends on the user selecting all the correct columns.

It does not look too difficult to select the base data required for the monthly log files. You would then have to split the data into months, re-order the columns, maybe change the date/time format, and finally save as monthly log files.

Then you would have to run the Create Missing utility to build the dayfile from the monthly files.

Then set the records began date in Cumulus back to the start of your imported data.

Finally set all the records using the various records editors.

I see several evenings of tinkering, tweaking and swearing! :lol:
Defenderdogs
Posts: 19
Joined: Mon 19 Apr 2021 6:32 am
Weather Station: Davis Vantage Pro2 Plus
Operating System: RPi3 Buster

Re: Converting weatherlink.com csv download files to Cumulus log files

Post by Defenderdogs »

Let me know if I should start another thread for this.

So the swearing has already started! I thought I would first mock up a single monthly file in Excel using the Cumulus data and the format for the Monthy file from here - https://cumuluswiki.org/a/ImportCumulusFile#

From this format it looks like the date and time fields (LogDate) should be a single field, but when I did that in Excel, formatting the field as a date-time field per the spec, the import failed telling me Field 2 of Row 1 is the wrong format. When I split the field into two, date and time, it worked. Also, if I look at the current, May21log.txt file in my data folder, the date and time are also split into date and time.

Is the import specification incorrect or am I missing something by a couple of post codes?

Marc
User avatar
mcrossley
Posts: 12685
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Converting weatherlink.com csv download files to Cumulus log files

Post by mcrossley »

In the log file the date and time are separate fields. That Wiki page refers to the MySQL database schema where they are combined into a single date/time value.

You should refer to this page for the monthly log file format: https://cumuluswiki.org/a/Standard_log_files
Post Reply