Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.5 (build 3282) - 23 February 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Weatherlink to Cumulus log converter

Discussion specific to Davis weather stations
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

ramsded wrote:Where my humidity results were --- in my WLK file I get an output of -3276 and whether I have had any rain or not, a figure of -6553.6 for my rainfall rate. This also I assume affects the dayfile text file as I have some dewpoint values of -1838.
As noted above, it doesn't currently check for all of the 'no reading' values. The Davis Weatherlink document doesn't actually define them; I think they must be as defined in the serial protocol spec.
Also, once the monthly files are included and the dayfile updated, will Cumulus automatically recalculate to display all the high / lows etc in all the various display options, or are additional steps needed to achieve this.
You will need to use the various high/low editors to scan the new logs and update the highs and lows as required.
Steve
ramsded
Posts: 20
Joined: Tue 05 Mar 2013 7:38 pm
Weather Station: Davis VP2
Operating System: W10
Location: Yorkshire, England

Re: Weatherlink to Cumulus log converter

Post by ramsded »

Steve

Thanks for your reply.

Presumably if I were to replace all the affected values with 0, this would affect the high/lows and averages, is there a value that can be used so as not to do this, ie outside a range depending on the parameter that Cumulus would then ignore ?

Dave
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

It's tricky, and it's one of the reasons (apart from being too busy) that I haven't done any more to the converter. What Cumulus does when it can't get new data for whatever reason is to re-use the last value that it had. While this has its disadvantages, it does have a number of advantages (like not affecting highs and lows and having minimal affect on averages). It means that Cumulus doesn't need and doesn't have 'null' values, so there is no figure you can insert which will get ignored.

So the converter will probably have to behave in the same way as Cumulus (it already does for UV and solar).
Steve
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

steve wrote:The Davis Weatherlink document doesn't actually define them; I think they must be as defined in the serial protocol spec.
Actually, it's clear that the .wlk file 'dash values' are not the same as the serial protocol dash values - they can't be. For example, in the archive records in the serial protocol, the dash value for humidity is 255 and humidity is held as a whole number. In the wlk files, humidity is held as tenths of a percent, and the dash value is apparently, but not actually documented anywhere, minus 32768.
Steve
User avatar
mcrossley
Posts: 12641
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Weatherlink to Cumulus log converter

Post by mcrossley »

steve wrote: the dash value is apparently, but not actually documented anywhere, minus 32768.
= 0xFFFF = -1 = false ???
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

-32768 is one of the 'dash values' in the serial protocol; but they also use +32767, 255 and 0. So the wlk files could use any of those for various fields, but from the post above it appears that it also uses -65536.
0xFFFF
Ah hold on, yes - maybe the 'dash value' fields just have all of the bits set to 1.
Steve
ramsded
Posts: 20
Joined: Tue 05 Mar 2013 7:38 pm
Weather Station: Davis VP2
Operating System: W10
Location: Yorkshire, England

Re: Weatherlink to Cumulus log converter

Post by ramsded »

Having now used the converter to convert all my WLK files back to Jan 2000, I am experiencing some problems ( no doubt of my own making ) whilst editing the high / low data. Initially this worked fine as I converted the files and edited the data month by month and then year by year. But eventually I realised I could do it all and paste all the monthly files and update the dayfile.txt in one go, when I then tried to fetch the files to edit, it only gets some data for Jan and none for the other months from the dayfile and only populates approx. 50% of that from the log data.

All the monthly files are there and the dayfile.txt contains all the data from 01/01/2000 ?

Other than when viewing this data from this period, all other options - view month, year and NOAA only allow me to select back to 2003 from the drop down pick list, I believed you could go back to 2000?

Is it possible to edit the NOAA headers for the UK ie be able to change State to County ?
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

ramsded wrote:Having now used the converter to convert all my WLK files back to Jan 2000, I am experiencing some problems ( no doubt of my own making ) whilst editing the high / low data. Initially this worked fine as I converted the files and edited the data month by month and then year by year. But eventually I realised I could do it all and paste all the monthly files and update the dayfile.txt in one go, when I then tried to fetch the files to edit, it only gets some data for Jan and none for the other months from the dayfile and only populates approx. 50% of that from the log data.

All the monthly files are there and the dayfile.txt contains all the data from 01/01/2000 ?
Sorry, but I can't follow that. Are you saying that you are using Edit -> Monthly Records? If you zip up your dayfile.txt and attach it, I'll take a look.
Other than when viewing this data from this period, all other options - view month, year and NOAA only allow me to select back to 2003 from the drop down pick list, I believed you could go back to 2000?
It's because I started writing Cumulus in 2003, so there was no point in allowing selection of earlier years. I can change that fairly easily.
Is it possible to edit the NOAA headers for the UK ie be able to change State to County ?
No; I followed the 'standard' format so that the files would be the same as produced by other software - Weatherlink in particular. Does WL allow you to change that?
Steve
ramsded
Posts: 20
Joined: Tue 05 Mar 2013 7:38 pm
Weather Station: Davis VP2
Operating System: W10
Location: Yorkshire, England

Re: Weatherlink to Cumulus log converter

Post by ramsded »

Thank you for the offer of checking my files, these are hopefully attached and go back to 01/01/2000 and Yes, I am using Edit - monthly records.

It would be good if you can allow selection back to 2000 in a future update.

Regards the NOAA setup, as you say you cannot change the headers in WL, but your program is streets ahead so I thought I would ask.

Dave
You do not have the required permissions to view the files attached to this post.
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

It's because of the problems mentioned above with the converter; it doesn't currently ignore all of the 'dash' values. Your Weatherlink files have data missing, so your converted Cumulus logs have invalid data (values and timestamps) in them, and hence so does your dayfile.txt, from the very first line:

01/01/00,11.0,0,01:00,-0.2,23:59,7.9,00:15,-1109.65,34:07,-1109.65,34:07,-8323.1,34:07,0.2,3.4,57.3,4.0,01:00,-3276,34:07,-3276,34:07,621.79,0.0,-1838.0,34:08,-70.8,00:00,-110.9,00:15,0.2,07:30,-0.3,08:15,-1838.0,34:07,-1838.0,34:07,112,14.9,0.0,0,00:00,0,00:00

I'm working on it, as time permits. But it's hard to know what to do in some cases.
Steve
oman4eva
Posts: 23
Joined: Wed 23 Oct 2013 4:42 pm
Weather Station: davis vantage pro 2 plus
Operating System: windows 7
Location: Canada

Re: Weatherlink to Cumulus log converter

Post by oman4eva »

Hey Steve,

I'm getting lots of "Floating Point Division by Zero" errors upon trying to convert some of my old data. The partially converted file ends here

20/01/03,19:30,-9.6,1,-56.0,7.2,16.1,292,-6553.6,0.0,989.44,0.0,20.4,6,16.1,-18.3,-9.6,0,0,252.60,0.00,-18.6,0,0.0,292,0.0,0.0

While an original export of the jan 2003 data shows no issues here

03/20/2003 7:30 p 6.1 6.1 2.4 6.4 5.7 100 6.1 11.3 27.4 E 0.0 990.5 22.1 23 30
03/20/2003 8:00 p 5.7 5.7 4.3 6.2 5.6 100 5.7 8.0 22.5 E 0.0 990.3 22.2 22 30

the same january exported file does have dashes down the end of the file

03/31/2003 5:00 p -- -- -- -2.3 -2.3 -- -- 12.9 22.5 S 0.0 992.9 23.2 12 30
03/31/2003 5:30 p -1.1 -1.1 -2.8 -1.1 -2.2 100 -1.1 8.0 25.7 SSW 0.0 992.3 23.3 12 30
03/31/2003 6:00 p -- -- -- -- -- -- -- 0.0 0.0 --- 0.0 991.9 23.2 12 30
03/31/2003 6:30 p -- -- -- -- -- -- -- 0.0 0.0 --- 0.0 991.8 23.2 11 30

etc


any ideas if there's a fix here.

cheers
oman
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

If you attach that Jan 03 wlk file (you may need to zip it so the forum doesn't object to the extension), I'll run it here so I can see what the problem is. The converter does clearly need work, though, and I don't know when I'll find time to do it.
Steve
Safety
Posts: 54
Joined: Fri 09 Apr 2010 7:32 pm
Weather Station: Davis Pro
Operating System: Windows 11

Re: Weatherlink to Cumulus log converter

Post by Safety »

Steve,
First of all thank you for taking the time to produce the converter utility programme.
In my case I have a number of years of data to convert (2003-2010). Back in 2003 I had reception problems from my ISS so each month I have a few erroneous values in my archive files.
Therefore when I run the converter I get errors as well documented within this group topic. In my case my objective is to use the data so I can compare each year of weather data using the graphs from “Select-A-Graph”.
To get the converter working all I needed to do is delete records that were missing data so I used a programme call “WLKReaderWriter” in my case I can live with losing a few 15 minute data grabs. This got me thinking, if you are looking for any ideas on new features for your program, I would like to suggest the following features:-
To be able to undertake a batch Load of a number of WLK files with the intention of detecting invalid records.
Each invalid record would then be flagged so that they could be highlighted for deletion.
The criteria for detecting if a record is invalid or not could be based on if the first temperature field (the field after the date / time fields in your txt file) has an erroneous value. In my case when I checked all the records that I needed to delete this is the field that always displays a value of -1838
oman4eva
Posts: 23
Joined: Wed 23 Oct 2013 4:42 pm
Weather Station: davis vantage pro 2 plus
Operating System: windows 7
Location: Canada

Re: Weatherlink to Cumulus log converter

Post by oman4eva »

Hi Steve

I've attached the March 2003 wlk file which was one of the month's having issues.


I've gone through and copied the monthly log files to the data directory and updated the dayfile.txt by copying the newly generated one into the original. I had a few questions.

a) Does the order of the dayfile matter? the entries appear to be a bit out of order, perhaps cause i didn't convert them in chronological order.

b) While Cumulus can display the past records if you load them or look at monthly records etc. However if i go to my website http://www.atmosp.physics.utoronto.ca/w ... record.htm it doesn't show any records from prior to when i set cumulus up initially. I.e. i've been swapping between cumulus and weatherlink since October 2013 but have just converted data back to 2001. I assume there must be a way to update this monthlyrecord.htm file automatically?

cheers
oman
You do not have the required permissions to view the files attached to this post.
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Weatherlink to Cumulus log converter

Post by steve »

oman4eva wrote:I've attached the March 2003 wlk file which was one of the month's having issues.
I'll have a play with it when I get chance.
Does the order of the dayfile matter?
Probably not; for most things (e.g. the 'This Month' etc displays) it scans the entire file looking for the entries it needs. I can't guarantee that will always be the case, and it may be that some of the code does already assume the correct order.
I assume there must be a way to update this monthlyrecord.htm file automatically?
Use the high/low editors to fetch the values from the dayfile, as mentioned earlier in the thread. But the off-scale values generated by the current version of the converter will probably render that ineffective.
Steve
Post Reply