Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

WDConverter

From build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

Post Reply
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

WDConverter

Post by 92merc »

When's the last time any one has used the WDConverter?

I am running both WD and Cumulus. But I have the daytime file PHP file on my site to read that end of day file. I really like that. What I was able to do is take my old data from the VWS days and import it into WD. My goal is to get all of my data back from 2004 until 2017 when I started Cumulus.

When I'm entering in all of my US units into the converter, point to the file, and save, I get in the log file "022017lg.txt line 1: ''79.0'' is not a valid integer value".

The first couple of lines of my data file are:
01 02 2017 00 05 13.4 78.0 7.8 30.260 7.0 11.0 325 0 0.000 0.000 0.000 13.4
01 02 2017 00 10 13.3 79.0 8.0 30.260 10.0 14.0 317 0 0.000 0.000 0.000 13.3

I've tried renaming the files like I saw in a previous post, but to no avail. Hoping someone has recently done a conversion that can give me some assistance.
User avatar
beteljuice
Posts: 3292
Joined: Tue 09 Dec 2008 1:37 pm
Weather Station: None !
Operating System: W10 - Threadripper 16core, etc
Location: Dudley, West Midlands, UK

Re: WDConverter

Post by beteljuice »

You don't make it clear if the couple of lines from the 'data' and the 'log' file are the same thing.

However: " line 1: "79.0'' is not a valid integer value". Is a correct statement ! - because of the quotes it is a string.

I don't know about line #1 as the figure appears in line#2, or is the log just an error report as things 'cycle' ?
Image
......................Imagine, what you will KNOW tomorrow !
freddie
Posts: 2434
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: WDConverter

Post by freddie »

I don't think that's quite correct, beteljuice. The error message has placed quotes around the value 79.0 to highlight it. But the error message is accurate in that 79.0 isn't a valid integer value - it is a floating point value. 79 would be an integer value. So the converter software didn't expect to find a floating point value at that position.
Freddie
Image
User avatar
beteljuice
Posts: 3292
Joined: Tue 09 Dec 2008 1:37 pm
Weather Station: None !
Operating System: W10 - Threadripper 16core, etc
Location: Dudley, West Midlands, UK

Re: WDConverter

Post by beteljuice »

... it is a floating point value
:bash: "fools rush in"

Edit: Re-think, but why is it happy with 78.0 in the previous line ??
Last edited by beteljuice on Wed 03 Jul 2019 2:11 pm, edited 1 time in total.
Image
......................Imagine, what you will KNOW tomorrow !
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

Re: WDConverter

Post by 92merc »

I'll see if I can take that one log file and open it up in Excel to change that column to a non-decimal number. If that works, I can live with that.
water01
Posts: 3215
Joined: Sat 13 Aug 2011 9:33 am
Weather Station: Ecowitt HP2551
Operating System: Windows 10 64bit
Location: Burnham-on-Sea
Contact:

Re: WDConverter

Post by water01 »

You would be better using an Editor such as Notepad++ https://notepad-plus-plus.org/ as this preserves all ASCII notations. plus Excel adds all sorts of extraneous Microsoft stuff.
David
Image
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

Re: WDConverter

Post by 92merc »

Problem with that is there is almost 20k lines for each log file. Editing all those lines will be time killing.

I did open them up in Excel. What's weird is the 79.0 opens as 79. It dropped the decimal. I'll have to play around with it to see if Excel can export it back out like that.
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

Re: WDConverter

Post by 92merc »

Actually, I think I figured out a workable solution. I did a Macro in NP++. It smoked that extra .0 out, then went to the next line. It actually converted most the data.

I didn't notice that down around lines 11979, that number mysteriously changed to not having the .0. So I'll have to visually scan the logs and see if there are weird anomalies like that. But I think that will be a workable solution.

Next thing, is what do I do with the resulting log file? I see I can plop the file into that data folder. But can those new logs be "processed" to make it into the "dayfile.txt" file?
User avatar
PaulMy
Posts: 3777
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: WDConverter

Post by PaulMy »

Glad you're making progress. I was looking for a past discussion on using Notepad++ RegEx (RegularExpression) plugin that helped me a few years to edit dayfile.txt. You may be past this but here is the link viewtopic.php?f=4&t=15949&p=122214#p122214

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www.komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX www.komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX www. komokaweather.com/cumulusmx4/index.htm

Image
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

Re: WDConverter

Post by 92merc »

Thanks. I'll look into RegEx.
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

Re: WDConverter

Post by 92merc »

Just as an FYI, I got the REGEX figured out with some assistance. I used ^.{17}\-{0,1}\d+\.\d+\h-{0,1}\K(\d+)\.0 with a replace with /1.

So I converted the missing parts of Feb2017, and added Jan2017 files to my data folder. I shutdown Cumulus while posting those files. Is there anything I have to do to get Cumulus to look at the logs and add the results to my dayfile.txt file? Trying to get a handle on the process before I add more data for the other years.
User avatar
PaulMy
Posts: 3777
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: WDConverter

Post by PaulMy »

Is there anything I have to do to get Cumulus to look at the logs and add the results to my dayfile.txt file?
Cumulus 1 has the Create Missing in the Edit Dayfile.txt but that is not available in CumulusMX. You could, if you're brave, run Cumulus 1 and have it create the missing data. I have used that edit feature in the past.

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www.komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX www.komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX www. komokaweather.com/cumulusmx4/index.htm

Image
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

Re: WDConverter

Post by 92merc »

HA HA! Freakin' A! Thanks Paul. I did it.

https://www.bismarckweather.net/dayfile2.php

For anyone else, I threw a copy of 1.9.4 on my PC. Copied the data folder over. Plopped a couple of new log files I had converted. In Cumulus 1.94., Edit, Dayfile.Txt. Had it "Create Missing". I then took the data that was added to that 1.94. version of the dayfile.txt, Edited it with Notepad++ and added the missing data. I've filled in my missing data to round out 2017.

Now I just need to convert the 2004-20016 data and put that up. Thanks for the help.
User avatar
PaulMy
Posts: 3777
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: WDConverter

Post by PaulMy »

You're very welcome.

Just a reminder,
The usual dayfile.txt has the days' actual highs and lows.
The monthly log files gives the actual data at each log interval, so may have missed a daily high or low if that occurred outside of the actual log recording. However, using the Create Missing to populate dayfile.txt from the monthly log is better than nothing.

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www.komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX www.komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX www. komokaweather.com/cumulusmx4/index.htm

Image
92merc
Posts: 59
Joined: Wed 15 Feb 2017 6:07 pm
Weather Station: Davis VP2
Operating System: Windows 11 Pro
Location: Bismarck, ND USA
Contact:

Re: WDConverter

Post by 92merc »

I was able to get all of my data converted into the dayfile. The one thing I did notice is the windrun value is off. Just guessing, but it's almost a factor of 2 or more. But the rest of the data looks good.

I'm satisfied. I got most of what I was looking to do. There are a few days in there where WUnderground was missing data. Not much I can do about that. I did have some of the data still from my old VWS to import into WD. But not enough to fill the single day holes.

Thanks for the assistance.
Post Reply