Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Edit Historical Data

From build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

Post Reply
AberDino
Posts: 8
Joined: Fri 22 Jul 2016 10:40 am
Weather Station: Davis Vue
Operating System: Windows 7
Location: Catterline

Edit Historical Data

Post by AberDino »

Hi All,

I've had a Davis weather station (VP2 ISS with Vue console with a WeatherLinkIP data logger) since 2011. As I live on the coast, the weather station regularly gets a hard time, and I've previously had to replace the outdoor temperature/humidity sensor and the anemometer. A little while ago, my outdoor humidity readings were starting to deviate again, so today I've replaced this sensor (previously replaced in December 2016, so it seems to last ~5 years).

Apologies if these has been asked previously. I did search the forum but couldn't find the answer I was looking for. What is the best way to edit the historical outdoor humidity readings? Should I simply remove these values, or replace them with dummy, more believable, values? What about the calculated data based on outdoor humidity, such as dew point? Can I force these to be recalculated, or should I simply delete/replace these values too? Last but not least, due to the extremely low outdoor humidity readings (0% at times), some of the records are false too. Can these be recalculated once the historical dataset has been corrected?

Thanks in advance for your help,

Pieter
AberDino
Posts: 8
Joined: Fri 22 Jul 2016 10:40 am
Weather Station: Davis Vue
Operating System: Windows 7
Location: Catterline

Re: Edit Historical Data

Post by AberDino »

So, after reading some of the reference information on the Cumulus Wiki I decided to have a go...

I stopped the running instance of Cumulus MX, and made copies of the monthly txt log files which contained the incorrect data. In Excel, I merged the "monthlyfileheader.txt" and monthly txt log files so I could see which columns contained the data I was looking for. I then proceeded to delete the erroneous outdoor humidity values, as well as all values which depend on the outdoor humidity values such as dew point, apparent temperature, etc. After removing the header, I saved as CSV files and renamed them back to txt files. I then took a similar approach with the dayfile.txt file (using the deayfileheader.txt file), and copied the updates files back to the Cumulus MX data directory.

After starting Cumulus MX, I used the 'Edit' menu in the web interface to correct the monthly / yearly / all-time records as required. I'm not sure whether this is the recommended approach, but it definitely seems to have worked :).

The only thing to be aware of, is that when you import the txt files into Excel, it automatically converts the date to the "dd/mm/yyyy" format, so you need to force this back to "dd/mm/yy" before you save the files. I found this out the hard way ;).

Thanks,

Pieter
AndyKF650
Posts: 704
Joined: Thu 20 Aug 2020 8:35 am
Weather Station: Davis VP2 Plus
Operating System: RPiv4 64bit Linux 12 Bookworm
Location: Jersey Channel Islands
Contact:

Re: Edit Historical Data

Post by AndyKF650 »

Saving as a .csv file gets around the formatting problem
AberDino
Posts: 8
Joined: Fri 22 Jul 2016 10:40 am
Weather Station: Davis Vue
Operating System: Windows 7
Location: Catterline

Re: Edit Historical Data

Post by AberDino »

That's what I did, but without formatting the date column, the dates in the saved CSV file were in the "dd/mm/yyyy" format.
User avatar
mcrossley
Posts: 12689
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Edit Historical Data

Post by mcrossley »

Yes, unless you force Excel to produce the date format you want it will go and do its own thing - probably based on your system wide locale settings.
Always be wary when editing in Excel.
Post Reply