Weather extreme records after data import

[Chris_Va]

Good morning,

I have an issue after data import I would like to run by everyone. I have searched the forum and really didn't find anything related but if I missed something, I apologize.

I recently upgraded from VWS 15/Weatherlink to CumulusMX on April 24th. Everything seems to be working well. Thank you to all that develop and support it.

I have begun to import my old Weatherlink data using the converter tool I found. I converted one month (Mar 2021). I copied the converted dayfile entries and added them to the top of my active dayfile and then I copied over the monthly data file (Mar21log.txt).

In the Cumulus admin dashboard, the Data Log viewer shows my data from March. The NOAA reports will also generate for March correctly. Everything seems correct except my March data is not included in the Records section (All Time, March, This Year). I was hoping an "end-of-day" routine would populate this data but it did not. Do I need to perform any manual task to have my imported (old) data included in the Records report section?

Thank you very much for any assistance.
Chris

[mcrossley]

Hi Chris, welcome to the forum!

At the moment, creating records from "added" historic data is a bit of a laborious manual process.

Before we talk about that though, you may want to take a look at the Create Missing utility.

If you run this against your imported log files, it will create additional data fields in your day file that were not present when the conversion utility was written.

Once you have done that, then you will need to use the various Editor screens in the Cumulus MX dashboard to re-create your records.

The All Time records are the simplest, just load the data from your monthly log files using the button at the bottom of the screen, then you have to cut'n'paste the records (either from the day file or the log files whichever is best). Be warned though, it may well throw up some issues in your monthly log files that you will need to fix (silly values etc).

Then repeat for each of the other records editors - the Monthly Records are a pain to do! Some automation of this process is on the ToDo list, but I've made little process on it.

[Chris_Va]

Thank you Mark, for the welcome and the information. I will work with your suggestions this evening and make some progress (or make a mess as I usually do). I have records in WeatherLink back to 2003 and it would be nice if I could include them. Back in the early 2000's in the winter the temp would actually fall below 0F occasionally...

What are these unfinished projects you speak of? Never heard of such a thing

Thank you again for the support.
Chris

[mcrossley]

If you still have the records from WL, then just enter them directly into the records editors. Cumulus only updates records when they are exceeded.

You could still load the log files in the editors to check them for silly values though.

[Chris_Va]

Oh, OK Mark,

So are you saying I could continue my data import for previous records relating to NOAA report generation & data review. Then just enter my record data through the editor for my "extremes" reporting?

Thanks

[mcrossley]

Yep, that is what I would do. Some of your records may have occurred "between" log entries, so if you have a records generated from live data, use those in preference.

[Chris_Va]

Just wanted to drop a note back Mark. Following your instructions and using the "Create Missing" utility, I have successfully imported a few years of WL data. I have also revised my "records" reports using your suggestions. All is working well. You are correct, the Monthly records can be a bit taxing but I just had to do Jan-Apr so not too bad.

Thank you again for the assistance and support.

Chris