Hello,
I just noticed that Cumulus MX was offline for me for several days (crashed maybe?) and obviously didn't record any weather data during that time. I have the missing data from EasyWeather in CSV format and also data from WeeWX (NOAA reports, etc.), but I'm unsure of the best way to import that data without messing up existing records. I've researched Dayfile.txt and can see the dates missing, but the number of rows is extensive and not sure if everything will line up neatly. The "Createmissing.exe" tool looks potentially useful, but the documentation doesn't seem to cover my exact use case.
Can anyone point me in the right direction for adding back this missing data?
Thanks!
Welcome to the Cumulus Support forum.
Latest Cumulus MX V4 release 4.0.1 (build 4023) - 16 May 2024
(Note that 4.1.0 (build 4024) - 05 June 2024 remains available, but usage of this version is not recommended for Davis stations - and the included utility in this distribution for migrating to v4 is known to contain errors affecting conversion of dayfile.txt)
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Latest Cumulus MX V4 release 4.0.1 (build 4023) - 16 May 2024
(Note that 4.1.0 (build 4024) - 05 June 2024 remains available, but usage of this version is not recommended for Davis stations - and the included utility in this distribution for migrating to v4 is known to contain errors affecting conversion of dayfile.txt)
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Cumulus MX offline for 3 days - how to import missing data
Moderator: mcrossley
- mcrossley
- Posts: 12966
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: Cumulus MX offline for 3 days - how to import missing data
Unfortunately the GW-1000 does not have a logger that records the data. So the usual method of winding Cumulus back and letting it catch up from the stations logger data will not work.
-
- Posts: 884
- Joined: Sat 16 Jul 2016 11:59 pm
- Weather Station: Davis VP2+ & GW1000 (Standalone)
- Operating System: Win10 Pro / rPi Buster
- Location: Australia
Re: Cumulus MX offline for 3 days - how to import missing data
Sounds like the dayfile will only need 3 records adding, but the Jul21Log.txt could be missing 3days x 24hours x RecordsPerHour lines.
I'd have a look at that file & see if you can rebuild the missing records in it from the CSV file.
Then test it all on an alternate machine with no station configured,
(Just not sure how possible that is in latest releases).
I'd have a look at that file & see if you can rebuild the missing records in it from the CSV file.
Then test it all on an alternate machine with no station configured,
(Just not sure how possible that is in latest releases).
:Now: :Today/Yesterday:
![Image](http://weather.inverellit.au/banner.php)
Main Station Davis VP2+ Running Via Win10 Pro.
Secondary Stations, Ecowitt HP2551/GW1000 Via rPi 3 & 4 Running Buster GUI.
:Local Inverell Ecowitt Station: :Remote Ashford Ecowitt Station:
Main Station Davis VP2+ Running Via Win10 Pro.
Secondary Stations, Ecowitt HP2551/GW1000 Via rPi 3 & 4 Running Buster GUI.
:Local Inverell Ecowitt Station: :Remote Ashford Ecowitt Station:
-
- Posts: 9
- Joined: Mon 08 Jun 2020 7:58 pm
- Weather Station: Ecowitt GW1000
- Operating System: Win10
Re: Cumulus MX offline for 3 days - how to import missing data
Hmm, ok, thanks both for your replies. It's only 3 days of data, so not a huge deal in the long run, but slightly annoying since it's rained quite a bit and now my yearly totals are off. (Not worried so much about temp/wind data.)
-
- Posts: 160
- Joined: Wed 06 Jan 2021 3:01 pm
- Weather Station: Ecowitt
- Operating System: Raspberry Pi
Re: Cumulus MX offline for 3 days - how to import missing data
I’ve been down this road rather a lot lately!! If you have the data, you can easily import the date, time and temperatures, without too much hassle. Get the data you do have in Cumulus format (date/time), do it in the month following as it’s much less complicated. Export the Cumulus file to a spreadsheet, cut the good data after the missing lines and save in a separate file. Paste the data you have from the alternative source into the bottom of the file, then paste back the rest of the data you previously saved. Save as a text file.
Delete the day records that aren’t complete from the day file for that month then run the ‘create missing’ program. This gives you entires for the days you’ve missed, albeit with some gaps which you can manually fill by editing the daylog. If you want a more in depth explanation let me know.
I’ve started building databases to make this sort of thing a lot easier, and have a database of ‘raw’ data in order to create missing as much as possible.
It’s time consuming, irritating and oddly addictive
. It would help of course, if all weather stations and weather programs used data in a nice, standardised format…….. but then what would I do on dark winter evenings
![Laughing :lol:](./images/smilies/icon_lol.gif)
Delete the day records that aren’t complete from the day file for that month then run the ‘create missing’ program. This gives you entires for the days you’ve missed, albeit with some gaps which you can manually fill by editing the daylog. If you want a more in depth explanation let me know.
I’ve started building databases to make this sort of thing a lot easier, and have a database of ‘raw’ data in order to create missing as much as possible.
It’s time consuming, irritating and oddly addictive
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
![Laughing :lol:](./images/smilies/icon_lol.gif)
![Laughing :lol:](./images/smilies/icon_lol.gif)
-
- Posts: 884
- Joined: Sat 16 Jul 2016 11:59 pm
- Weather Station: Davis VP2+ & GW1000 (Standalone)
- Operating System: Win10 Pro / rPi Buster
- Location: Australia
Re: Cumulus MX offline for 3 days - how to import missing data
Would be curious to see the csv file you have.
Is it from an SD card that is installed in the console?
You could upload a zip of it out of interest, along with details of it's origin.
Phil.
Is it from an SD card that is installed in the console?
You could upload a zip of it out of interest, along with details of it's origin.
Phil.
:Now: :Today/Yesterday:
![Image](http://weather.inverellit.au/banner.php)
Main Station Davis VP2+ Running Via Win10 Pro.
Secondary Stations, Ecowitt HP2551/GW1000 Via rPi 3 & 4 Running Buster GUI.
:Local Inverell Ecowitt Station: :Remote Ashford Ecowitt Station:
Main Station Davis VP2+ Running Via Win10 Pro.
Secondary Stations, Ecowitt HP2551/GW1000 Via rPi 3 & 4 Running Buster GUI.
:Local Inverell Ecowitt Station: :Remote Ashford Ecowitt Station: